Re: introduced an auth callback

On Thu, May 17, 2012 at 08:46:26PM +0000, Joe Mason wrote:

> I've pushed 3 new patches to
> https://github.com/JoeNotCharles/curl/commits/authcallback_api2, and
> also updated
> https://github.com/JoeNotCharles/curl/commits/authcallback_api2_squashed
> with a force push.
> 
> - added a "succeeded" member to curl_auth_info, which is always 0.  (I
> tried to find a place to add a call to the auth callback once auth is
> successful, but couldn't find the right place.  Someone with more
> knowledge of curl may have to do this)
> - renamed CURLAUTH_TYPE_HOST to CURLAUTH_TYPE_HTTP so we can add
> TYPE_FTP, TYPE_SMTP, etc later
> - fixed the bug with CURLOPT_FAILONERROR

Sorry for the slow response. I got a chance tonight to play more with my
proof-of-concept patch to make git use the auth callback. I confirmed
that the FAILONERROR bug is fixed. However, I did run into a new bug.

If the requested URL is something like:

  https://example.com/

then it works fine. But if it contains a username, like:

  https://user <at> example.com/

then the callback is never invoked, and curl returns a 401. The problem
seems to be that the logic in Curl_http_auth_act checks

failed tests building curl 7.26

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Re: DELELE occurence error later

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

RE: introduced an auth callback

> From: curl-library-bounces <at> cool.haxx.se [curl-library-bounces <at> cool.haxx.se] on
>  behalf of Jeff King [peff <at> peff.net]
> Sent: Friday, June 01, 2012 2:40 AM
> To: libcurl development
> Subject: Re: introduced an auth callback
> 
> Sorry for the slow response. I got a chance tonight to play more with my
> proof-of-concept patch to make git use the auth callback. I confirmed
> that the FAILONERROR bug is fixed. However, I did run into a new bug.

No problem.  I haven't had time to work on the auth callback myself recently.  I hope to get back to it soon. 
(Since it's on github, feel free to submit patches if you can fix anything!)

> If the requested URL is something like:
> 
>   https://example.com/
> 
> then it works fine. But if it contains a username, like:
> 
>   https://user <at> example.com/
> 
> then the callback is never invoked, and curl returns a 401. The problem
> seems to be that the logic in Curl_http_auth_act checks
> conn->bits.user_passwd; if it is set, then we assume we don't need to
> gather more credentials. But we do; we don't actually have a password.

Yes, I thought it was very strange that user_passwd only checks username.  But I followed the way curl
already did it since I didn't want to risk changing bits I didn't fully understand.

> Furthermore, specifying this URL:
> 
>   https://user:wrong <at> example.com/
> 
> also does not trigger the callback; it just returns a 401. Shouldn't it
> trigger the callback (possibly with the retries count incremented)?
> If we did that, it would fix both issues (when we have just a username,
> presumably we send the username and a blank password in the first
> request, so it can be considered a special case of having the wrong
> password).

That's an interesting bug.  I agree, that would be the best fix.  I'll look into it.

> > I didn't touch the url, since I think we've agreed to do that through
> > curl_easy_getinfo.
> 
> Has anybody been working on that?

Not to my knowledge.

Joe
---------------------------------------------------------------------
This transmission (including any attachments) may contain confidential information, privileged
material (including material protected by the solicitor-client or other applicable privileges), or
constitute non-public information. Any use of this information by anyone other than the intended
recipient is prohibited. If you have received this transmission in error, please immediately reply to
the sender and delete this information from your system. Use, dissemination, distribution, or
reproduction of this transmission by unintended recipients is not authorized and may be unlawful.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Re: failed tests building curl 7.26

Hi Neil,
Am 01.06.2012 10:41, schrieb Neil Bowers:
> I'm trying to build curl 7.26. Everything seems to go OK, but two tests
> failed:
>
>     TESTDONE: 547 tests out of 549 reported OK: 99%
>     TESTFAIL: These test cases failed: 554 587
>     TESTDONE: 681 tests were considered during 261 seconds.
>
>
> The things of note while running make test:
>
>     test 554...sh: line 1: 34194 Bus error: 10 ./libtest/lib554
>     http://127.0.0.1:8990/554 > log/stdout554 2> log/stderr554
>     FAILED
>     test 563...OK (389 out of 678, remaining: 01:53)
>     Will not run ssh server as root to mitigate security risks
>     RUN: failed to start the SSH server
>     test 587...sh: line 1: 34638 Segmentation fault: 11 ./libtest/lib587
>     http://127.0.0.1:8990/587 > log/stdout587 2> log/stderr587
>     FAILED
>
>
> Are these something I should worry about?
please run the tests as non-root user, and make sure stunnel is 
installed. Also IIRC we had at some time some bus errors and segfaults 
with one of my older autobuild machines, but with recent OSes it doesnt 
happen anymory ...
if you have some spare CPU cycles then perhaps consider to run some 
autobuilds [1] from cron - we dont have actually any MacOSX builds; that 
would greatly help with fixing such issues.

thanks, Gün.

[1] http://curl.haxx.se/dev/builds.html

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Re: failed tests building curl 7.26

Hej Gün,

please run the tests as non-root user, and make sure stunnel is installed. Also IIRC we had at some time some bus errors and segfaults with one of my older autobuild machines, but with recent OSes it doesnt happen anymory ...

Am 01.06.2012 10:41, schrieb Neil Bowers:

I'm trying to build curl 7.26. Everything seems to go OK, but two tests

failed:

   TESTDONE: 547 tests out of 549 reported OK: 99%

   TESTFAIL: These test cases failed: 554 587

   TESTDONE: 681 tests were considered during 261 seconds.

I did as requested, and now get:

TESTDONE: 581 tests out of 583 reported OK: 99%

TESTFAIL: These test cases failed: 554 587 

TESTDONE: 681 tests were considered during 436 seconds.

So more tests were run, and none of the new ones failed

The two tests which failed did so for the same reasons as before:

The things of note while running make test:

   test 554...sh: line 1: 34194 Bus error: 10 ./libtest/lib554

   http://127.0.0.1:8990/554 > log/stdout554 2> log/stderr554

   FAILED

   test 563...OK (389 out of 678, remaining: 01:53)

   Will not run ssh server as root to mitigate security risks

   RUN: failed to start the SSH server

   test 587...sh: line 1: 34638 Segmentation fault: 11 ./libtest/lib587

   http://127.0.0.1:8990/587 > log/stdout587 2> log/stderr587

   FAILED

if you have some spare CPU cycles then perhaps consider to run some autobuilds [1] from cron - we dont have actually any MacOSX builds; that would greatly help with fixing such issues.

I'm just setting that up. I noticed that in the first run, one more test failed:

TESTFAIL: These test cases failed: 554 564 587 

This is running on my mac laptop, so I'll play round with the best time for the cron to run. I'll force one now so you'll have a first set of data from it.

Let me know if there's anything more I can do to help?

Tack,

Neil

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

Re: DELELE occurence error later

On Fri, Jun 01, 2012 at 04:02:09PM +0200, M. Shinkaze wrote:
> Sorry for the late reply, I change the way to proceed but got an error
> "CURLE_QUOTE_ERROR" on the file "FILE.txt" to delete in the directory "dir" :
> 
> firstly, I connect directly the the directory which contains the file to
> suppress :
> CurlErrorCode = curl_easy_setopt(pCurl, CURLOPT_URL, "ftp://server/dir");

This URL specifies a *file* named dir, not a directory. You need a
trailing slash to specify a directory.

> CurlErrorCode = curl_easy_setopt(pCurl, CURLOPT_CONNECT_ONLY, 1L);

Why set this? It's not only documented to be useful for HTTP only, but
even if it were usable for FTP, it would prevent anything from
being sent.

> CurlErrorCode = curl_easy_setopt(pCurl, CURLOPT_USERNAME, user);
> CurlErrorCode = curl_easy_setopt(pCurl, CURLOPT_PASSWORD, passwd);
> CurlErrorCode = curl_easy_setopt(m_esayH, CURLOPT_CONNECT_ONLY, 1L);  -> here

You don't say what m_esayH is, but it doesn't seem relevant when calling
curl_easy_perform(pCurl);

> without this option I got an error CURLE_REMOTE_FILE_NOT_FOUND
> CurlErrorCode = curl_easy_perform(pCurl);
> 
> then on another function I am trying to delete the file :
> 
> CurlErrorCode = curl_easy_setopt(pCurl, CURLOPT_POSTQUOTE, "DELE File.txt");
> CurlErrorCode = curl_easy_perform(pCurl);
> 
> this last perform return "CURLE_QUOTE_ERROR"

If you enable debugging, you'll likely notice that libcurl never enters
the directory "dir", because "dir" is specified in the URL to be a file.

>>> Dan
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

RE: POP3 Authentication

Hi Daniel,

On Sun, 28 May 2012, Daniel Stenberg wrote:

> > As this introduces a new command I have updated the test cases as I see 
> > appropriate - however, being a Windows Developer and not having a Linux 
> > style console to hand I have been unable to verify these. I would
appreciate 
> > if someone could run them please and provide appropriate feedback -
thank 
> > you. 
>
> I've run them and they seem to run fine! 

Thank you - I think I missed Test 800, so I'm not sure how that continued
working or if the additional AUTH replies were even necessary :-/

> > * To change pop3_endofresp() depending on the feedback I receive  
> > * Introduce PLAIN and NTLM authentication mechanisms 
> > * Add new test cases for these 
>
> Sounds awesome. I'll take off on a little work trip to China this week so
I 
> might not respond to anything in a while, but don't let that stop you! 

I hope you had a good trip.

I have added support for the AUTH command and for PLAIN, LOGIN and NTLM
mechanisms - I think would also be worthwhile adding support for DIGEST-MD
and CRAM-MD5 as we have in SMTP - so I guess you could say I'm just over
half way through 

As such I am thinking about the test cases more and have attached my first
attempt at Test 816 (for PLAIN authentication) if someone would be so kind
to take a look.

I see Yang has now pushed some changes to the server code to support the
AUTH command, which is great, but I am a little lost as to what I need to do
to support some AUTH mechanisms in the test cases for POP3.

The first question is... Do I still need to have a replycmd for the AUTH
query in the xml? For example:

REPLY AUTH +OK\r\nPLAIN\r\n.

If so, how do I add a reply for the "AUTH PLAIN" that the client will send
when choosing plain as the mechanism? The server needs to respond with the +
command for continue. Would I need the following as a replycmd::

REPLY AUTH PLAIN +

With this would the server interpret that as a "PLAIN +" reply to AUTH or a
"+" reply to "AUTH PLAIN" ?

Or is this simply not needed and I need to tell the server what
authentication mechanisms are supported with another config option?

In this example, I would expect the full communication between the client
and server to be:

S: +OK cURL POP3 server ready to serve
C: AUTH
S: +OK
S: PLAIN
S: .
C: AUTH PLAIN
S: +
C: dGVzdAB0ZXN0ADEyMzQ=
S: +OK User authenticated
C: LIST
S: +OK No messages
C: QUIT
S: +OK byebye

Cheers in advance

Steve

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

RE: introduced an auth callback

On Fri, 1 Jun 2012, Joe Mason wrote:

>>> I didn't touch the url, since I think we've agreed to do that through 
>>> curl_easy_getinfo.
>>
>> Has anybody been working on that?
>
> Not to my knowledge.

I plan to get back in the game again properly on this topic, sorry for being a 
bit off for a while.

I believe this feature is setting a direction for how future applications will 
want to do authentication with libcurl so it is worth spending time and effort 
on making it right.

I would like us to get the basics working and then merge that into the master 
branch and then continue to work with Steve, Jeff and everyone else to polish 
it further and make sure it works for git, for non-HTTP protocols and that it 
provides a smooth API for applications.

It will also make sure we don't break any existing functionality (thanks to 
the auto-tests) and with our daily snapshots etc it will allow early adopters 
and willing testers to have a go at it and try it out before we release it in 
a public release.

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html

a sasl warning

Hi Steve,

Since Curl_ntlm_sspi_cleanup() is defined to nothing on some systems we get a 
warning in curl_sasl.c and I was thinking perhaps we can add something like 
the following to avoid the warning:

curl_sasl.c:195:44: error: unused parameter 'conn' [-Werror=unused-parameter]

The patch that works for me and I think would work on windows as well:

diff --git a/lib/curl_sasl.c b/lib/curl_sasl.c
index e9a3c18..407386d 100644
--- a/lib/curl_sasl.c
+++ b/lib/curl_sasl.c
 <at>  <at>  -198,10 +198,12  <at>  <at>  void Curl_sasl_cleanup(struct connectdata *conn, 
unsigned int auth
    /* Cleanup the ntlm structure */
    if(authused == SASL_AUTH_NTLM) {
      Curl_ntlm_sspi_cleanup(&conn->ntlm);
+    (void)conn; /* avoid warnings on systems where Curl_ntlm_sspi_cleanup()
+                   is just a nop */
    }
  #else
    /* Reserved for future use */
    (void)conn;
    (void)authused;

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette:  http://curl.haxx.se/mail/etiquette.html