Everything that concerns curl. Questions, bug reports, scripts and related tools and subjects welcome

Martin Maurer | 1 Jun 2011 13:12

curl (on Windows Vista 64 Bit edition) does only write partial file when doing download via redirect

Hello,

i am using

curl 7.21.6 (i386-pc-win32) libcurl/7.21.6 zlib/1.2.5
Protocols: dict file ftp gopher http imap ldap pop3 rtsp smtp telnet tftp
Features: AsynchDNS IPv6 Largefile libz

on Windows Vista 64 Bit edition.

I am calling curl as following:

once

curl http://192.168.178.145/ab/cde/fg/hijk.js > hijk.org

and later multiple times

curl http://192.168.178.145/ab/cde/fg/hijk.js > hijk.js

fc hijk.org hijk.js

I downloaded file hijk.js is equal to original file, i do the second url call again (i do this via batch file).

Around every two hours i see that hijk.js is a bit too small. curl itself reports the amount of bytes, which the original file had.

The written content seems to be on a 4 KByte (not completely sure, but at least on a 1 KByte boundary).

The last pieces from boundary to end of file is missing. It looks like writing to stdout which is redirected to hijk.js is not yet complete, when file compare is done.

According to wireshark it looks like the file was completely downloaded, with no error, also the last missing content is displayed as correctly transfered.

This leads me to the opinion, that something on PC side is not working.

Is there perhaps a flush missing ?

I am currently trying the following call instead:

curl -D log -o hijk.js -v -S http://192.168.178.145/ab/cde/fg/hijk.js

and the error has not occured yet. But it is too early to say, this way, it is working.

Best regards,

Martin

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

headers

Daniel Stenberg | 1 Jun 2011 13:35

Re: curl (on Windows Vista 64 Bit edition) does only write partial file when doing download via redirect

On Wed, 1 Jun 2011, Martin Maurer wrote:

> Around every two hours i see that hijk.js is a bit too small. curl itself 
> reports the amount of bytes, which the original file had. The written 
> content seems to be on a 4 KByte (not completely sure, but at least on a 1 
> KByte boundary). The last pieces from boundary to end of file is missing. It 
> looks like writing to stdout which is redirected to hijk.js is not yet 
> complete, when file compare is done. According to wireshark it looks like 
> the file was completely downloaded, with no error, also the last missing 
> content is displayed as correctly transfered. This leads me to the opinion, 
> that something on PC side is not working.
>
> Is there perhaps a flush missing ?

Since curl exits when done, all file handles are flushed automatically.

You can use --trace-ascii or similar to see exactly what curl has received.

In Windows land, lots of the stuff that is related to files and filesystems 
are a bit peculiar so it wouldn't at all surprise me if there at times could 
be so that the kernel hasn't yet completely finished writing all the data to 
the file when you try to read it. Possibly you should add a short pause 
between the curl invoke and the use of the file.

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

headers

zin tun kyaw | 2 Jun 2011 09:14

how to get "--data " and "-T" value from curl at the same time in a method

hi,
   I have a method to upload a file.
   This the method below.
   "directory" variable and contents are for file "-T" to upload.
   "data" variable is for "--data" for "metadata".

This is my curl command
curl -T C:\Users\a.jpg --data " { "metadata" : "username":"name"} -H "Content-Type:application/.." http://localhost:8080/user/folder/a.jpg -v

But I can't get the "--data" values from data variable.How can i do that?

<at> PUT
    <at> Path("/{container:.+}")
    <at> Consumes(MediaTypes.DataObject)
    <at> Produces(MediaTypes.DataObject)
    public Response doputDataObject( <at> PathParam("container")String directory,byte[]contents,String data) throws IOException, Exception
    {
        ...........
        if(bool==true){
           return Response.ok().header("X-CDMI-Specification", "1.0").build();
        }else{
            return Response.status(Response.Status.BAD_REQUEST).build();
        }

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

headers

Daniel Stenberg | 2 Jun 2011 22:14

Re: how to get "--data " and "-T" value from curl at the same time in a method

On Thu, 2 Jun 2011, zin tun kyaw wrote:

>  curl -T C:\Users\a.jpg --data " { "metadata" : "username":"name"} -H
> "Content-Type:application/.." http://localhost:8080/user/folder/a.jpg -v
>
> *But I can't get the "--data" values from data variable.How can i do that?*

You can't. You can only use -T _or_ --data. For a HTTP URL, -T makes a PUT and 
--data makes a POST.

But to do PUT and get data from two sources, you can do something like this:

  curl -X PUT --data  <at> file --data "metadata" ...

but that will append a & in there that I don't think you'll like.

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

headers

zin tun kyaw | 3 Jun 2011 02:56

Re: how to get "--data " and "-T" value from curl at the same time in a method

hi daniel,

     how about i used --upload-file with data ?
    It is also same with -T and --data.
    If i can use how to get --upload-file value from curl in web service method and how to set that into variable.

curl --upload-file C:\Users\a.jpg --data " { "metadata" : "username":"name"} -H
"Content-Type:application/.." http://localhost:8080/user/folder/a.jpg -v

thanks.

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

headers

Daniel Stenberg | 4 Jun 2011 00:17

Re: how to get "--data " and "-T" value from curl at the same time in a method

On Fri, 3 Jun 2011, zin tun kyaw wrote:

>     how about i used --upload-file with data ?

--upload-file and -T is the same option, just with two different names. See 
the man page.

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

headers

Richard Silverman | 6 Jun 2011 22:42

inappropriate GSSAPI delegation

Hello,

I'm writing to report a security bug in curl, which probably should not go
to the general list.  How should I go about this?

Thanks,

--

-- 
   Richard E. Silverman
   res <at> qoxp.net
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

headers

Daniel Stenberg | 6 Jun 2011 22:49

Re: inappropriate GSSAPI delegation

On Mon, 6 Jun 2011, Richard Silverman wrote:

> I'm writing to report a security bug in curl, which probably should not go 
> to the general list.  How should I go about this?

Hi Richard,

Thanks for asking and wanting to help us out. To report security issues, 
please email "curl-security at haxx.se". That's a list of trusted project team 
members, as we prefer to first work out the details and level of the problem 
in private so that we can release a fix and security alert at the same time 
(if necessary) to reduce the impact for our millions of users.

This is mentioned, and our previous vulnerability reports are collected here:
http://curl.haxx.se/docs/security.html

--

-- 

  / daniel.haxx.se
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

headers

Emerson Espínola | 7 Jun 2011 03:27

Using curl to send/receive email through Hotmail

Hi.

Is that possible to use curl lib to send and receive emails through my hotmail account? In other words, if I want to write a simple email client that uses my hotmail account to send and receive emails, is that possible to use curl to do that?

If so, what should I do?

[]'s
Emerson de Lira Espínola

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

headers

Daniel Stenberg | 7 Jun 2011 22:46

Re: Using curl to send/receive email through Hotmail

On Mon, 6 Jun 2011, Emerson Espínola wrote:

> Is that possible to use curl lib to send and receive emails through my 
> hotmail account? In other words, if I want to write a simple email client 
> that uses my hotmail account to send and receive emails, is that possible to 
> use curl to do that?
>
> If so, what should I do?

Yes it is possible. It can involve little to a lot of hard work. Start here:

 	http://curl.haxx.se/docs/httpscripting.html

--

-- 

  / daniel.haxx.se

-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-users
FAQ:        http://curl.haxx.se/docs/faq.html
Etiquette:  http://curl.haxx.se/mail/etiquette.html

26	May 2013
36	April 2013
44	March 2013
39	February 2013
27	January 2013
18	December 2012
47	November 2012
41	October 2012
37	September 2012
40	August 2012
30	July 2012
78	June 2012
122	May 2012
101	April 2012
69	March 2012
43	February 2012
72	January 2012
51	December 2011
50	November 2011
36	October 2011
43	September 2011
89	August 2011
51	July 2011
62	June 2011
55	May 2011
84	April 2011
78	March 2011
55	February 2011
98	January 2011
43	December 2010
66	November 2010
90	October 2010
65	September 2010
73	August 2010
67	July 2010
46	June 2010
89	May 2010
112	April 2010
83	March 2010
58	February 2010
38	January 2010
59	December 2009
104	November 2009
42	October 2009
67	September 2009
76	August 2009
91	July 2009
79	June 2009
75	May 2009
122	April 2009
121	March 2009
134	February 2009
157	January 2009
96	December 2008
84	November 2008
88	October 2008
110	September 2008
167	August 2008
183	July 2008
113	June 2008
124	May 2008
141	April 2008
105	March 2008
86	February 2008
98	January 2008
107	December 2007
127	November 2007
155	October 2007
92	September 2007
113	August 2007
63	July 2007
129	June 2007
101	May 2007
108	April 2007
132	March 2007
148	February 2007
171	January 2007
86	December 2006
154	November 2006
141	October 2006
165	September 2006
78	August 2006
87	July 2006
83	June 2006
84	May 2006
77	April 2006
91	March 2006
105	February 2006
132	January 2006
102	December 2005
152	November 2005
138	October 2005
123	September 2005
169	August 2005
95	July 2005
145	June 2005
107	May 2005
153	April 2005
153	March 2005
168	February 2005
175	January 2005
130	December 2004
172	November 2004
105	October 2004
108	September 2004
80	August 2004
132	July 2004
78	June 2004
139	May 2004
73	April 2004
147	March 2004
101	February 2004
101	January 2004
110	December 2003
139	November 2003
205	October 2003
122	September 2003
169	August 2003
207	July 2003
133	June 2003
181	May 2003
231	April 2003
139	March 2003
145	February 2003
171	January 2003
124	December 2002
123	November 2002
202	October 2002
81	September 2002
108	August 2002
76	July 2002

Mon	Tue	Wed	Thu	Fri	Sat	Sun

		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30