headers
Tim Hunter | 1 Jan 2008 01:20
Picon

Re: Best way to download >1GB files

thefed wrote:
> But doesn't open-uri download the whole thing to your compy? I was about 
> to use it, but then I ran it in irb and saw it returned a file object.

Isn't that what you want to happen? I thought your question was about 
how to download it in small chunks so it's not all in memory at the same 
time. This code downloads the whole file, but 8kb at a time.

--

-- 
RMagick: http://rmagick.rubyforge.org/
RMagick 2: http://rmagick.rubyforge.org/rmagick2.html
Permalink | Reply | 5 comments |
headers
Bryan Duxbury | 1 Jan 2008 01:23
Gravatar

Re: Best way to download >1GB files

Is there some reason to not use wget or curl? Those are both written  
already. What are you hoping to do with the files you download?

-Bryan

On Dec 31, 2007, at 2:04 PM, thefed wrote:

> What is the best way to download files from the internet (HTTP)  
> that are greater than 1GB?
>
> Here's the story in whole....
> I was trying to use Ruby Net::HTTP to manage a download from  
> wikipedia... Specifically all current versions of the english  
> one... But anyways, as I was downloading it, I got a memory error  
> as I ran out of RAM.
>
> My current code:
>       open( <at> opts[:out], "w") do |f|
>         http = Net::HTTP.new( <at> url.host,  <at> url.port)
>         c = http.start do |http|
>           a = Net::HTTP::Get.new( <at> url.page)
>           http.request(a)
>         end
>         f.write(c.body)
>       end
>
> I was hoping there'd be some method that I can attach a block to,  
> so that for each byte it will call the block.
>
> Is there some way to write the bytes to the file as they come in,
(Continue reading)

Permalink | Reply | 13 comments |
headers
Ian Whitlock | 1 Jan 2008 03:25
Picon

Re: Why not call Thread.join?

Craig Beck wrote:
>>   module Enumerable
>>   print "Completed operation for #{e}!\n"
>> end
>> # Completed operation for 8!
>> # Completed operation for 7!
>> # Completed operation for 9!
>> Time.now - start_time                 # => 5.009334
> 
> try looking at the crude timeline below...
> 
> sec  0         1         2         3         4         5
> 6         7
>   |---------|---------|---------|---------|---------|---------|---------|
> main    ==== <at> =================================================
> t[1]     ===================================================
> t[2]    ===================================================
> t[3]     ===================================================
> 
> The  <at>  on the main thread represents when the t.join gets called. It
> waits in this simple case for t[1] to finish it's work (sleeping for 5
> seconds), then waits for t[2]. As t[2] has also been doing work all
> this time, it only blocks the main thread for another 0.1 sec before
> finishing. Same for t[3]. So this contrived example it takes 5 seconds
> + whatever overhead for starting threads.
> 
> You could throw more instrumentation in there if you wish and do
> things like adding additional calls to sleep to simulate extra thread
> overhead to make it more obvious.
(Continue reading)

Permalink | Reply | 2 comments |
headers
Matt Todd | 1 Jan 2008 03:51
Picon
Gravatar

[ANN] Halcyon 0.3.18 Release

= Halcyon
== JSON Server App Framework

Halcyon is a JSON Web App Server Framework built on Rack for speed and
light weight.

For more information, check out our website at
http://halcyon.rubyforge.org/ or go straight to the RDocs:
http://halcyon.rubyforge.org/doc/

=== Announcement

I'm very pleased to release a heap of improvements to the initial
public version of Halcyon, now version 0.3.18, a beta release
candidate.

Also, stay tuned for release 0.4.0 which should occur in the next few
weeks, adding support for Analogger, daemonization and cluster
management, thorough testing, more examples, and some improvements to
signal handling.

Release Name: halcyon-0.3.18
Notes:
Halcyon has gone through a great deal of improvement, including
supporting proper logging, extensive configuration file improvements
like config files. Halcyon is now releasing to the wild for public
testing version 0.3.18.

Changes:
* Fixed many minor issues with dependencies and auto loading.
(Continue reading)

Permalink | Reply | 0 comments |
headers
Giles Bowkett | 1 Jan 2008 03:59
Picon
Gravatar

how to store a block in a var?

Mil perdones, but my brain seems to have gone awol. I have multiple
places in a spec where I need to run the same block of code. I would
be much happier storing this block in a var as a Proc as opposed to
wrapping it up in a method. Is there a way I can do this? My spec
looks like this:

do_stuff do
  stuff
end

where "stuff" is the part I want to wrap up into some tidy thing. I
tried it with lambda but my brain was too tired.

--

-- 
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com
Permalink | Reply | 3 comments |
headers
Tim Hunter | 1 Jan 2008 04:02
Picon

Re: how to store a block in a var?

Giles Bowkett wrote:
> Mil perdones, but my brain seems to have gone awol. I have multiple
> places in a spec where I need to run the same block of code. I would
> be much happier storing this block in a var as a Proc as opposed to
> wrapping it up in a method. Is there a way I can do this? My spec
> looks like this:
> 
> do_stuff do
>   stuff
> end
> 
> where "stuff" is the part I want to wrap up into some tidy thing. I
> tried it with lambda but my brain was too tired.
> 

var = Proc.new { stuff }

--

-- 
RMagick: http://rmagick.rubyforge.org/
RMagick 2: http://rmagick.rubyforge.org/rmagick2.html
Permalink | Reply | 2 comments |
headers
Giles Bowkett | 1 Jan 2008 04:06
Picon
Gravatar

Re: how to store a block in a var?

> var = Proc.new { stuff }

yah, but can I just pass it as a block of code?

do_stuff do
  var.call
end

??

the thing is that do_stuff is the only thing that makes the code in
var meaningful, so when I do that now, I'm getting NoMethodError: no
method "call" for nil

--

-- 
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com
Permalink | Reply | 1 comment |
headers
Chuck Remes | 1 Jan 2008 04:13
Picon

Zed Shaw - Ruby has dodged a bullet

Much like watching a car accident in slow motion, I could scarcely  
believe my eyes when reading Zed Shaw's screed on Ruby & Rails on his  
blog [1].

I know I shouldn't give an attention whore that which he so  
desperately seeks, but I can't help myself. I'm the guy down on the  
street shouting for the idiot on the ledge to jump. Not only has he  
jumped, but he has promised "future installments" to complete the  
tragedy. I cannot look away; I don't *want* to look away.

Any how... I think Kevin Clark labeled him correctly. See Zed's rant  
for the background on this reference.

If I was in a position to do so, I am sorely tempted to hire him just  
to fire him right away. :-)

Thankfully the mongrel project is in sane hands. He did a wonderful  
job developing mongrel for which he deserves full credit. His rant and  
apparent abandonment of Ruby is just peculiar in light of his past  
contributions.

A community with a significant reputation for helping novices and  
welcoming newbies doesn't have a lot of room for cranks. I hope Zed is  
welcomed by the community(ies) focused on his next language choice;  
based on his personality traits, I'd say he should join the LISP  
community ASAP. Even they might find him to be a bit much.

Happy New Year! Happy New Rants!

[1] http://www.zedshaw.com/rants/rails_is_a_ghetto.html
(Continue reading)

Permalink | Reply | 111 comments |
headers
Giles Bowkett | 1 Jan 2008 04:15
Picon
Gravatar

Re: def if ; end - basically impossible?

> You should be able to check the code out of svn.

Whoops! Duh. Will do.

> > Right now I'm passing blocks to a code generator, like this:
> >
> > Generator.module_eval(&block)
> >
> > I could probably do something which treats the code as strings, and
> > either passes them off to module_eval(&block) like this, *or* passes
> > them to send(:if) instead. Even a simple case statement might work.
>
> I'm having a hard time imaging what you are describing.
>
> You should be able to use self.if instead of send(:if).

The thing is, I want to be able to do this:

generating_code(ForSomething) do
  if variable != some_condition
    do :stuff
  else
    do(:other => stuff)
  end
end

And basically *not* use the if or else built into Ruby. I can do it
very easily if I do this:

gen_code(Etc) do
(Continue reading)

Permalink | Reply | 1 comment |
headers
M. Edward (Ed) Borasky | 1 Jan 2008 04:36
Favicon

Re: Zed Shaw - Ruby has dodged a bullet

Chuck Remes wrote:
> Much like watching a car accident in slow motion, I could scarcely
> believe my eyes when reading Zed Shaw's screed on Ruby & Rails on his
> blog [1].
> 
> I know I shouldn't give an attention whore that which he so desperately
> seeks, but I can't help myself. I'm the guy down on the street shouting
> for the idiot on the ledge to jump. Not only has he jumped, but he has
> promised "future installments" to complete the tragedy. I cannot look
> away; I don't *want* to look away.
> 
> Any how... I think Kevin Clark labeled him correctly. See Zed's rant for
> the background on this reference.
> 
> If I was in a position to do so, I am sorely tempted to hire him just to
> fire him right away. :-)
> 
> Thankfully the mongrel project is in sane hands. He did a wonderful job
> developing mongrel for which he deserves full credit. His rant and
> apparent abandonment of Ruby is just peculiar in light of his past
> contributions.
> 
> A community with a significant reputation for helping novices and
> welcoming newbies doesn't have a lot of room for cranks. I hope Zed is
> welcomed by the community(ies) focused on his next language choice;
> based on his personality traits, I'd say he should join the LISP
> community ASAP. Even they might find him to be a bit much.
> 
> Happy New Year! Happy New Rants!
>
(Continue reading)

Permalink | Reply | 67 comments |

Gmane