1 Apr 2011 09:14
1 Apr 2011 12:38
Re: Big dumps
I've been running them by hand and keeping an eye out for stalls (after we started seeing hangs at the abstractdump stage) but it looks like I nailed that bug. I'll start running them in endless loops soon. Ariel Στις 01-04-2011, ημέρα Παρ, και ώρα 09:14 +0200, ο/η Andreas Meier έγραψε: > Hello, > it seems to me, that you have to start each big dump manually. At the > moment there are only two big dumps produced, de and ru. pl is finished, > but no new dump is started. Couldn't you start they automatically like > the non-big dumps. > > Best regards, > Andreas > > _______________________________________________ > Xmldatadumps-l mailing list > Xmldatadumps-l@... > https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
3 Apr 2011 11:47
3 Apr 2011 14:42
Re: Reading 7z files with C#
Reading that the recommendation ends up at http://www.7-zip.org/sdk.html is this what you mean? It's a maze of source code so I am not immediately sure what to use. :)
On Sun, Apr 3, 2011 at 12:15 PM, Andrew Dunbar <hippytrail <at> gmail.com> wrote:
On 3 April 2011 19:47, White Cat <wikipedia.kawaii.neko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:There seems to be a bunch of answers to this on Stackoverflow:
> Hi, I am open to suggestions on library recommendations for reading 7z files
> with c# :)
http://stackoverflow.com/questions/3203701/sample-c-net-code-for-zipping-a-file-using-7zip
Andrew Dunbar (hippietrail)
> _______________________________________________
> Xmldatadumps-l mailing list
> Xmldatadumps-l <at> lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
>
>
3 Apr 2011 17:41
Re: Reading 7z files with C#
I am not a C# developer but this seems like a good start: http://www.eggheadcafe.com/tutorials/aspnet/064b41e4-60bc-4d35-9136-368603bcc27a/7zip-lzma-inmemory-com.aspx
Best
Diederik
Sent from my iPhone
On 2011-04-03, at 8:42, White Cat <wikipedia.kawaii.neko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Reading that the recommendation ends up at http://www.7-zip.org/sdk.html is this what you mean? It's a maze of source code so I am not immediately sure what to use. :)On Sun, Apr 3, 2011 at 12:15 PM, Andrew Dunbar <hippytrail-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:On 3 April 2011 19:47, White Cat <wikipedia.kawaii.neko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:There seems to be a bunch of answers to this on Stackoverflow:
> Hi, I am open to suggestions on library recommendations for reading 7z files
> with c# :)
http://stackoverflow.com/questions/3203701/sample-c-net-code-for-zipping-a-file-using-7zip
Andrew Dunbar (hippietrail)
> _______________________________________________
> Xmldatadumps-l mailing list
> Xmldatadumps-l-RusutVdil2icGmH+5r0DM0B+6BGkLq7r@public.gmane.org
> https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
>
>
_______________________________________________
Xmldatadumps-l mailing list
Xmldatadumps-l-RusutVdil2icGmH+5r0DM0B+6BGkLq7r@public.gmane.org
https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
3 Apr 2011 21:28
Re: Reading 7z files with C#
I really do not see a way to stream it like bz2's. :(
On Sun, Apr 3, 2011 at 5:41 PM, Diederik van Liere <dvanliere-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
I am not a C# developer but this seems like a good start: http://www.eggheadcafe.com/tutorials/aspnet/064b41e4-60bc-4d35-9136-368603bcc27a/7zip-lzma-inmemory-com.aspxBestDiederik
Sent from my iPhone
On 2011-04-03, at 8:42, White Cat <wikipedia.kawaii.neko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Reading that the recommendation ends up at http://www.7-zip.org/sdk.html is this what you mean? It's a maze of source code so I am not immediately sure what to use. :)On Sun, Apr 3, 2011 at 12:15 PM, Andrew Dunbar <hippytrail-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:On 3 April 2011 19:47, White Cat <wikipedia.kawaii.neko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:There seems to be a bunch of answers to this on Stackoverflow:
> Hi, I am open to suggestions on library recommendations for reading 7z files
> with c# :)
http://stackoverflow.com/questions/3203701/sample-c-net-code-for-zipping-a-file-using-7zip
Andrew Dunbar (hippietrail)
> _______________________________________________
> Xmldatadumps-l mailing list
> Xmldatadumps-l-RusutVdil2icGmH+5r0DM0B+6BGkLq7r@public.gmane.org
> https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
>
>
_______________________________________________
Xmldatadumps-l mailing list
Xmldatadumps-l-RusutVdil2icGmH+5r0DM0B+6BGkLq7r@public.gmane.org
https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
3 Apr 2011 22:13
Re: Reading 7z files with C#
Didn't realize you were looking for a streaming implementation......
Sent from my iPhone
Sent from my iPhone
On 2011-04-03, at 15:28, White Cat <wikipedia.kawaii.neko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
I really do not see a way to stream it like bz2's. :(On Sun, Apr 3, 2011 at 5:41 PM, Diederik van Liere <dvanliere-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:I am not a C# developer but this seems like a good start: http://www.eggheadcafe.com/tutorials/aspnet/064b41e4-60bc-4d35-9136-368603bcc27a/7zip-lzma-inmemory-com.aspxBestDiederik
Sent from my iPhone
On 2011-04-03, at 8:42, White Cat <wikipedia.kawaii.neko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
Reading that the recommendation ends up at http://www.7-zip.org/sdk.html is this what you mean? It's a maze of source code so I am not immediately sure what to use. :)On Sun, Apr 3, 2011 at 12:15 PM, Andrew Dunbar <hippytrail-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:On 3 April 2011 19:47, White Cat <wikipedia.kawaii.neko-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:There seems to be a bunch of answers to this on Stackoverflow:
> Hi, I am open to suggestions on library recommendations for reading 7z files
> with c# :)
http://stackoverflow.com/questions/3203701/sample-c-net-code-for-zipping-a-file-using-7zip
Andrew Dunbar (hippietrail)
> _______________________________________________
> Xmldatadumps-l mailing list
> Xmldatadumps-l-RusutVdil2icGmH+5r0DM0B+6BGkLq7r@public.gmane.org
> https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
>
>
_______________________________________________
Xmldatadumps-l mailing list
Xmldatadumps-l-RusutVdil2icGmH+5r0DM0B+6BGkLq7r@public.gmane.org
https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
4 Apr 2011 03:12
Re: Reading 7z files with C#
Hi,
I haven't seen a streaming C# implementation for 7z files (would be nice to have though), but here is some code in python that does it from Dmitry Chickov:
7z support added to xmlreader:
http://lists.wikimedia.org/pipermail/pywikipedia-l/2010-April/006169.html
interfacing to the pages-meta-history file for dump analysis:
http://code.google.com/p/wrdese/source/browse/trunk/b/verify-wiki-dump-print-empty.py
"Somewhat cleaned up version of the MediaWiki Dump Analysis code from the wredese project"
http://code.google.com/p/pymwdat/
cheers,
Jamie
----- Original Message -----
From: Diederik van Liere <dvanliere <at> gmail.com>
Date: Sunday, April 3, 2011 1:13 pm
Subject: Re: [Xmldatadumps-l] Reading 7z files with C#
To: White Cat <wikipedia.kawaii.neko <at> gmail.com>
Cc: "xmldatadumps-l <at> lists.wikimedia.org" <xmldatadumps-l <at> lists.wikimedia.org>
> Didn't realize you were looking for a streaming implementation......
>
> Sent from my iPhone
>
> On 2011-04-03, at 15:28, White Cat
> <wikipedia.kawaii.neko <at> gmail.com> wrote:
>
> > I really do not see a way to stream it like bz2's. :(
> >
> > On Sun, Apr 3, 2011 at 5:41 PM, Diederik van Liere
> <dvanliere <at> gmail.com> wrote:
> > I am not a C# developer but this seems like a good start:
> http://www.eggheadcafe.com/tutorials/aspnet/064b41e4-60bc-4d35-
> 9136-368603bcc27a/7zip-lzma-inmemory-com.aspx
> >
> > Best
> > Diederik
> >
> > Sent from my iPhone
> >
> > On 2011-04-03, at 8:42, White Cat
> <wikipedia.kawaii.neko <at> gmail.com> wrote:
> >
> >> Reading that the recommendation ends up at http://www.7-
> zip.org/sdk.html is this what you mean? It's a maze of source
> code so I am not immediately sure what to use. :)
> >>
> >> On Sun, Apr 3, 2011 at 12:15 PM, Andrew Dunbar
> <hippytrail <at> gmail.com> wrote:
> >> On 3 April 2011 19:47, White Cat
> <wikipedia.kawaii.neko <at> gmail.com> wrote:
> >> > Hi, I am open to suggestions on library recommendations for
> reading 7z files
> >> > with c# :)
> >>
> >> There seems to be a bunch of answers to this on Stackoverflow:
> >>
> >> http://stackoverflow.com/questions/3203701/sample-c-net-code-
> for-zipping-a-file-using-7zip
> >>
> >> Andrew Dunbar (hippietrail)
> >>
> >> > _______________________________________________
> >> > Xmldatadumps-l mailing list
> >> > Xmldatadumps-l <at> lists.wikimedia.org
> >> > https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
> >> >
> >> >
> >>
> >> _______________________________________________
> >> Xmldatadumps-l mailing list
> >> Xmldatadumps-l <at> lists.wikimedia.org
> >> https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
> >
>
4 Apr 2011 04:28
Re: Reading 7z files with C#
On 3 April 2011 21:47, White Cat <wikipedia.kawaii.neko@...> wrote: > Hi, I am open to suggestions on library recommendations for reading 7z files > with c# :) I usually use the command line "7za" program with the -so command line switch, which writes decompressed output to standard output. For example, a command line would be something like: 7za x -so http://download.wikimedia.org/enwiki/20110317/enwiki-20110317-pages-articles.xml.bz2 Execute this command line and capture the standard output as a stream. (For example, in Python this could be done with "subprocess.Popen()"). Greg Hewgill http://hewgill.com
8 Apr 2011 21:46
en history dumps hiccup in current (April) run
Observant dump watchers will notice that history piece 9 of the April en wikipedia dumps suddenly got tiny. That's because I restarted it. The compressor had gone out to lunch. I'll keep an eye out the next few days and see what happens; the rest continue to run uninterrupted. Ariel
RSS Feed