Marcin Cieslak | 1 Jan 18:28 2009
Picon

IPv6 on stable - thank you

Hi,

Big thank you for providing IPv6 for the stable toolserver.

I think I was using it unknowingly since around Dec 14th, excellent idea 
- so thanks again.

--Marcin

River Tarnell | 4 Jan 16:07 2009
Picon

Re: [Toolserver-announce] maintenance 28th to 31st Dec: all servers


yarrow (s1/s3) is now up.  as it wasn't replicating during the downtime, the
replication lag is fairly high for the moment.

	- river.
Frédéric Schütz | 4 Jan 21:06 2009
Picon

Statistics files on the toolserver (was: Re: 1200 MB every night)

DaB. wrote:

I am following up on a discussion here in October:

>> I run a script that downloads 1200 MB every night
> 
> if you do this, please save the data at 
> 
> /mnt/user-store/
> 
> (create a directoy there). So every usercan use the data and they have to 
> downloaded one 1 time.

Since I am becoming involved with statistics too, I have setup such a 
scheme in /mnt/user-store/stats.  Data files starting from 1 October 
2008 are currently available (emijrp asked if I could get older files 
too, which should be doable but I haven't looked into it yet). I still 
have to fine tune the update process, but basically a cron task will 
take care of this at least every day (probably more often, but I have to 
see when the original files are actually updated)

Let me know if anyone else is interested in using this data.

> Perhaps there is a better way (rsync or something) to get the data from the 
> source.

I use wget; it will not download files twice unless they have been 
modified (which should not happen). Also, files are already gz'ipped, so 
compression would not be of much use here. Even though rsync is a better 
solution on paper, all in all, I don't think it would improve the 
(Continue reading)

River Tarnell | 5 Jan 02:51 2009
Picon

s3 replication is halted


hi,

as Wikimedia deleted the MySQL binlogs required for replicating s3 on the
Toolserver, replication is now halted.  a full dump/import is required to fix
replication.  because disk space on yarrow is limited at the moment, and we are
about to receive new servers, i might wait until this before reimporting.
(however, if that's likely to take a long time, i might do it sooner.)

	- river.
Mashiah Davidson | 5 Jan 11:50 2009
Picon

Re: s3 replication is halted

Hello, River,


could you please to clarify "about to receive" and "long time" let say selecting a proper time measure: days, weeks, months. Our services depend on replicated data, so the delay might influence us and our users somehow.

Mashiah

2009/1/5 River Tarnell <river <at> loreley.flyingparchment.org.uk>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

hi,

as Wikimedia deleted the MySQL binlogs required for replicating s3 on the
Toolserver, replication is now halted.  a full dump/import is required to fix
replication.  because disk space on yarrow is limited at the moment, and we are
about to receive new servers, i might wait until this before reimporting.
(however, if that's likely to take a long time, i might do it sooner.)

       - river.
-----BEGIN PGP SIGNATURE-----

iD8DBQFJYWeSIXd7fCuc5vIRAo03AJ9ouEHS98oTwKPugzcUsc8Lp+LmSgCffFmd
gK3xul+k9UinKPPHH5Q3PWg=
=pnlB
-----END PGP SIGNATURE-----

_______________________________________________
Toolserver-l mailing list
Toolserver-l <at> lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/toolserver-l

<div>
<p>Hello, River,</p>
<div><br></div>
<div>could you please to clarify "about to receive" and "long time" let say selecting a proper time measure: days, weeks, months. Our services depend on replicated data, so the delay might influence us and our users somehow.</div>
<div><br></div>
<div>Mashiah<br><br><div class="gmail_quote">2009/1/5 River Tarnell <span dir="ltr">&lt;<a href="mailto:river <at> loreley.flyingparchment.org.uk">river <at> loreley.flyingparchment.org.uk</a>&gt;</span><br><blockquote class="gmail_quote">
-----BEGIN PGP SIGNED MESSAGE-----<br>
Hash: SHA1<br><br>
hi,<br><br>
as Wikimedia deleted the MySQL binlogs required for replicating s3 on the<br>
Toolserver, replication is now halted. &nbsp;a full dump/import is required to fix<br>
replication. &nbsp;because disk space on yarrow is limited at the moment, and we are<br>
about to receive new servers, i might wait until this before reimporting.<br>
(however, if that's likely to take a long time, i might do it sooner.)<br><br>
 &nbsp; &nbsp; &nbsp; &nbsp;- river.<br>
-----BEGIN PGP SIGNATURE-----<br><br>
iD8DBQFJYWeSIXd7fCuc5vIRAo03AJ9ouEHS98oTwKPugzcUsc8Lp+LmSgCffFmd<br>
gK3xul+k9UinKPPHH5Q3PWg=<br>
=pnlB<br>
-----END PGP SIGNATURE-----<br><br>
_______________________________________________<br>
Toolserver-l mailing list<br><a href="mailto:Toolserver-l <at> lists.wikimedia.org">Toolserver-l <at> lists.wikimedia.org</a><br><a href="https://lists.wikimedia.org/mailman/listinfo/toolserver-l" target="_blank">https://lists.wikimedia.org/mailman/listinfo/toolserver-l</a><br>
</blockquote>
</div>
<br>
</div>
</div>
Daniel Kinzler | 5 Jan 12:23 2009
Picon

Re: s3 replication is halted

Mashiah Davidson schrieb:
> Hello, River,
> 
> could you please to clarify "about to receive" and "long time" let say
> selecting a proper time measure: days, weeks, months. 

It will be about a month until the new boxes are online.

-- daniel

Pietrodn | 5 Jan 12:30 2009
Picon

Re: s3 replication is halted

Il giorno 05/gen/09, alle ore 12:23, Daniel Kinzler ha scritto:

> Mashiah Davidson schrieb:
>> Hello, River,
>>
>> could you please to clarify "about to receive" and "long time" let  
>> say
>> selecting a proper time measure: days, weeks, months.
>
> It will be about a month until the new boxes are online.
>
> -- daniel

Hmm... that's a long time. Can you restart the replication before  
please?

Pietrodn
powerpdn <at> gmail.com

Daniel Kinzler | 5 Jan 12:34 2009
Picon

Re: s3 replication is halted

Pietrodn schrieb:
> Il giorno 05/gen/09, alle ore 12:23, Daniel Kinzler ha scritto:
> 
>> Mashiah Davidson schrieb:
>>> Hello, River,
>>>
>>> could you please to clarify "about to receive" and "long time" let  
>>> say
>>> selecting a proper time measure: days, weeks, months.
>> It will be about a month until the new boxes are online.
>>
>> -- daniel
> 
> Hmm... that's a long time. Can you restart the replication before  
> please?

It's not a matter ofrestarting. The process is: take a slave server out of
rotation and make a dump (I don't have access on the main cluster, so I can't do
that), then copy the dump over, import it, and *then* start replication. This
takes quite some time (a week, i'd guess), and is a lot of hassle. Also, during
import, s3 is going to be completly unavailable for several days, instead of
just having old data.

Doing this twice is no fun. So it's not an easy choice. I leave it to river to
make it :)

-- daniel

Pietrodn | 5 Jan 13:14 2009
Picon

Re: s3 replication is halted

Il giorno 05/gen/09, alle ore 12:34, Daniel Kinzler ha scritto:

> Pietrodn schrieb:
>> Il giorno 05/gen/09, alle ore 12:23, Daniel Kinzler ha scritto:
>>
>>> Mashiah Davidson schrieb:
>>>> Hello, River,
>>>>
>>>> could you please to clarify "about to receive" and "long time" let
>>>> say
>>>> selecting a proper time measure: days, weeks, months.
>>> It will be about a month until the new boxes are online.
>>>
>>> -- daniel
>>
>> Hmm... that's a long time. Can you restart the replication before
>> please?
>
> It's not a matter ofrestarting. The process is: take a slave server  
> out of
> rotation and make a dump (I don't have access on the main cluster,  
> so I can't do
> that), then copy the dump over, import it, and *then* start  
> replication. This
> takes quite some time (a week, i'd guess), and is a lot of hassle.  
> Also, during
> import, s3 is going to be completly unavailable for several days,  
> instead of
> just having old data.
>
> Doing this twice is no fun. So it's not an easy choice. I leave it  
> to river to
> make it :)
>
> -- daniel

Oh, I understand. I didn't think it was such a long and difficult  
process.

Pietrodn
powerpdn <at> gmail.com

Simon Walker | 5 Jan 13:34 2009

Re: s3 replication is halted

How long are the binlogs kept for on Wikimedia servers?

Surely it would be possible to take a dump now, import it to s3, start
replication, then import the same dump onto the new server, and let it
catch up from a month of replag?

Of course, this wouldn't be possible if the binlogs are not kept for that long.

2009/1/5 Pietrodn <powerpdn <at> gmail.com>:
> Il giorno 05/gen/09, alle ore 12:34, Daniel Kinzler ha scritto:
>
>> Pietrodn schrieb:
>>> Il giorno 05/gen/09, alle ore 12:23, Daniel Kinzler ha scritto:
>>>
>>>> Mashiah Davidson schrieb:
>>>>> Hello, River,
>>>>>
>>>>> could you please to clarify "about to receive" and "long time" let
>>>>> say
>>>>> selecting a proper time measure: days, weeks, months.
>>>> It will be about a month until the new boxes are online.
>>>>
>>>> -- daniel
>>>
>>> Hmm... that's a long time. Can you restart the replication before
>>> please?
>>
>> It's not a matter ofrestarting. The process is: take a slave server
>> out of
>> rotation and make a dump (I don't have access on the main cluster,
>> so I can't do
>> that), then copy the dump over, import it, and *then* start
>> replication. This
>> takes quite some time (a week, i'd guess), and is a lot of hassle.
>> Also, during
>> import, s3 is going to be completly unavailable for several days,
>> instead of
>> just having old data.
>>
>> Doing this twice is no fun. So it's not an easy choice. I leave it
>> to river to
>> make it :)
>>
>> -- daniel
>
> Oh, I understand. I didn't think it was such a long and difficult
> process.
>
> Pietrodn
> powerpdn <at> gmail.com
>
>
> _______________________________________________
> Toolserver-l mailing list
> Toolserver-l <at> lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l
>

--

-- 
Regards,

Simon Walker
User:Stwalkerster on all public Wikimedia Foundation wikis
Administrator on the English Wikipedia
Developer of Helpmebot and the ACC tool

Your donations keep Wikipedia running! Support the Wikimedia
Foundation today: http://www.wikimediafoundation.org/wiki/Donate


Gmane