[stable] new Python

hi,

a new Python, version 2.6.2, is now available on stable.  this is *not* the
default Python; it is installed in /opt/ts/python/2.6, and to use it, you
should write something like this at the top of your script:

#! /opt/ts/python/2.6/bin/python

all previously-installed modules are also available with the new Python.  the
problem with $PYTHONPATH is also fixed, so there is no longer any need to set
that to use any modules.

i recommend all projects which use Python change to 2.6, as any new module
requests will only be built for this version.

	- river.

Re: [stable] new Python

River Tarnell ha scritto:
[...]
> ld write something like this at the top of your script:
> 
> #! /opt/ts/python/2.6/bin/python

What about creating "/usr/bin/python2.6" as symbolic link?

Python version 2.4 and 2.5 are already available as "/usr/bin/python2.4" 
and "/usr/bin/python2.5".

Thanks,
--

-- 
Francesco Cosoleto

What gets us into truoble is not what we don't know. It's what we know 
for sure that just ain't so. (Mark Twain)

potential downtime today

hi,

during today new Toolserver hardware will be installed.  there is a possibility
that this will cause some downtime.

	- river.

Servers installed, setup pending

Hi all

Just a quick update on the new servers: they are installed now, but need some
more setup. This should be complete by next week, or maybe the week after.

Regards,
Daniel

CPU use and 'nice'

hi,

please remember that 'nice' is not a magical fix for programs that use too much
CPU.  if a program uses a lot of CPU, it should be reniced; but that doesn't
mean that you can use as much CPU as you like as long as you renice it.

if you write a tool that uses 50% CPU constantly, you are using 6.25% of the
total available CPU on the Toolserver for yourself.  if everyone did this, we
could only support 16 users before running out of CPU.  we have a lot more than
16 users, so please be considerate, and monitor your CPU use.

	- river.

Replication is back after master-switch and schema-update

Hello all,

yesterday at highnoon the master-db-servers were switched because of an huge 
schema-update. Because we have only one cluster of db-servers at the moment we 
couldn't do the schema-update the same way like the foundation (the foundation 
stops one of their slaves, does the schema-update, waits, start it again, 
stops another slave, does the schema-update and so on until all slaves are 
updated, and declare one slave as new master, runs the schema-update on the 
old master = no downtime) and so it took several hours until the schema-update 
was done and we could switch to the new master-servers.
I deleted the view on the logging-table during the update to speed the update 
up and killed several long-running queries too tonight.
The update was finish before I wake-uped, and I just finished the 
masterswitch, restarted replication and recreated the logging-tables (see 
maintaince-log for details). So in a few hours everything should be normal 
again.

Happy Corpus Christi (it's an holiday here :)).

Sincerly,
DaB.

--

-- 
wp-blog.de

Hello all,

yesterday at highnoon the master-db-servers were switched because of an huge 
schema-update. Because we have only one cluster of db-servers at the moment we 

Problem

Hello all,

I need help. I have a perl programme (for Check Wikipedia). This can 
scan a dump of a language very fast. 2200 pages per minute is no problem.

I will daily scan with the same script the page text of the live 
Wikipedia. Not all pages, but maybe 20000 per day per language. Normally 
this need only 10 minutes with the dump, but with the live Wikipedia 
this need many time. I use the Wikipedia-API to get the text of an 
article and so my script can only scan 120 pages per minute. So this 
scan need at the moment in enwiki 300 minutes or in dewiki 134 minutes. 
The most time my script is waiting. This is a problem because there is a 
high CPU usage.

I need a faster way to get the text from the live Wikipedia. So I can 
reduce the CPU usage.

Maybe someone know a faster way! Or have an other idea.

Thanks for every help,
Stefan (sk)

More Info:
http://de.wikipedia.org/wiki/Benutzer:Stefan_Kühn/Check_Wikipedia
http://en.wikipedia.org/wiki/Wikipedia:WikiProject_Check_Wikipedia
http://toolserver.org/~sk/checkwiki/checkwiki.pl

Re: Problem

Stefan Kühn:
> The most time my script is waiting. This is a problem because there is a high
> CPU usage.

that doesn't make sense.  if it was waiting, it wouldn't be using any CPU.  if
it's using CPU, it's not waiting, it's doing something.

	- river.

Re: Problem

Stefan Kühn schrieb:
> The most time my script is waiting. This is a problem because there is a 
> high CPU usage.
> 
> I need a faster way to get the text from the live Wikipedia. So I can 
> reduce the CPU usage.

If waiting causes CPU usage, you are doing something VERY wrong. Waiting should
mean NO CPU usage.

-- daniel

Re: Problem

Maybe a guru for perl can make a look at the script. I hope anyone have 
a idea.

Stefan

River Tarnell schrieb:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Stefan Kühn:
>> The most time my script is waiting. This is a problem because there is a high
>> CPU usage.
> 
> that doesn't make sense.  if it was waiting, it wouldn't be using any CPU.  if
> it's using CPU, it's not waiting, it's doing something.
> 
> 	- river.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (HP-UX)
> 
> iEYEARECAAYFAkoxbiIACgkQIXd7fCuc5vL1bQCgihqAbqvlw+IJ+/HtAAyVR4fK
> n7kAoJA4ZyBi3dr6Qy1Y4yw1JIQF1vFo
> =TPyX
> -----END PGP SIGNATURE-----
> 
> _______________________________________________
> Toolserver-l mailing list
> Toolserver-l <at> lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/toolserver-l