Tim Alder | 5 Sep 19:38 2011
Picon

replag is growing /no contact to planet.osm.org

Hello,
maintenance on OSM.org is now over but our replag of ptolemy is still 
growing.

We are not able to contact the diff-server from OpenStreetMap:
	osm <at> ptolemy:~$ ping planet.openstreetmap.org
	no answer from planet.openstreetmap.org

Ping works fine from nightshade or my laptop.

I hope nobody block us.
We change nothing on the config.

Greetings Kolossos

Grant Slater | 5 Sep 22:42 2011

Re: replag is growing /no contact to planet.osm.org

On 5 September 2011 18:38, Tim Alder
<tim.alder@...> wrote:
> Hello,
> maintenance on OSM.org is now over but our replag of ptolemy is still
> growing.
>
> We are not able to contact the diff-server from OpenStreetMap:
>        osm <at> ptolemy:~$ ping planet.openstreetmap.org
>        no answer from planet.openstreetmap.org
>
> Ping works fine from nightshade or my laptop.
>
> I hope nobody block us.
> We change nothing on the config.
>

I am part of the OSM sysadmin team. There are no blocks from our side,
although ping replies are rate limited which may be causing the ping
issue.

Tried (or similar) from the host?: w3m http://planet.osm.org/

/ Grant

Picon

Re: Maps-l Digest, Vol 29, Issue 1


----------
Отправлено с моего телефона Nokia

------Исходное сообщение------
От: <maps-l-request <at> lists.wikimedia.org>
Кому: <maps-l <at> lists.wikimedia.org>,<maps-l-request <at> lists.wikimedia.org>
Дата: 5 Сентябрь 2011 г. 20:43:16 GMT+00:00
Тема: Maps-l Digest, Vol 29, Issue 1

Send Maps-l mailing list submissions to
	maps-l <at> lists.wikimedia.org

To subscribe or unsubscribe via the World Wide Web, visit
	https://lists.wikimedia.org/mailman/listinfo/maps-l
or, via email, send a message with subject or body 'help' to
	maps-l-request <at> lists.wikimedia.org

You can reach the person managing the list at
	maps-l-owner <at> lists.wikimedia.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of Maps-l digest..."

Today's Topics:

   1. Re: Starting of re-rendering all  tiles (Peter K?rner)
   2. Re: Maps-l Digest, Vol 28, Issue 4 (maksumial8biz <at> yandex.ru)
   3. Re: Starting of re-rendering all  tiles (Tim Alder)
   4. replag is growing /no contact to planet.osm.org (Tim Alder)
(Continue reading)

Tim Alder | 6 Sep 07:27 2011
Picon

Re: replag is growing /no contact to planet.osm.org

Thanks for the answer.
I  try now a "wget planet.openstreetmap.org" and this works fine.
So the problem seems on our side.
osm2pgsql is also running sometimes for short time.
But something seem confused.

Greetings Kolossos

Am 05.09.2011 22:42, schrieb Grant Slater:
> On 5 September 2011 18:38, Tim
Alder<tim.alder@...>  wrote:
>> Hello,
>> maintenance on OSM.org is now over but our replag of ptolemy is still
>> growing.
>>
>> We are not able to contact the diff-server from OpenStreetMap:
>>         osm <at> ptolemy:~$ ping planet.openstreetmap.org
>>         no answer from planet.openstreetmap.org
>>
>> Ping works fine from nightshade or my laptop.
>>
>> I hope nobody block us.
>> We change nothing on the config.
>>
>
> I am part of the OSM sysadmin team. There are no blocks from our side,
> although ping replies are rate limited which may be causing the ping
> issue.
>
> Tried (or similar) from the host?: w3m http://planet.osm.org/
(Continue reading)

Peter Körner | 6 Sep 09:29 2011
Picon

Re: replag is growing /no contact to planet.osm.org

Am 06.09.2011 07:27, schrieb Tim Alder:
> Thanks for the answer.
> I  try now a "wget planet.openstreetmap.org" and this works fine.
> So the problem seems on our side.
> osm2pgsql is also running sometimes for short time.
> But something seem confused.

I figured out that osm2pgsql is crashing with this error message:

Processing: Node(277k) Way(42k) Relation(0)get_way failed: ERROR:  could 
not access status of transaction 2920577984
DETAIL:  Could not open file "pg_clog/0AE1": No such file or directory.
(7)
Arguments were: 70616681,
Error occurred, cleaning up

looks like a crashed postgres database. Maybe opening a ticket in jira 
could help. Maybe.

Peter

Tim Alder | 6 Sep 18:36 2011
Picon

Re: replag is growing /no contact to planet.osm.org

Done:
https://jira.toolserver.org/browse/TS-1171

Tim

Am 06.09.2011 09:29, schrieb Peter Körner:
> Maybe opening a ticket in jira could help. Maybe.
>
> Peter
>

Kai Krueger | 7 Sep 03:00 2011
Picon

Re: replag is growing /no contact to planet.osm.org

On 09/06/2011 01:29 AM, Peter Körner wrote:
> Am 06.09.2011 07:27, schrieb Tim Alder:
>> Thanks for the answer.
>> I  try now a "wget planet.openstreetmap.org" and this works fine.
>> So the problem seems on our side.
>> osm2pgsql is also running sometimes for short time.
>> But something seem confused.
>
>
> I figured out that osm2pgsql is crashing with this error message:
>
> Processing: Node(277k) Way(42k) Relation(0)get_way failed: ERROR:  could
> not access status of transaction 2920577984
> DETAIL:  Could not open file "pg_clog/0AE1": No such file or directory.
> (7)
> Arguments were: 70616681,
> Error occurred, cleaning up
>
>
> looks like a crashed postgres database. Maybe opening a ticket in jira
> could help. Maybe.

As mentioned in the jira ticket, it is still crashing, and it is always 
crashing at the exact same point: when trying to process way 70616681 ( 
http://www.openstreetmap.org/browse/way/70616681/history )

I have seen similar errors  about "pg_clog" files missing a while ago 
already. I think back then it was when manually sending queries to the 
postgres db on ptolemy.

(Continue reading)

Tim Alder | 7 Sep 07:18 2011
Picon

Re: replag is growing /no contact to planet.osm.org

I started vacuum full.
Tim

Am 07.09.2011 03:00, schrieb Kai Krueger:

>
>
> Things people have suggested to try and fix it:
>
> 1) Run a vacuum full on the database.
>
> What is the current strategy for vacuuming the database? Is Auto-vacuum
> turned on, or manual vacuums run?
>
> 2) Create the missing pg_clog file as an empty file
>
> dd if=/dev/zero of=/var/lib/pgsql/data/pg_clog/0AE1 bs=256K count=1
>
> I think that might cause some data loss, although it might help to
> recover the rest of the database (which in our case isn't necessary as
> one can recreate it from other sources)
>
> 3) Dump the database and then do a full restore. In our case, it would
> be probably easier to simply drop the db and do a fresh import via
> osm2pgsql.
>
> If 3) turns out to be the only possibility, it might be a good time to
> upgrade to postgresql 9.0
>
> Kai
(Continue reading)

Tim Alder | 8 Sep 17:18 2011
Picon

Starting of re-rendering all tiles

Hello,
we haved now reached with the re-rendering the point z-curve=0.25.
So we are now rendering the emptiness of north norway. But this has no 
visible effect on the rendering speed. The rendering performance 
correlate very good with the queue length. I found now the parameter 
-num in tirex-batch to limit the queue length. I will restart the 
process for tiles with z>0.25 and limit the queue to a value of around 
5.000.

"vacuum full" is now running more than 1 day. If it's not done tomorrow, 
I believe I will stop it. Ok?

Greetings Kolossos

Kay Drangmeister | 8 Sep 18:51 2011
Picon

Re: Starting of re-rendering all tiles

Hi all,

Am 08.09.2011, 17:18 Uhr, schrieb Tim Alder <tim.alder@...>:

> The rendering performance correlate very good with the queue length.

Do you mean this graph:
http://munin.toolserver.org/OSM/ptolemy/tirex_status_queued_requests.html

> I found now the parameter
> -num in tirex-batch to limit the queue length. I will restart the
> process for tiles with z>0.25 and limit the queue to a value of around
> 5.000.

I fail to see the point (sorry for my tone being provocative, I
really don't intend to be rude). Currently the only effect I have
is: my changes are not being rendered. This is probably because the
queue is not respecting a first-come-first-served order, otherwise
the max-tile age would not be so high. And because of this I can
see no difference if my changes are not rendered because there are
constantly 5000 or growing 558879 (as of now) in the queue.

How about limiting it to *1* (or maybe 2) in order to give other
tiles a chance, too?

Why do you fill the queue quicker than it can be rendered? Just
adding a delay to slow it down would not harm, would it?

Sorry for whining, I just want *some* tiles rendered! :-)))
(And no, it has nothing to do with the replag being horrible, too
(Continue reading)


Gmane