Sandor Seres | 17 Aug 15:04 2014
Picon

[OSM-dev] How to handle these issues?

I am not quite shore how to handle the following two dilemmas:

1.If a rendering system renders the planet-sea area objects (instead of planet-land area objects), is then Antarctica a hole in the global ocean object or the global ocean object simply ends by the upper border-line of Antarctica?

2.There are around 25460 objects in the class defined by the tag natural=land in the latest dump. The wiki documentation for the same tag says “This tag should not be used”. How to interpret this?

Are editors refusing data with this tag? Should we ignore the data with this tag in the dumps? Looking at different OSM based maps I see that I am not the only one confused with the issue. Unfortunately, the class contains area objects related to all water types/classes so, this is not just

rendering them at certain stage. In vector mapping, as you certainly know, it is much more complicated.

Thanks for suggestions/meanings.

Sandor

_______________________________________________
dev mailing list
dev <at> openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
nbu | 8 Aug 14:11 2014
Picon

[OSM-dev] Auto Response

I am no longer using this email as it is receiving vast ammounts of spam. Please send your email to one of my other email addresses or contacte me by some other means. Neil

_______________________________________________
dev mailing list
dev <at> openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Richard Welty | 7 Aug 22:19 2014
Picon

[OSM-dev] current documentation on using osmosis to populate an empty osm database?

i'm trying to import an osm extract into an empty osm database using
osmosis and i'm not at all clear on where current documentation is.
there are some howtos and examples out there covering what appear
to be obsolete versions which isn't helping.

can anyone point me at something current?

thanks,
   richard

--

-- 
rwelty <at> averillpark.net
 Averill Park Networking - GIS & IT Consulting
 OpenStreetMap - PostgreSQL - Linux
 Java - Web Applications - Search

_______________________________________________
dev mailing list
dev <at> openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Matthijs Melissen | 7 Aug 00:53 2014
Picon

[OSM-dev] Release openstreetmap-carto v2.18.0

Dear all,

Today, v2.18.0 of the openstreetmap-carto stylesheet has been released.

Changes include:

* Remove transparency from certain landuse tags
(https://github.com/gravitystorm/openstreetmap-carto/pull/792)
* Render lake names on lower zoom levels
(https://github.com/gravitystorm/openstreetmap-carto/pull/816)
* Rewrite code for road ordering, resolving various road layering bugs
(https://github.com/gravitystorm/openstreetmap-carto/pull/626)
* Various bug fixes

For a full list of commits, see
https://github.com/gravitystorm/openstreetmap-carto/compare/v2.17.0...v2.18.0.

We welcome any bug reports at
https://github.com/gravitystorm/openstreetmap-carto/issues.

-- Matthijs
Rhys Gareth | 6 Aug 11:29 2014
Picon

[OSM-dev] Isochrone generation with mode=CAR starting or ending in private roads

When generating Car/drive isochrones, I found that OTP returned empty GeoJSON for some locations which were on private roads and not near-enough to make a "jump" to a public road. I then discovered it's not possible to have a trip of any transport mode starting at that point at all. 

Example start point which fails to generate isochrones: http://www.openstreetmap.org/#map=18/1.24930/103.84794&layers=T

After thinking about it, I think I'd want isochrones to be able to start or end in private roads, but not use them as "through roads" in case there are ones which have more than one connection (there are some like this in Singapore, such as large government complexes). The logic I think could work is quite simple:
1. It's OK to start in a private road and connect to public
2. It's OK to end in a private road
3. If you have gone from public-private it's not OK to go back to public - the trip ends there.

I prefer this over attempting to make a large jump to the nearest public road. 

Alternatively, can this be solved by "correcting" the OSM road classification type, or adjusting OTP's settings for which road types are allowed? My own assessment was that private was the correct road type, and enabling OTP to use private roads would just result in problems with it generating isochrones which drive right through private complexes and out the other side.

Rhys
_______________________________________________
dev mailing list
dev <at> openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Lynn W. Deffenbaugh (Mr | 3 Aug 05:21 2014
Picon

[OSM-dev] Minutely updates format error?

For the past 8-12 (maybe 24) hours, my tile server has been failing to 
update.  The error reported is:

> Processing: Node(0k 0.0k/s) Cache(Ch:0.0% <at> 0.0%u fh:0.0%) Way(0k 
> 0.00k/s) Relation(0 0.00/s) Why:NodeStart
> Processing: Node(360k 2.0k/s) Cache(Ch:0.0% <at> 2.9%u fh:98.9%) Way(0k 
> 0.00k/s) Relation(0 0.00/s)

> Entity: line 395041: parser error : Input is not proper UTF-8, 
> indicate encoding !
> Bytes: 0xD8 0x51 0x72 0x20
>       <tag k="name" v="Ù¾Ø²Ø´Ú©Û ÙاÙÙÙÛ Ø§Ø³ØªØ§Ù Ø®Ø±Ø§Ø³Ø§Ù
ØQr  <483
> ^
> Entity: line 395041: parser error : Unescaped '<' not allowed in 
> attributes values
>       <tag k="name" v="Ù¾Ø²Ø´Ú©Û ÙاÙÙÙÛ Ø§Ø³ØªØ§Ù Ø®Ø±Ø§Ø³Ø§Ù
ØQr  <483
> ^
> Entity: line 395041: parser error : attributes construct error
>       <tag k="name" v="Ù¾Ø²Ø´Ú©Û ÙاÙÙÙÛ Ø§Ø³ØªØ§Ù Ø®Ø±Ø§Ø³Ø§Ù
ØQr  <483
> ^
> Entity: line 395041: parser error : Couldn't find end of Start Tag tag
>       <tag k="name" v="Ù¾Ø²Ø´Ú©Û ÙاÙÙÙÛ Ø§Ø³ØªØ§Ù Ø®Ø±Ø§Ø³Ø§Ù
ØQr  <483
> ^
> /var/lib/mod_tile/changes.osc.gz : failed to parse

Is there an issue in the data from the database or where might this be 
coming from?  My state.txt file is:

> #Fri Aug 01 20:03:01 EDT 2014
> sequenceNumber=985291
> timestamp=2014-08-02T00\:01\:02Z

Lynn (D) - KJ4ERJ
Éric Gillet | 2 Aug 12:52 2014
Picon

Re: [OSM-dev] OSM data import using osm2pgsql... what's next.

On Sat, Aug 2, 2014 at 9:52 AM, Christian Quest <cquest <at> openstreetmap.fr> wrote:

By default osm2pgsql reprojects into web mercator, but you can avoid it and ask to keep the spherical coordinates if this is a better choice for your use.

 
May I ask how to use spherical coordinates ? I've hit the problem about distances being distorded with projections, and never managed to do it right with osm2pgsql based db.

_______________________________________________
dev mailing list
dev <at> openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Loic Duros | 29 Jul 22:51 2014
Picon

[OSM-dev] OSM data import using osm2pgsql... what's next.

I am completely new to both OSM data and PostGIS, and I have a few questions, which I thought would be best asked in this mailing list. If you can think of another mailing list where these questions would be more appropriate, please let me know.

I am currently using osm2pgsql to import the planet osm data. This is taking a while, expectedly, and I'm using --flat-nodes in order to save space, as well as -k, which according to -h is for the hstore.
For reference, this is the full command I am running:
osm2pgsql -c -d mapdatabase -U myuser -W --slim -C 18000 -k --flat-nodes=/var/planet/planet_nodes.cache --number-processes 4 -v

I would like to use my server later on for tile-serving purposes (using Mapnik and mod-tile, and/or TileStache), but I would also like to be able to run queries that give me interesting data for an Android app I will build.
Because I am using flat-nodes, will I still be able to make queries to calculate distances, find points of interest (such as detecting hills around a point, ...)

The PostGIS in Action book has an interesting example of query:
"Suppose the task at hand is to find the total length of all interstate highways in the state of Utah. We search for two tables: one with the polygons for all the states and one with all interstate highways in the United States, represented as linestrings. Next we extract the polygon that represents Utah from the states table and perform an SQL join with the highways table using
the geometric intersects function as the join operator and geometric intersection as
our output function. The output of that query would be those portions of all highways
within the state of Utah."

Is such a query doable with the postgis database generated by osm2pgsql? I understand osm2pgsql is focused on tile rendering, but I'm curious to see how much can be done in terms of analysis with it, not just for roads but for looking for distinct features in the environment (forest, parks, hills, ...) within a given range. Does using flat-nodes for the import limit the capability to do spatial queries in any way?

Finally, I would also like to create a custom design for rendering tiles. I have taken a look at osm.xml located in my local directory, and it looks like this is the right file to override to design tiles a little differently in terms of look and feel. However, I've also tried using TileMill, and was wondering if this is something that can be used with Mapnik as well.

Thank you for your interest and for your guidance.

Loic Duros
_______________________________________________
dev mailing list
dev <at> openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Paul Norman | 23 Jul 03:51 2014
Picon

[OSM-dev] Upcoming openstreetmap-carto changes

v2.17.0 of the openstreetmap-carto stylesheet has been released, though 
not yet deployed on tile.osm.org.

Significant changes include

* Rendering other shop values with a generic icon 
(https://github.com/gravitystorm/openstreetmap-carto/pull/604)
* Rendering wider road shields, and converting the road shields to SVGs 
(https://github.com/gravitystorm/openstreetmap-carto/pull/694)
* Cleaning up path rendering on low zooms 
(https://github.com/gravitystorm/openstreetmap-carto/pull/747)
* Improving highway=track name rendering 
(https://github.com/gravitystorm/openstreetmap-carto/pull/748)
* Rendering access=permissive the same as access=yes (or no access) 
(https://github.com/gravitystorm/openstreetmap-carto/pull/742)
* Waterway cleanups 
(https://github.com/gravitystorm/openstreetmap-carto/pull/722, and others)
* Assorted code cleanups

https://github.com/gravitystorm/openstreetmap-carto/compare/v2.16.0...v2.17.0 
has a full list of commits.

We are still very much cleaning up a lot of messy CartoCSS brought over 
from the Mapnik XML

Some interesting stuff under discussion

* Multi-line road shield rendering: 
https://github.com/gravitystorm/openstreetmap-carto/pull/750
* Rendering ref from route relationships: 
https://github.com/gravitystorm/openstreetmap-carto/issues/596
* Moving ordering of road rendering to SQL: 
https://github.com/gravitystorm/openstreetmap-carto/pull/626
* Moving to YAML for project layers: 
https://github.com/gravitystorm/openstreetmap-carto/issues/711
Christian Quest | 20 Jul 12:10 2014
Picon

[OSM-dev] recursion added to render_list

I've talked about this idea with some of you at last SOTM and SOTM-EU.

The idea is to have render_list taking advantage of disk cache by rendering tiles in a more "pyramidal" way than the actual full horizontal.

1) better use of disk cache to reduce disk I/O needs

Up to now, render_list renders a tile at zoom N but when getting to zoom N+1, most of the needed data are usually the same but not in the disk cache anymore causing a lot of I/O when rendering several zoom level.

With my changes, render_list immediately enqueue the 4 underlying N+1 tiles (at the top of the queue). This allows to reduce disk I/O by taking advantage of the caches still containing most of the needed data.

That's fine but where should be stop recursion ?


2- pre-render only what's worth prerendering...

For this I've added a rendering time limit.

If a metatile takes less than this limit, it is not worth pre-rendering it as it will be quickly generated if requested.
This allows to limit recursion and at the same time pre-render what can be done fast enough upon request.


To use this new feature, I'v only added on option to render_list to control the rendering time limit (in ms). With it, its behavior is unchanged.


Example:
render_list -m osm -a -f -z 9 -Z 9 --max-time 5000


On my todo list:
- when rendering several zoom levels (example: -z 9 -Z 12), check if a metatile has not been already recursively rendered by comparing its timestamp to the render_list start time

- add a maximum zoom limit (for the moment recursion can go down to zoom 18)

My commit is on github, comments are welcome... before my first mod_tile pull request

--
Christian Quest - OpenStreetMap France
_______________________________________________
dev mailing list
dev <at> openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev
Paul Norman | 17 Jul 17:03 2014
Picon

Re: [OSM-dev] Tuning render_list: beware of hyperthreading and RAID cards latency...


On 2014-07-17 6:39 AM, Christian Quest wrote:
I'm currently doing some rendering benchmark to tune a new tile server we're about to put online.

A strange thing I've found last night: render_list with 16 thread takes more time than with 8 .

The server has 2 quad-core Xeon (X5570), this means 8 physical cores. It looks like hyperthreading adds additionnal context switches or something similar.
With "-z 9 -Z 9 -n 16 -l 32" it takes 3016s, and with "-z 9 -Z 9 -n 8 -l 12" 2647s, 10% less.

I'll disable HT on my next run in the server bios and see if it is consistent.

Is there a chance you're running into IO bottlenecks?

Another thing I found... do not put your SSD behind a RAID card.
The RAID card (Dell Perc 6i) add too much latency.
On the same render_list, the iowait threads are increasing a lot and iops decreases a lot.
The Perc 6 is an older lower-end RAID card using a LSI 10xx chip. It doesn't surprise me it doesn't work ideally with SSDs. I'm also not sure if it supports all the SATA commands a SSD uses, which can be a serious issue for consistency, the bane of SSDs.

A more modern card would probably perform better, but if you're not RAIDing SSDs, you should really be using any card you have in HBA mode, unless you have a battery-backed cache and can therefor tune write barriers and related settings better.
_______________________________________________
dev mailing list
dev <at> openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev

Gmane