Martin Schafran | 30 Apr 15:18 2013

uml for ducumentation

hi,

for osmosis newbies who don't have the ultimate version of idea its helpful to 
have an uml diagram. 
and eclipse plugins just don't work.
feel free to improve this script and include some graph to the repository:

#!/bin/bash

# download umlgraph from http://www.umlgraph.org/

# edit the path to your UMLGraph directory 
umlhome=/home/m/bin/UMLGraph-5.6

drawgraph () {
  # files to be included into graph
  filetree=$(tree -fiP *.java | grep "/main/" | grep "\.java")
  # create dot file
  java -classpath $umlhome/lib/UmlGraph.jar:$JAVA_HOME/lib/tools.jar 
org.umlgraph.doclet.UmlGraph -private $filetree
  # convert dot file to png
  dot -Tpng -ograph.png graph.dot
}

cd osmosis-apidb               
drawgraph
cd ../osmosis-areafilter      
drawgraph
cd ../osmosis-core            
drawgraph
(Continue reading)

Fedor Snegov (MTS phone | 28 Apr 11:11 2013
Picon

--read-xml and --sort osmosis operations of Srtm2osm output file failed with error: "...Cannot represent 70730 as a char"

Hi, dear osmosis-dev list members

I am trying to merge osm map file containing Srtm2osm contours of 
interested area ( 
boundaries='http://www.openstreetmap.org/?lat=60.22&lon=32.04&zoom=7&layers=M' 
) with osm.pbf Openstreetmaps map file in order to obtain Garmin map.

I employ osmosis (great tool) for merging procedure. The merging 
procedure fails every time with errors which are presented below.

For test reasons I am trying simply to convert Srtm2osm program output 
file to pbf format. The command line invoking osmosis is the following:

"./osmosis-0.43.1/bin/osmosis \
--read-xml file=$contours_file_name.$contours_file_extension \
--sort \
--write-pbf file=$contours_file_name.osm.pbf omitmetadata=true 
compress=deflate"

But it stops and gives me again the same error log like this:

"Apr 27, 2013 11:35:22 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.43.1
Apr 27, 2013 11:35:22 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Apr 27, 2013 11:35:22 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Apr 27, 2013 11:35:22 PM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
Apr 27, 2013 11:35:42 PM 
(Continue reading)

Felix Schneider | 26 Apr 10:41 2013
Picon

tagtransform: copy-matched does not removes tags when no tag on entity is matched

Hi,

I'm  trying to filter all tags in an .osm file on a whitelist basis. This works fine when at least one tag on an
entity is matched, but fails when an entity has only tags that should be removed. 

I think the problem is around TransformHelper.java:132, but I have no clue how to fix this. 

Sample files and sample outputs are attached.

Regards,
Felix Schneider

Attachment (input_failing.osm): application/octet-stream, 705 bytes
Attachment (input_working.osm): application/octet-stream, 750 bytes
Attachment (output): application/octet-stream, 3276 bytes
Attachment (translation.xml): application/xml, 308 bytes
_______________________________________________
osmosis-dev mailing list
osmosis-dev@...
http://lists.openstreetmap.org/listinfo/osmosis-dev
fin | 23 Apr 15:20 2013

feature request: used-relations

Hi there,

i'd like to request a feature: please allow users to _include_
relations which are referenced by relations, much like --used-way and
--used-node filter out ways and nodes that are not referenced.

my use case is simple: i'm trying to extract boundary data for one
admin_level in austria and some of those boundaries reference
subareas:
http://wiki.openstreetmap.org/wiki/Relation:boundary

i can't seem to filter them out and i can't have osmosis include them,
so most tools i've tried end up barfing on that data.

including all entities - either by default or as an option, since that
would add complexity & computing time - that are referenced seems like
the most reasonable course of action.

thanks!
-fin
Simon Nuttall | 22 Apr 23:52 2013
Picon

Password contains a percent character

I'm calling osmosis from php, and when the password contains a %
character it fails to login.

E.g when the last character of the password is % I get this:

java.lang.IllegalArgumentException: URLDecoder: Incomplete trailing
escape (%) pattern

and if the % char is in the middle of the password:

java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters
in escape (%) pattern - For input string: "es"

Could this be due to lack of percent encoding in any GET request that
is being made?
Martin Schafran | 22 Apr 11:29 2013

wrong intermediary timestamp

hi,

I noticed that there is a wrong timestamp in the intermediary diff file when 
populating diffs from api db to simple db via a diff.osm file.

at the end it does not impact the snapshot but only the diff.osm file

steps to reproduce:

make a change at 10:00:00Z and upload to api db (josm)
api db has the correct time 10Z

make diff file:  --write-xml-change intervalBegin=10:00:00Z
diff file has the wrong time 08:00:00Z

import diff file to snapshot or simple schema
snapshot db has the correct time 10Z

my local machine has UTC+2 time.

martin
Paul Norman | 19 Apr 14:14 2013
Picon

pgsnapshot alternative query optimizations: ways using nodes

In part 1 of a N+1 part series, I'm intending to outline some of the
benchmarking I have done into different queries and indexes with a
pgsnapshot database.

Meta: Where on the wiki should this type of stuff go?

All benchmarking times are with a hot cache, but similar trends were
observed with a cold cache. I tested with two areas: the large area
-87.75,41.80,-87.65,41.85 (322k nodes, 58.1k ways) and the small area
-111.92,33.82,-111.91,33.83 (159 nodes, 30 ways)

My testing machine is a 6 core AMD 1090T with all databases on a 6 x 7200
RPM drive array, 32GB ram.

The traditional way of running a query to get all ways which contain a set
of nodes as their children is to do two joins. This type of query is used in
extracting a bbox from the database.

First you need a set of nodes. This could be generated by this query for a
bbox

CREATE TEMPORARY TABLE bbox_nodes AS 
  SELECT * FROM nodes 	
    WHERE geom &&
ST_SetSRID(ST_MakeBox2D(ST_Point(-87.75,41.80),ST_Point(-87.65,41.85)),4326)
;
	
The exact query doesn't matter much - it could be a polygon and
ST_Intersects(geom, area_geom) and everything below should still apply. I am
assuming that some kind of geometry query is used. A random list of nodes
(Continue reading)

Paul Norman | 18 Apr 09:36 2013
Picon

Wrapper scripts for streaming replication

I'm considering making use of streaming replication to feed change data to a
process, but I'm not quite sure how to script it to handle interruptions
like power outages and such correctly without getting in advance of itself.
Does anyone have any wrapper scripts that would help with this?

I'd rather not reinvent the wheel so I can focus on the logic of
interpreting the osc data and putting it into my database.

Alternately, I could re-implement a streaming replication reader in python,
which is what I intend to be writing my osc to DB code in.
Brett Henderson | 16 Apr 13:50 2013

Re: FW: question

On 15 April 2013 00:28, mahmoud ghareebh <m.soliman_2010-PkbjNfxxIARBDgjK7y7TUQ@public.gmane.org> wrote:
yes an old version , do i need to remove it?

The plugin is failing to load, so yes I'd remove it.

Again, do you really need to compile Osmosis yourself?
 



Hi Mahmoud,

Is there a reason you're not using the pre-compiled version?
http://bretth.dev.openstreetmap.org/osmosis-build/osmosis-latest.zip

The errors you're receiving are unusual.  I see the following message in your logs.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: Cannot load JPF-plugin 'org.mapsforge.MapFileWriter' for extensionpoint 'Task'
Do you have some custom Osmosis plugins installed somewhere?

Brett
 
_______________________________________________
osmosis-dev mailing list
osmosis-dev@...
http://lists.openstreetmap.org/listinfo/osmosis-dev
Paul Norman | 16 Apr 10:31 2013
Picon

Bad PostgreSQL plans with pgsnapshot

I ran across a PostgreSQL query planner bug today when benchmarking
pgsnapshot. This may affect others so I'm documenting it.

A common task is to extract a bounding box or polygon from the database,
equivalent to a map? call.

The way I was benchmarking was a multi-step process, but there's two that
are relevant for this.

The first is getting all the nodes in the area. The query for this is

CREATE TEMPORARY TABLE bbox_nodes AS 
  SELECT * FROM nodes 
    WHERE geom &&
ST_SetSRID(ST_MakeBox2D(ST_Point(-87.75,41.80),ST_Point(-87.65,41.85)),4326)
;
ANALYZE bbox_nodes;

The next is to get the ways with any of those nodes as members.

I started this with 

SELECT DISTINCT ... FROM ways
  JOIN way_nodes ON (ways.id=way_nodes.way_id)
  JOIN bbox_nodes ON (way_nodes.node_id=bbox_nodes.id);

This was basically the query I intended to benchmark, but I was getting
horrible query plans and run times so I split the query up, arriving at
this: 

SELECT DISTINCT way_nodes.way_id
  FROM way_nodes 
  JOIN bbox_nodes ON (way_nodes.node_id=bbox_nodes.id);

I was still getting bad query plans involving a sequential scan of
way_nodes, a 2.2 billion row table.

After much help from RhodiumToad from #postgresql the cause was tracked down
to a bug in the query planner where a fixed fudge factor was accounting for
almost all of the cost of fetching the random pages. The cost estimate was
about 356 of which 348 was from the fudge factor. This was causing it to
vastly overestimate the cost of the index lookups, forcing it to do a
sequential scan.

This bug has already been fixed in 9.3
(http://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;f=src/backend/u
tils/adt/selfuncs.c;h=31f38f28b00cbe2b9267205359e3cf7bafa1cb97) but not
backported to earlier versions.

The workaround was to increase cpu_tuple_cost to 0.1. This makes it use the
indexes, resulting in a 50s query for a 350k node 60k way bbox instead of a
query aborted at 10 minutes.

If you are running a query against a small bbox (50k nodes) this bug does
not matter as the incorrect fudge factor is not significant enough.

If you are running it against a very large bbox (~10-100 million nodes?) it
also does not matter, a sequential scan will be faster there.

I do not know if this bug impacts the osmosis --dataset-bounding-box task,
jxapi or pyxapi. 

I do not know if set enable_seqscan=false is an appropriate solution. If you
have an index on bbox_nodes (id) it is not as it will force index usage on
that table when it needs to fetch 100% of the table anyways.
mahmoud ghareebh | 13 Apr 12:43 2013
Picon

FW: question

hi 
i`m trying to install osmosis on windows 7 , i have jdk 6 , and i follow this instructions :
./gradlew assemble
success
Perform a complete build including unit tests:
./gradlew build
failure , and give me errors i`m attach the report , could u help me?
<!-- .ExternalClass .ecxhmmessage P { padding:0px; } .ExternalClass body.ecxhmmessage { font-size:12pt; font-family:Calibri; } -->
Attachment (tests.zip): application/zip, 36 KiB
_______________________________________________
osmosis-dev mailing list
osmosis-dev@...
http://lists.openstreetmap.org/listinfo/osmosis-dev

Gmane