Mike Flannigan | 28 Mar 02:58 2014
Picon
Picon

Re: HOW-TO convert USGS topo to GTiff


Nice write up.  Thanks for sharing that.
I didn't know about turning off layers.

This is the GDAL command I like to use.

gdal_translate -of GTiff bm.pdf bm.tif --config GDAL_PDF_DPI 300 -co 
"COMPRESS=JPEG" -co "JPEG_QUALITY=85"

It appears to provide a good trade-off between file
size and image quality.

Mike

On 3/27/2014 2:00 PM, mapserver-users-request <at> lists.osgeo.org wrote:
> I have had several people send me follow-up emails about converting the USGS eTopos (you can download
these from store.usgs.gov), so I thought it might make sense to just post this to the list as a HOW-TO. 
Apologies for cross-post.
>
> The challenge is to take the GeoPDF that you download from USGS and make it into a series of TIFFs you can
mosaic together.  This requires at least GDAL 1.8 and for full functionality you want 1.10.
>
> First, the simplest syntax would be
>
> gdal_translate topo.pdf topo.tif
>
> That would give you a GeoTiff with all the layers turned on, rendered at 150 dpi.  Easy enough.
>
> However, depending on what you want, you may want to alter that.  For example the GeoPDF renders
differently as you zoom in; it has scale-dependent rendering.  So you may want to change the dpi to a higher
(Continue reading)

Smith, Michael | 27 Mar 15:21 2014

HOW-TO convert USGS topo to GTiff

I have had several people send me follow-up emails about converting the USGS eTopos (you can download these from store.usgs.gov), so I thought it might make sense to just post this to the list as a HOW-TO.  Apologies for cross-post.

 

The challenge is to take the GeoPDF that you download from USGS and make it into a series of TIFFs you can mosaic together.  This requires at least GDAL 1.8 and for full functionality you want 1.10.

 

First, the simplest syntax would be

 

gdal_translate topo.pdf topo.tif

 

That would give you a GeoTiff with all the layers turned on, rendered at 150 dpi.  Easy enough.

 

However, depending on what you want, you may want to alter that.  For example the GeoPDF renders differently as you zoom in; it has scale-dependent rendering.  So you may want to change the dpi to a higher number.  I did all mine at 300 dpi which seemed to fit a 24k rendering in ArcMap and looks nice on the screen.

 

gdal_translate --config GDAL_PDF_DPI 300 topo.pdf topo.tif

 

That still has all the layers turned on though.  You may want some of them off.  For example, you may want to turn off the NAIP data and the UTM grid (like I did).  If so you can specify certain LAYERS to turn off, first you have to figure out what they are with this (this requires GDAL 1.10):

 

gdal_info –mdd LAYERS topo.pdf

 

will return the metadata and a list of layers something like this:

<snip>

Metadata (LAYERS):

  LAYER_00_NAME=Map_Collar

  LAYER_01_NAME=Map_Collar.Map_Elements

  LAYER_02_NAME=Map_Frame

  LAYER_03_NAME=Map_Frame.Projection_and_Grids

  LAYER_04_NAME=Map_Frame.Projection_and_Grids.Projection_Coordinate_Values

  LAYER_05_NAME=Map_Frame.Projection_and_Grids.Geographic_and_Grid_Ticks

  LAYER_06_NAME=Map_Frame.Projection_and_Grids.Projection_Line_Mask

  LAYER_07_NAME=Map_Frame.Projection_and_Grids.Grid_Lines

<snip>

  LAYER_28_NAME=Map_Frame.Land_Cover.Woodland

  LAYER_29_NAME=Images

  LAYER_30_NAME=Images.Orthoimage

<snip>

 

So I’m looking to get rid of layer 7 and layer 30.  The syntax for that uses the layer name like this:

 

gdal_translate --config GDAL_PDF_DPI 300 --config GDAL_PDF_LAYERS_OFF Map_Frame.Projection_and_Grids.Grid_Lines,Images.Orthoimage topo.pdf topo.tif

 

This only works for the new 2011/12/13 maps.  The historic maps are scanned images, so they don’t have any real layers.  They are just one raster image in a PDF.  So you would just do

gdal_translate --config GDAL_PDF_DPI 300 topo.pdf topo.tif.

 

What you get in both cases is an output GeoTIFF of the topo map.  The map will have all the collar info, so you will then have to crop the images, remove the collars, and mosaic the slivers to get a seamless mosaic, but that is just done the same way as any other imagery collection.  That process is to simply:

                Crop the image to its minimum size with gdalwarp

                Burn some null value to the remaining collar slivers with gdal_rasterize –i –burn <etc.>

                Mosaic the neighboring tile data into the sliver with gdalwarp

 

One other cool thing about the GeoPDF with its scale-dependent rendering is that you could create topos at different dpis (scales) and thus make your overviews look like what a 100K or 250K topo would look like (well with the new ones anyway, not the historic ones).

 

Don’t forget to RTF(riendly!)M http://www.gdal.org/frmt_pdf.html

 

 

===============================
Michael Smith MS GISP
State GIS Manager, Maine Office of GIS
State of Maine, Office of Information Technology
michael.smith _at_ maine.gov 207-215-5530

Board Member, Maine GeoLibrary
Education Chair, Maine GIS Users Group
State Rep, National States Geographic Information Council



State House Station 145
51 Commerce Drive
Augusta, ME 04333-0145
69o 47' 58.9"W  44o 21' 54.8"N

_______________________________________________
mapserver-users mailing list
mapserver-users <at> lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
TDS | 26 Mar 20:58 2014
Picon

Combine GetFeatureInfo for layers with same data source

Hello,

is it possible to join two layers in GetFeatureInfo with same template and same shape file to output only one combined html file?

Example:
Streets_Main => streets.shp => template.html
Streets_Sub => streets.shp => template.html

OUTPUT:

Current:
<html>Output for Streets_Mains</html>
<html>Output for Streets_Sub</html>

Wish:
<html>
Streets_Mains unique id Streets_sub (e.g. main street and sub street have both id=1 in shape file)
</html>

--
Bye, TDS mailto:tds <at> tds-net.de
1+1=10
You have a question? - 42 or RTFM.
_______________________________________________
mapserver-users mailing list
mapserver-users <at> lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
Picon

MapServer bug when calling GDAL MiniDriver

Hi,

It seems like there is a problem when calling the GDAL "WorldWind" MiniDriver, at least for the GISInternals build with Mapserver V6.4.1.   If I include
<Projection>EPSG:4326</Projection>
in the xml file I’m using, I get the following error:

 

[Tue Mar 25 14:20:30 2014].665000 msDrawRasterLayerLow(Land_Sea): entering.

[Tue Mar 25 14:20:30 2014].667000 msDrawRasterLayerLow(): Unable to access file. Corrupt, empty or missing file 'c:\MapData\Land-Sea.xml' for layer 'Land_Sea'. GDALWMS: Bad projection specified.

[Tue Mar 25 14:20:30 2014].667000 msDrawMap(): Image handling error. Failed to draw layer named 'Land_Sea'.

[Tue Mar 25 14:20:30 2014].667000 freeLayer(): freeing layer at 000000000027CB20.

 

If I remove the Projection declaration from the xml file I don’t get an error. 

 

GDAL_Translate works just fine on this xml with or without the Projection declaration, but when called from a mapfile by mapserver, I get an error.  The GDAL info page here:  http://www.gdal.org/frmt_wms.html  says that the parameter is optional, but it does work under older versions of Mapserver from MS4W using Apache.  This is probably very rarely used, so I’m not sure if it is even worth putting in a bug.

James

 

 

 

Attachment (smime.p7s): application/pkcs7-signature, 9 KiB
_______________________________________________
mapserver-users mailing list
mapserver-users <at> lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
Jakub Štambachr | 24 Mar 12:14 2014
Picon

WMS GetFeatureInfo as JSON/GeoJSON

Hi all,

 

I’ve been trying to make mapserver return wms getfeatureinfo as json  but to no success (so far I’ve been using text/html header, content, footer templates).

I followed instructions here: http://mapserver.org/output/template_output.html

My mapfile looks like this:

 

MAP

OUTPUTFORMAT

  NAME "geojson"

  DRIVER "TEMPLATE"

  MIMETYPE "application/json"

  FORMATOPTION "FILE=template/geojson.js"

END

  WEB

  QUERYFORMAT "geojson"

  LAYER

TEMPLATE "template/geojson.js"

       METADATA

wms_enable_request" "*"

wms_feature_info_mime_type" "gml text/html text/plain"

 

As a template I used a slightly modified version of JSON example found here http://mapserver.org/output/template_output.html#examples:

// mapserver template

[resultset layer=mums]

{

  "type": "FeatureCollection",

  "features": [

    [feature trimlast=","]

    {

      "type": "Feature",

      "id": "[myuniqueid]",

      "geometry": {

        "type": "PointLineString",

        "coordinates": [

          {

            "type": "Point",

            "coordinates": [[x], [y]]

          }

        ]

      },

      "properties": {

        "description": "[description]",

        "venue": "[venue]",

        "year": "[year]"

      }

    },

    [/feature]

  ]

}

[/resultset]

 

According to the instruction I added this  layer=mums&mode=nquery&qformat=geojson to the querystring. The problem is that now the query returns all the features in the layer (the template gets filled correctly though). I use INFO_FORMAT=text/html in the wms getfeatureinfo query.

 

Any help to get this working would be appreciated, thanks

 

Jakub

 

_______________________________________________
mapserver-users mailing list
mapserver-users <at> lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
Marcin Wociór | 24 Mar 08:50 2014
Picon

mapserver 6.2+ OGR output - not working

Hi,
I'm trying to output data from mapserver using OGR (CSV, SHP, GML).
I tried map server versions (windows):
	- OSGeo4w mapserver (6.4.1-2), mapserver-dev (6.5.0.47),
mapserver-stable-dev (6.4.1.47), mapserver6 (6.2.0-1)
	- sdk versions: release-1600-gdal-1-9-2-mapserver-6-2-0.zip,
release-1600-x64-gdal-1-10-1-mapserver-6-4-1.zip

All versions give this error:
msDrawMap(): Image handling error. Unable to initialize image.
msPrepareImage(): GD library error. Unable to initialize image.

And log file:
[Fri Mar 21 23:08:30 2014].75000 CGI Request 1 on process 12224
[Fri Mar 21 23:08:30 2014].75000 msPrepareImage(): GD library error. Unable
to initialize image.
[Fri Mar 21 23:08:30 2014].75000 msDrawMap(): Image handling error. Unable
to initialize image.
[Fri Mar 21 23:08:30 2014].75000 mapserv request processing time (msLoadMap
not incl.): 0.000s
[Fri Mar 21 23:08:30 2014].75000 msFreeMap(): freeing map at 0249C7F8.

All versions have INPUT=OGR in mapserv -v output.

Any clues?
Regards, Marcin

My config looks like this (everything except ogrs output works well):

MAP
  CONFIG 'MS_ERRORFILE' "d:/map_test/mapserver.log"
  DEBUG 5

  IMAGETYPE   OGRGML
  EXTENT      139.562947 35.524705 139.917821 35.817635
  SIZE        550 450

  WEB
	TEMPPATH  "d:/map_test/"
	IMAGEPATH "d:/map_test/"
	IMAGEURL "d:/map_test/"
  END

  IMAGECOLOR  255 255 255
  SHAPEPATH   "../data"

OUTPUTFORMAT
    NAME "DXF"
	DRIVER "OGR/DXF"
	FORMATOPTION "STORAGE=filesystem"
	FORMATOPTION "FORM=zip"
  FORMATOPTION "FILENAME=result.csv"
END   

OUTPUTFORMAT
  NAME "CSV"
  DRIVER "OGR/CSV"
  MIMETYPE "text/csv"
  FORMATOPTION "LCO:GEOMETRY=AS_WKT"
  FORMATOPTION "STORAGE=memory"
  FORMATOPTION "FORM=simple"
  FORMATOPTION "FILENAME=result.csv"
END

OUTPUTFORMAT
  NAME "OGRGML"
  DRIVER "OGR/GML"
  FORMATOPTION "STORAGE=filesystem"
  FORMATOPTION "FORM=simple"
  FORMATOPTION "FILENAME=result.gml"
  IMAGEMODE "PC256"
END

OUTPUTFORMAT
  NAME "SHAPEZIP"
  DRIVER "OGR/ESRI Shapefile"
  FORMATOPTION "STORAGE=filesystem"
  FORMATOPTION "FORM=simple"
  FORMATOPTION "FILENAME=result.zip"
END

OUTPUTFORMAT
  NAME "png"
  DRIVER AGG/PNG
  MIMETYPE "image/png"
  IMAGEMODE RGB
  EXTENSION "png"
  FORMATOPTION "GAMMA=0.75"
END
OUTPUTFORMAT
  NAME "gif"
  DRIVER GD/GIF
  MIMETYPE "image/gif"
  IMAGEMODE PC256
  EXTENSION "gif"
END
OUTPUTFORMAT
  NAME "png8"
  DRIVER AGG/PNG8
  MIMETYPE "image/png; mode=8bit"
  IMAGEMODE RGB
  EXTENSION "png"
  FORMATOPTION "QUANTIZE_FORCE=on"
  FORMATOPTION "QUANTIZE_COLORS=256"
  FORMATOPTION "GAMMA=0.75"
END
OUTPUTFORMAT
  NAME "jpeg"
  DRIVER AGG/JPEG
  MIMETYPE "image/jpeg"
  IMAGEMODE RGB
  EXTENSION "jpg"
  FORMATOPTION "GAMMA=0.75"
END
OUTPUTFORMAT
  NAME "svg"
  DRIVER CAIRO/SVG
  MIMETYPE "image/svg+xml"
  IMAGEMODE RGB
  EXTENSION "svg"
END
OUTPUTFORMAT
  NAME "pdf"
  DRIVER CAIRO/PDF
  MIMETYPE "application/x-pdf"
  IMAGEMODE RGB
  EXTENSION "pdf"
END
OUTPUTFORMAT
  NAME "GTiff"
  DRIVER GDAL/GTiff
  MIMETYPE "image/tiff"
  IMAGEMODE RGB
  EXTENSION "tif"
END
OUTPUTFORMAT
  NAME "kml"
  DRIVER KML
  MIMETYPE "application/vnd.google-earth.kml.xml"
  IMAGEMODE RGB
  EXTENSION "kml"
END
OUTPUTFORMAT
  NAME "kmz"
  DRIVER KMZ
  MIMETYPE "application/vnd.google-earth.kmz"
  IMAGEMODE RGB
  EXTENSION "kmz"
END
OUTPUTFORMAT
  NAME "cairopng"
  DRIVER CAIRO/PNG
  MIMETYPE "image/png"
  IMAGEMODE RGB
  EXTENSION "png"
END
  OUTPUTFORMAT
  NAME png8_t
  DRIVER "GD/PNG"
  IMAGEMODE PC256
  TRANSPARENT OFF
END
  # Start of LAYER DEFINITIONS-------------------------------
  LAYER 
    NAME ""ą˜H"
    DATA dourokukan
    STATUS DEFAULT
    TYPE LINE

    CLASS
      NAME ""ą˜H"    
      STYLE
        COLOR 227 227 127
      END
    END
  END 
  # End of LAYER DEFINITIONS -------------------------------
END

_______________________________________________
mapserver-users mailing list
mapserver-users <at> lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Picon

Running a .aspx script

HI,
I'm trying to get mapserver (from GISInternals) running under IIS7.  I need to figure out how to get it to
execute a .aspx script that I'm using as a tile server.  I use the gdal-minidrivers to call the script, and
that part seems to be working fine, but I get an error 404 from what looks like a valid URL.  

Under MS4W under Apache, I added .aspx to the list of .php extensions in the httpd.conf for application type
"application/x-httpd-php" which ran /cgi-bin/php-cgi.exe.  What's the equivalent under IIS?  Any
clues would be appreciated!
Thanks,
James
_______________________________________________
mapserver-users mailing list
mapserver-users <at> lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Mike Flannigan | 21 Mar 16:06 2014
Picon
Picon

Converting PDF files to GeoTiff


On a perhaps related subject, I was wondering if there
was a preferred way to convert PDF map files to georeferenced
GeoTiffs?  I have thousands (many thousands) of these to
convert.

The tools I would like to use in order of preference are:
Perl
QGIS
Any open source software
Global Mapper

I am on Win7.

Here is an example of a file I want to convert:
http://www.mflan.com/temp/ny_west point_140214_1941_62500_geo.pdf

I am also not adverse to downloading the entire country (USA)
of GeoTiff 15 minute topos if they exist somewhere, but I have
not found a location that serves them up for free.

Mike
Houston, TX

On 3/21/2014 8:53 AM, mapserver-users-request <at> lists.osgeo.org wrote:
> Date: Fri, 21 Mar 2014 12:21:30 +0000
> From: "EVANS, JAMES R GS-13 USAF ACC 84 RADES/SCZE"
> 	<james.evans.4 <at> us.af.mil>
> To: "Rahkonen Jukka  (Tike)"<jukka.rahkonen <at> mmmtike.fi>,
> 	"'mapserver-users <at> lists.osgeo.org'"  <mapserver-users <at> lists.osgeo.org>
> Subject: Re: [mapserver-users] Processing JP2000 files
> Message-ID:
> 	<20EA793B1BC710448656BDC9BD25C4A104C5B771 <at> 52ZHTX-D07-04D.area52.afnoapps.usaf.mil>
> 	
> Content-Type: text/plain; charset="us-ascii"
>
> Hi Jukka,
> I have Global Mapper, and used it to do a batch convert to geotiff on a test set of images.  The converted files
work fine with the OpenJPEG drive, but the size almost doubles and on my 10 year old pc at home it takes about 3
minutes per file to convert.  I will look at all the options you mentioned below.
> Thanks!
> James

_______________________________________________
mapserver-users mailing list
mapserver-users <at> lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Steve.Toutant | 21 Mar 15:49 2014
Picon

WFS filter creates a query using a number instead of text


The problem is when we send a string that represents a number, example:
Filter=<Filter><PropertyIsEqualTo><PropertyName>code_cs</PropertyName><Literal>864000</Literal></PropertyIsEqualTo></Filter>

I get:
ERROR:  operator does not exist: character varying = integer
"code_cs"= 864000

DescribeFeatureType returns
<element name="code_cs" type="string" />

In posgis, code_cs is defined as character varying(18)

My workaround is using Like instead of EqualTo
Filter=<Filter><PropertyIsLike wildcard='*' singleChar='.' escape='!'><PropertyName>code_cs</PropertyName><Literal>864000</Literal></PropertyIsLike></Filter>

Is this a bug in mapserver, creating the query using a number, even if DescribeFeatureType returns 'string' as type?

thanks
Steve

_______________________________________________
mapserver-users mailing list
mapserver-users <at> lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users
jcwalz | 20 Mar 21:42 2014
Picon
Picon

Reference Map

Hello,

Is there any way to add a reference map to GeoMOOSE 2.6.1?

Thanks!

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/Reference-Map-tp5130208.html
Sent from the Mapserver - User mailing list archive at Nabble.com.
_______________________________________________
mapserver-users mailing list
mapserver-users <at> lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Picon

Processing JP2000 files

Hi,
I'm trying to transition from MS4W to the latest GISInternals 64bit build. 
Most of my image sets transferred over just fine, but I'm having trouble 
getting the USDA NAIP set to work.    Here are the errors I'm seeing:
One request from the log file:
[Wed Mar 19 16:37:27 2014].812000 msDrawRasterLayerLow(NAIP): Filename is: 
m_4111264_ne_12_1_20110715_20111011.jp2
[Wed Mar 19 16:37:27 2014].812000 msDrawRasterLayerLow(NAIP): Path is: 
c:\MapData\NAIP\m_4111264_ne_12_1_20110715_20111011.jp2
[Wed Mar 19 16:37:27 2014].818000 msResampleGDALToMap in effect: cellsize = 
2980.000000
[Wed Mar 19 16:37:27 2014].818000 msDrawGDAL(NAIP): using RAW_WINDOW=0 0 5960 
7630, dst=0,0,2,3
[Wed Mar 19 16:37:27 2014].818000 msDrawRasterLayerGDAL(): 
red,green,blue,alpha bands = 1,2,3,4

My applications logs this:
Start time: 3/19/2014 4:42:43 PM
Total download time: 00:00:01.3010000
Average transfer rate: 0.0 kbit/s
Url: 
http://localhost/mapserver/mapserv.exe?request=GetMap&layers=NAIP&TRANSPARENT=TRUE&BGCOLOR=0x000000&map=C:/Mapdata/servemap.map&srs=EPSG:4326&width=512&height=512&bbox=-112.5,22.5,-90,45&format=image/jpeg&version=1.1.1&styles=
Target file: C:\WorldWindCache\Earth\RADES_WMS_LOCAL2\NAIP\0\5\5_3.jpeg.tmp
Progress: 0/0 bytes
Status: The server committed a protocol violation. Section=ResponseHeader 
Detail=CR must be followed by LF
Proxy: 
http://localhost/mapserver/mapserv.exe?request=GetMap&layers=NAIP&TRANSPARENT=TRUE&BGCOLOR=0x000000&map=C:/Mapdata/servemap.map&srs=EPSG:4326&width=512&height=512&bbox=-112.5,22.5,-90,45&format=image/jpeg&version=1.1.1&styles=

Response headers:
========================================

If I try to open that request URL in a web browser, I get several pages of 
this text:

Warning 1: Empty SOT marker detected: Psot=12. Warning 1: Empty SOT marker 
detected: Psot=12. Warning 1: Empty SOT marker detected: Psot=12. Warning 1: 
Empty SOT marker detected: Psot=12. Warning 1: Empty SOT marker detected: 
Psot=12. Warning 1: Empty SOT marker detected: Psot=12. Warning 1: Empty SOT 
marker detected: Psot=12. Warning 1: Empty SOT marker detected: Psot=12. 
Warning 1: Empty SOT marker detected: Psot=12. Warning 1: Empty SOT marker 
detected: Psot=12. Warning 1: Empty SOT marker detected: Psot=12. Warning 1: 
Empty SOT marker detected: Psot=12. Warning 1: Empty SOT marker detected: 
Psot=12. Warning 1: Empty SOT marker detected: Psot=12.

Is this a GDAL problem?  How can I get this to work?  Any suggestions would be 
greatly appreciated.
Thanks!
James

Attachment (smime.p7s): application/pkcs7-signature, 9 KiB
_______________________________________________
mapserver-users mailing list
mapserver-users <at> lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users

Gmane