wiki:Cookbook gdal

Context Navigation

Version 80 (modified by cedric, 15 years ago) (diff)
--

back to first page ..
back to Recipes for EDA CookBook Eda

Using GDAL - Geospatial Data Abstraction Library and rgdal

Using GDAL - Geospatial Data Abstraction Library and rgdal
Connecting PostGIS with R using Rgdal

gdal url:http://www.gdal.org/
is a translator library for raster geospatial data formats that is released under an X/MIT style Open Source license by the Open Source Geospatial Foundation. As a library, it presents a single abstract data model to the calling application for all supported formats. It also comes with a variety of useful commandline utilities for data translation and processing.
Cédric => What I've found at this site is not really helpfull, it is only python and c++ code.
url:http://www.bostongis.com/?content_name=ogr_cheatsheet => Cedric: very good site !
url:http://trac.osgeo.org/gdal/ Cedric : a bit too hard for me

Quick Start for OSGeo4W Users ¶

Note: Step 2 must be run on a computer with internet access as the installer downloads individual packages as needed. You may also need to enable your software firewall to allow the installer to have outbound access.

Download the OSGeo4W Installer. http://download.osgeo.org/osgeo4w/osgeo4w-setup.exe
Run the installer.
Select Express Install, and Next.
Pick one or more packages to install, and Next.
The selected packages and their required subpackages will be downloaded and installed automatically.

After installation:

Desktop applications will be found in Start -> All Programs -> OSGeo4W
Commandline applications will be available in the OSGeo4W Shell command window.

Below we suppose that you have the Qgis tool installed, so your path is C:\OSGeo4W Qgis has gdal included. Gdal comes with two main tools (executable files), the files for gdal are in

C:\OSGeo4W\share\gdal

call >cmd

C:\OSGeo4W\bin\ogr2ogr

C:\OSGeo4W\bin\ogrinfo

Translating .mdb into shapes outside ESRI

C:\OSGeo4W\bin\ogr2ogr -f "ESRI Shapefile" C:\base\basesig\Czhyd.shp C:\base\basesig\Czhyd.mdb

Cedric : this one works but warning there should be no space in the path (ex : no "document and settings/"...)

Conversion from PostGIS to ESRI Shape

adapted from url: http://www.bostongis.com/?content_name=ogr_cheatsheet The pgsql2shp and shp2pgsql are usually the best tools for converting back and forth between PostGIS and ESRI for 2 main reasons.

It has fewer idiosyncracies when converting data.
It has a lot fewer dependencies so can fit on your floppy.

If you really want to use Ogr2Ogr for this kind of conversion, below is the standard way to do it Here the example should export the table riversegments
Cedric

C:\OSGeo4W\bin\ogr2ogr -f "ESRI Shapefile" C:\base\basesig\essai.shp PG:"host=localhost user=postgres dbname=CCM password=postgres port=5433" "riversegments"

Celine (pwd to be precised)

C:\OSGeo4W\bin\ogr2ogr -f "ESRI Shapefile" C:\base\basesig\essai.shp PG:"host=localhost user=postgres dbname=CCM password=PWD" "riversegments"

Cedric : celui ci marche mais j'ai eu du mal, ne marche pas une deuxième fois si les shape sont déjà crées dans le répertoire

C:\OSGeo4W\bin\ogr2ogr -f "ESRI Shapefile" C:\base\basesig\essai PG:"host=localhost user=postgres dbname=CCM password=postgres port=5433" "riversegments"

Cedric : OK marche de nouveau
Selecting specific fields, sets of data and Geometry
Sometimes you have more than one geometry field in a table, and ESRI shape can only support one geometry field per shape. Also you may only want a subset of data. In these cases, you will need to select the geometry field to use. The most flexible way to do this is to use the -sql command which will take any sql statement.

C:\OSGeo4W\bin\ogr2ogr -f "ESRI Shapefile" C:\base\basesig\essai1 PG:"host=locahost user=postgres dbname=CCM password=postgres port=5433" -sql "SELECT gid, the_geom FROM riversegments"

Cedric : ne marche pas, j'obtiens un message du type Unable to open datasource with following drivers, celui là on l'a très très souvent, j'ai essayé plusieurs trucs je ne comprends pas, la requete marche dans pgadmin

One way in which ogr2ogr excels above using the pgsql2shp tool is that ogr2ogr can export multiple tables at once. This is pretty handy for sharing your postgis data with others who do not have a postgis database.

The code below will export all your postgis tables out into a folder called mydatadump in ESRI shape (shp) format.

C:\OSGeo4W\bin\ogr2ogr -f "ESRI Shapefile" essai2 PG:"host=localhost user=postgres dbname=CCM password=postgres port=5433"

Now most of the time you probably only want to output a subset of your postgis tables rather than all your tables. This code exports only the riversegments and rivernodes tables to a folder called c:/base/basesig/mydatadump in ESRI shapefile format

C:\OSGeo4W\bin\ogr2ogr -f "ESRI Shapefile" c:/base/basesig/mydatadump PG:"host=localhost user=postgres dbname=CCM password=postgres port=5433" riversegments rivernodes

Cedric Celui là marche même si il n'y a pas de guillemet"

Notes plus tard... Dans Qgis, le Rgdal fait la même chose, mais je pense qu'il faut lancer le C:\OSGeo4W\bin\gdal16.bat pour mettre à niveau les classpath ...

Changing the projection of any source

C:\OSGeo4W\bin\ogr2ogr -- help

gets :

Usage: ogr2ogr [--help-general] [-skipfailures] [-append] [-update] [-gt n]
               [-select field_list] [-where restricted_where]
               [-sql <sql statement>]
               [-spat xmin ymin xmax ymax] [-preserve_fid] [-fid FID]
               [-a_srs srs_def] [-t_srs srs_def] [-s_srs srs_def]
               [-f format_name] [-overwrite] [[-dsco NAME=VALUE] ...]
               dst_datasource_name src_datasource_name
               [-lco NAME=VALUE] [-nln name] [-nlt type] [layer [layer ...]]

 -f format_name: output file format name, possible values are:
     -f "ESRI Shapefile"
     -f "MapInfo File"
     -f "TIGER"
     -f "S57"
     -f "DGN"
     -f "Memory"
     -f "BNA"
     -f "CSV"
     -f "GML"
     -f "GPX"
     -f "KML"
     -f "GeoJSON"
     -f "Interlis 1"
     -f "Interlis 2"
     -f "GMT"
     -f "SQLite"
     -f "ODBC"
     -f "PostgreSQL"
     -f "MySQL"
 -append: Append to existing layer instead of creating new if it exists
 -overwrite: delete the output layer and recreate it empty
 -update: Open existing output datasource in update mode
 -select field_list: Comma-delimited list of fields from input layer to
                     copy to the new layer (defaults to all)
 -where restricted_where: Attribute query (like SQL WHERE)
 -sql statement: Execute given SQL statement and save result.
 -skipfailures: skip features or layers that fail to convert
 -gt n: group n features per transaction (default 200)
 -spat xmin ymin xmax ymax: spatial query extents
 -dsco NAME=VALUE: Dataset creation option (format specific)
 -lco  NAME=VALUE: Layer creation option (format specific)
 -nln name: Assign an alternate name to the new layer
 -nlt type: Force a geometry type for new layer.  One of NONE, GEOMETRY,
      POINT, LINESTRING, POLYGON, GEOMETRYCOLLECTION, MULTIPOINT, MULTILINE,
      MULTIPOLYGON, or MULTILINESTRING.  Add "25D" for 3D layers.
      Default is type of source layer.
 -a_srs srs_def: Assign an output SRS
 -t_srs srs_def: Reproject/transform to this SRS on output
 -s_srs srs_def: Override source SRS

 Srs_def can be a full WKT definition (hard to escape properly),
 or a well known definition (ie. EPSG:4326) or a file with a WKT
 definition.

Note l'argument -s_srs ne fonctionne pas pour un export de shapes si la table geometry_columns est mal renseignée

C:\OSGeo4W\bin\ogr2ogr -f "ESRI Shapefile" c:/Base PG:"host=localhost user=postgres dbname=qeauvil" "prelevement_geo" -s_srs "EPSG:2154" 
C:\OSGeo4W\bin\ogr2ogr -f "ESRI Shapefile" c:/Base PG:"host=localhost user=postgres dbname=qeauvil" "station_sta" -s_srs "EPSG:2154"

Connecting PostGIS with R using Rgdal

The rddal is made of functions which point out to the c code functions of gdal, it seems to have been developped for intamap eu project which is also an R library and the most interesting page can be found at the following location url: http://wiki.intamap.org/index.php/PostGIS et bien sûr ça marche pas ...
A nice example script follows which I translated into the following code

library(rgdal)
layer = readOGR("PG:dbname=CCM", "riversegments")

but it doesn't work So after a quite long time spent in gdal, I can translate into the more complex dsn

require(rgdal)
readOGR(dsn="PG:host=locahost user=postgres dbname=CCM password=postgres port=5433",layer="riversegments")

However the R invariably returns an absolutely not helpful code, and the same with R2.10 or R2.10.1

> require(rgdal)
Le chargement a nécessité le package : rgdal
Le chargement a nécessité le package : sp
Geospatial Data Abstraction Library extensions to R successfully loaded
Loaded GDAL runtime: GDAL 1.6.2, released 2009/07/31
Path to GDAL shared files: C:/Program Files/R/R-2.10.0/library/rgdal/gdal
Loaded PROJ.4 runtime: Rel. 4.6.1, 21 August 2008
Path to PROJ.4 shared files: C:/Program Files/R/R-2.10.0/library/rgdal/proj
Message d'avis :
le package 'sp' a été compilé avec la version R 2.10.1 
> readOGR(dsn="PG:host=localhost dbname=eda2.0 user=postgres password=postgres port=5433",layer="ccm21.riversegments_bretagne")
Erreur dans ogrInfo(dsn = dsn, layer = layer, input_field_name_encoding = input_field_name_encoding) : 
  Cannot open file

Interestingly it is the pointer to ogrinfo() which fails so we will try this with ogrinfo (which is found at the same place than ogr2ogr

C:\OSGeo4W\bin\ogrinfo -f "PostgreSQL" PG:"host=localhost user=postgres dbname=CCM password=postgres port=5433"

which returns

C:\base\basesig>C:\OSGeo4W\bin\ogrinfo -f "PostgreSQL" PG:"host=localhost user=postgres dbname=CCM password=postgres port=5433"
Usage: ogrinfo [--help-general] [-ro] [-q] [-where restricted_where]
               [-spat xmin ymin xmax ymax] [-fid fid]
               [-sql statement] [-al] [-so] [--formats]
               datasource_name [layer [layer ...]]

There does not seem to be any problem and also nothing is returned ... The next example for writing

library(rgdal)
writeOGR(meuse[1:10,], "PG:dbname=postgis", "meuse3", "PostgreSQL")

returns

Erreur dans writeOGR(meuse[1:10, ], "PG:dbname=postgis", "meuse3", "PostgreSQL") : 
  No such driver: PostgreSQL

All examples which do not imply postgres work

logo <- system.file("pictures/logo.jpg", package="rgdal")[1]
x <- new("GDALReadOnlyDataset", logo)
x
getDriverLongName(getDriver(x))
displayDataset(x)
ogrDrivers()
dsn <- system.file("vectors", package = "rgdal")[1]
ogrInfo(dsn=dsn, layer="cities")
cities <- readOGR(dsn=dsn, layer="cities")
summary(cities)
ogrInfo(dsn=dsn, layer="kiritimati_primary_roads")
kiritimati_primary_roads <- readOGR(dsn=dsn, layer="kiritimati_primary_roads")
summary(kiritimati_primary_roads)
ogrInfo(dsn=dsn, layer="scot_BNG")
scot_BNG <- readOGR(dsn=dsn, layer="scot_BNG")
summary(scot_BNG)
if ("GML" %in% ogrDrivers()$name) {
	dsn <- system.file("vectors/airports.gml", package = "rgdal")[1]
	airports <- try(readOGR(dsn=dsn, layer="airports"))
	if (class(airports) != "try-error") summary(airports)
}
dsn <- system.file("vectors/ps_cant_31.MIF", package = "rgdal")[1]
ogrInfo(dsn=dsn, layer="ps_cant_31")
ps_cant_31 <- readOGR(dsn=dsn, layer="ps_cant_31")
summary(ps_cant_31)
dsn <- system.file("vectors/Up.tab", package = "rgdal")[1]
ogrInfo(dsn=dsn, layer="Up")
Up <- readOGR(dsn=dsn, layer="Up")
summary(Up)
dsn <- system.file("vectors/test_trk2.gpx", package = "rgdal")[1]
test_trk2 <- try(readOGR(dsn=dsn, layer="tracks"))
if (class(test_trk2) != "try-error") summary(test_trk2)
test_trk2pts <- try(readOGR(dsn=dsn, layer="track_points"))
if (class(test_trk2pts) != "try-error") summary(test_trk2pts)
dsn <- system.file("vectors", package = "rgdal")[1]
ogrInfo(dsn=dsn, layer="trin_inca_pl03")
birds <- readOGR(dsn=dsn, layer="trin_inca_pl03")
summary(birds)

mmmh ... and PostgreSQL in not listed in ogrDrivers()

ogrDrivers()

> ogrDrivers()
             name write
1          AVCBin FALSE
2          AVCE00 FALSE
3             BNA  TRUE
4             CSV  TRUE
5             DGN  TRUE
6  ESRI Shapefile  TRUE
7      Geoconcept  TRUE
8         GeoJSON  TRUE
9             GML  TRUE
10            GMT  TRUE
11            GPX  TRUE
12            KML  TRUE
13   MapInfo File  TRUE
14         Memory  TRUE
15            REC FALSE
16            S57  TRUE
17           SDTS FALSE
18          TIGER  TRUE
19        UK .NTF FALSE
20            VRT FALSE
21         XPlane FALSE

in url:http://cran.r-project.org/web/packages/rgdal/index.html rgdal.pdf read.ogr details we read :

"The drivers available will depend on the installation of GDAL/OGR, and can vary; the ogrDrivers() function shows which are available, and which may be written (but all are assumed to be readable). Note that stray ﬁles in data source directories (such as *.dbf) may lead to suprious errors that ac- companying *.shp are missing.
"

After sending a mail to the R-Sig-Geo list, the answer is as follows (by Roger Bivand and we thank him !)

If you mean the Windows binary rgdal package, then the answer is in

file.show(system.file("README", package="rgdal"))
file.show(system.file("README.windows", package="rgdal"))

The Windows binary package builds the minimal set of drivers, with only one external dependency (expat for reading GPX and KML). The file refered to describes how you might build rgdal from source, using FWTools or OSGEO4W as the provider of GDAL and its dependent DLLs. If you do try this out (look for the chunk beginning: "Initial notes for OSGeo4W"), and if:

source(system.file("OSGeo4W_test", package="rgdal"), echo=TRUE)

works, as well as PostGIS access, please let me know.

Installing FWTools is a convenient way to get started (but will be limited to the - extensive - range of drivers built into the binary package): cedric => however we are using Qgis and OSGeo4W tool so working with FWtools does not interest us url:http://fwtools.maptools.org/

url:http://www.microsoft.com/express/Downloads/#2008-Visual-CPP

To understand the script from the readme file (below), one has to be familiar with visual C++ The following pages are helpfull url:http://trac.osgeo.org/gdal/wiki/BuildingWithMinGW for building with mingwin
url:http://trac.osgeo.org/gdal/wiki/MakeFileProjects for building with windows
url:http://vterrain.org/Distrib/gdal.html to build against vc++
url:http://www.eoinmurphy.org/blog/2008/07/09/setting-up-mingw-and-msys/

Below is adapted from Roger Bivand

## Cedric notes for building gdal against OSGeo4W

## 1- download rgdal sources

## 2- Run in OSGeo4W console, after setting:

set OSGEO4W_BUILD=yes
set GDAL_HOME=%OSGEO4W_ROOT%


## 3-get the Rdll.lib
## MinGW is avalaible with Rtools
## install pexports (available at http://sourceforge.net/projects/mingw/files/ somewhere down the long list ...
cd C:\Program Files\R\R-2.10.1\bin
C:\Rtools\MinGW\bin\ pexports R.dll > R.exp

## using visual c++ (vc9) command line
cd C:\Program Files\R\R-2.10.1\bin
lib /def:R.exp /out:Rdll.lib
cd C:/rgdal/src
cl /MT /Ox /EHsc /D "WIN32" /c /I "C:/Program Files/R/R-2.10.1/include" /I C:/OSGeo4W/include /I "C:\Program Files\Microsoft Visual Studio 9.0\VC\include" /D OSGEO4W *.cpp
link /dll /out:rgdal.dll /def:rgdal.def *.obj "C:/Program Files/R/R-2.10.1/bin/Rdll.lib" /libpath:"C:\Program Files\Microsoft Visual Studio 9.0\VC\lib" /libpath:"C:\Program Files\Microsoft SDKs\Windows\v6.0A\Lib" /libpath:"C:\OSGeo4W\lib" gdal_i.lib proj_i.lib
cd ../..

## 4. Ensure the configure.win, Makefile.win and RGDAL.def are all correct. 

## set the location of GDAL_HOME in configure.win (mine is GDAL_HOME="C:/OSGeo4W")

-//-- start configure.win

#!/bin/sh
if test -z "$GDAL_HOME" ; then
    GDAL_HOME="C:/OSGeo4W"
fi
echo "GDAL_HOME=${GDAL_HOME}" > src/Makeconf.win

echo "using GDAL_HOME: ${GDAL_HOME}"

if test -z "$OSGEO4W_BUILD" ; then

  if test -z "$R_PACKAGE_DIR" ; then
    R_PACKAGE_DIR=${DPKG}
 fi

  mkdir "${R_PACKAGE_DIR}/libs"
  cp ${GDAL_HOME}/bin/adrg.dll "${R_PACKAGE_DIR}/libs"
  cp ${GDAL_HOME}/bin/cblas.dll "${R_PACKAGE_DIR}/libs"
  cp ${GDAL_HOME}/bin/dted.dll "${R_PACKAGE_DIR}/libs"
  cp ${GDAL_HOME}/bin/fribidi.dll "${R_PACKAGE_DIR}/libs"
  cp ${GDAL_HOME}/bin/gdiplus.dll "${R_PACKAGE_DIR}/libs"
  cp ${GDAL_HOME}/bin/gdal15.dll "${R_PACKAGE_DIR}/libs"
  cp ${GDAL_HOME}/bin/gdal16.dll "${R_PACKAGE_DIR}/libs"
  echo "OSGEO= " >> src/Makeconf.win

else
    echo "OSGEO=-DOSGEO4W" >> src/Makeconf.win
fi
-//-- end configure.win


## [rgdal]/src  
##   - includes a Makefile.win with one line:   all:
##   - and RGDAL.def as supplied in rgdal/src

## Place Makefile.win in src/, containing a single line:

all:

##5. Install package (my rdal sources are still in c:/)

## CD C:\

C:\"Program Files"\R\R-2.10.1\bin\Rcmd INSTALL --build rgdal
## Note if this step does not work look if you have the Rtools properly installed and configured (especially the path)
## Then copy manually all dll of c:/OSGeo4W/bin in the newly rgdal/libs 

##Post-installation, start R from the OSGeo4W console (command line). If you want
##to check that the OSGeo4W-aware rgdal is present, check the reported support
##file locations, and consider running:
source(system.file("OSGeo4W_test", package="rgdal"), echo=TRUE)

I don't think that you need to copy any OSGEO4W DLLs to the package tree. 
I do think that you have to start R from within an OSGEO4W console or environment, which has set up the PATH for the executables. I don't think that you need to modify configure.win, just make sure that the package build process is being run from within OSGEO4W. I think that running everything within your target environment is the key.

ogrDrivers()
source(system.file("OSGeo4W_test", package="rgdal"), echo=TRUE)
readOGR(dsn="PG:host=localhost dbname=eda2.0 user=postgres password=postgres port=5433",layer="ccm21.riversegments_bretagne")

...urlhttp://download.osgeo.org/fdo/3.4.1/binaries/

help for rgdal package
url:file:///C:/Program%20Files/R/R-2.10.0/library/rgdal/html/00Index.html

Céline : "J'ai recherché dans mes bouquin et sur le Applied Spatial Data Analysis with R de Roger Bivand dont tu peux retrouver (une partie) des codes sur ce site" : url:http://www.asdar-book.org/ Plus précisément sur cette page url:http://www.asdar-book.org/book/die.R ou à url:http://www.asdar-book.org/book/die_mod.R

Download in other formats:

Plain Text