mobile and gis dev notes » GIS

Nominatim + homebrew on OSX + OSM data + PHP = open sourced reverse geocoder

rupert — Mon, 14 Nov 2011 05:45:20 +0000

This installation guide (at the time of writing) was tested on SVN trunk of OSM2PGSQL and running on latest/stable Postgres/Postgis versions on OSX via homebrew.

1. Summary

OSX Snow Leopard
 
OSM2PGSQL: 
Head http://svn.openstreetmap.org/applications/utils/export/osm2pgsql Revision: 27034
Last Changed Author: frederik
Last Changed Rev: 27030
Last Changed Date: 2011-11-09 10:57:49 +1100 (Wed, 09 Nov 2011)
 
POSTGRES: 9.0.4
 
POSTGIS: "POSTGIS="1.5.3" GEOS="3.3.1-CAPI-1.7.1" PROJ="Rel. 4.7.1, 23 September 2009" LIBXML="2.7.3" USE_STATS" # SELECT POSTGIS_FULL_VERSION();
 
PHP: 5.3.8 (cli) (built: Nov 14 2011 14:41:52) #php -v

2. Installation
Most of the software is installed via homebrew.

# Install homebrew
% /usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"
 
# Install postgresql
% brew install postgresql
% initdb -E utf8 -D /usr/local/var/postgres
% cp /usr/local/Cellar/postgresql/9.0.4/org.postgresql.postgres.plist ~/Library/LaunchAgents/
% launchctl load -w ~/Library/LaunchAgents/org.postgresql.postgres.plist
% psql -d postgres -f /usr/local/Cellar/postgresql/9.0.4/share/postgresql/contrib/adminpack.sql
 
# Install postgis
% brew install proj
% brew install geos
% brew install postgis
 
# Create template_postgis_osm
% createdb -E utf8 template_postgis_osm
% psql -d template_postgis_osm -f "/usr/local/Cellar/postgresql/9.0.4/share/postgresql/contrib/pg_trgm.sql"
% psql -d template_postgis_osm -f /usr/local/Cellar/postgis/1.5.3/share/postgis/postgis.sql
% psql -d template_postgis_osm -f /usr/local/Cellar/postgis/1.5.3/share/postgis/spatial_ref_sys.sql
 
# Install osm2pgsql. Can skip this.
# % brew install osm2pgsql

For detail instructions on installing Postgres/Postgis via Homebrew, read this homebrew + postgresql9.0.4 + postgis.1.5.3 + proj4 + geos3.3.1 + osm2pgsql. If you are having problems installing GEOS, then read that link as it shows you how to upgrade GEOS to 3.3.1.

OSM2PGSQL needs GEOS as well. Note that brew only install the osm2pgsql binary. Don’t worry, we will compile this via source later.

3. More Installation.

We need to get PHP installed to run gazetteer http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/gazetteer/website/

# Install PHP
% brew install php --with-mysql --with-pgsql --with-apache
 
# Hookup with Apache
# Edit httpd.conf to LoadModule
 
# Install PEAR DB
% pear install db

4. OSM2PGSQL
Read this wiki: http://wiki.openstreetmap.org/wiki/Osm2pgsql. Well, we eventually need the whole OSM2PGSQL source as it contains the website (gazetteer).

% svn co http://svn.openstreetmap.org/applications/utils/export/osm2pgsql osm2pgsql
% cd osm2pgsql
% ./autogen.sh
% ./configure
% make
 
# At this point there should be an osm2pgsql binary.

We need to compile gazetteer for gazetteer.so which is used by gazetteer-functions.sql

gis/osm2pgsql/gazetteer% make clean
gis/osm2pgsql/gazetteer% make 
gis/osm2pgsql/gazetteer% make install
test -z "/usr/local/lib/osm2pgsql" || .././install-sh -c -d "/usr/local/lib/osm2pgsql"
 /bin/sh ../libtool --mode=install /usr/bin/install -c  'gazetteer.la' '/usr/local/lib/osm2pgsql/gazetteer.la'
libtool: install: /usr/bin/install -c .libs/gazetteer.so /usr/local/lib/osm2pgsql/gazetteer.so
libtool: install: /usr/bin/install -c .libs/gazetteer.lai /usr/local/lib/osm2pgsql/gazetteer.la
----------------------------------------------------------------------
Libraries have been installed in:
   /usr/local/lib/osm2pgsql
 
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the `-LLIBDIR'
flag during linking and do at least one of the following:
   - add LIBDIR to the `DYLD_LIBRARY_PATH' environment variable
     during execution
 
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
test -z "/usr/local/share/gazetteer" || .././install-sh -c -d "/usr/local/share/gazetteer"
 /usr/bin/install -c -m 644 'extract_countrynames.sql' '/usr/local/share/gazetteer/extract_countrynames.sql'
 /usr/bin/install -c -m 644 'gazetteer-index.sql' '/usr/local/share/gazetteer/gazetteer-index.sql'
 /usr/bin/install -c -m 644 'gazetteer-loaddata.sql' '/usr/local/share/gazetteer/gazetteer-loaddata.sql'
 /usr/bin/install -c -m 644 'gazetteer-tables.sql' '/usr/local/share/gazetteer/gazetteer-tables.sql'
 /usr/bin/install -c -m 644 'import_country_name.sql' '/usr/local/share/gazetteer/import_country_name.sql'
 /usr/bin/install -c -m 644 'import_country_osm_grid.sql' '/usr/local/share/gazetteer/import_country_osm_grid.sql'
 /usr/bin/install -c -m 644 'import_gb_postcodearea.sql' '/usr/local/share/gazetteer/import_gb_postcodearea.sql'
 /usr/bin/install -c -m 644 'import_gb_postcode.sql' '/usr/local/share/gazetteer/import_gb_postcode.sql'
 /usr/bin/install -c -m 644 'import_specialwords.sql' '/usr/local/share/gazetteer/import_specialwords.sql'
 /usr/bin/install -c -m 644 'import_us_statecounty.sql' '/usr/local/share/gazetteer/import_us_statecounty.sql'
 /usr/bin/install -c -m 644 'import_us_state.sql' '/usr/local/share/gazetteer/import_us_state.sql'
 /usr/bin/install -c -m 644 'import_worldboundaries.sql' '/usr/local/share/gazetteer/import_worldboundaries.sql'
 /usr/bin/install -c -m 644 'gazetteer-functions.sql' '/usr/local/share/gazetteer/gazetteer-functions.sql'

5. Download Data
You can get some regional OSM data from cloudmade. http://downloads.cloudmade.com/oceania/australia_and_new_zealand/australia/victoria

I suggest you download a regional extract prior to downloading/testing with the whole planet-osm. If you don’t believe me that it will take long, you can read http://wiki.openstreetmap.org/wiki/Nominatim/Installation

6. Load and Index Data
Basically, this is the summary of commands taken from http://wiki.openstreetmap.org/wiki/Nominatim/Installation At the time of writing this, I had issues such as “planet_osm_ways” (and several tables) does not exist. So I did a pg_dump and restored the tables afterwards. Be very careful with using the script below, you can comment the indexing part just to speed up on loading and see if you have errors, etc.

DATABASE_NAME=gazetteer_vic
OSM2PGSQL_HOME=/Users/rupert/projects/gis/osm2pgsql
SOURCE_DATA=/Users/rupert/Desktop/australia/victoria.osm
DUMP_DIR=/Users/rupert/Desktop/pg_dumps/streetlookup
 
dropdb $DATABASE_NAME 
#dropuser www-data
 
createdb $DATABASE_NAME -E UTF8 -T template_postgis_osm
createuser -SDR www-data
 
# This will create the planet_osm_ways, etc
$OSM2PGSQL_HOME/osm2pgsql --create --latlong --database $DATABASE_NAME --username rupert --slim --prefix planet_osm --cache 2048 $SOURCE_DATA
 
pg_dump --host 127.0.0.1 --port 5432 --username rupert --format custom --file "$DUMP_DIR/planet_osm_ways.backup" --table public.planet_osm_ways $DATABASE_NAME
pg_dump --host 127.0.0.1 --port 5432 --username rupert --format custom --file "$DUMP_DIR/planet_osm_nodes.backup" --table public.planet_osm_nodes $DATABASE_NAME
pg_dump --host 127.0.0.1 --port 5432 --username rupert --format custom --file "$DUMP_DIR/planet_osm_rels.backup" --table public.planet_osm_rels $DATABASE_NAME
 
# This will create the place table
$OSM2PGSQL_HOME/osm2pgsql --latlong -O gazetteer --database $DATABASE_NAME --username rupert --slim --prefix planet_osm --cache 2048 $SOURCE_DATA
 
pg_restore --host 127.0.0.1 --port 5432 --username rupert --dbname $DATABASE_NAME "$DUMP_DIR/planet_osm_ways.backup"
pg_restore --host 127.0.0.1 --port 5432 --username rupert --dbname $DATABASE_NAME "$DUMP_DIR/planet_osm_nodes.backup"
pg_restore --host 127.0.0.1 --port 5432 --username rupert --dbname $DATABASE_NAME "$DUMP_DIR/planet_osm_rels.backup"
 
rm -Rf $DUMP_DIR/*.backup
 
psql -d $DATABASE_NAME -q -f import_country_osm_grid.sql
psql -d $DATABASE_NAME -q -f import_worldboundaries.sql
psql -d $DATABASE_NAME -q -f import_country_name.sql
psql -d $DATABASE_NAME -q -f import_gb_postcode.sql
psql -d $DATABASE_NAME -q -f import_gb_postcodearea.sql
psql -d $DATABASE_NAME -q -f import_us_state.sql
psql -d $DATABASE_NAME -q -f import_us_statecounty.sql
 
psql -d $DATABASE_NAME -f gazetteer-functions.sql
 
psql -d $DATABASE_NAME -f gazetteer-tables.sql
 
psql -d $DATABASE_NAME -f gazetteer-functions.sql
 
psql -d $DATABASE_NAME -f gazetteer-loaddata.sql
 
#Indexing
psql -d $DATABASE_NAME -f gazetteer-index.sql

Save this as run.sh in /Users/rupert/projects/gis/osm2pgsql/gazetteer

Where do you run this?

% cd /Users/rupert/projects/gis/osm2pgsql/gazetteer
% sh run.sh

6. Test
If you are successful, you should have a “placex” table. Now that we have a postgis database running, you can now run spatial statements thru pgadmin. See the guts of reverse.php

SELECT * 
FROM placex
WHERE ST_DWithin( ST_SetSRID(ST_Point(145.234377, -37.856320),4326), geometry, 0.0001)
AND ST_GeometryType(geometry) NOT IN ('ST_Polygon', 'ST_MultiPolygon')

This one took only 21 ms.

7. Website

Make sure www-data have permissions to the tables. Rememeber to replace gazetteer_vic with your DATABASE_NAME.

for tbl in `psql -qAt -c "select tablename from pg_tables where schemaname = 'public';" gazetteer_vic` ; do psql -c "alter table $tbl owner to \"www-data\"" gazetteer_vic; done

Assuming you have PHP and PEAR DB installed. Then update the data connection settings found in http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/gazetteer/website/.htlib/settings.php

Run the same query but using reverse.php.

http://127.0.0.1/nominatim/reverse.php?format=xml&lat=-37.856320&lon=145.234377&zoom=18&addressdetails=1

osx + homebrew + postgresql9.0.4 + postgis.1.5.3 + proj4 + geos3.3.1 + osm2pgsql

rupert — Mon, 14 Nov 2011 02:02:21 +0000

For the impatient

% /usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"
% brew install postgresql
% initdb -E utf8 -D /usr/local/var/postgres
% cp /usr/local/Cellar/postgresql/9.0.4/org.postgresql.postgres.plist ~/Library/LaunchAgents/
% launchctl load -w ~/Library/LaunchAgents/org.postgresql.postgres.plist
% psql -d postgres -f /usr/local/Cellar/postgresql/9.0.4/share/postgresql/contrib/adminpack.sql
% brew install proj
% brew install geos %you should really read below before running this
% brew install postgis
% createdb -E utf8 template_postgis
% psql -d template_postgis -f /usr/local/Cellar/postgis/1.5.3/share/postgis/postgis.sql
% psql -d template_postgis -f /usr/local/Cellar/postgis/1.5.3/share/postgis/spatial_ref_sys.sql

1. Install homebrew

%open https://github.com/mxcl/homebrew
%open https://github.com/mxcl/homebrew/wiki/Installation
%/usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"

Install python first if you need PL/Python in postgres. The –framework option tells Homebrew to compile a Framework-style Python build, rather than a UNIX-style build. Read this INSTALLATION GUIDE.

%brew install python --framework

2. Install Postgres
Note that postgres installation will pick up the latest Python Configuration, so it is important to ensure that we want the specific PYTHON interpreter to compile with our postgresql installation. You can turn the flag -v to enable verbose installation. Watch for the configuration log output for PYTHON to check if it loads the PYTHON environment that we expect

%PYTHON=/usr/local//bin/python brew install postgresql -v
....
checking for python... /usr/local/bin/python
checking for Python distutils module... yes
checking Python configuration directory... /usr/local/Cellar/python/2.7.2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config
...
If builds of PostgreSQL 9 are failing and you have version 8.x installed,
you may need to remove the previous version first. See:
  https://github.com/mxcl/homebrew/issues/issue/2510
 
To build plpython against a specific Python, set PYTHON prior to brewing:
  PYTHON=/usr/local/bin/python  brew install postgresql
See:
  http://www.postgresql.org/docs/9.0/static/install-procedure.html
 
 
If this is your first install, create a database with:
  initdb /usr/local/var/postgres
 
If this is your first install, automatically load on login with:
  mkdir -p ~/Library/LaunchAgents
  cp /usr/local/Cellar/postgresql/9.0.4/org.postgresql.postgres.plist ~/Library/LaunchAgents/
  launchctl load -w ~/Library/LaunchAgents/org.postgresql.postgres.plist
 
If this is an upgrade and you already have the org.postgresql.postgres.plist loaded:
  launchctl unload -w ~/Library/LaunchAgents/org.postgresql.postgres.plist
  cp /usr/local/Cellar/postgresql/9.0.4/org.postgresql.postgres.plist ~/Library/LaunchAgents/
  launchctl load -w ~/Library/LaunchAgents/org.postgresql.postgres.plist
 
Or start manually with:
  pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start
 
And stop with:
  pg_ctl -D /usr/local/var/postgres stop -s -m fast
 
 
Some machines may require provisioning of shared memory:
  http://www.postgresql.org/docs/current/static/kernel-resources.html%SYSVIPC
 
If you want to install the postgres gem, including ARCHFLAGS is recommended:
    env ARCHFLAGS="-arch x86_64" gem install pg
 
To install gems without sudo, see the Homebrew wiki.
==> Summary
/usr/local/Cellar/postgresql/9.0.4: 2577 files, 35M, built in 3.1 minutes
brew install postgresql  188.73s user 62.38s system 106% cpu 3:55.06 total

3. Postgres Post Installation. Initialize DB.

% initdb -E utf8 -D /usr/local/var/postgres
The files belonging to this database system will be owned by user "rupert".
This user must also own the server process.
 
The database cluster will be initialized with locale en_AU.UTF-8.
The default text search configuration will be set to "english".
 
creating directory /usr/local/var/postgres ... ok
creating subdirectories ... ok
selecting default max_connections ... 20
selecting default shared_buffers ... 2400kB
creating configuration files ... ok
creating template1 database in /usr/local/var/postgres/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
 
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the -A option the
next time you run initdb.
 
Success. You can now start the database server using:
 
    postgres -D /usr/local/var/postgres
or
    pg_ctl -D /usr/local/var/postgres -l logfile start

4. Postgres Startup

% cp /usr/local/Cellar/postgresql/9.0.4/org.postgresql.postgres.plist ~/Library/LaunchAgents/
% launchctl load -w ~/Library/LaunchAgents/org.postgresql.postgres.plist
%telnet 127.0.0.1 5432
Trying 127.0.0.1...
Connected to rupert-mbp.
Escape character is '^]'.
% psql -d postgres -f /usr/local/Cellar/postgresql/9.0.4/share/postgresql/contrib/adminpack.sql

UPDATE:
I have now disabled the automatic startup of Postgres as it provides me more control especially during development. Extract Postgres9.1.2.tar.gz to /Library/StartupItems.

To start/stop

/Library/StartupItems/Postgres9.1.2/Postgres9.1.2 start

5. Postgis
Note this will install dependencies, PROJ4 and GEOS. At the time of writing this, we have a problem with GEOS. Need to update GEOS formula to 3.3.1. For more info read:

% open https://github.com/mxcl/homebrew/issues/8151
% open https://gist.github.com/1306088
% brew edit geos

% brew install proj
% brew install geos
% brew install postgis
% createdb -E utf8 template_postgis
% psql -d template_postgis -f /usr/local/Cellar/postgis/1.5.3/share/postgis/postgis.sql
% psql -d template_postgis -f /usr/local/Cellar/postgis/1.5.3/share/postgis/spatial_ref_sys.sql

6. osm2pgsql

~% brew install osm2pgsql
==> Checking out http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/
==> ./autogen.sh
==> ./configure
==> make
/usr/local/Cellar/osm2pgsql/HEAD: 6 files, 328K, built in 70 seconds

7. Check for loaded/configured PLPYTHON environment for POSTGRES
If you get import module errors, you should check your PYTHON version and if the module is loaded in your python’s site-packages. The function below will provide which version of PYTHON is compiled with your POSTGRESQL.

CREATE PROCEDURAL LANGUAGE
'plpythonu' HANDLER plpython_call_handler;
 
CREATE OR REPLACE FUNCTION pyver() returns text AS
$$
import sys
RETURN sys.version
$$ LANGUAGE 'plpythonu';
 
SELECT pyver();
"2.7.2 (default, Dec 30 2011, 09:25:27) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.1.00)]"

Installing Postgres8.3,8.4,9.0, Postgis1.3.3,1.4.1,2.0, pgRouting on OSX (updated)

rupert — Tue, 15 Feb 2011 03:31:08 +0000

(Updated):Postgres9.0.3 Postgis2.0svn

Before you go any further. Dump data before migrating and installing. Warning: Most of the packages listed below is installed by source.

1. Download the current postgres source. http://www.postgresql.org/ftp/source/

8.3:
$./configure --with-prefix=/usr/local/pgsql --with-python
 
8.4:
$./configure --prefix=/usr/local/pgsql --with-python
 
$make
$sudo make install

2. Don’t delete the postgres folder. You might need this later on for future compilations. See pgadmin3 admin pack below.

3. Add a postgres user from System Preferences > Accounts

4. Initialize the data directory

$mkdir /usr/local/pgsql/data
$chown postgres /usr/local/pgsql/data/
$su - postgres
$/usr/local/pgsql/bin/initdb -E utf8 -D /usr/local/pgsql/data

The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale C.
The default text search configuration will be set to "english".

fixing permissions on existing directory /usr/local/pgsql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 20
selecting default shared_buffers/max_fsm_pages ... 2400kB/20000
creating configuration files ... ok
creating template1 database in /usr/local/pgsql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the -A option the
next time you run initdb.

Success. You can now start the database server using:

/usr/local/pgsql/bin/postgres -D /usr/local/pgsql/data
or
/usr/local/pgsql/bin/pg_ctl -D /usr/local/pgsql/data -l logfile start

4. Install Geos , PROJ4, and Postgis

GEOS:

- install geos with the Kyngchaos installer

- installing by source

svn checkout http://svn.osgeo.org/geos/trunk trunk
cd trunk/
sh autogen.sh
./configure --prefix=/usr/local/geos --enable-python
make
make install
sudo make install

PROJ4:

svn co http://svn.osgeo.org/metacrs/proj/trunk/proj/ proj
cd proj
sh autogen.sh
./configure --prefix=/usr/local/proj4
make
sudo make install

POSTGIS:

8.3:
$./configure --with-pgsql=/usr/local/pgsql/bin/pg_config --with-geos=/usr/local/bin/geos-config
 
8.4 using Kyngchaos GEOS Framework:
$./configure --with-pgsql=/usr/local/pgsql/bin/pg_config --with-geosconfig=/Library/Frameworks/GEOS.framework/unix/bin/geos-config
 
9.0:using Kyngchaos GEOS + PROJ framework:(Update Feb 15, 2011)
./configure --with-pgsql=/usr/local/pgsql/bin/pg_config --with-geosconfig=/Library/Frameworks/GEOS.framework/unix/bin/geos-config --with-projdir=/Library/Frameworks/PROJ.framework/unix/ --with-gettext=/usr/local/share/gettext/
 
Note: libintl.h error. You need gettext
gcc -g -O2  -fno-common -DPIC  -Wall -Wmissing-prototypes  -DUSE_NLS -DLOCALEDIR=\"/usr/local/pgsql/share/locale\" -Iyes/include  -c shp2pgsql-core.c
In file included from shp2pgsql-core.c:15:
shp2pgsql-core.h:17:21: error: libintl.h: No such file or di
 
9.0.4: using src from GEOS + PROJ
./configure --with-pgconfig=/usr/local/pgsql/bin/pg_config --with-geosconfig=/usr/local/geos/bin/geos-config --with-projdir=/usr/local/proj4
 
$make
$sudo make install

If everthing is successful, you should see files in /usr/local/pgsql/share/contrib/.

cd /usr/local/pgsql/share/contrib/postgis-2.0
- legacy.sql
- postgis.sql
- postgis_upgrade_20_minor.sql
- spatial_ref_sys.sql
- uninstall_legacy.sql
- uninstall_postgis.sql

5. Starting postgres on boot. Download postgresstartup.tar.gz then extract to your /Library/StartupItems

6. Creating the database

createdb -E utf8 template_postgis
createlang plpgsql template_postgis
 
8.3:
psql -d template_postgis -f /usr/local/pgsql/share/lwpostgis.sql
psql -d template_postgis -f /usr/local/pgsql/share/spatial_ref_sys.sql
 
8.4:
psql -d template_postgis -f /usr/local/pgsql/share/contrib/postgis.sql 
psql -d template_postgis -f /usr/local/pgsql/share/contrib/spatial_ref_sys.sql
 
9.0 + postgis2.0:
psql -d template_postgis -f /usr/local/pgsql/share/contrib/postgis-2.0/postgis.sql 
psql -d template_postgis -f /usr/local/pgsql/share/contrib/postgis-2.0/spatial_ref_sys.sql

7. Install and download pgAdmin3 for MacOS X

http://www.postgresql.org/ftp/pgadmin3/release/

Update Feb 15, 2011: Use pgadmin1.12 to connect with postgres9. Other versions (1.8.4, 1.6.3 don’t work)

8. Startup pgadmin3. You will notice there is a window stating…

Server instrumentation
The server lacks instrumentation functions.
pgAdmin III uses some support functions that are not available by default in all PostgreSQL versions. These enable some tasks that make life easier when dealing with log files and configuration files.

9. Compile the adminpack. Go to $postgresql_install_directory/contrib/adminpack

$ make
gcc -no-cpp-precomp -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Winline -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv  -bundle -multiply_defined suppress  adminpack.o  -L../../src/port -bundle_loader ../../src/backend/postgres  -o libadminpack.0.0.so
rm -f libadminpack.0.so
ln -s libadminpack.0.0.so libadminpack.0.so
rm -f libadminpack.so
ln -s libadminpack.0.0.so libadminpack.so
 
rupert:adminpack rupert$ sudo make install
Password:
mkdir /usr/local/pgsql/share/contrib
/bin/sh ../../config/install-sh -c -m 644 ./uninstall_adminpack.sql '/usr/local/pgsql/share/contrib'
/bin/sh ../../config/install-sh -c -m 644 adminpack.sql '/usr/local/pgsql/share/contrib'
/bin/sh ../../config/install-sh -c -m 755  libadminpack.0.0.so '/usr/local/pgsql/lib/adminpack.so'

10. Load the adminpack.sql into your maintenance and template database

$ psql -U postgres -d postgres -f /usr/local/pgsql/share/contrib/adminpack.sql 
CREATE FUNCTION
CREATE FUNCTION
CREATE FUNCTION
...

11. Disconnect and Reconnect from pgAdmin3. You shouldn’t see the window again.

Continue only if you want pgRouting

12. Essentially we would need Boost Graph Library (BGL) a.k.a boost, Genetic Algorithm Utility Library (or, GAUL for short), and GEOS (which we installed earlier when we installed postgis).

13. The easiest way to install boost is using MacPorts.

$ sudo port install boost-jam
--->  Fetching boost-jam
--->  Attempting to fetch boost-jam-3.1.16.tgz from http://downloads.sourceforge.net/boost
--->  Verifying checksum(s) for boost-jam
--->  Extracting boost-jam
--->  Configuring boost-jam
--->  Building boost-jam
--->  Staging boost-jam into destroot
--->  Installing boost-jam 3.1.16_0
--->  Activating boost-jam 3.1.16_0
--->  Cleaning boost-jam
--->  Fetching gmake
--->  Attempting to fetch make-3.81.tar.bz2 from http://ftp.gnu.org/gnu/make
--->  Verifying checksum(s) for gmake
--->  Extracting gmake
--->  Configuring gmake
--->  Building gmake with target all
--->  Staging gmake into destroot
--->  Installing gmake 3.81_0
--->  Activating gmake 3.81_0
--->  Cleaning gmake
--->  Fetching boost
--->  Attempting to fetch boost_1_34_1.tar.bz2 from http://downloads.sourceforge.net/boost
...
$ sudo port install boost

Note: I encountered an error when I directly installed “sudo port install boost” on my first run. A quick workaround is to install boost-jam, then install boost afterwards. For more details:

Error: Target org.macports.build returned: shell command ” cd “/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_devel_boost/work/boost_1_34_1″ && gmake all ” returned error 2 Command output: Makefile:2: *** missing separator. Stop.

14. You can check if boost was successfully installed by…

$ sudo port -v installed boost boost-jam
The following ports are currently installed:
  boost @1.34.1_3+darwin_9 (active)
  boost-jam @3.1.16_0 (active)

15. For GAUL, we don’t need slang base on http://pgrouting.postlbs.org/wiki/1.x/InstallationManual.

$ ./configure --enable-slang=no
$ make
$ make install

16. Ok, so this is the heartbreaker. I was able to get pass cmake on pgRouting on version 1.02 however, I received “undefined symbols” when linking the librouting.dylib

Linking CXX shared library ../../lib/librouting.dylib
Undefined symbols:
  "_errcode", referenced from:
      _shortest_path in dijkstra.o
      _shortest_path_astar in astar.o
      _shortest_path_shooting_star in shooting_star.o

For the complete error details, see pgrouting_problem.txt

17. Thanks to www.kyngchaos.com, we can install http://www.kyngchaos.com/files/software/unixport/pgRouting-1.0.1-4.dmg binary from http://www.kyngchaos.com/wiki/software:postgres

18. Now we can load the pgRouting functions to our template database

psql -U postgres -d template_postgis -f /usr/local/pgsql/share/contrib/routing_core.sql 
psql -U postgres -d template_postgis -f /usr/local/pgsql/share/contrib/routing_core_wrappers.sql 
psql -U postgres -d template_postgis -f /usr/local/pgsql/share/contrib/routing_dd.sql
psql -U postgres -d template_postgis -f /usr/local/pgsql/share/contrib/routing_dd_wrappers.sql 
psql -U postgres -d template_postgis -f /usr/local/pgsql/share/contrib/routing_dd_tsp.sql
psql -U postgres -d template_postgis -f /usr/local/pgsql/share/contrib/routing_tsp.sql
psql -U postgres -d template_postgis -f /usr/local/pgsql/share/contrib/routing_tsp_wrappers.sql

References:

http://developer.apple.com/internet/opensource/postgres.html

http://www2.russbrooks.com:8080/2007/11/4/install-postgresql-on-mac-os-x-10-5-leopard

Migrating postgres/postgis data (updated)

rupert — Tue, 15 Feb 2011 00:51:56 +0000

One of my hurdles recently is migrating data from my Debian Lenny Desktop Box (Postgres8.2.7/Postgis1.3.1) to my new MacBookPro Leopard(Postgres8.3.1/Postgis1.3.3). I found it out the hard way by inspecting the dump files manually.

1. pg_dump is your friend.

pg_dump --help

2. I strongly suggest if you have a big dump file (mine is 500MB) to split the schema from the data.

Add “-s” to create the schema:

pg_dump -C -s -h 127.0.0.1 -U lbs -W beijing > beijing_20080507_schema.sql

Add “-a” to dump the data only:

pg_dump -a -h 127.0.0.1 -U lbs -W beijing > beijing_20080507_schema.sql

3. pg_dump from an 8.2 would have statically linked liblwgeom to /usr/lib/postgresql/8.2/liblwgeom. You should change that to whereever your liblwgeom resides, mine is on /usr/local/pgsql/lib/liblwgeom. Just do a simple search and replace using vim on your file_schema.sql

4. After editing the schema, we can now restore the structure of the database. Check for errors and manually update the schema if needs be.

psql -h 127.0.0.1 -U beijing_4326 < beijing_20080507_schema.sql

5. Ok, so now we have the structure ready, we can also check this from pgAdmin3. Have a good look on the functions and table structures if they are fully restored.

6. Let’s load the data.

psql -h 127.0.0.1 -U beijing_4326 < beijing_20080507_data.sql

Postgres PostGIS CheatSheet v2

rupert — Thu, 11 Nov 2010 02:29:12 +0000

This is a quick-command list of Postres. If you want detailed instructions, please visit the Postgres Manual.

How do I know the version of Postgis?

SELECT POSTGIS_FULL_VERSION();

How do I Show all databases?
1. Using “psql -l”

2. Using

postgres=# \l
List of databases
Name       |  Owner   | Encoding
------------------+----------+----------
postgis          | postgres | UTF8
postgres         | postgres | UTF8
template0        | postgres | UTF8
template1        | postgres | UTF8
template_postgis | postgres | UTF8
(5 rows)

Note: Do not drop template databases if not necessary.

How do I run a script from the prompt?

psql -d cybersoftbj -u user -f myfile.sql

Its very useful in reloading user-defined functions.

How do I create a user/role?

CREATE ROLE lbs WITH LOGIN PASSWORD 'mypassword' SUPERUSER INHERIT CREATEDB CREATEROLE;

How do I change the password for a user/role?

ALTER ROLE lbs PASSWORD 'mynewpassword';

How to provide/restrict access privileges to tables?

GRANT SELECT ON TABLE TABLE TO user;
REVOKE SELECT ON TABLE TABLE FROM user;

How to dump database in a text file?

pg_dump -U lbs -d cybersoftbjv1 -h 127.0.0.1 -W &gt; cybersoftbjv1.sql

How to dump database cleanly?

% pg_dump -c  -d -E UTF8 -h 127.0.0.1 -U lbs -W platform_v1 &gt; platform_v1.sql

How to rename a database?

ALTER DATABASE beijing_app RENAME TO beijing_app_20080801;

How to update using two tables?

UPDATE road_for_update u
SET the_geom = r.the_geom
FROM roads r
WHERE r.rd_id = u.rd_id;

DROP TABLE

DROP TABLE IF EXISTS "my_table";

How to change a column type with Cast?

ALTER TABLE roads ALTER COLUMN class_new TYPE integer USING class_new::integer;

How to add a geometry column to a table?

SELECT AddGeometryColumn('public', 'poi', 'the_geom', 4326, 'POINT', 2)

Changing column names with spaces?

ALTER TABLE class_aroundme RENAME "level 1" TO level_1;

Setting kernel shmmax for postgres

% sysctl -w kernel.shmmax=134217728

Note: For permanent changes see /etc/sysctl.cfg

11. How to backup table(s) from pg_dump?

% pg_dump poi_beijing -t class -t poi_class -f $BACKUPDIR/test_$MYDATE.sql

Change integer primary key to serial

CREATE SEQUENCE seq_job_id INCREMENT 1 MINVALUE 1000 MAXVALUE 2147483648 START 1000 CACHE 1;
 
ALTER TABLE job ALTER COLUMN job_id SET DEFAULT NEXTVAL('seq_job_id'::regclass);
 
SELECT * FROM job
 
SELECT NEXTVAL('seq_job_id')

Date and Time Function Helper: date_add

CREATE OR REPLACE FUNCTION date_add(diffType Character Varying(15), incrementValue bigint, inputDateTime timestamp without time zone) RETURNS timestamp AS $$
DECLARE
   YEAR_CONST Char(15) := 'year';
   MONTH_CONST Char(15) := 'month';
   DAY_CONST Char(15) := 'day';
   HOUR_CONST Char(15) := 'hour';
   MIN_CONST Char(15) := 'minute';
   SEC_CONST Char(15) := 'second';
 
   dateTemp timestamp without time zone;
   intervals interval;
BEGIN
   IF lower($1) = lower(YEAR_CONST) THEN
       SELECT cast(cast(incrementvalue AS character varying) || ' year' AS interval) INTO intervals;
   ELSEIF lower($1) = lower(MONTH_CONST) THEN
       SELECT cast(cast(incrementvalue AS character varying) || ' months' AS interval) INTO intervals;
   ELSEIF lower($1) = lower(DAY_CONST) THEN
       SELECT cast(cast(incrementvalue AS character varying) || ' day' AS interval) INTO intervals;
   ELSEIF lower($1) = lower(HOUR_CONST) THEN
       SELECT cast(cast(incrementvalue AS character varying) || ' hours' AS interval) INTO intervals;
   ELSEIF lower($1) = lower(MIN_CONST) THEN
       SELECT cast(cast(incrementvalue AS character varying) || ' minutes' AS interval) INTO intervals;
   ELSEIF lower($1) = lower(SEC_CONST) THEN
       SELECT cast(cast(incrementvalue AS character varying) || ' seconds' AS interval) INTO intervals;            
   END IF;
 
   dateTemp:= inputDateTime + intervals;
 
   RETURN dateTemp;
END;
$$ LANGUAGE plpgsql;

How to set the current timezone in postgres?

# Session based only
SET time zone 'utc';
SELECT  current_setting('TIMEZONE');
# Permanent
# Edit /usr/local/var/postgres/postgresql.conf (#postgres installed via homebrew)
timezone = 'UTC'

Date/Time Functions

SELECT current_setting('TIMEZONE'); 
--"Australia/Victoria"
 
SELECT Now(), 
timezone('UTC', now()), 
EXTRACT(EPOCH FROM CURRENT_TIMESTAMP(0)),
to_timestamp(EXTRACT(EPOCH FROM CURRENT_TIMESTAMP(0))),
to_timestamp(EXTRACT(EPOCH FROM CURRENT_TIMESTAMP(0)))::TIMESTAMP
 
--"2011-11-14 09:29:14.249427+11" 
--"2011-11-13 22:29:14.249427"
--1321223354
--"2011-11-14 09:29:14+11"
--"2011-11-14 09:29:14"

How to specify the id of a sequence to prevent SQL Error: PGRES_FATAL_ERROR:ERROR: duplicate key value violates unique constraint “gps_histories_pkey”?
This happens when the maximum number of records in gps_histories is not in sync with the sequence id.

SELECT SETVAL('dfms_4000.gps_histories_id_seq', (SELECT MAX(id) FROM dfms_4000.gps_histories)+1)

iPhone Note #20:Integrating Mapserver/TileCache to RouteMe

rupert — Thu, 14 Jan 2010 01:37:57 +0000

Below is a summary of how I was able to implement Mapserver, TileCache and Route-Me iPhone Mapping Framework.

1. Assuming you have a working Mapserver/TileCache setup. Take note of the ff parameters: resolution and bbox. Below is my tilecache.cfg:

 61 [mapserver_australia_3857]
 62 type=MapServerLayer
 63 mapfile=/Users/rupert/projects/pelicancorp/DMOB/trunk/map/australia_3857.map
 64 layers=all
 65 extension=jpg
 66 bbox=-20037508.34, -20037508.34, 20037508.34, 20037508.34
 68 maxResolution=156543.033928041
 70 levels=20
 71 srs=EPSG:3857
 72 tms_type=google
 73 extent_type=loose
 74 spherical_mercator=true

2. Grab the RMGenericMercatorWMSSource from http://groups.google.com/group/route-me-map/browse_thread/thread/b58fa1d20cf15823/e30c42d9c90a8170?lnk=gst&q=Generic#e30c42d9c90a8170

3. By default without any changes, the RMGenericMercatorWMSSource could display Mapserver WMS Tiles. This is possible by passing an NSDictionary *parameters, which contains arrayValues and arrayKeys for creating an http 256×256 image request to the http://127.0.0.1/cgi-bin/mapserv binary.

//This would request to http://127.0.0.1/cgi-bin/mapserv
NSArray *arrayValues = [[NSArray alloc] initWithObjects:@"/path-to/australia_3857.map", @"all", @"png", @"EPSG:3857",nil];
NSArray *arrayKeys = [[NSArray alloc] initWithObjects:@"MAP", @"LAYERS", @"FORMAT", @"SRS", nil];
NSDictionary *wmsParameters = [[NSDictionary alloc] initWithObjects: arrayValues forKeys:arrayKeys ];

Resulting http requests to the mapserv binary:

http://192.168.1.193:81/tilecache/tilecache.py?LAYERS=australia_3857&SRS=EPSG:3857&REQUEST=GetMap&SERVICE=WMS&STYLES=&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&FORMAT=png&VERSION=1.1.1&WIDTH=256&HEIGHT=256&BBOX=16828376.000000,-4006523.000000,16833268.000000,-4001631.000000

Note that the bbox values does not contain decimal places. Nevertheless, it still works on Mapserver.

4. Now, assuming we have a valid tilecache running. And it is tested from browser, i.e http://127.0.0.1/map/tilecache_3857.html, open up firebug to see the requests. Below is a sample…

http://192.168.1.193:81/tilecache/tilecache.py?LAYERS=australia_3857&FORMAT=jpg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A3857&BBOX=16123932.497377,-4539747.9811239,16143500.376618,-4520180.1018829&WIDTH=256&HEIGHT=256

But it seems, route-me is not supplying the bbox values accurately, as the decimal values is truncated (BBOX=16828376.000000,-4006523.000000,16833268.000000,-4001631.000000) from our previous request(3).

After changing the wmsParameter values to create a tilecache request, I noticed that route-me is not displaying the tiles correctly.

//Testing for Windows:TileCache - ?
NSArray *arrayValues = [[NSArray alloc] initWithObjects:@"australia_3857", @"png", @"EPSG:3857",nil]; //for WIndows
 
//TileCache URL Parameters:
NSArray *arrayKeys = [[NSArray alloc] initWithObjects:@"LAYERS", @"FORMAT", @"SRS", nil];

5. The workaround is to use “double” instead of “floats” in the RMGenericMercatorWMSSource. You can download the zip from RMGenericMercatorWMSSource.zip. I have modified “initialResolution” and “originShift” to both use “double”.

typedef struct {
	double x;
	double y;
} CGDoublePoint;
 
typedef struct { 
	CGDoublePoint ul; 
	CGDoublePoint lr; 
} CGXYRect;

Afterwards, I copied the maxResolution and bbox (tilecache) and specified it for the initialResoultion and originShift respectively.

#import "RMGenericMercatorWMSSource.h"
 
CGFloat DegreesToRadians(CGFloat degrees) {return degrees * M_PI / 180;}; 
CGFloat RadiansToDegrees(CGFloat radians) {return radians * 180/ M_PI;}; 
 
@implementation RMGenericMercatorWMSSource
 
-(id) initWithBaseUrl:(NSString *)baseUrl parameters:(NSDictionary *)params
{ 
	if (![super init]) 
		return nil; 
 
	// 156543.03392804062 for sideLength 256 pixels 
	// initialResolution = 2 * M_PI * 6378137 / [self tileSideLength];
	// specify here whatever the resolution is from tilecache.
	initialResolution = 156543.033928041;
 
	// 20037508.342789244 
	//originShift = 2 * M_PI * 6378137 / 2;
	//originShift = 20037508.342789244f;
	// specify here whatever the bbox is from tilecache.
	originShift = 20037508.32;
 
	NSLog(@"test initialResolution:%f originShift:%f", initialResolution, originShift);
 
	// setup default parameters
	// use official EPSG:3857 by default, user can override to 900913 if needed.
	wmsParameters = [[NSMutableDictionary alloc] initWithObjects:[[[NSArray alloc] initWithObjects:@"EPSG:3857",@"image/png",@"GetMap",@"1.1.1",@"WMS",nil] autorelease] 
											  forKeys:[[[NSArray alloc] initWithObjects:@"SRS",@"FORMAT",@"REQUEST",@"VERSION",@"SERVICE",nil] autorelease]];
	[wmsParameters addEntriesFromDictionary:params];
 
	// build WMS request URL template
	urlTemplate = [NSString stringWithString:baseUrl];
	NSEnumerator *e = [wmsParameters keyEnumerator];
	NSString *key;
	NSString *delimiter = @"";
	while (key = [e nextObject]) {
		urlTemplate = [urlTemplate stringByAppendingFormat:@"%@%@=%@",
					   delimiter,
					   [[key uppercaseString] stringByAddingPercentEscapesUsingEncoding:NSASCIIStringEncoding], 
					   [[wmsParameters objectForKey:key] stringByAddingPercentEscapesUsingEncoding:NSASCIIStringEncoding]];
		delimiter = @"&";
	}
	int sideLength =  [self tileSideLength];
	urlTemplate = [[urlTemplate stringByAppendingFormat:@"&WIDTH=%d&HEIGHT=%d",sideLength,sideLength] retain];
	return self;
}
 
// implement in subclass?
-(NSString*) uniqueTilecacheKey
{
	return @"AbstractMercatorWMSSource";
}
 
-(NSString *)shortName
{
	return @"Generic WMS Source";
}
-(NSString *)longDescription
{
	return @"Generic WMS Source";
}
-(NSString *)shortAttribution
{
	return @"Generic WMS Source";
}
-(NSString *)longAttribution
{
	return @"Generic WMS Source";
}
 
-(float) minZoom
{
	return 1.0f;
}
-(float) maxZoom
{
	return 19.0f;
}
 
// Converts given lat/lon in WGS84 Datum to XY in Spherical Mercator EPSG:3857 
-(CGPoint) LatLonToMeters: (CLLocationCoordinate2D) latlon 
{ 
	CGPoint meters; 
	meters.x = latlon.longitude * originShift / 180; 
	meters.y = (log( tan((90.0 + latlon.latitude) * M_PI / 360.0 )) / (M_PI / 180.0)) * originShift / 180; 
	return meters; 
}
 
//Converts XY point from Spherical Mercator EPSG:3857 to lat/lon in WGS84 Datum 
-(CLLocationCoordinate2D) MetersToLatLon: (CGPoint) meters 
{ 
	CLLocationCoordinate2D latlon; 
	latlon.longitude = (meters.x / originShift) * 180.0; 
	latlon.latitude = (meters.y / originShift) * 180.0; 
	//latlon.latitude = - 180 / M_PI * (2 * atan( exp( latlon.latitude * M_PI / 180.0)) - M_PI / 2.0); 
	latlon.latitude = 180 / M_PI * (2 * atan( exp( latlon.latitude * M_PI / 180.0)) - M_PI / 2.0); 
	return latlon; 
} 
 
// Converts pixel coordinates in given zoom level of pyramid to EPSG:3857 
-(CGDoublePoint) PixelsToMeters: (int) px PixelY:(int)py atZoom:(int)zoom 
{ 
	double resolution = [self ResolutionAtZoom: zoom]; 
	CGDoublePoint meters; 
	double x = (px * resolution - originShift); 
	double y = (py * resolution - originShift); 
 
	meters.x = x;
	meters.y = y;
 
	NSLog(@"px: %d py: %d resolution: %f originShift: %f x: %f y: %f", px, py, resolution, originShift, x, y);
 
	return meters; 
} 
 
-(NSString*) tileURL: (RMTile) tile 
{ 
	//RMLatLongBounds tileBounds = [self TileLatLonBounds:tile];
	// Get BBOX coordinates in meters
	CGXYRect tileBounds = [self TileBounds:tile];
 
	NSString *url = [urlTemplate stringByAppendingFormat:@"&BBOX=%f,%f,%f,%f",
					 tileBounds.ul.x,
					 tileBounds.lr.y,
					 tileBounds.lr.x,
					 tileBounds.ul.y];
 
	NSLog(@"Tile %d,%d,%d yields %@",tile.zoom, tile.x, tile.y, url); 
 
	return url; 
} 
 
 
//Returns bounds of the given tile in EPSG:3857 coordinates 
-(CGXYRect)  TileBounds: (RMTile) tile 
{
	int sideLength =  [self tileSideLength];
 
	int zoom = tile.zoom;
	long twoToZoom = pow(2,zoom);
	CGXYRect tileBounds; 
	tileBounds.ul = [self PixelsToMeters: (tile.x * sideLength) 
								  PixelY: ((twoToZoom-tile.y) * sideLength) 
								  atZoom: zoom ]; 
	tileBounds.lr = [self PixelsToMeters: ((tile.x+1) * sideLength) 
								  PixelY: ((twoToZoom-tile.y-1) * sideLength) 
								  atZoom: zoom];
	return tileBounds; 
} 
 
//Resolution (meters/pixel) for given zoom level (measured at Equator) 
-(double) ResolutionAtZoom : (int) zoom 
{ 
	return initialResolution / pow(2,zoom); 
} 
 
@end

6. Download TileCache_RouteMe.zip Note: You need a valid mapserver and tilecache running.

TileCache: Getting Cutoff on the edges

rupert — Fri, 24 Oct 2008 03:37:43 +0000

In Tilecache, metaTiling creates larger tiles and chops them up.
Note the difference between using metatile=yes and metatile=no.

Figure 1: metatile=no

Figure 2: metatile=yes

Recaching single tiles in Tilecache

rupert — Mon, 22 Sep 2008 04:16:12 +0000

Got this one from IRC this morning.. In order to recache single tiles in TC, as long as it is not metatile (metatile=false), then we can simply append “FORCE=1″ in the URL..

http://192.168.2.14/tilecache/tilecache.py?FORCE=1&LAYERS=beijing_900913_wide_en&MAP=%2Fmyhome%2Fmap%2Fbeijing%2Fnew%2Fbeijing_900913_wide_en%2Fbeijing.map&FORMAT=jpg&TRANSPARENT=true&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A900913&BBOX=12951949.03168994,4852222.893873828,12952101.905746482,4852375.767930372&WIDTH=256&HEIGHT=256

Well for completeness, I gave my ofcmate a DELETE tool which wipes out everything in the /tilecache/map_dir directory..

Transform 900913 to 4326

rupert — Wed, 03 Sep 2008 14:37:18 +0000

Stumbled upon an error in reprojecting data from 900913 (Google Speherical Mercator) to 4326 (WGS84). I’m recalling this from my head now, the error was something related to “NAD sth”. Workaround was to convert it as follows:

1. 900913
2. 32650 (UTM for your area)
3. 4326

I’ll get the SQL from my ofcmate and paste it here…

Postgres PostGIS CheatSheet v2

rupert — Fri, 29 Aug 2008 05:10:32 +0000

This is a quick-command list of Postres. If you want detailed instructions, please visit the Postgres Manual.

How do I Show all databases?
1. Using “psql -l”

2. Using

postgres=# \l
List of databases
Name       |  Owner   | Encoding
------------------+----------+----------
postgis          | postgres | UTF8
postgres         | postgres | UTF8
template0        | postgres | UTF8
template1        | postgres | UTF8
template_postgis | postgres | UTF8
(5 rows)

Note: Do not drop template databases if not necessary.

How do I run a script from the prompt?
psql -d cybersoftbj -u user -f myfile.sql

Its very useful in reloading user-defined functions.

How do I create a user/role?
CREATE ROLE lbs WITH LOGIN PASSWORD 'mypassword' SUPERUSER INHERIT CREATEDB CREATEROLE;

How do I change the password for a user/role?
ALTER ROLE lbs PASSWORD 'mynewpassword';

How to provide/restrict access privileges to tables?
GRANT SELECT ON TABLE table TO user; REVOKE SELECT ON TABLE table FROM user;

How to dump database in a text file?
pg_dump -U lbs -d cybersoftbjv1 -h 127.0.0.1 -W > cybersoftbjv1.sql

How to dump database cleanly?
pg_dump -c -d -E UTF8 -h 127.0.0.1 -U lbs -W platform_v1 > platform_v1.sql

How to rename a database?
ALTER DATABASE beijing_app RENAME TO beijing_app_20080801;

How to update using two tables?
UPDATE road_for_update u SET the_geom = r.the_geom FROM roads r WHERE r.rd_id = u.rd_id;

How to change a column type with Cast?
ALTER TABLE roads ALTER COLUMN class_new TYPE integer USING class_new::integer;

How to add a geometry column to a table?
EXAMPLE: SELECT AddGeometryColumn(‘public’, ‘poi’, ‘the_geom’, 4326, ‘POINT’, 2)

Changing column names with spaces?
ALTER TABLE class_aroundme RENAME "level 1" TO level_1;

Setting kernel shmmax for postgres
sysctl -w kernel.shmmax=134217728
Note: For permanent changes see /etc/sysctl.cfg11. How to backup table(s) from pg_dump?
pg_dump poi_beijing -t class -t poi_class -f $BACKUPDIR/test_$MYDATE.sql

Using GDALWARP to reproject raster that will fit with Google Satellite

rupert — Tue, 29 Jul 2008 12:48:21 +0000

Just a couple of notes to onself using gdal: Use gdalwarp to reproject your GeoTIFF files! I wanted to use my own satellite images acquired from GeoEye, however, on some areas I wanted to use google sat images as well since I don’t have the coverage. In order to do so, I need to reproject the sat images to 900913. Note you need to specify this in your epsg file in my previous post.

rupert:beijing_900913_satellite rupert$ gdalinfo Mosaic_RGB.tif
Driver: GTiff/GeoTIFF
Files: Mosaic_RGB.tif
Size is 4248, 4553
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.2572235630016,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433],
    AUTHORITY["EPSG","4326"]]
Origin = (116.291476140000000,40.025198500000002)
Pixel Size = (0.000046860000000,-0.000035970000000)
Metadata:
  AREA_OR_POINT=Area
  TIFFTAG_XRESOLUTION=100
  TIFFTAG_YRESOLUTION=100
Image Structure Metadata:
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  ( 116.2914761,  40.0251985) (116d17'29.31"E, 40d 1'30.71"N)
Lower Left  ( 116.2914761,  39.8614271) (116d17'29.31"E, 39d51'41.14"N)
Upper Right ( 116.4905374,  40.0251985) (116d29'25.93"E, 40d 1'30.71"N)
Lower Right ( 116.4905374,  39.8614271) (116d29'25.93"E, 39d51'41.14"N)
Center      ( 116.3910068,  39.9433128) (116d23'27.62"E, 39d56'35.93"N)
Band 1 Block=4248x1 Type=Byte, ColorInterp=Red
Band 2 Block=4248x1 Type=Byte, ColorInterp=Green
Band 3 Block=4248x1 Type=Byte, ColorInterp=Blue

rupert:beijing_900913_satellite rupert$ gdalwarp -s_srs epsg:4326 -t_srs epsg:900913 Mosaic_RGB.tif sat_4m_rgb.tif
Creating output file that is 4245P x 4556L.
Processing input file Mosaic_RGB.tif.
0...10...20...30...40...50...60...70...80...90...100 - done.

rupert:beijing_900913_satellite rupert$ gdalinfo sat_4m_rgb.tif
Driver: GTiff/GeoTIFF
Files: sat_4m_rgb.tif
Size is 4245, 4556
Coordinate System is:
PROJCS["Google Maps Global Mercator",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.2572235630016,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0],
        UNIT["degree",0.0174532925199433],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Mercator_1SP"],
    PARAMETER["central_meridian",0],
    PARAMETER["scale_factor",1],
    PARAMETER["false_easting",0],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]]]
Origin = (12945507.907502911984921,4869604.732793668285012)
Pixel Size = (5.219801430503303,-5.219801430503303)
Metadata:
  AREA_OR_POINT=Area
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (12945507.908, 4869604.733) (116d17'29.31"E, 40d12'53.10"N)
Lower Left  (12945507.908, 4845823.317) (116d17'29.31"E, 40d 3'2.78"N)
Upper Right (12967665.965, 4869604.733) (116d29'25.89"E, 40d12'53.10"N)
Lower Right (12967665.965, 4845823.317) (116d29'25.89"E, 40d 3'2.78"N)
Center      (12956586.936, 4857714.025) (116d23'27.60"E, 40d 7'58.12"N)
Band 1 Block=4245x1 Type=Byte, ColorInterp=Red
Band 2 Block=4245x1 Type=Byte, ColorInterp=Green
Band 3 Block=4245x1 Type=Byte, ColorInterp=Blue

Installing Mapserver on MacOSX (by source)

rupert — Sat, 26 Jul 2008 02:03:16 +0000

Just noticed that William of kyngchaos has updated the mapserver binary for MacOSX.

But right now, I need to tile these images bought from GeoEye, so I need TIFF support. Below is a summary of getting Mapserver installed by source. Note that I have the necessary GEOS, GDAL from kyngchaos as well from this ealier post.

1. Download the ff files:
-rw-r--r--@ 1 rupert admin 564313 Jul 26 10:32 agg-2.5.tar.gz -rw-r--r--@ 1 rupert admin 1345700 Jul 26 10:22 gd-2.0.35.tar.gz -rw-r--r--@ 1 rupert admin 613261 Jul 26 10:22 jpegsrc.v6b.tar.gz -rw-r--r--@ 1 rupert admin 796551 Jul 26 10:22 libpng-1.2.29.tar.gz -rw-r--r--@ 1 rupert admin 1948751 Jul 26 09:55 mapserver-5.2.0.tar.gz -rw-r--r--@ 1 rupert admin 1336295 Jul 26 11:11 tiff-3.8.2.tar.gz

2. Install in the ff order:

- jpegsrc
- libpng
- gd (if you have trouble installing gd, then follow this pdf:installing_gd2_on_os_x_server)
- agg (make only)
- tiff
- mapserver

3. For mapserver, please install using the ff configure switches:

./configure \
--with-agg=/myhome/rupert/mapserver/agg-2.5 \
--with-jpeg \
--with-gd \
--with-freetype \
--with-png \
--with-ogr \
--with-proj \
--with-gd \
--with-httpd=/usr/local/apache2/bin/httpd \
--with-tiff \
--with-wfs \
--with-wcs \
--with-sos \
--with-wmsclient \
--with-wfsclient \
--with-tiff \
--with-gdal=/usr/local/bin/gdal/gdal-config \
--with-geos=/usr/local/bin/geos-config \
--with-postgis=/usr/local/pgsql/bin/pg_config

4. Mapserver output

rupert:mapserver-5.2.0 rupert$ ./mapserv -v
MapServer version 5.2.0 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG OUTPUT=WBMP OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=ICONV SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=GEOS INPUT=TIFF INPUT=EPPL7 INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE

Postgres PostGis PostInstall

rupert — Fri, 06 Jun 2008 23:34:36 +0000

1. Edit pg_hba.conf

# "local" is for Unix domain socket connections only
local   all         all                               trust
# IPv4 local connections:
host    all         all         127.0.0.1/32          md5
host    all         all         192.168.1.0 255.255.255.0       md5
# IPv6 local connections:
#host    all         all         ::1/128               trust

2. Edit postgres.conf

55
56 listen_addresses = '*' # what IP address(es) to listen on;
57 # comma-separated list of addresses;
58 # defaults to 'localhost', '*' = all
59 # (change requires restart)

4. For CentOS5.1, create symbolic links:
ln -s /usr/local/lib/libproj.so.0 /usr/lib/libproj.so.0
ln -s /usr/local/lib/libgeos_c.so.1 /usr/lib/libgeos_c.so.1
ldconfig
/etc/init.d/postgresql stop
/etc/init.d/postgresql start

3. Postgis Post Install

createdb -E utf8 template_routing
createlang plpgsql template_routing
psql -d template_routing -f /usr/share/postgresql-8.3-postgis/lwpostgis.sql 
psql -d template_routing -f /usr/share/postgresql-8.3-postgis/spatial_ref_sys.sql 
psql -d template_routing -f /usr/share/postlbs/routing_core.sql 
psql -d template_routing -f /usr/share/postlbs/routing_core_wrappers.sql

Installing Postgresql, Postgis, pgRouting on Debian

rupert — Sun, 25 May 2008 03:55:57 +0000

Operating System: Debian sid

Versions:

postgres 8.3.1
postgis 1.3.3
pgRouting1.02

1. Install base system and ssh

#vi /etc/apt/sources.list to include
deb http://debian.cn99.com/debian etch main
deb-src http://debian.cn99.com/debian etch main
#apt-get update
#apt-get upgrade libc6

2. Install the required packages for postgres8.3 and postgis1.3.3

#apt-get install sudo nmap telnet
#apt-get install python2.5 python2.5-dev python-setuptools
#apt-get install g++
#apt-get install build-essential cmake ibboost-graph-dev
#apt-get install libreadline5 libreadline5-dev
#apt-get install zlib-bin zlib1g-dev
#apt-get install libkrb5-dev
#apt-get install libcurl3
#apt-get install libssl-dev
#apt-get install postgresql-8.3
#apt-get install postgresql-8.3-postgis
#apt-get install postgresql-server-dev-8.3

3. Installing pgRouting

# tar -zxvf pgRouting-1.02.tgz
# cmake .
# make 
# make install

Jpeg decoder problem on TileCache on MacOSX Leopard

rupert — Mon, 12 May 2008 07:26:07 +0000

There is a problem with a “jpeg decoder” because Python Imaging Library (PIL) cannot find the jpeg libraries. Remember that we installed kyngchaos UnixIO libraries, therefore the PIL setup.py script should point to use those libraries.

Actually, I even installed the jpegsrc manually. I believe there is no need to do this, since UnixIO Image libraries is sufficient enough for PIL to install. What is important is to ensure that python tests succeeded. Once you get passed that, then installing PIL should work.

1. Extract Imaging-1.1.6.tar.gz
2. Edit setup.py to reflect:
JPEG_ROOT="/Library/Frameworks/UnixImageIO.framework/unix"
3. python setup.py build_ext -i
4. If no test fails, then go ahead and install
5. python setup.py install

Installing Mapserver on MacOSX Leopard (the easy way)

rupert — Tue, 06 May 2008 13:01:09 +0000

The easiest way to install mapserver on your Leopard is by downloading and installing dmg files from www.kyngchaos.com (courtesy of William Kyngesburye). These binaries were also noted from Mapserver’s Download Page.

1. Download the ff binaries in order (please note the version numbers at the time of writing):

1. UnixImageIO_Framework-1.0.22a.dmg
2. FreeType_Framework-2.3.5-3.dmg
3. GEOS_Framework-3.0.0-2.dmg
4. PROJ_Framework-4.6.0-1.dmg
5. SQLite3_Framework-3.5.7-1.dmg
6. MapServer-5.0.2-2.dmg

2. Once installed, you can copy the mapserv binaries to your apache cgi-bin

sudo cp /Library/WebServer/CGI-Executables/mapserv /usr/local/apache2/cgi-bin/

3. Check the mapserv output

rupert:~ rupert$ /usr/local/apache2/cgi-bin/mapserv -v
MapServer version 5.0.2 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG OUTPUT=WBMP OUTPUT=SWF OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI SUPPORTS=GEOS INPUT=EPPL7 INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE

This is actually the first time I was able to install mapserver NOT BY SOURCE and still achieve the same binaries that I wanted (with AGG support). Full credit should be given to William.

Interesting Location Technologies

rupert — Fri, 25 Apr 2008 02:32:46 +0000

Been very busy for the past few days/weeks in the office. Just read this one from Google Reader, Nokia develops navigating system based on image recognition, landmarks. In summary, base from Nokia’s Palo Alto Research center, providing direction guiding varies from gender and culture. However, providing examples such as “Turn Right after the next gas station” seems to be a universal and a normal behavior of providing directions, the person can relate more to the advice than providing line routes above synthetic maps.

This is interesting facts as it supports our theory in Call Center Directed Guiding. Although, a 360 or panoramic view of any point along the line would be ideal, an easier and more realistic approach for the time being would be to include important landmark snapshots along the route. However, I believe our organisation’s project is one step ahead since we have a warm body who already gave a summary of the route during the conversation. This means the caller have an idea of the route complexity beforehand. Then, we could also send this landmark images to the subscriber’s mobile as his future reference (which means additional VAS).

Another interesting location technology, fruitful for direction guiding, would be iPointer. I believe, base on the coordinates and orientation, the server could algorithmically predict which POI you are looking at or infront of. They mentioned they have a thin client on the mobile device, so I guess the S60 client tries to get the orientation base on the bearing. Also check this one out, “users can now walk down the street anywhere in Japan and point at over 700,000 objects such as buildings, shops, restaurants, banks, historical sites and instantly retrieve information on what they are looking at or find what they are looking for just by pointing their phone.”. Niche eh? Just compells me more to start up our mobile pursuit…

But what rocked me the most was indoor tracking by Ubisense using ultrawideband technology which is close to a foot?!?. This I have to see, as it opens new business cases for direction guiding indoors such as big malls, etc. Imagine mapjack views inside malls with Ubisense technology. Sweet.

Other References:
http://www.openlandmark.com/

Using TileCache, OpenLayers, Mapserver for Projection 900913

rupert — Tue, 08 Apr 2008 01:50:12 +0000

I had a few problems with TileCache the other week which I am eager to blog about, since I knew for sure that later on, I might encounter the same. I don’t have the exact errors with me right now, so I’m jotting this down from my head…

Classic Resolutions problem. Use extent_type=loose
Can not set image type

UPDATED (JAN 11, 2010): Classic Resolutions problem:

How are resolutions calculated? Assuming we have:

Original:
Lower Left (LL) or minx, miny: 12453557, -5434940
Upper Right (UR) or max, maxy: 16980842, -1180729

maxResolution = (max – minx)/tilesize = (16980842 – 12453557)/512 = 8842.353
where tilesize = 512.

Therefore, we can set/guess for max so that we have maxResolution as a whole number.

Adjusted:
minx, miny: 12453557, -5434940
maxx, maxy: 16980661, -1180729

gives a maxResolution (whole number) of 8842.

Now, you can use 8842 in both the TileCache.cfg and OpenLayers Javascript.
A. Google BaseLayer + Road WMS Overlay = Validation approach before we proceed to TC.

1. OpenLayers (OL) Code: http://www.gisnotes.com/platform/docs/examples/GoogleWMSRoads.cfm

2. Map File Configuration: /home/map/beijing/new/beijing_google_roads

Well the above files are not exactly for TC, but it is a good demonstration that we could overlay our custom layers (WMS) on top of Google as the base layer. For a more detailed explanation regarding Google + OL, please read my previous gisnote. Also, this is a good test to see if our Mapserver mapfile configuration is correct. I do suggest not to proceed with OL + TC not unless you have validated the mapfile through OL (Google + WMS).

B. Google900913 Tilecache as baselayer

Ok.. so here it is..

1. OpenLayers Code: http://www.gisnotes.com/platform/docs/examples/TilecacheGoogle900913Base.cfm

        window.onload = function() {
            var options = {
                        projection: new OpenLayers.Projection("EPSG:900913"),
                        units: "m",
 
                        //maxResolution: 156543.0339,
                        resolutions : [156543.03390000001, 78271.516950000005, 39135.758475000002, 19567.879237500001, 9783.9396187500006, 4891.9698093750003, 2445.9849046875001, 1222.9924523437501, 611.49622617187504, 305.74811308593752, 152.87405654296876, 76.43702827148438, 38.21851413574219, 19.109257067871095, 9.5546285339355475, 4.7773142669677737, 2.3886571334838869, 1.1943285667419434, 0.59716428337097172, 0.29858214168548586],
                        //zoom: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19] @ numZoomLevels: 20
                        numZoomLevels: 20,
                        //maxExtent: new OpenLayers.Bounds(12823075.86334, 4800551.12375, 13101918.14248, 5021301.26141)
                        //maxExtent: new OpenLayers.Bounds(12823075, 4800551, 13101918, 5021301)
                        //maxExtent: new OpenLayers.Bounds(12848138, 4785083, 13080212, 5021118.5)
                        maxExtent: new OpenLayers.Bounds(-20037508, -20037508,20037508, 20037508.34)
                    };
 
            OpenLayers.IMAGE_RELOAD_ATTEMPTS = 3;
            OpenLayers.Util.onImageLoadErrorColor = "transparent";
 
            map = new OpenLayers.Map('mapdiv',options);
 
            var serverURL = "/tilecache/tilecache.py";
            // start custom layer here
            var layer_obj = new OpenLayers.Layer.WMS(
                "Beijing",
                serverURL,
                {
                    layers: 'beijing_900913',
                    map: '/home/map/beijing/new/beijing_900913/beijing.map',
                    format: 'image/jpg',
                    'transparent': 'off'
                }
            );
            layer_obj.setIsBaseLayer(true);
            layer_obj.setVisibility(true);
            map.addLayer(layer_obj);

2. TileCache Configuration: /wwwroot/tilecache/tilecache.cfg

[beijing_900913]
type=WMSLayer
url=http://127.0.0.1/cgi-bin/mapserv?map=/home/map/beijing/new/beijing_900913/beijing.map
layers=beijing_900913
metatile=true
extension=jpg
levels=20
resolutions=156543.03390000001, 78271.516950000005, 39135.758475000002, 19567.879237500001, 9783.9396187500006, 4891.9698093750003, 2445.9849046875001, 1222.9924523437501, 611.49622617187504, 305.74811308593752, 152.87405654296876, 76.43702827148438, 38.21851413574219, 19.109257067871095, 9.5546285339355475, 4.7773142669677737, 2.3886571334838869, 1.1943285667419434, 0.59716428337097172, 0.29858214168548586
srs=EPSG:900913
bbox=-20037508, -20037508, 20037508, 20037508.34
#maxResolution=156543.0339
extent_type=loose
debug=on

You can either set resolutions or maxResolutions. I always wanted to be explicit, so setting the resolutions is my preference.

3. MapFile Configuration: /home/map/beijing/new/beijing_900913/beijing.map
I have a long mapfile, and using includes helped a lot, see my previous gisnote. For debugging your mapfile, read this post.

Always Remember…I believe when they made TC, I presume they always tested on WGS84(lonlat), this means if you don’t set anything explicit, it would implicitly set settings in WGS84(lon/lat)…

Ensure that you have the vector data in the necessary projection, in my case it’s 900913.

Ensure that the resolutions from OL (B.1) is reflected in TC configuration (B.2).

Ensure that the maxextents from OL (B.1) is reflected in TC configuration (B.2) as bbox.

C. Still can’t get it running?

1. Narrow down the error.

2. tail -f /var/log/apache2/error.log and access.log is your bestfriend. It’s a good starting point to trace down what went wrong. If you can’t see it from Apache’s error.log then ensure that python debugging is turned on.

3. Sometimes TC [a] gives you an obvious error or [b] its just a link to the mapserv request.

Examples of [a]:

Classic Resolution problem stated above. - If you followed my final notes above, then this should not happen. If you’ve set up things correctly but it’s still broken and you want to pull your hair out, then do a clean test stated below in D.
Cannot set image type – Check the extensions from the mapfile and TC config. Are they both in jpg’s?

The latter [b] is harder to debug. Most likely its a mapserver mapfile mishap and the easiest way to hunt this down is only thru your error.log (note: you need mapserver to have a debug=true when you compiled it, that’s another story.)

D. Doing a clean test as always
1. Always disable any caching when testing. Disable cache from your browser, for FF I use the webdeveloper toolbar. It’s as easy as Disable -> Disable Cache.

2. Stop/Start Apache. Hoping Python get’s a clean start as well. If you found any pyc (python compiled) under TC source directory… then stop Apache; remove the pycs; start Apache.

3. Seeing the same images? Always remove the image caches! rm -Rf /data/tilecache/beijing_900913

Installing Mapserver on Debian (reprise)

rupert — Fri, 28 Mar 2008 11:31:43 +0000

As noted from my previous blog post regarding Mapserver on Debian, you don’t get AGG with Mapserver when installing directly from Debian packages. Thus, it would be better to install Mapserver by source. However, debian still helps because it would install all the necessary libraries needed for compiling mapserver.

./configure \
--with-agg \
--with-jpeg \
--with-gd \
--with-freetype \
--with-png \
--with-ogr \
--with-proj \
--with-gdal \
--with-httpd=/usr/sbin/apache2 \
--with-tiff \
--with-wfs \
--with-wcs \
--with-threads \
--with-wmsclient \
--with-wfsclient \
--with-geos \
--with-postgis \
--enable-debug

When I started installing yesterday, I did not know which packages/libraries are installed in my box. I dove off immediately by running my configure.sh above and received an error: Could not find gd.h or libgd.a/libgd.so in /usr/local..

The best way to figure out if you’re ready is reading thru the configure output.

checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ANSI C... none needed
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for ranlib... ranlib
checking for flex... no
checking for lex... no
checking for yywrap in -lfl... no
checking for yywrap in -ll... no
checking for bison... no
checking for byacc... no
checking if compiler supports -R... no
checking if compiler supports -Wl,-rpath,... yes
checking for exp in -lm... yes
checking for __gxx_personality_v0 in -lstdc++... yes
checking how to run the C preprocessor... gcc -E
checking for egrep... grep -E
checking for ANSI C header files... yes
checking for strcasecmp... yes
checking for strncasecmp... yes
checking for strdup... yes
checking for strlcat... no
checking for vsnprintf... yes
MapServer Version from mapserver.h: '5.0.2'
configure: checking where FreeType 2.x is installed...
checking for freetype-config... /usr/bin/freetype-config
using libfreetype from -lfreetype -lz
checking for FT_Init_FreeType in -lfreetype... yes
using libfreetype -lfreetype from system libs.
configure: checking where Zlib is installed...
checking for zlibVersion in -lz... yes
using libz from system libs (-DUSE_ZLIB).
configure: checking where PNG is installed...
checking for png_init_io in -lpng... yes
using libpng from system libs.
configure: checking whether we should include JPEG support...
checking for jpeg_read_header in -ljpeg... yes
using libjpeg from system libs.
configure: checking where libXpm is installed...
checking for XpmFreeXpmImage in -lXpm... yes
using libXpm from system libs.
configure: checking where libiconv is installed...
checking for iconv_open in -lc... yes
checking for libiconv_open in -liconv... no
using libiconv from system libs.
libiconv found. Enabling internationalization (-DUSE_ICONV)
configure: checking for GD 2.0.16 or higher...
checking for gdFontCacheSetup in -lgd... yes
using libgd 2.0.16 (or higher) from system libs (-L/usr/lib -lgd -ljpeg -lfreetype -lpng -lz -lXpm -lX11 ).
checking for gdImageCreate in -lgd... yes
checking for gdImageGif in -lgd... yes
checking for gdImagePng in -lgd... yes
checking for gdImageJpeg in -lgd... yes
checking for gdImageWBMP in -lgd... yes
checking for gdImageStringFT in -lgd... yes
checking for gdImageOpenPolygon in -lgd... yes
checking for gdImageGifPtr in -lgd... yes
checking for gdFontGetTiny in -lgd... yes
using GD ( -DUSE_GD_GIF -DUSE_GD_PNG -DUSE_GD_JPEG -DUSE_GD_WBMP -DUSE_GD_FT -DGD_HAS_FTEX_XSHOW -DGD_HAS_GDIMAGEGIFPTR -DGD_HAS_GETBITMAPFONTS) from system libs.
configure: checking whether we should include PDF support...
checking for PDF_setlinewidth in -lpdf... no
checking for PDF_setrgbcolor in -lpdf... no
checking for PDF_moveto in -lpdf... no
checking for PDF_curveto in -lpdf... no
checking for PDF_show_xy in -lpdf... no
checking for PDF_load_font in -lpdf... no
libpdf not found or too old... PDF support not included.
checking if AGG support requested... looking for agg libs in /usr
using libagg from /usr
using libaggfontfreetype from /usr
configure: checking whether we should include EPPL7 support...
including EPPL7 support.
configure: checking whether we should include PROJ.4 support...
checking for pj_init in -lproj... yes
checking for pj_transform in -lproj... yes
using PROJ.4 from system libs.
configure: checking whether we should include thread safe support...
checking for pthread_create in -lpthread... yes
using -lpthread from system libs.
configure: checking whether we should include ESRI SDE support...
ESRI SDE support not requested.
checking if GEOS support requested... yes
checking for geos-config... /usr/bin/geos-config
checking for GEOS version &gt;= 2.2.2... yes. Found version 2.2.3
configure: checking whether we should include OGR support...
checking for gdal-config... /usr/bin/gdal-config
OGR enabled (-DUSE_OGR).
checking if GDAL support requested... yes
checking checking ms JPEG output... no we, have GDAL available.
configure: checking whether we should include TIFF support...
checking for TIFFOpen in -ltiff... yes
using libtiff from system libs.
checking if PostGIS support requested... yes
checking for pg_config... /usr/bin/pg_config
yes, user supplied pg_config
checking if MyGIS support requested... no
checking if OracleSpatial support requested... no
checking if MING/Flash support requested... no
configure: checking whether we should include WMS Server support...
OGC WMS compatibility enabled (-DUSE_WMS_SVR).
configure: checking whether we should include WFS Server support...
OGC WFS Server support enabled (-DUSE_WFS_SVR).
configure: checking whether we should include WCS Server support...
OGC WCS Server support enabled (-DUSE_WCS_SVR).
configure: checking whether we should include WMS Client Connections support...
configure: checking whether we should include WFS Client Connections support...
configure: checking whether we should include OGC SOS Server support...
configure: checking for curl-config...
checking for curl-config... /usr/bin/curl-config
found libcurl version 7.18.0
OGC WMS Client Connections enabled (-DUSE_WMS_LYR).
OGC WFS Client Connections enabled (-DUSE_WFS_LYR).
configure: checking for xml2-config...
configure: checking whether FastCGI support should be enabled...
FastCGI support not enabled.
configure: checking HTTPD server (Apache) version...
using user-supplied httpd (/usr/sbin/apache2)
/usr/sbin/apache2 version is Apache/2.2.8 (2002008).
Your system is apparently running Apache/2.2.8. Setting stderr to non-blocking for msDebug() due to Apache 2.x bug (see MapServer bug 458 or Apache bug 22030).
Compiling with fast MS_NINT
checking compiler warnings... basic
configure: checking whether we should enable debug features...
Enabling debug features: -g in CFLAGS.
configure: checking for PHP/MapScript module options...
PHP/MapScript module not configured.
checking for g++ -shared ... yes
checking if --enable-runpath requested... no
checking if --with-java-include-os-name specified... no, autodetected linux
configure: creating ./config.status
config.status: creating Makefile
config.status: creating mapscript/java/Makefile
config.status: creating mapscript/csharp/Makefile
MapServer is now configured for
-------------- Compiler Info -------------
C compiler: gcc -g -O2 -fPIC -Wall
C++ compiler: g++ -g -O2 -fPIC -Wall
Debug: -g -DNEED_NONBLOCKING_STDERR
Generic NINT:
Threading support: -DUSE_THREAD
-------------- Renderer Settings ---------
zlib support: -DUSE_ZLIB
png support:
jpeg support:
iconv support: -DUSE_ICONV
AGG support: -DUSE_AGG
AGG Freetype support: -laggfontfreetype
Ming(flash) support:
PDFLib support:
-------------- Data Format Drivers -------
native tiff support: -DUSE_TIFF
PostGIS support: -DUSE_POSTGIS
Proj.4 support: -DUSE_PROJ
EPPL7 support: -DUSE_EPPL
ArcSDE support:
OGR support: -DUSE_OGR
GDAL support: -DUSE_GDAL
GEOS support: -DUSE_GEOS
Oracle Spatial support:
FastCGI support:
-------------- OGC Services --------------
WMS Server: -DUSE_WMS_SVR
WMS Client: -DUSE_WMS_LYR
WFS Server: -DUSE_WFS_SVR
WFS Client: -DUSE_WMS_LYR
WCS Server: -DUSE_WCS_SVR
SOS Server:
-------------- MapScript -----------------
PHP MapScript: no

I was able to weed out the problem above by checking on the libraries noted with “yes” above if they are installed or not. To overcome this, I needed to install the ff debian packages on ‘lenny(testing)’:

libagg-dev libgd2-xpm libgd2-xpm-dev libtiff4 libtiff4-dev libpng libpng-dev libxpm4 libxpm4-dev libfreetype6-dev libapache2-mod-python python-imaging

Results:

debsexy:/usr/lib/cgi-bin# ./mapserv -v
MapServer version 5.0.2 OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG OUTPUT=WBMP OUTPUT=SVG SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE SUPPORTS=WMS_SERVER SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER SUPPORTS=WFS_CLIENT SUPPORTS=WCS_SERVER SUPPORTS=THREADS SUPPORTS=GEOS INPUT=TIFF INPUT=EPPL7 INPUT=POSTGIS INPUT=OGR INPUT=GDAL INPUT=SHAPEFILE

Installing PostGres8.3 and Postgis1.3.2 on Windows

rupert — Tue, 04 Mar 2008 00:15:05 +0000

It seems very nice for the new PostGres8.3 installer to bundle up third party libraries installation with the new admin pack feature. Using this feature, you can install PostGis, nPgsql, etc. The installation was painless and smooth this time. I have to backup all my data though using pg_dump.

To lessen the learning curve on installing PostGres + PostGis together, I made a short flash movie here. There is no audio in the tutorial. The idea was to introduce PostGres to my Chinese staff. I have to remove PostGreSQL 8.2 first before installing 8.3.