Archive

Archive for June, 2007

Installing QNavigator in QGIS

June 27th, 2007 rupert Comments off
[root@rupert-linux qnavigator]# mkdir build && cd build
[root@rupert-linux build]# cmake -D QGIS_PREFIX=/usr/local/qgis ..
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Check size of void*
-- Check size of void* - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Looking for Q_WS_X11
-- Looking for Q_WS_X11 - found
-- Looking for Q_WS_MAC
-- Looking for Q_WS_MAC - not found.
-- Looking for Q_WS_WIN
-- Looking for Q_WS_WIN - not found.
-- Found Qt-Version 4.2.3
-- Configuring done
-- Generating done
-- Build files have been written to: /home/install/qgis-svn/src/plugins/qnavigator/build
[root@rupert-linux build]# make
Scanning dependencies of target dglib
[  9%] Building CXX object dglib/CMakeFiles/dglib.dir/dgraph.o
[ 18%] Building CXX object dglib/CMakeFiles/dglib.dir/dijkstra.o
[ 27%] Building CXX object dglib/CMakeFiles/dglib.dir/fheap.o
[ 36%] Building CXX object dglib/CMakeFiles/dglib.dir/routing_core.o
Linking CXX static library libdglib.a
Creating makefiles...
[ 36%] Built target dglib
Scanning dependencies of target dgbuild
[ 45%] Building CXX object dglib/CMakeFiles/dgbuild.dir/dgbuild.o
Linking CXX executable dgbuild
[ 45%] Built target dgbuild
Scanning dependencies of target dgpath
[ 54%] Building CXX object dglib/CMakeFiles/dgpath.dir/dgpath.o
Linking CXX executable dgpath
[ 54%] Built target dgpath
[ 54%] Generating ../../python/ui/mainwindow_ui.py
[ 54%] Generating ../../python/ui/findstreetdialog_ui.py
[ 54%] Generating ../../python/ui/aboutdialog_ui.py
[ 54%] Generating ../../python/ui/roadinfodialog_ui.py
[ 54%] Generating ../../python/ui/qnavigator_rc.py
[100%] Built target ui
[root@rupert-linux build]#
Categories: GIS Tags: ,

Installing Trac on CentOS4

June 27th, 2007 rupert 1 comment

There is a problem with clearsilver on CentOS systems. I get neo_cgi.so module not found. Don’t force yourself, it would simply not work (Trac0.10.4 on CentOS, not unless we try the DAG repositories).

So, I have to use Trac 0.11, setuptools and Genshi.

Prerequisites:
- Must have python 2.3+
- Must have mod_python-3.3.1.tgz
- Must have httpd2.0.x+ or httpd2.2.x

A. Core Trac Installation

1. Install setuptools-0.6c6-py2.3.egg. Download the setuptools according to your python distribution and simply run:

sh setuptools-0.6c4-py2.3.egg

2. Install Genshi. There are numerous ways of installing Genshi.

easy_install Genshi

or


#wget http://ftp.edgewall.com/pub/genshi/Genshi-0.4.2-py2.3.egg
#chmod +x Genshi-0.4.2-py2.3.egg
#easy_install Genshi-0.4.2-py2.3.egg

3. Install Trac 0.11. We need to install trac from svn
#svn co http://svn.edgewall.org/repos/trac/trunk trac
#python ./setup.py install

This would install trac-admin and tracd in /usr/bin.

Trac Post Intall

#mkdir -p /var/www/trac
#cd /var/www/trac
#trac-admin /var/www/trac/poimgr initenv
#chown -Rf nobody:root /var/www
#chmod -Rf 755 /var/www

Just accept the defaults for now, we can change it later on /var/www/trac/poimgr/conf/trac.ini.

C. Apache Configuration

LoadModule python_module /usr/local/apache2/modules/mod_python.so
 
<location>
SetHandler mod_python
PythonHandler trac.web.modpython_frontend
PythonOption TracEnv /var/www/trac/poimgr
PythonOption TracUriRoot /trac/poimgr
#Order deny,allow
#Deny from all
#Allow from 192.168.1.0/24
AuthType Basic
AuthName "TRAC-POIMGR"
AuthUserFile "/repos/svn-auth-file"
require valid-user
</location>

Notes:
http://www.yolinux.com/TUTORIALS/LinuxSubversionAndTracServer.html

Depending on your installation, you may encounter some problems. I had problems with expat errors:


ExpatError (null): line 1,column 0.

Turns out that it was confused between the link libraries of apache and that of python.

#ps aux | grep http | head -3
#sof -p 29982 | grep expat
# strings libexpat.so.0.1.0 | grep expat_
expat_1.95.2
# strings /usr/lib/python2.3/lib-dynload/pyexpat.so | grep expat_
expat_1.95.7
 
Quickfix is to use symbolic links:
#cd /usr/local/apache2/lib/
#cp -Rf /usr/lib/libexpat.so.0.5.0 /usr/local/apache2/lib/
#ln -s libexpat.so.0.5.0 libexpat.so
#ln -s libexpat.so.0.5.0 libexpat.so.0
#su -l
#/etc/init.d/httpd restart

Configuring Trac.
1. I have to move /etc/svn-auth-file to /repos-auth-file. Change the permissions so “nobody” could access it.

2. How to configure my users? Well, since its an intranet application and my trac-users and developers is both in the svn-auth-file, I need a way to configure users easily. Follow these steps from trac-hack.

easy_install http://trac-hacks.org/svn/accountmanagerplugin/trunk

Role base authorisation in trac

trac-admin /var/www/trac/poiclient permission remove anonymous \
TICKET_CREATE TICKET_MODIFY WIKI_CREATE WIKI_MODIFY
 
for n in rupert andrew; do
trac-admin /var/www/trac/{app_name} permission add $n TRAC_ADMIN
done
 
for n in rupert andrew; do
trac-admin /var/www/trac/{app_name} permission add $n WIKI_DELETE
done
 
trac-admin /var/www/trac/poiclient permission add authenticated \
BROWSER_VIEW CHANGESET_VIEW FILE_VIEW LOG_VIEW MILESTONE_VIEW \
REPORT_SQL_VIEW REPORT_VIEW ROADMAP_VIEW SEARCH_VIEW \
TICKET_CREATE TICKET_MODIFY TICKET_VIEW TIMELINE_VIEW \
WIKI_CREATE WIKI_MODIFY WIKI_VIEW

3. Here’s my trac.ini. Things to look at is the notification and account-manager.

4. Deleting a Trac Ticket:

# sqlite3 trac.db
delete from ticket_custom where ticket = [ticketID];
delete from ticket_change where ticket = [ticketID];
delete from ticket where id = [ticketID];

Other References:
http://www.dscpl.com.au/wiki/ModPython/Articles/ExpatCausingApacheCrash
http://www.yolinux.com/TUTORIALS/LinuxSubversionAndTracServer.html
Trac and Multiple Subversion Repositories

Future Work:
http://insurrection.tigris.org/

Categories: linux, subversion Tags: , , , ,

HTTP access_log analysis

June 21st, 2007 rupert 2 comments

1. In order to report execution times, include a “%T” in your httpd.conf as follows:

    #
    # The following directives define some format nicknames for use with
    # a CustomLog directive (see below).
    #
    LogFormat "%h %l %u %t \"%r\" %&gt;s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %&gt;s %b %T" common
 
    <ifmodule>
      # You need to enable mod_logio.c to use %I and %O
      LogFormat "%h %l %u %t \"%r\" %&gt;s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
    </ifmodule>
 
    #
    # The location and format of the access logfile (Common Logfile Format).
    # If you do not define any access logfiles within a <virtualhost>
    # container, they will be logged here.  Contrariwise, if you *do*
    # define per-</virtualhost><virtualhost> access logfiles, transactions will be
    # logged therein and *not* in this file.
    #
    CustomLog logs/access.log common
</virtualhost>

2. There are many log analysis softwares (webalizer, awstats, etc) out there, but most of them are outdated. I found this nifty little tool called visitor from www.hping.org. Freely distributed for unix but demo versions for windows build. Here’s a demo.

Categories: linux Tags:

Mapinfo Utility for gdal_translate.

June 21st, 2007 rupert Comments off

I made a small perl utility to automate the gcp’s from Mapinfo Raster TABS to gdal_translate command line. Currently your tabfile would have:

!table
!version 300
!charset WindowsLatin1
 
Definition Table
  File "beijing_6th_1.jpg"
  Type "RASTER"
  (116.522865,40.016316) (347,184) Label "Pt 1",
  (116.681215,40.015286) (7729,243) Label "Pt 2",
  (116.679777,39.777904) (7666,14674) Label "Pt 3",
  (116.523827,39.779108) (397,14606) Label "Pt 4"
  CoordSys Earth Projection 1, 104
  Units "degree"
  RasterStyle 4 1
  RasterStyle 7 1677695

How to use
gdal_mapinfo

1. ls *.TAB > init.sh

2. vi init.sh to reflect the ff:

perl gdal_mapinfo.pl Beijing_6th_1.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_10.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_11.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_12.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_13.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_2.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_3.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_4.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_5.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_6.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_7.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_8.TAB &gt;&gt; final.bat
perl gdal_mapinfo.pl Beijing_6th_9.TAB &gt;&gt; final.bat

3. The resulting final.bat should have the ff:
gdal_translate -gcp 347 184 116.522865 40.016316 -gcp 7729 243 116.681215 40.015286 -gcp 7666 14674 116.679777 39.777904 -gcp 397 14606 116.523827 39.779108 -of GTiff Beijing_6th_1.jpg I:\\satimages\translated\Beijing_6th_1_translated.tif

gdalwarp -s_srs epsg:4326 -t_srs epsg:4326 I:\\satimages\translated\Beijing_6th_1_translated.tif
I:\\satimages\warped\Beijing_6th_1.tif

Categories: GDAL/OGR Tags: , , , ,

Processing Mapinfo Raster JPEG Images using GDAL

June 14th, 2007 rupert Comments off

I have a couple of sat images (raw jpegs) from Google that I want to use with Openlayers/Mapserver. The raw jpegs were registered using Mapinfo via GCP (Ground Control Points).

Mapinfo Raster JPEG Images example:

rupert@rupert-winxp /e/home/map/beijing/new/satimages$ ll
-rw-r–r– 1 rupert None 358 Jan 30 03:23 2NE1.TAB
-rw-r–r– 1 rupert None 7.1M Jan 30 02:38 2NE1.jpg
-rw-r–r– 1 rupert None 356 Feb 1 18:56 2NE2a.TAB
-rw-r–r– 1 rupert None 3.8M Feb 1 08:57 2NE2a.jpg

You cannot fully reference 2NE1.TAB as a Mapserver Layer. I tried to use 2NE1.jpg, but the problem its not georeferenced.

rupert@rupert-winxp /e/home/map/beijing/new/satimages
$ gdalinfo 2NE1.jpg
Driver: JPEG/JPEG JFIF
Size is 8650, 6744
Coordinate System is `'
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0, 6744.0)
Upper Right ( 8650.0,    0.0)
Lower Right ( 8650.0, 6744.0)
Center      ( 4325.0, 3372.0)
Band 1 Block=8650x1 Type=Byte, ColorInterp=Red
Band 2 Block=8650x1 Type=Byte, ColorInterp=Green
Band 3 Block=8650x1 Type=Byte, ColorInterp=Blue

The georeference coordinates of 2NE1.jpg, just like an ESRI World File, is found in 2NE1.TAB…

rupert@rupert-winxp /e/home/map/beijing/new/satimages
$ cat 2NE1.TAB
!table
!version 300
!charset WindowsLatin1
 
Definition Table
  File "2ne1.jpg"
  Type "RASTER"
  (116.38575,39.906105) (349,6619) Label "Pt 1",
  (116.390072,39.93201) (1160,317) Label "Pt 2",
  (116.42786,39.932296) (8210,253) Label "Pt 3",
  (116.4295878,39.90722318) (8522,6358) Label "Pt 4"
  CoordSys Earth Projection 1, 0
  Units "degree"

I found hurting myself in trying to create an ESRI world file from the current MAPINFO Raster TABS. So, I decided to go for GeoTIFF since its native in Mapserver. Using GDAL utilities my only problem is how to put a coordinate system and reference to the raster.

On windows, you could use Frank’s FWTools. For Linux, compile GDAL by source with python would be extremely helpful later on. For installation of GDAL on Linux, we can use Mapserver’s Verbose Installation in Linux Guide.

GDAL – the saviour!.

GDAL utilities is extremely helpful in reprojection, scaling, image mosaics, etc. For now, we will use gdal_translate and gdal_warp. Please RTFM the utilities.

1. Using gdal_translate to specify the gcp’s registered in Mapinfo.

gdal_translate -gcp pixel line easting northing
Add the indicated ground control point to the output dataset. This option may be provided multiple times to provide a set of GCPs.



$ gdal_translate -gcp 349 6619 116.38575 39.906105 -gcp 1160 317 116.390072 39.93201 -gcp 8210 253 116.42786 39.932296 -gcp 8522 6358 116.4295878 39.90722318 -of GTiff 2NE1.jpg 2NE1translated.tif

Input file size is 8650, 6744
0...10...20...30...40...50...60...70...80...90...100 - done.

Note: even if we specify the gcp’s, gdal_translate would not specify the corner coordinates of the tiff.

rupert@rupert-winxp /e/home/map/beijing/new/satimages
$ gdalinfo 2NE1translated.tif
Driver: GTiff/GeoTIFF
Size is 8650, 6744
Coordinate System is `'
GCP Projection =
GCP[  0]: Id=1, Info=
          (349,6619) -&gt; (116.38575,39.906105,0)
GCP[  1]: Id=2, Info=
          (1160,317) -&gt; (116.390072,39.93201,0)
GCP[  2]: Id=3, Info=
          (8210,253) -&gt; (116.42786,39.932296,0)
GCP[  3]: Id=4, Info=
          (8522,6358) -&gt; (116.4295878,39.90722318,0)
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0, 6744.0)
Upper Right ( 8650.0,    0.0)
Lower Right ( 8650.0, 6744.0)
Center      ( 4325.0, 3372.0)
Band 1 Block=8650x1 Type=Byte, ColorInterp=Red
Band 2 Block=8650x1 Type=Byte, ColorInterp=Green
Band 3 Block=8650x1 Type=Byte, ColorInterp=Blue

2. Use gdalwarp to reproject using the gcp and specify the coordinates.

The gdalwarp utility is an image mosaicing, reprojection and warping utility. The program can reproject to any supported projection, and can also apply GCPs stored with the image if the image is “raw” with control information.


$ gdalwarp -s_srs epsg:4326 -t_srs epsg:4326 2NE1translated.tif warped.tif
Creating output file that is 9422P x 5631L.
Processing input file 2NE1translated.tif.
:0...10...20...30...40...5050...60...70...80...90...

Let’s check after gdalwarp using gdalinfo…

$ gdalinfo warped.tif
Driver: GTiff/GeoTIFF
Size is 9422, 5631
Coordinate System is:
GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.2572235630016,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0],
    UNIT["degree",0.0174532925199433],
    AUTHORITY["EPSG","4326"]]
Origin = (116.383841930499160,39.933342296341991)
Pixel Size = (0.000004927579869,-0.000004927579869)
Metadata:
  AREA_OR_POINT=Area
Corner Coordinates:
Upper Left  ( 116.3838419,  39.9333423) (116d23'1.83"E, 39d56'0.03"N)
Lower Left  ( 116.3838419,  39.9055951) (116d23'1.83"E, 39d54'20.14"N)
Upper Right ( 116.4302696,  39.9333423) (116d25'48.97"E, 39d56'0.03"N)
Lower Right ( 116.4302696,  39.9055951) (116d25'48.97"E, 39d54'20.14"N)
Center      ( 116.4070558,  39.9194687) (116d24'25.40"E, 39d55'10.09"N)
Band 1 Block=9422x1 Type=Byte, ColorInterp=Red
Band 2 Block=9422x1 Type=Byte, ColorInterp=Green
Band 3 Block=9422x1 Type=Byte, ColorInterp=Blue

Sweet. Now, all we need to do is display the raster images in Mapserver/OpenLayers.

Specifying a raster image in Mapserver

LAYER
NAME “2NE1″
DATA “satimages/2NE1.tif”
TYPE RASTER
STATUS DEFAULT
END

LAYER
NAME “2NE2″
DATA “satimages/2NE2a.tif”
TYPE RASTER
STATUS DEFAULT
END

Here is the end result…
.

Lessons learned, I tried to specify coordinate extents using gdal_translate -a_ullr ulx uly lrx lry. Specifying the coordinates was subjective by just looking at the cursor location of the registered image in Mapinfo. It is still best to use the GCP’s. Simply put, we need to be accurate in specifying corner coordinates in raster images to project them accurately.

.