mobile and gis dev notes

I had a problem when using ogr2ogr and converting a postgres table to a road table. My postgres table containa a utf-8 road name which is all in chinese. The mapinfo road table created by ogr2ogr seems to contain the correct geometry and other fields that is in utf-8. However, all my chinese characters is all messed up. So, I have to export the file and open it to mapinfo.

1. In Postgres, to export to a file..

cybersoftbjv1=# set client_encoding = gbk;
SET
cybersoftbjv1=# \o road.txt;
cybersoftbjv1=# select rd_id, cn_name from roads where cn_name <> '';
cybersoftbjv1=# \q


2. Open the file in vim, and do a “%s/ //g”. This would replace all ” ” to “”.
Note: This is reasonable for chinese since chinese dont have spaces. However english prases and sentences differ.

3. Open the file in mapinfo and replace the other columns using Table -> Update.

If anybody has any other way to specify the client encoding in ogr2ogr that would be perfect…

Grabbing the latest ms4w-2.2.6, I was now able to get debugging output by specifying below in my mapfile.


CONFIG MS_ERRORFILE "stderr"


This would log all requests to your Apache error log. I can now play with different debugging modes as described from RFC28. Here is a simple output of my error.log with a DEBUG 2 set..


[error] [client 127.0.0.1] CGI Request 1 on process 4136
[error] [client 127.0.0.1] msDrawMap(): Layer 0 (district), 0.172s
[error] [client 127.0.0.1] msDrawMap(): Layer 1 (water_200k), 0.187s
[error] [client 127.0.0.1] msDrawMap(): Layer 2 (greens_200k), 0.454s
[error] [client 127.0.0.1] msDrawMap(): Layer 6 (roads_150_01), 0.906s
[error] [client 127.0.0.1] msDrawMap(): Layer 9 (roads_270_01), 0.515s
[error] [client 127.0.0.1] msDrawMap(): Layer 10 (roads_180_01), 0.563s
[error] [client 127.0.0.1] msDrawMap(): Layer 11 (roads_280_01), 0.531s
[error] [client 127.0.0.1] msDrawMap(): Layer 12 (roads_400_01), 0.516s
[error] [client 127.0.0.1] msDrawMap(): Layer 13 (roads_140_03), 0.547s
[error] [client 127.0.0.1] msDrawMap(): Layer 14 (roads_140_04), 1.297s
[error] [client 127.0.0.1] msDrawMap(): Layer 15 (roads_141_02), 0.703s
[error] [client 127.0.0.1] msDrawMap(): Layer 16 (roads_150_02), 0.890s
[error] [client 127.0.0.1] msDrawMap(): Layer 17 (roads_boundary_160_170_270), 0.953s
[error] [client 127.0.0.1] msDrawMap(): Layer 18 (roads_160_02), 0.829s
[error] [client 127.0.0.1] msDrawMap(): Layer 19 (roads_170_02), 0.640s
[error] [client 127.0.0.1] msDrawMap(): Layer 20 (roads_270_02), 0.531s
[error] [client 127.0.0.1] msDrawMap(): Layer 22 (roads_180_02), 0.563s
[error] [client 127.0.0.1] msDrawMap(): Layer 23 (roads_280_02), 0.516s
[error] [client 127.0.0.1] msDrawMap(): Layer 24 (roads_400_02), 0.531s
[error] [client 127.0.0.1] msDrawMap(): Layer 26 (subway), 0.109s
[error] [client 127.0.0.1] msDrawMap(): Layer 27 (subwaystops), 0.125s
[error] [client 127.0.0.1] msDrawMap(): Layer 28 (subway_transfer_stops), 0.110s
[error] [client 127.0.0.1] msDrawMap(): Layer 29 (district_boundary), 0.156s
[error] [client 127.0.0.1] msDrawMap(): Layer 33 (400_280_180_170_160_150_labels_01), 4.859s
[error] [client 127.0.0.1] msDrawMap(): Layer 35 (subwaystops_labels), 0.125s
[error] [client 127.0.0.1] msDrawMap(): Layer 36 (district_labels), 0.125s
[error] [client 127.0.0.1] msDrawMap(): Drawing Label Cache, 5.860s
[error] [client 127.0.0.1] msDrawMap() total time: 23.329s


Im trying out ways to create a topology of edges and vertices for a road network. Currently, there are three ways I know of:

1. Using ArcGIS build coverage line.
– This includes the use of ArcGIS. Exporting the feature into tics, arcs and nodes, then afterwards assembling them all together. Its functional but have not fully tested the quality of the road topology. Also, assembling them back together through the spatial objects comparison will take some time. It would be better *If* ArcGIS could create the nodes wrt to the base table.

2. Using PostGres, postlbs functions.
– Use of SELECT ASSIGN_VERTEX_ID(‘table_name’, double_precision_distance. So far this bails on me on my first test on my win-xp laptop. We can test this on Linux if the response is the same.

3. Using Mapinfo Basic Scripts provided by J.
– Haven’t gone indepth with these for now, but looking forward to it.

4. Using Grass
– As documented in pgrouting.postlbs.org site’s Topology Creation

Summary
One of the upcoming power features of Mapserver 5.0 is its AGG Rendering engine. Above is an example image of the AGG rendering capabilities of Mapserver. Below is a list of my experiences with Mapserer 5.0 AGG

• Slower than GD ofcourse, but cartographic quality is really impressive…
• You need to have SUPPORT=AGG to be able to use AGG. So far, I was not able to compile mapserver source with AGG, however using mapserver-beta-3 for linux and ms4w betas worked fine.
• Layer Transparency. I was not able to get transparency for my roads overlaying on top of my basemap. Tbonfort from IRC suggested I use RGBA for the outputformat, but it still breaks. Hope Im not missing anything important here…

The Mapfile
Currently, the mapfile ( map_all_agg.map ) contains 2,587 lines! The lengthy mapfile consists of the ff: districts or political boundaries, water bodies, greens or open spaces, roads, subway lines, subway stops. The roads is classified into 12 categories ( Expressway, Class-1, Class-2, Class-3, Class-4, Major Road, Main, Secondary, Minor, CommunityRoad(w/name), CommunityRoad, ParkRoads). Class1-4 is a classification for Roads outside the 5th Ring Road of Beijing. While Major Roads – Minor Roads is a classification inside the 5th Ring. The roads were drawn as a line layer and not as a polygon.

How to achieve the overlapping of road intersections?
The technique is to seperate the road boundary layer from the actual road line. A mixture of widths and color management was employed to achieve the effect. Essentially, we started with the ff:

1. road boundary layer
2. road layer

Further , trial and error resulted in refining the road boundary layer with different minscales and maxscales. The same was adapted with the road layer. Download map_all_agg.txt. My next iteration, would be to refactor some of the road layers if possible.

A couple of notes to myself, we could speed things up by trying out the ff:

• Using SHPTREE
• Simplifying the geometry as you change scales

Testing Methodology
1. Use Firefox and install firebug.
2. Download YSlow from Yahoo.
3. View load times using “net” tab

Using SHPTREE
1. The utility specifies indexes as it slices the shapefile into quadrants as specified from its definition. To run shptree, do the ff:

$ shptree roads.shp
creating index of new LSB format


2. It would generate a roads.qix. No changes need to be made with the mapfile. Mapserver loads the shapefile without the extension.

3. Testing… Significant speed improvements were made as shown below:

• Before: 29.44 secs
• After: 5.91 secs

I had the liberty of upgrading to ColdFusion 8. Works smoothly…

• Install took some time.
• Memory peaks instantly at 250M
• Binding to Apache2.2 was seamless. No need to use the updated wsconfig.jar
• New ColdFusion Administrator. Debugging looks promising…