mobile and gis dev notes

As noted from my previous blog post regarding Mapserver on Debian, you don’t get AGG with Mapserver when installing directly from Debian packages. Thus, it would be better to install Mapserver by source. However, debian still helps because it would install all the necessary libraries needed for compiling mapserver. Read more…

I was bitten 4 hours searching for this in the mailing list. I have a postgis layer defined in mapserver map file as follows:

LAYER
        NAME "pois"
        STATUS DEFAULT
		GROUP "pois"
        TYPE POINT
 
		CONNECTIONTYPE postgis
		PROCESSING "CLOSE_CONNECTION=DEFER"
		CONNECTION "user=lbs password=xtlme15n dbname=beijing_stat host=192.168.1.211 port=5432"
		#DATA "the_geom from (SELECT poi_id, the_geom FROM poi WHERE new_block_id = 7) as foo USING UNIQUE poi_id USING SRID=4326"
		DATA "the_geom from poi as foo USING UNIQUE poi_id USING SRID=4326"
 
		FILTER "new_block_id=%myid%"
 
		CLASS
			NAME "block-pois"
    		        STYLE
				SYMBOL "circle"
     			        COLOR 255 0 0
				SIZE 10
   			END
  		END
END

Traditionally, I could append and change the FILTER attribute by passing it to the Mapserver CGI as follows:
map.pois.filter=new_block_id%3D700.

Apparently, this changed with Mapserver 5.0. Please see MapServer 4.10 to 5.0 Migration Guide. Thanks to this mailing list thread, it turned out that we need to pass a value to a custom variable set in the mapfile for security reasons. Hope this one, goes in to the docs. I was hoping to comment out in the Mapserver Documentation but registration is holding me off with a ‘Connection Refused’.

Grabbing the latest ms4w-2.2.6, I was now able to get debugging output by specifying below in my mapfile.


CONFIG MS_ERRORFILE "stderr"


This would log all requests to your Apache error log. I can now play with different debugging modes as described from RFC28. Here is a simple output of my error.log with a DEBUG 2 set..


[error] [client 127.0.0.1] CGI Request 1 on process 4136
[error] [client 127.0.0.1] msDrawMap(): Layer 0 (district), 0.172s
[error] [client 127.0.0.1] msDrawMap(): Layer 1 (water_200k), 0.187s
[error] [client 127.0.0.1] msDrawMap(): Layer 2 (greens_200k), 0.454s
[error] [client 127.0.0.1] msDrawMap(): Layer 6 (roads_150_01), 0.906s
[error] [client 127.0.0.1] msDrawMap(): Layer 9 (roads_270_01), 0.515s
[error] [client 127.0.0.1] msDrawMap(): Layer 10 (roads_180_01), 0.563s
[error] [client 127.0.0.1] msDrawMap(): Layer 11 (roads_280_01), 0.531s
[error] [client 127.0.0.1] msDrawMap(): Layer 12 (roads_400_01), 0.516s
[error] [client 127.0.0.1] msDrawMap(): Layer 13 (roads_140_03), 0.547s
[error] [client 127.0.0.1] msDrawMap(): Layer 14 (roads_140_04), 1.297s
[error] [client 127.0.0.1] msDrawMap(): Layer 15 (roads_141_02), 0.703s
[error] [client 127.0.0.1] msDrawMap(): Layer 16 (roads_150_02), 0.890s
[error] [client 127.0.0.1] msDrawMap(): Layer 17 (roads_boundary_160_170_270), 0.953s
[error] [client 127.0.0.1] msDrawMap(): Layer 18 (roads_160_02), 0.829s
[error] [client 127.0.0.1] msDrawMap(): Layer 19 (roads_170_02), 0.640s
[error] [client 127.0.0.1] msDrawMap(): Layer 20 (roads_270_02), 0.531s
[error] [client 127.0.0.1] msDrawMap(): Layer 22 (roads_180_02), 0.563s
[error] [client 127.0.0.1] msDrawMap(): Layer 23 (roads_280_02), 0.516s
[error] [client 127.0.0.1] msDrawMap(): Layer 24 (roads_400_02), 0.531s
[error] [client 127.0.0.1] msDrawMap(): Layer 26 (subway), 0.109s
[error] [client 127.0.0.1] msDrawMap(): Layer 27 (subwaystops), 0.125s
[error] [client 127.0.0.1] msDrawMap(): Layer 28 (subway_transfer_stops), 0.110s
[error] [client 127.0.0.1] msDrawMap(): Layer 29 (district_boundary), 0.156s
[error] [client 127.0.0.1] msDrawMap(): Layer 33 (400_280_180_170_160_150_labels_01), 4.859s
[error] [client 127.0.0.1] msDrawMap(): Layer 35 (subwaystops_labels), 0.125s
[error] [client 127.0.0.1] msDrawMap(): Layer 36 (district_labels), 0.125s
[error] [client 127.0.0.1] msDrawMap(): Drawing Label Cache, 5.860s
[error] [client 127.0.0.1] msDrawMap() total time: 23.329s


Summary
One of the upcoming power features of Mapserver 5.0 is its AGG Rendering engine. Above is an example image of the AGG rendering capabilities of Mapserver. Below is a list of my experiences with Mapserer 5.0 AGG

• Slower than GD ofcourse, but cartographic quality is really impressive…
• You need to have SUPPORT=AGG to be able to use AGG. So far, I was not able to compile mapserver source with AGG, however using mapserver-beta-3 for linux and ms4w betas worked fine.
• Layer Transparency. I was not able to get transparency for my roads overlaying on top of my basemap. Tbonfort from IRC suggested I use RGBA for the outputformat, but it still breaks. Hope Im not missing anything important here…

The Mapfile
Currently, the mapfile ( map_all_agg.map ) contains 2,587 lines! The lengthy mapfile consists of the ff: districts or political boundaries, water bodies, greens or open spaces, roads, subway lines, subway stops. The roads is classified into 12 categories ( Expressway, Class-1, Class-2, Class-3, Class-4, Major Road, Main, Secondary, Minor, CommunityRoad(w/name), CommunityRoad, ParkRoads). Class1-4 is a classification for Roads outside the 5th Ring Road of Beijing. While Major Roads – Minor Roads is a classification inside the 5th Ring. The roads were drawn as a line layer and not as a polygon.

How to achieve the overlapping of road intersections?
The technique is to seperate the road boundary layer from the actual road line. A mixture of widths and color management was employed to achieve the effect. Essentially, we started with the ff:

1. road boundary layer
2. road layer

Further , trial and error resulted in refining the road boundary layer with different minscales and maxscales. The same was adapted with the road layer. Download map_all_agg.txt. My next iteration, would be to refactor some of the road layers if possible.

A couple of notes to myself, we could speed things up by trying out the ff:

• Using SHPTREE
• Simplifying the geometry as you change scales

Testing Methodology
1. Use Firefox and install firebug.
2. Download YSlow from Yahoo.
3. View load times using “net” tab

Using SHPTREE
1. The utility specifies indexes as it slices the shapefile into quadrants as specified from its definition. To run shptree, do the ff:

$ shptree roads.shp
creating index of new LSB format


2. It would generate a roads.qix. No changes need to be made with the mapfile. Mapserver loads the shapefile without the extension.

3. Testing… Significant speed improvements were made as shown below:

• Before: 29.44 secs
• After: 5.91 secs

Here’s a quick overview of how I upgraded my custom MS4W binded with ColdFusion..

1. Extract the new ms4w_new.zip to E:\ms4w_new

2. diff E:\ms4w_new\Apache\conf\httpd.conf E:\ms4w\Apache\conf\httpd.conf

If the changes is minimal, then proceed.

3. Uninstall ColdFusion connector from Apache.

D:\CFusionMX\bin\connectors\Remove_ALL_connectors.bat


4. Uninstall Apache from services.msc. Run

E:\ms4w\apache-uninstall.bat


5. Move

mv E:\ms4w E:\ms4w_old
mv E:\ms4w_new E:\ms4w


6. Install Apache as a windows services to services.msc

E:\ms4w\apache-install.bat


7. Bind ColdFusion again to Apache

D:\CFusionMX\bin\connectors\Apache_connector.bat