The Ghost Map – A Tale of Victorian GIS Technology

The Story of London’s Most Terrifying Epidemic–And How It Changed Science, Cities, and the Modern World

I just got a copy of Steven Johnson’s The Ghost Map which came out in paper last week. The reason I’m really excited about this book is that it is amongst others about history of analytical cartography and solving urban sustainability problems as seen by Victorian London in the mid 19th century. In fact, as Johnson mentioned, there was an open question that time about whether such huge cities like London (over 2.5 million population in1850) will be still sustainable or whether it will collapse just as the ancient Rom did at a certain point.

The Story takes place in August 1854 when a cholera outbreak was raging in London. Dr. John Snow, a private physician and kind of Sherlok Holmes-like cross-disciplinary thinker, was able to link the epidemic to contaminated drinking water source from public pumps. He was drawing a map with elements like fatalities, water pumps and streets to advance an argument about the nature of the disease.

Yes, it’s a remarkable example of how people started to recognize the importance of spatial dimensions for better understanding complex problems in an urbanizing world. But it’s also a remarkable historical example of how map-making pretends to be objective, where there is a clear author’s intention behind. In this particular case, Snow’s intention was to reinforce his theory on the cause of the disease. In fact, his map was rather a graphic description or summary of his studies than an analytical tool.

The Ghost in the Map

Nevertheless, there is a memorable GIS story coming out of the book: the way of taking advantage of local knowledge and (amateur) cross-disciplinary thinking on a local level. It shows the importance to empower communities through GIS technology and participation.

For instance the 2005 Hurricane Katrina GISCorps disaster response is a good example for the importance of participatory GIS and community mapping. Teams of volunteers organized by the GISCorps responded very quickly and produced maps for the initial responders and briefing maps showing things like power outages or road closures. They also helped rescue teams through geocoding (translating) addresses into GPS coordinates. The nub of the matter: all these efforts could only be accomplished by the community itself (in close collaboration with private industry).

PS: Seems that the guys from Riverhead Books produced a little trailer cartoon, which by the word of Steven Johnson can be described as “Yellow Submarine meets 28 Days Later” :)

Intergeo – 2 days to go

Intergeo is now just around the corner, bags packed and ready to go to Leipzig this week. Unfortunately, this year the event clashes with FOSS4G in Victoria, Canada, so i’ll miss out on some GIS fellows.

In the run-up to the Intergeo, I was glad to read a good in-depth article on Geoweb from the german weekly newspaper Die Zeit.

After last years success, OSGeo will organize another Open Source Park as part of the exhibition.

I don’t like to park my car in East Berlin

This map (from brennende-autos.de) shows recent arson attacks on cars in the city of Berlin. So, if you’re about to visit Berlin – please first check where it is safe to leave your car (especially the luxurious ones).

Deprecation within Database Schemas

Java developers already know about deprecating API’s – the ability to mark specific classes or methods as “deprecated” thus informing API users that they should stop referencing those parts, because they might be removed in future releases. Introducing such a transition period allows code refactoring being less painful (mainly for the public API parts) and preserve backward compatibility.

Along with the rise of new approaches to more agile database techniques, came the idea of using similar practices also in DB development. That means for example, if you want to remove a column then you don’t drop it immediately but mark the column as deprecated. After a transition period of some weeks or months, the column will be dropped. This strategy is of particular interest to database refactoring, because usually database schemas are highly coupled to a wide variety of things. And unlike Java API’s where we can distinguish between protected and public API parts, items within a database schema are potentially always public.

What to deprecate?

The following database artifacts are items that might be deprecated:

• Tables
• Views
• Single Columns
• Stored Procedures, Functions, Sequences

When to deprecate database items?

• Item will be dropped in future
• Is inefficient or insecure to use

How to deprecate?

A deprecation mechanism should fulfill the following requirements

• Deprecation markers should be attached to their items in a manner that they are preserved even if the items will be moved (import/export etc)
• Markers should refer to an explanation why the item has been deprecated and suggest what to use instead
• Throw warning messages when deprecated items are used

Techniques to deprecate database items

In case of Java, the deprecation is carried out through @deprecation comments within the source code. The Java Compiler throws warning messages when it finds references to deprecated items. Obviously within a database we cannot issue compile warnings. The only way to inform about calls to deprecated items is at runtime. That means every time a deprecated table, column, view or so is accessed, a warning message should be written to a log table. But how can we know when those items are accessed?

Oracle databases are offering the following options:

1) Triggers: It’s the standard way to attach DML monitoring to database tables. But triggers cannot monitor on SELECT’s and they are not working on Stored Procedures, Sequences and things of this nature.

2) Database Auditing: With Oracle’s Regular Auditing one can monitor also the use of Views, Functions and Sequences plus one can even log the SQL statement that causes the audit event. But to monitor table access on column-level, Fine-Grained Auditing (FGA) is needed. FGA is only available with Oracle Enterprise Edition.

Issuing deprecation warnings

Another problem is about the deprecation warning messages content itself. The logged messages should help developers and admins to figure out from where deprecated items are accessed. That means at least the program name and SQL statement should be logged. Unfortunately Oracle doesn’t offer something like a stack trace (as Java).

Conclusion

Wouldn’t it be nice if databases offered a standard mechanism to deprecate all the variety of items within a database? I think that will be a huge help towards achieving a more agile way of database design.

Using Ant properties within SQL scripts with Incanto’s <sqlplus> Task

In a previous article i wrote about a solution for passing values from the database via select statements to Ant properties inside Ant build files.

Now I’ve extended the Incanto <sqlplus> Task to do the other way round and pass Ant property sets into the SQL scripts as defined variables.

Within SQL*Plus it is possible to work with defined variables in the form of:

define myusername = SCOTT
define usertable = ALL_USERS
select * from &usertable where username='&myusername';

In case of running SQL scripts from Ant with Incanto one can pass single properties into SQL this way:

<property name="db.username" value="SCOTT" />
<ora:sqlplus silent="true" logon="${db.userid}" failonerror="yes">
 <![CDATA[
    define myusername = '${db.username}'
    select * from all_users where username='&myusername';
 ]]>
</ora:sqlplus>

Ant will replace all ${} style constructions with the given value of the property. But this works only for text content (SQL scripts) inside the tag (CDATA section) not with SQL script files that are called using the start attribute. You have to call the SQL file from within the CDATA section to workaround this drawback:

<property name="db.username" value="SCOTT" />
<ora:sqlplus silent="true" logon="${db.userid}" failonerror="yes">
 <![CDATA[
    define myusername = '${db.username}'
    @mysqlfile.sql
 ]]>
</ora:sqlplus>

I have extended Incanto’s <sqlplus> task to simplify this. It is now possible to include a <propertyset> element within the <sqlplus> task.

<property name="db.username" value="SCOTT" />
<ora:sqlplus silent="true" logon="${db.userid}" failonerror="yes">
  <propertyset>
    <propertyref prefix="db"/>
  </propertyset>
  <![CDATA[       
    select * from all_users where username='&db_username';
  ]]>
</ora:sqlplus>

All properties included by the <propertyset> element now can be used inside the SQL script. The <sqlplus> Task is passing these as defined variables into SQL*Plus. But note: since SQL*Plus has some rigid naming conventions for variable names, the <sqlplus> Task converts all dot (.) characters to underlines. Thus in the prior example I reference the property db.username by db_username.

Equinox as a Service

… “Bug 176928″

No doubt about it, Equinox has high potentials for server-side use. Therefore the question arises: what would be a convenient way to run Equinox applications as a service on Windows platforms (just like Tomcat does)?

On the one hand, we’ve got the Tanuki service wrapper as a weapon of choice to run java apps as a Windows Service or Unix Daemon. And on the other hand?