Manifold GIS set to leverage GPU's much more widely in v9 ?

If you are interested in the path the next version of Manifold likely is taking, the following screencast of Dimitri's recent presentation at the Nvidia GPU Conference gives us some hints! Altough Dimitri  doesn't in this presentation go into any great specific detail about Manifold v9 CUDA capabilities (expected as that the presentation was not covered by an Non Disclosure Agreement), nonetheless, this presentation seems to be the closest one can get to a technical presentation by Manifold at a User Meeting, minus the NDA!

First, Dimitri goes into a lot of detail about the fundamental software development challenges for GPU programming, much of which is over my head in terms of technical detail. Sadly, all the examples in this presentation refer to raster processing, which is already present in Manifold v8.

The second half of the talk is clearly more interesting, as he presents fundamental architectural work developing a lightweight processing scheduler. This scheduler seems to be the key infrastructure element enabling efficient workload paralellisation, enabling Manifold to optimally leverage a heterogeneous environment of multiple CPU's and GPU's. One thing that seems to be clear from this presentation is that Manifold over the past two years have been very busy rewriting large parts of their core code to enable the paralellisation of almost all GIS tasks inside Manifold. Particularly interesting is the mention at the end that they are in a position to take advantage of any GPGPU platform (Nvidia and AMD), which implies the adoption of OpenCL by Manifold for their next release.

PostGIS Developments Presentation

If you didn't get a chance to fly down to Sydney to attend the FOSSG4 conference (I certainly didn't!), they now have video casts up of most presentations at http://blip.tv/search?q=fosslc. One I found interesting was a great video cast of the presentation by Paul Ramsey on development progress on PostGIS at the recent FOSSG4.

A great mention as well for Manifold as one of the GIS packages supporting PostGIS in the video by Paul Ramsey, although he alleges to some FUD that Manifold the company spread regarding PostGIS ( at 13:15 in the video, although I don't remember what particular Dimitri postings he was referring too :-)


According to Paul's presentation, PostGreSQL (with PostGIS) certainly seems to be at least equal to other spatially enabled databases in terms of feature completeness, performance and robustness, and its free, making procurement much easier (19:00 min in video). And the new release of PostGIS seems to be all about speed improvements, which is obviously a good thing. A overview of the roadmap ahead for PostGIS promises a lot of things to come...

Now, my interest in PostGIS has only started lately, mostly because I was asked to develop a "Introduction to Spatial Databases" e-learning course. Given the need for students to be able to run the practicals on their home computers, we chose PostGIS and Quantum GIS as the software tools for the tutorials. I must say I have been impressed by the functionality of PostGIS, and it certainly hasn't been a very steep learning curve for me, thanks mostly to my previous Spatial SQL knowledge learned from Manifold's Spatial SQL experience. Although the syntax and some of the commands are slightly different between PostGIS SQL and Manifold Spatial SQL, knowledge of either is really helpful when trying to write queries! Wider issues of optimisation such as spatial indexes though present another level of complexity/power in PostGIS,  which Manifold's internal SQL engine doesn't expose to the user.

"Introduction to GIS & Cartography" Course Dates announced

Just a quick note to say we have finalised dates for the next session of our "Introduction to GIS and Cartography" course using Manifold GIS in February (18th and 19th) 2010 here at UCL.  Please find below the detailed invitation:

The invitation is also available in PDF format with a detailed agenda

The Department of Civil, Environmental and Geomatic Engineering, University College London, will be hosting an Introduction to Geographical Information Systems and Cartography Course on the 18^th and 19^th of February 2010. This course is aimed at novice or potential GIS Users interested in key concepts of geographical data capture, storage and analysis. After course completion participants will be able to generate, manipulate and analyse geographic information confidently and create high-quality cartographic outputs.

The course is organised into modules containing comprehensive overviews of fundamental topics relating to Geographical Information Systems, databases and cartography, alongside hands-on tutorials teaching participants the most important functionalities of GIS.

The course will introduce users to and use Open Street Map (OSM) data and Manifold GIS software. Participants will be tutored by leading GIS lecturers and researchers with extensive GIS expertise in a commercial and academic context.

Participants will receive a comprehensive training manual containing all of the course content such as presentation slides, tutorial worksheets, project files and datasets used. This training manual will act as a valuable reference guide after the course is completed.

Each participant can expect:

• Experienced academic tutors


• A workstation preloaded with all software and data for the tutorials


• State-of-the-art air-conditioned computer room


• Comprehensive course documentation folder


• Course Certificate from UCL on completion


• Lunch and refreshments provided

The course fee is £650 (incl. VAT) per participant. Please note that we have arranged a discount for organisations sending two or more participants. The course will be held on UCL’s main campus in Bloomsbury, Central London.

For booking and any further enquiries, please email Patrick Weber at p.weber@ucl.ac.uk or you can phone +44 (0)20 7679 2745 .

Apologies for the lack of posts!

To alll the people visiting and wondering why I havent posted over the summer, I just want to apologise. I know this blogging thing needs some love and new content, but I have been just too busy trying to finish my Engineering Doctorate work lately.  I wont make any promises here about when exactly I'll be finishing, but leave you with the excellent PhD Comics by Jorge Cham.

Just a hint: I am approaching on the motivation chart the point entitled: "Advisor runs out of funding" ....

User Interface Mockup for GIS

As part of my research work, I am developing a spatial decision support system for inward investment promotion. The design and user interface for the system needs careful consideration and development, given that the end product will be used primarly by non experts.

Having decided on a basic methodology for the decision making process, we are now in the process of developing UI mockups, which we can present to stakeholders to get their opinion of the system. UI mockups allow an early view on the look & feel , i.e. usability of the system, and allow the designer to easily collect feedback from users and integrate changes without having to modify/rewrite code.  For my project, I also hope that I can give stakeholders a better understanding of the system development process and of the project's main aims through this showcase.

I have been looking at different methodologies for producing such UI mockups, and how they have been applied to GIS & SDSS type projects. I must say that I have only found sparse evidence of systematic efforts by GIS practitioners to apply usability principles through the use of mockups  in GIS development activities. My view also is corroborated by Muki in his recent blog post, where he argues that usability and the processes in software development that ensure good usability still seem to be considered a nice to have, and the GIS industry lacks a "usability culture", inherent other industry sectors.

But I digress...  As always, there are multiple methods of doing UI mockups. One of the most straightforward and widely used methods uses paper, pen and scissors to physically create the UI elements, which can then be arranged into an UI mockup. This of course is very flexible as you can easily create and assemble together whatever user interface elements one needs or wants.A digital equivalent is to use PowerPoint or any other general purpose drawing tool to generate the UI mockup. But there are also drawbacks. Physical UI mockups can get lost/destroyed, are not easy to revise, and changes can't easily be tracked. Digital mockups made with PowerPoint are also cumbersome to generate, and one needs to invest a lot of time into generating basic UI elements (altough some templates to get you started are available). With PowerPoint mockups, you can even generate basic interactivity inside the mockup by creating links between slides, altough this is still cumbersome.

Obviously, a specific quick and dirty design tool for mockups would be great, and thats exactly what I found with Balsamiq Mockups. They have both a free online tool version as well as a desktop application version available, which supports features such as dynamic links between screens. Also, Balsamiq Mockups intentionally uses hand-drawn UI elements to generate "paper quality mockups", so that people don't get attached to “that pretty color gradient” or think that your mockup has actual code behind it and is “practically done”.

I just spent the afternoon creating such a mockup for my project, and I am more than happy with the results, even though they only offer a basic "map" element which is specific to GIS. But the rest of the template of pre defined UI elements is rich enough to model most GIS related UI.

I would urge any GIS developer who hasn't used mockups for their application development process before to give it a try and enjoy the benefits of better usability for their applications and happier users!

Five low hanging fruit: quick usability fixes Manifold should consider

Over the past four years that I have been using Manifold on a daily basis for almost all of my professional work, I have been able to develop a good understanding of its strengths, idiosyncrasies  and weaknesses.

Even so, it is when working with other Manifold users, and specifically teaching Manifold to new users, that one can often find basic usability issues at hand that go unnoticed by more seasoned Manifold users like myself. In this post, I do not want to address larger and more complicated issues, but instead focus on providing a small list of low hanging fruit that could easily be fixed by the Manifold development team:

1) Setting Area of Interest in Layouts

During our training course, almost all new users needed help to understand how to set the area of interestof a layout. The way to change the AOI of a map component in a layout might be powerful, but is complex and well hidden from the user in multiple steps. Most users will upon the creation of a layout from a map, expect the zoom extent of the layout to be the same as what they see in the map component. Confusion sets in when the whole component, or worse a blank page, greets them when they open their newly created layout. "Hey where's my nice map gone?"

The process then is less than straighforward and involves in total 8 steps and at least as many mouseclicks : right-click on the layout (1) and choose properties (2), set the Scope to Locked Rectangle (3), go back to the map component (4), goto View -> Panes -> Layers (5) (as that pane is hidden by default!), using the Layers Pane, enable the Layout Component extent rectangle (6), resize to the AOI the extent rectangle (7), finally go back to the Layout (8).

Ideally, users would expect after selecting a layout map component that it would behave like a regular map component where users can pan and zoom to set a AOI for the layout.

2) Formatting toolbar

Most new users will stumble over one of the more fundamental idiosyncrasies the Manifold user interface has to offer. The concept of the formatting toolbar relies on the fact that, as opposed to other GIS packages, one drawing can contain different geometry types (points, lines, areas). This is reflected in the formatting toolbar, which always offers the user formatting options for all three geometry types. Because the formatting options look very similar to one another, new users are often deeply confused and unshure which icon they need to click to influence for example the line foreground color.



One straightforward solution would be to blank out formatting options for geometry types not currently present in a specific drawing, thus simplifying the toolbar considerably (for a lines only drawing, this would reduce the number of icons from 17 icons down to 4!).

Another possible solution would be to indicate through a small text box what geometry type a set of formatting icons refers to, instead of relying on users to find the right option by hovering over an icon or trial and error.

3) Label Display Options

When working with labels, users are often at a loss on where to change advanced options such as callouts, label overlap conflicts and line label placement. Users in general will look first in the component specific menu, i.e. the Labels Menu, where they wont find a entry for such advanced display options. Instead, Label display options are "hidden from the user" in the generic View Menu. The same applies to for the display options of surfaces and images.

IMHO it would make much more sense to group all options relating to a specific component type in the specific Menu for that component, i.e. the "Labels" Menu or "Surfaces" Menu.

4) Queries in Layouts

A table dragged into a layout results in the creation of a layout component displaying table records. This makes sense to the user and he is pleased to be able to include tabular data into his layout to generate reports.

Unfortunately, the same doesn't happen for queries dragged into a layout. From Manifold's perspective, the software sticks to its principles and generates a layout component containing the SQL Query text. 99% of the time, this isn't the most useful behaviour, and a user would rather expect the query results table to be generated and placed into the layout.

It would be much better if users were given a choice if they wanted the query text, or the query results to be included in the layout. (The workaround I currently employ in this situation is to run a query which generates a table which in turn is included in a layout, which is fine except you can't update the table easily without deleting it and rerunning the query, thus loosing the layout position).

5) The Query Toolbar

Finally, this one was already touched upon by Muki on his blog,  but it is worth repeating his comment here, as the change is the most straightforward to implement on this list:

The way the query toolbar works is that you select a field in the left drop-down list, an operator at the central drop-down and a value in the text box on the right and click on select to see the result. For example, if you enter 5 in the toolbar in the picture, it will lead to a selection of the 5 polygons on the map with the smallest area.
The confusing part of the interface is the ‘not’ between the left drop-down and the central one. For a new user, the interface reads ‘find objects on the map where the field Area (I) are not the bottom X’. The ‘not’ in this case is a toggle button that can be activated to negate the operation that was selected in the central drop-down. Clearly, it would be better if, when not activated, it had the word ‘is’ (Area is the bottom 5) and ‘not’ appeared only when it was active. This is one of the cases where usability enhancement could be carried out in less than a minute of a programmer’s time – and surely makes life less confusing to many novice users…

As you might have guessed, these are only some of the usability issues one can find in Manifold, and I would appreciate any comments here if you can think of others. I have already send in suggestions to Manifold for most of these issues, and I would urge you to do the same if you can agree with my analysis.

History of GIS - the Canada GIS

Part 2 of documentation ; Part 3 of documentation

I just wanted to draw attention to this great set of videos that give a great overview of one of the first large scale Geographical Information Systems, in this case the Canada Geographic Information System (CGIS). Dr Roger Tomlinson was the initiator, planner and director of this for the time very ambitious project, which meant that he has gone down in history as the "father" of GIS.  I feel quite proud to note that he wrote his PhD thesis here at University College London!

From the Wikipedia article:

The Canada Geographic Information System (CGIS) was developed in the 1960s and 1970s to assist in regulatory procedures of land-use management and resource monitoring. At that time, Canada was beginning to realize problems associated with its seemingly endless boundaries, in combination with natural resource availability. The government therefore decided to launch a national program to assist in management and inventory of its resources. The simple automated computer processes designed to store and process large amounts of data enabled Canada to begin a national land-use management program and become a foremost promoter of geographic information systems (GIS).

OpenStreetMap Tiles now available for Manifold!

Just saw this post on the forum. After a long time where the idea of a OpenStreetMap ISI Driver for Manifold was thrown around on the forum, James K. finally stepped up to the plate!
James did a stellar job, not only generating a ISI Dll for the default OSM Mapnik Tile Layer, but also adding the Tiles@Home and Cloudmade Cycle Map. The Dll works fine in both x32 and x64, and finally allows map makers to quickly and easily add a high quality OSM background map to their products.

This is a significant step for map makers, as the OSM tiles offer a level of freedom in terms of usage license. Altough a acknowledgement of the OSM license is needed in any mapping products which contain any OSM data, this is a simple requirement when compared with the grey and murky licensing requirements for Google, Microsoft Virtual Earth and Yahoo tiles.

It remains to be seen with the imminent update to 9.0, if Manifold.net are going to support OSM XML as a Import/Export format. Altough there are existing scripts to enable the import of OSM XML data, these scripts have issues and don't scale efficiently to bulk downloads. A native solution hopefully would enable bulk imports and the ability to save (or even upload directly to OSM) to OSM XML for update of OSM.

PS: AFAIK planet.osm, representing a database dump in XML form of the whole OSM dataset, is one of the largest publicily available vector datasets out there (over 364 million elements). Import and display of such a huge dataset could be a great benchmark test for upcoming versions of Manifold!

Manifold & R for spatial statistics: an unlikely couple!

There have been numerous discussions in the past in the Manifold User Forums regarding a lack of (exploratory) spatial analysis/statistics tools, such as measures and visualisations, in the Manifold GIS. Altough it seems likely that a number of users have been sending in detailed suggestions for spatial analysis functionality, so far it seems that Manifolds development focus has been devoted to other more fundamental areas such as interfacing with spatial databases and the ability to efficiently use multithreading and CUDA.

Given that in the short to medium term, we most probably won't see the integration of significant spatial analysis functionality into Manifold, a pragmatic approach is the integration of external software packages with Manifold. There are a number of software packages that offer spatial analysis and statistics capabilities such as for example Crimestat, Geoda and the R project. R, an open source project benefits from a wide support in academia as a platform for the implementation of statistical computing, and thus provides a very rich environment for the analysis of spatial data through a combination of free packages. R is a command-line environment, and although the syntax is relatively accessible, it does present a significant learning curve for any beginners.

Recently, my research project led me to investigate the spatial distribution of foreign investors into London. I needed to do a density analysis of historic investment patterns to identify likely agglomeration or dispersion processes between investors. Although Manifold doesn't offer any relevant density estimation algorithms, R, and specifically the spatstat package, allows for the creation of Kernel Density Estimation (for the estimation of density) grids.

I took the opportunity to write a script that gives users a point and click front-end to both the kernel smoothed intensity function from a point pattern (KDE), and spatial smoothing (interpolation) of numeric values observed at a set of irregular locations (GKS) from inside Manifold.  The script takes care of building and maintaining the interface between Manifold and R, running the analysis in the background and creating a result surface component. I must acknowledge here the help and inspiration of numerous users on the forum which have been working with R.

[caption id="attachment_44" align="aligncenter" width="527" caption="Screenshot example of interface and output"][/caption]



Although R is completely hidden from the user of the script once installed, the successful installation relies on a basic understanding of the concepts of R and the installation of a few prerequisite software tools and R packages. Along with R , you need to have installed R(D)Com A all in one package for R and R(D)Com can be found at statconn. You also need a C:\temp\ directory (temporary files are stored there). You also need to have the following R packages installed:

• spatstat, the main analysis package


• maptools, helper package for the conversion of data to a spatstat compatible format


• rgdal, package allowing the import and export of data from R to Manifold

I strongly advise anyone wanting to use this script to first read and understand the algorithms and outputs involved by consulting the relevant help pages from the spatstat package.  I also include two datasets that are suitable for experimentation with the script, one each for KDE and GKS.

Finally, there are some caveats to this script. I do not make any guarantees as to the output of the script, and I have to repeat that you need to have an understanding of what the algorithms do to fully comprehend the analysis.  Also, the script at the moment doesn't take account of projections at all. I have personally only tested the script with projected point patterns (British National Grid). In the case of BNG, assigning the BNG projection to the created Image (Preserve Local Values ticked) should be sufficient. Your mileage may vary when using other projections.

You can find the download of the Manifold .map file with the script here!

As you may have guessed, this is only a first stab at the integration of Manifold with R, and it is still an unlikely couple which sometimes can have communication difficulties. Clearly is a lot of work left in integrating other most basic functionality, a few examples of functionalities being other interpolators such as IDW, Geographically Weighted Regression, LISA and Moran's I ... .

But this proof of concept shows the potential for added functionality to boost Manifold's power from a pure GIS to an exploratory spatial statistics toolset.

Manifold 9: A world record and release date

Just saw this press release on the Manifold website:

Carson City, NV USA — 16 March 2009 — Manifold.net today announced a new world record for the number of processors used in a personal computer for Geographic Information Systems (GIS) processing. At the company's 2009 European User Meeting in London, Manifold demonstrated an upcoming new software product that simultaneously utilized over 1440 processor cores to perform a remote sensing image computation at supercomputer speed with over 3.5 teraflops of performance. Manifold demonstrated the new software on a desktop 64-bit Windows PC equipped with three NVIDIA GTX 295 GPU cards costing less than $500 each. (Illustration at right shows the demonstration hardware.)

Wow, I didnt know that we witnessed a world record at the User Meeting back in February here at UCL! Well I am glad that Manifold came and did their demo of Release 9.0, even though they did so in their usual hyberbole style.

Also, good to see that they are showing commitment to a release date around June. This should imply a beta starting in the next weeks...!!!

Heatmaps for Mashups ... too easy?

HeatMapApi.com is a new service which allows Google Maps mashups to integrate heat map representations easily. Heat maps,  or more generally point to raster interpolations allow the graphical representation of point patterns through the use of continuous colors identifying areas of higher or lower density of points. Areas where this has been employed are crime hotspots analysis or economic activity analysis.

A novel concept in Web 2.0 mashups, I was interested in finding out what the methodology was behind the generation of these rasters. Sadly, I couldn't find a definitive answer on the algorithm that the website uses to generate its hotspot maps. They do expose in their API two variables that can influence the generation of the heatmaps, decay and boost, but without information on the algorithm behind it, the setting of these values remains a pure exercise in trial and error, and seeing what "looks" best. Also, because the parameters are set as "optional", most developers will be tempted into a one size fits all approach, smoothing out interesting patterns in the data, or creating hotspots that are not statistically viable, creating masses of effectively meaningless maps.

Mashup developers thus will more than ever the spatial analysis literacy skills to understand the processes, models and algorithms that lie behind the pretty maps.

Note: This is not a new problem, but has been present all through the development of Exploratory Spatial Data Analysis over the past 20-30 years in academia and commercial settings, and a lot can be learned from this past experience.