Kartta Labs and re.city was a really cool Google Research project that I was involved in a few years ago. It enabled users to view, interact and contribute to historical streetscapes with a timeline. Working from historical maps, geo-rectifying them, tracing and digitizing buildings, adding historical photographs of buildings and with a catalogue of three dimensional models it produced a fascinating and engaging time enabled view into our past. The project was much bigger than I write about here so feel free to explore the videos at the end and linked blog posts and papers produced.
Three dimensional city streetscapes were created with a timeslider control. The buildings were modelled using photographs of the frontages of the buildings, and the footprints from historical maps.
I work with museums, galleries and archives on a number of historical crowdsourced mapping projects. Notably the New York Public Library’s Map Warper platform. Today this lives on in https://mapwarper.net and on Wikimedia’s Map Warper https://warper.wmflabs.org. I’m also familiar with the OpenStreetMap stack of applications, vector and raster tiles, cartography and a bit of server infrastructure – all these came into play during this project.
The components and applications of the project all ran on Google Cloud Project and with Kubernetes via the Google Kubernetes Engine (GKE). It used Google single sign on so users need only log into one place and authenticate for the different applications.
The front page showed an interactive map with pan and zoom control, a time slider at the top, links to the other applications, and help and about pages etc. The main applications are the Warper, Editor and Reservoir. This front page is where users can explore the map, select buildings to see or add historic photos of buildings and see the building footprints, and enter the interactive 3D view.
The maps layer was a custom style – to look vintage – using the colours inspired by 19th century street maps and with a subtle paper texture background.
Noter – application to allow a user to connect a building footprint and a face of a building based on geometry of a historic photo of a building
Reservoir – a repository of 3D models of buildings.
Warper was enhanced
Look for some of these improvements coming to the main mapwarper application soon. If there’s some you think are better than others let me know.
Quick Place
A new interface was added to help users quickly and easily place map images geographically. Users would see the collection of maps and could choose one to “quick place”.
There’s a simple scale and drag rotate the image on a map. You can either hit one “looks good” button, skip to do another or manually find a map to work on.
OCR for Geocoding
Using image processing + Google Cloud Vision for Optical Character Recognition, the map images are processed to extract any road names and any place names. Together with any map metadata, this can be help automatically locate the map, and could potentially be used for editing the features. This also helps users to rectify the maps, giving a good starting location.
I imagine that today’s more advanced AI image reading tools might give much better results.
Versioning
Maps had versions, enabling maps to be rolled back to other versions after a change in adding control points.
Social improvements
The users who worked on the map were stored and linked to so that a user could see who helped on the map, or see a call to action if no users had started on it.
Vector Tiles
Within the warper, vector tiles were added for rectification, and with a time slider, so users could see and rectify historic maps while looking at this map.
Other updates
The warper was updated to work via Docker and with Kubernetes. The Rails version was updated and OpenLayers was updated. As before, the vector tiles and time slider was integrated. (I note: One thing users of mapwarper.net suggest is that control points could be snapped to vector features). The Trace tab was changed so that you could load the rectified map into the Editor’s ID editor as a background image with the time tags so the user can quickly get started tracing over the historical map directly into the database. Mapwarper’s code was also updated to run with a later version of OpenLayers and the whole system with Docker.
Editor website
The editor website was a fork of the OpenStreetMap website and stack with the Id Editor and was Dockerized to work with Kubernetes and a GCP Cloud SQL Postgres server. This serves as the API where geometries of the buildings are kept. One edit for the website was to pass through some query parameters to the iD editor, so for example to load a rectified historic map or default historic dates to the editor. The id editor was customised to have a better interface for start and end times for features.
Vector Tiles
Tegola vector tile server was used and also run in Docker and in Kubernetes. The data was exported from the API as a planet and replication files created, and added into another GCP Cloud SQL Postgres server for vector tiles. Postgres was also uses to make the replication and planet service. The vector features had time properties within them so the maps are able to filter it.
AI / Machine Learning / Neural Networks
The output of the crowdsourcing mapping process fed into a workflow that heavily used AI to help create the 3D rendered cityscapes. We also considered using AI to assist with the georectification process.
Starting with footprints on maps and façade regions in historical images (both are annotated by crowdsourcing or detected by automatic algorithms), the footprint of one input building is extruded upwards to generate its coarse 3D structure. The height of this extrusion is set to the number of floors from the corresponding metadata in the maps database.
In parallel, instead of directly inferring the detailed 3D structures of each façade as one entity, the 3D reconstruction pipeline recognizes all individual constituent components (e.g., windows, entries, stairs, etc.) and reconstructs their 3D structures separately based on their categories. Then these detailed 3D structures are merged with the coarse one for the final 3D mesh. The results are stored in a 3D repository and ready for 3D rendering.
The key technology powering this feature is a number of state-of-art deep learning models:
Faster region-based convolutional neural networks (RCNN) were trained using the façade component annotations for each target semantic class (e.g., windows, entries, stairs, etc), which are used to localize bounding-box level instances in historical images.
DeepLab, a semantic segmentation model, was trained to provide pixel-level labels for each semantic class.
A specifically designed neural network was trained to enforce high-level regularities within the same semantic class. This ensured that windows generated on a façade were equally spaced and consistent in shape with each other. This also facilitated consistency across different semantic classes such as stairs to ensure they are placed at reasonable positions and have consistent dimensions relative to the associated entry ways.
Videos
The project is over now but a bunch of videos covering all the components can be found on the YouTube channel: https://www.youtube.com/@re-city8789
Want to do the same kind of project today? I’d suggest to start using OpenHistoricalMap. I was involved in development and hosting of the OpenHistoricalMap project from the beginning and evangelize for the project at a number of conferences and whenever I can!
In some ways this project and OHM’s redesign happened at the same time, but for different purposes. re.city’s data is Open Data License (ODBL) which enables the use of OpenStreetMap’s data which is useful for existing city streets and buildings. Users can then work back from today’s map into the past. This project was focused on cities. OpenHistoricalMap data is CC0 / Public Domain which potentially limits it’s ability to use OSM data but which which focuses on much wider historical timeframes and lots of existing historical data is public domain.
The walk started from the theatre cafe and across the road was the Rock Garden. This rock garden was full of specimens of rocks around the region and theres a Rock Map in the middle.
From there it was on the search for rock sticks, but I first passed a hippy shop selling rocks, minerals and crystals.
Next stop was the Anglican Cathedral where the stoneyard worked rock and made them into gargoyles. I chatted with some american tourists about them.
Next stop was the first sweet shop for Sticks of Rock. I chose a “carnival” type the multicoloured one. For one pound. I then opened it and begun sucking it and licking it as I went.
I visited the cathedral shop and was told off for eating the rock. I left quickly.
Next I wandered the streets, holding the rock in my hand like a tourist might hold a “magic” wand from a harry potter themed shop.
At one point the stick suddenly broke and snapped off. And part of it crashed to the road and splintered. Tourists kicked bits of it down to the crossroads. At the exact point where it snapped was another sweet shop which sold sticks of rock. I took this as a sign but as what I dont know.
I think if there was many in the group it would be like a baton race. at this point someone else would buy one and take over. It could be good and performative if the group were all licking sticks of rock. I didn’t see anyone eating rock, but there were a few with ice creams. Walking around the shops just licking the rock stick (which lasted a long time and I didnt finish) is a kind of odd thing to do.
The “York Rock” had york lettering and a picture of the cathedral on it.
FOSS4G:UK was on last week, here in Leeds and I really enjoyed the two days. Here are some highlights and thoughts.
I think the highlight talk for me was Weiming Huang “Pretraining Geospatial Foundation Models with OpenStreetMap”. Weiming is a new geographer in Leeds University School of Geography. The talk was about AI model and OSM data. Foundation models are not generative models (like chatGPT for example) and they are not task specific models. They are representative model, self supervised. The models are not focused on the use but on the data. They can then be used for multiple tasks. CityFM City Foundation Model.
For each feature in the model (e.g. from OSM) we can look at it from 3 perspectives, visual, textual and locational. For each road we can see that it contains nearby Points of interest, roads and routes. We can associate features with other features. e.g. hospital vs cafe. The models have to be trained by area. I asked Weiming if his models could run on a browser. Potentially. He also used LORA to fine tune a llama to determine OSM Points of Interest. E.g. what are the names and location of cafes in this city. Something to look into. The models generally cant be used to compare between different areas. I wondered if they could be used to express the folksonomy, the way things are mapped and categorised – essentially making a taginfo of sorts?
Yuchi Lai did an interesting talk looking at bike trips in London. The tfl publish data about bike hire but only time and start and end destination. She used bike routing APIs to make routes between the two points. h3 polygons was used. Could be used to determine where best to put in new cycle paths. Cycle routes in area with trips.
Asli Doga Kanturk talked about geotagged and sentiment analysis of social media posts. Foursquare reviews were looked at for analysis of leisure hubs. It reminded me of the work with Geoiq on the twitter firehose a decade ago.
There were two historic talks, Unlocking Historic Planning Data which was about extracting plans from static pdfs, using computer vision. Georeferencing was a hard problem for them. There were many questions related to data quality and AI hallucinations. And Barry Rowlingson did a talk on “how I accidentally become an expert in an obscure WW2 coordinate system” with meditations on his life and how cities change over time.
Corridor chat pointed me to Lonboard: https://developmentseed.org/lonboard/latest/ to look at. Corridor chat – look at taxi driver brain scan navigation. Oxford circus tube station navigation studies by UCL. I also enjoyed talks on remote sensing (hyperspectral) and postgis tips and tricks (views and event triggers) amongst many others
Overall the conference was very interesting and inspiring. I remembered and was remembered by several attendees from past events! The conference was energizing. Many people mentioned psychogeography – and my gut tells me there may be some interesting AI applications here enough to propose a session for next time.
Geobase, a serverless geospatial platform has a vector tile server, database functions, PostGIS, H3 and the MobilityDB extensions (amongst much more!) built in, and this quickstart blueprint uses all of them. On the front end we are using deck.gl and maplibre and typescript and react.
The application shows animated trails representing taxi trips in Porto, Portugal:
In the above animation you can see pulsing blue circles. These are where 2 taxis have passed close by next to each other within 10 metres. The point is actually the shortest distance between the two trips, where the taxis got the closest. Often it’s passing by on the street, and starting off by taxi ranks or dropping off in the same locations.
MobilityDB was mostly used, to represent the taxi trips as lines-with-time, in the vector tileserver function to serve these temporally aware lines, in the outlier detection and cleaning steps and in the close pass analysis steps.
The outlier detection step identifies and removes parts of the trip with impossible speeds. In the following image, the blue and red lines are the taxi trips, but the blue ones have many very straight and very long segments. These will have very high speeds. The red lines is the cleaned dataset
I’m impressed with MobilityDB, in the next image, the green polygon is a h3 hexagon. The purple line is one taxi trip, the light blue is the trip clipped to the hexagon, and the dark blue is the trip within the hexagon and clipped to a particular time frame.
Moreover, because Geobase is also PostGIS in the cloud, we can connect to it in QGIS, and here are the query layers in QGIS
The line is clipped to the timespan:
{[POINT(-965739.7745940761 5036826.935711992)@2013-07-01 09:27:07+00, POINT(-966081.4141113205 5036898.801939623)@2013-07-01 09:27:22+00, POINT(-966272.7055242998 5036942.099465625)@2013-07-01 09:27:30+00]}
The processing is mostly performed with PostGIS. For example, to identify trips with outlier periods:
whenTrue(speed(temporal_geom) #> 55) AS high_speed_periods
And then to remove these “high_speed_periods” from the trips:
WHEN high_speed_periods IS NOT NULL THEN
minusTime(temporal_geom, spanset(span(high_speed_periods)))
ELSE
temporal_geom
I hope you enjoy the video, I used OBS for the screen recording which gave me more control over it with the audio. My voice can sometimes be quiet so I hope the video is engaging for you.
I helped make a video about how to get started with Geobase https://geobase.app/ with the ship movement blueprint quickstart. It’s using AIS (ship tracking) data, kind of like GPS for ships but with lots of extra info.
Geobase has automatically creates and serves vector tiles from any table or database function. With MobilityDB (and many other extensions) installed and enabled, we use a function to create vector tiles of the ship movements trajectories and because we have the timestamps we can have a timeline control too. On the front end it uses deck.gl to animate the trails.
The ship movements is a MobilityDB type, basically a special linestring of points-with-timestamp. And we can do things like work out the speed of the ship and compare to the reported speed in knots and see any difference. Simplification functions can be done on the data taking into consideration time and movement. So we could simplify based on the distance travelled, or the time taken, or just good old classic douglas peucker!
There’s also a function to query the ship data to create h3 hexagons of activity by drawing on the map. I hope it gives a quick and easy introduction into how Geobase can be used.
Geobase.app itself is very cool. There’s no free tier at the moment however – it’s just getting started. In a nutshell it’s Supabase (cloud based postgres) + PostGIS + Tile Server.
I’ll be needing better enunciation or a throat sweet for the future, but in the meantime you will have to listen to my softly spoken voice!
thinkwhere