Featured

Fuel shortages, Google Maps and adapting your tools

Archaeologists have always used tools invented for other purposes. Our ‘signature’ tool, the trowel, was designed for brick-laying; our hoes, mattocks, shovels and wheelbarrows are gardening tools; and I doubt the inventors of the JCB dreamt that with the right bucket and a skilled operator their heavy earth-moving equipment could delicately strip off a few tens-of-centimetres of topsoil to cleanly expose archaeology in the subsoil beneath. What is true of physical tools is also true of technological ones; total stations and computer-aided design were created for construction; satellite imagery had its genesis in espionage; GIS were first used by geographers. Even our theoretical and methodological foundations are often borrowed from others. As archaeologists we specialise in taking something made for something else, and tweaking it to answer our questions. Adapting an existing tool to answer our research questions is as archaeological as brushing dirt from a precious artefact and, as I recently discovered, is widely applicable to a variety of situations beyond the excavation, lab or library.

Seeking petrol

At the time of writing, the UK is currently experiencing a temporary fuel shortage, resulting from a combination of panic-buying and a shortage in HGV drivers. I live in a suburban area, but public transport is limited and I have a pre-schooler, so I have always done a modest amount of driving. The fuel shortage caught us unawares with limited petrol in the car and after a few days of taking my little one to school we were running seriously short. At this time fuel deliveries were still being made, but the number of people seeking fuel and the intermittent nature of the deliveries meant that it was impossible to know which petrol station would have fuel. I needed to find a way to determine exactly which petrol station might have what I needed before I left. Where could I find real-time information on which petrol stations had fuel? How could I locate the closest one?

Real-time feedback in Google maps

Thanks to years of archaeological research, I have a long history of looking at maps and satellite imagery and I’m used to combining data, maps and visual imagery to address research questions. So when I was faced with a very real problem my brain automatically thought along the same lines. Since I was looking for the closest petrol station with fuel, the problem was obviously a question of mapping. Google maps is one of the most powerful mapping tools available for simple tasks and offers real-time information, it was a sensible place to start looking for a solution.

Google maps image showing the Southend-on-sea area, with petrol stations marked.
Google Maps showing petrol stations near my home in Leigh-on-sea. Screenshot taken at 12.30pm on Monday 4 October (Google Map data ©2021).

When teaching basic GIS I often use Google maps as an example of a deceptively simple but immensely powerful GIS. Behind the Google maps interface is an incredibly complex programme collating vast quantities of data, from satellite imagery and road maps, to data about speed limits and real-time information about traffic movements. Despite this complexity and vast processing power, it appears simple to us because the interface places substantial limits on what we can do with it; see a map of the area around us, search for a location, navigate from one place to another, or find a simple amenity, like a petrol station. Google Maps is immensely powerful but setup to be used in certain specific ways. It contains the locations of nearby petrol stations (as in the image above) and real-time traffic information (image below), but its not setup to find petrol stations with fuel. If I wanted do that I would need use it in a way its designers had never intended. I would need to adapt this tool to access the data I needed, just like generations of archaeologists before.

Google maps image showing the Southend-on-sea area, with traffic information turned on.
The borough of Southend-on-sea in Google Maps, with traffic information turned on. Screenshot taken at 12.45pm on Monday 4 October (Google Map data ©2021)

Adapting the tools

Adapting the data from Google Maps to answer my question necessitated current background knowledge of the situation and its impact on my society. The one thing everyone in any urban or suburban area knows about the petrol shortage is that any petrol station with fuel rapidly creates a jam along the adjacent road as people queue up to get fuel and block the road. So I combined the petrol station locations with the traffic information to identify petrol stations with queues outside. Since people don’t queue outside a petrol station without fuel, those with queues must have fuel. In the image to the right Tesco Petrol Station and the West Street BP to the right of the map just above the ‘Southend’ label, both have queues outside. The Shell to the right of Tesco does not. We promptly went to the West Street BP and fuelled-up without difficulty.

Google maps image showing a detail of the Southend-on-sea area, with petrol stations marked and traffic information turned on. Some petrol stations show a red queue of traffic approaching their entrances, while others do not.
Google Maps extract from 28 September 2021, 8pm. (Google Map data ©2021)

‘Improper use may cause damage!’

As when using any tool in a way its creators did not intend, a certain amount of local knowledge, experience and common sense is required. In the image to the right the BP with a Wild Bean Cafe in the bottom left also has queues shown to the west of it, but I can’t be sure those indicate it has petrol. It’s on the A13 and close to a junction. The A13 is a major road that is often slow and can have queues and jams almost anywhere on it for any reason, and the traffic lights often produce short local jams like those shown in the image. So if I was really desperate to go straight to a petrol station with fuel, seek out queues at petrol stations on a minor roads some distance from intersections. The BP petrol station on West Street, Prittlewell, is an excellent example. The West Street BP is centre right, just above the ‘Southend’ label in the image above right. When I checked traffic information against petrol stations I found a dark red tail leading away from the West Street BP (centre right in the image below). West Street is not a major road and the BP is not close to a major intersection. The only likely reason there’s a queue leading straight to that petrol station at 8pm is that those people are waiting for fuel.

Checks and balances

Google maps image showing a detail of the Southend-on-sea area, with petrol stations marked and traffic information turned on. Some petrol stations show a red queue of traffic approaching their entrances, while others do not.
Google Maps of the area near Leigh-on-sea showing petrol station locations and traffic information. Screenshot taken at 1pm on Monday 4 October (Google Map data ©2021)

Its possible to double-check that a petrol station has fuel using the powerful statistics held in Google maps. At the time of writing it is 1pm on a sunny weekday. There should not be a lot of unusual traffic or unexpected jams that are unrelated to fuel queues and might confuse me, but if it was rush-hour I’d have to be more careful. Checking the petrol situation at the present (image above) it seems that the Tesco Petrol Station and Shell in the centre right of the image have fuel, while the Esso MFG Kent Elms does not.

Details of the Tesco Petrol Station in Southend on sea showing its address, opening hours, phone number and times when it is busiest.
Tesco Petrol Station statistics from Google Maps on 4 October 2021 at 1.25pm.
Details of the Tesco Petrol Station in Southend on sea showing its address, opening hours, phone number and times when it is busiest.
West Street BP petrol station statistics from Google Maps on 4 October 2021 at 1.25pm..

I can confirm that the traffic information reflects queues for fuel rather than any other traffic incident by checking the real-time statistics in Google Maps (this was Paul Barrett’s idea). If you click on the little marker balloon for a likely petrol station and scroll down, there’s a little graph showing popular times. If the queue on the traffic information is for fuel, the graph should show that the petrol station is currently ‘busier than usual’. If we check on the Tesco Petrol Station for now 1pm on 4 October 2021, we find it is ‘busier than usual’ (image above left), confirming the evidence of the traffic information, that showed a queue outside it. This contrasts quite nicely with the BP on West Street, which is now running out or is out of fuel and is therefore ‘less busy than usual’ (image above right).

Adapting and mis-using tools

Aside from being an interesting use of an incredibly powerful free GIS, why is my solution to a personal fuel crisis of archaeological or Egyptological significance? I believe there are two reasons. The real-world value of humanities and social science disciplines like archaeology and Egyptology are often questioned and those who study them expected to explain how their supposedly arcane subject prepares them for the real world of work. I developed this particular method of solving my real-world fuel shortage problem because of my long history of looking at maps and satellite imagery and combining various sources of data and visual imagery to address research questions. While students learn many valuable skills studying archaeology or Egyptology, the ability to marshal varied sources of data in textual, statistical and visual formats; and judiciously adapt tools for new purposes, is highly valuable in many professions.

Secondly, my method of finding petrol stations with fuel follows the same model of adaptation that is common in archaeology and Egyptology. I took an existing, common tool (Google Maps) and combined two established features of that tool (real-time traffic information and the local search function) with specific cultural knowledge (that petrol stations with fuel attract queues) to extract information on which petrol stations were likely to have fuel. I improved the rigour of my method with additional background cultural and cartographic knowledge (of the main roads and traffic hotspots) and checked it against an independent dataset (Google Maps real-time statistics on how busy locations are). The result is an efficient and effective method of obtaining information I did not previously have direct access to but which was present in existing datasets, all without developing new software or diagnostic tools. For me this is what GIS, landscape and archaeological research is all about!

Featured

Free high-resolution satellite imagery and how to find it.

The resolution of satellite imagery is crucially important to its usefulness to the archaeologist because it directly impacts the features you can see and the precision with which you can georeference other data. For some purposes (such as overviews or larger maps) we might only need low or medium resolution imagery, but in many cases high resolution imagery is essential to our research objectives. During my recent georeferencing project, it was necessary to use the highest resolution imagery I could obtain to locate the historic maps as precisely as possible.

Unfortunately, although you can purchase high resolution satellite imagery from various providers, it is often expensive and the contract comes with quite strict restrictions regarding what you can do with it and with whom you can share it. This makes free high resolution imagery a very valuable resource for the archaeologist, although it does have limitations. There are various ways to obtain free high resolution satellite imagery, but here I compare the imagery provided by three of the most common providers Google Earth, ESRI ArcGIS World Imagery and Bing Maps. Imagery from all these providers can be accessed online in their applications, where you can also access archived imagery through Google Earth historical imagery or the ESRI Wayback app. But if you want to do much more than look at the imagery it is advisable to access it through a GIS. If you are using ESRI ArcGIS World Imagery you can access it through the basemapping layers of ArcGIS, but all three providers can also be accessed through the Contributed Services of the QuickMapServices plugin of QGIS.

Comparing imagery of Dra Abu el-Naga

Dra Abu el-Naga, at the north-eastern end of the Theban Necropolis, is a an important area where 17th Dynasty royals and Ramesside nobles were buried. It continues to be excavated by various teams, including the Djehuty Project, the German Archaeological Institute (DAI) and the University of Pisa and is the subject of several recent publications, concerning its geomorphology and landscape archaeology.

The following images compare three different open-source high resolution satellite images of Dra Abu el-Naga at the same scale. You can also see a more dynamic comparison in this video on my YouTube channel. first satellite image comes from Bing maps. It is reasonably clear at a scale of 1:2000, revealing the structures of the excavated tombs and the open shafts in front of them, but when you zoom in it becomes blurry and the features are difficult to distinguish (from 1:59 minutes in the video).

Image showing a satellite image of the northern part of Dra Abu el-Naga. Tomb structures and open shafts are relatively clear.
The northern part of Dra Abu el-Naga in the Theban necropolis, as rendered by Bing maps on 2 March 2021 in QGIS using the QuickMapServices plugin ©2021 Maxar, ©2021Microsoft.

The Google Earth satellite image is much brighter and somewhat clearer than the the Bing maps image (from 2.45 in the video). It’s worth noting that the Bing maps image appears to be older than the Google Earth imagery. The Google Earth imagery shows tombs in the bottom left corner, which are covered with debris in the Bing maps image (above).

A satellite image showing the same northern part of Dra Abu el-Naga as it appears in Google Earth imagery. The image is brighter and tomb structures and shafts are visible. More tombs are visible in the bottom left of the image where debris has been removed.
The northern part of Dra Abu el-Naga, as shown in Google Earth satellite imagery on 2 March 2021, displayed in QGIS using the QuickMapServices plugin ©2021 ORION-ME, ©2021 Maxar Technologies, ©2021Google.

In my opinion the ESRI ArcGIS World Imagery is the best of the three, although it is not as bright as the Google Earth imagery. The tomb structures and shafts are still visible, but the imagery is sharper and easier to understand (from 4:10 in the video). It also stands up to zooming better than either the Bing maps or the Google Earth imagery. It’s worth noting that you may need to use ArcGIS to export ESRI World Imagery. Although its possible to view it in QGIS, it failed to export as an image.

Image of the northern part of Dra Abu el-Naga showing tomb structures and shafts. The image is of a good contrast, recent and shows the features well.
The northern part of Dra Abu el-Naga as shown by ESRI ArcGIS World Imagery on 2 March 2021, displayed in ArcGIS 10.

Limitations of free imagery

Bing, Google and ESRI do not own their own satellites, instead they provide imagery from commercial and government sources. These free high resolution images will not be as good as if you purchased them for yourself, but provided we’re aware of the limitations they can still be incredibly helpful.

The first problem with these images is that they are often provided without metadata, so we don’t know which satellite took them or when. Nor do we know the precise resolution, whether the image is pan-sharpened or which electromagnetic bands are included. Based on how clear they appear I would guess that all the images in this blog post have a resolution of 30-60cm, with the Bing maps image being the lowest resolution and the ESRI image the highest. But it is just a guess. Similarly I believe the Bing image is older than the Google Earth and ESRI images, based on the greater exposure of archaeology in the latter, but I cannot say precisely from the imagery.

Another difficulty with free imagery is that it is updated by the provider intermittently. An image that covered your site last year might have been replaced by now with a less useful version, perhaps with cloud cover, dust or an error in georeferencing. Even if there isn’t an obvious error or kink in the georeferencing, you may find that the new image doesn’t match the rest of your data as well. Global imagery providers use a global coordinate system to display satellite imagery, which may be different from the local coordinate system and projection you are using. As a result you may find a feature does not appear in exactly the same location in the new satellite image as in the old one. The difference is not likely to be large and may not be significant for your project, but if you are using free high resolution imagery for survey or other precision tasks, a shift of a few metres could represent a serious problem.

A lot of the value of high resolution satellite imagery is in the multiple multi-spectral bands provided and the opportunities for raster analysis and pan-sharpening. Free high resolution imagery does not provide access to the individual bands, permit bands to be recombined or allow for raster analysis. While such imagery can form a useful basemap for display or georeferencing, it isn’t suitable for various remote sensing applications or analyses.

Don’t forget to reference

Referencing your satellite imagery is just as important as citing written sources, and perhaps more so, because the ‘fair use’ carve outs are much less clear and tested with this relatively new type of data. ArcGIS provides relatively clear instructions on citing their products. ArcGIS, Bing and Google Earth programmes watermark downloads, but these watermarks may not be included when using 3rd party software, such as QGIS. You may therefore need to check relevant information for how to cite Bing and Google Earth maps in your captions. You may also need to include acknowledgements or further references depending on the terms of service.

Acknowledgements and References

The first two images in this blogpost were created in QGIS using Bing and Google Earth satellite imagery respectively. The third image was created using ArcGIS® software by Esri. ArcGIS® and ArcMap™ are the intellectual property of Esri and are used herein under license. Copyright © Esri. All rights reserved. For more information about Esri® software, please visit http://www.esri.com.

Featured

Finishing the map, georeferencing the pyramid of Djedefre

In a previous post I described how I georeferenced a difficult map of Abu Rawash. During that process I had to ignore the pyramid of Djedefre because it was drawn at a different scale to the rest of the map. Here and in this video I discuss how I subsequently georeferenced the pyramid of Djedefre at the correct scale.

The map of the pyramid of Djedefre was cropped from map I in Porter and Moss’ (1932) Topographical Bibliography volume IIIi, which is featured in my previous post.

Comparison of the angle of the causeway after accurately georeferencing the pyramid (left) and the angle of the causeway in the satellite imagery (right). Map cut from Porter and Moss 1932, volumne IIIi, map I.

First, I identified the archaeological features in the map and the satellite image. Although the pyramid is very clear in the satellite imagery, this process was complicated by the long ramp or causeway heading north from the pyramid. Initially I assumed this was the pyramid’s causeway to the Valley Temple, but the angle of this feature in the map, and the angle in the satellite imagery were different. This made me wonder if the feature I could see in the satellite imagery was some kind of recent Decauville track for removing spoil from the pyramid, and the actual causeway ran at a different angle and had been removed. Having georeferenced the map I think the feature is probably the pyramid causeway in both cases, the angle in the map being incorrect due to the differences in scale between the drawing of the pyramid and the rest of the map. If you review the image of the georeferenced map in my previous post you will see that the end of the causeway as drawn in the map lines up with the end of the feature in the satellite image, even though the pyramid is incorrectly drawn. This suggests that the angle of the causeway became misaligned when the pyramid and the rest of the map were joined together. Nevertheless I chose to ignore the causeway when georeferencing the pyramid, because its questionable accuracy would make georeferencing more difficult and it was readily visible in the satellite image anyway.

I then scaled the image to the correct scale to align it to the satellite imagery (from 1.20 minutes in the video). As I note at 4.59 in the video, one thing to be aware of when georeferencing maps is that the lines of the map occupy space within the GIS – so the line of the enclosure wall of the pyramid complex represents 3m on the ground after georeferencing. This can make it difficult to align the map with satellite imagery, particularly if the map only covers a small area and/or is cropped from a much larger map.

Once I determined approximately the correct scale (1:2250) I began linking the ground control points in the map and satellite imagery (from 4.15 minutes in the video). This revealed further inaccuracies and forced me to make decisions about which points in the map I believed were more accurate than others. In this previous post I discussed the importance and limitations of RMSE. The georeferencing of the map of the pyramid of Djedefre really emphasises how RMSE and residuals can be used to improve georeferencing, and also the limitations of the process. I used the RMSE and residuals, combined with the visual position of the map on the satellite image, to test the ground control points (from 10.08 in the video). They rapidly revealed that parts of the pyramid complex had been drawn inaccurately in relation to each other. After noting that the satellite pyramid and south-west corner of the enclosure were in dashed lines, I opted to set the ground control points elsewhere as it seemed likely that the satellite pyramid was more speculatively drawn. Testing various ground control points also revealed that the enclosure wall around the complex was drawn closer to the pyramid than it really is, forcing me to chose whether to include the complex enclosure wall in the ground control points or concentrate on the pyramid. I chose to focus upon the pyramid and mortuary temple, and rectified the map with an RMSE of 3.59, which was an improvement on a previous attempt, but still far from the 0.75m RMSE which would represent the 1:3000 ideal (Conolly and Lake 2006, 82-83). The inaccuracy in the map, its scale and the resolution of the satellite imagery are all contributors to this high RMSE. Depending on what I need to do with the map, I may seek out a more recent map or re-georeference it. Georeferencing a map this small, with this many inaccuracies, to satellite imagery, is always going to be difficult and likely to produce a high RMSE.

A satellite image of Djedefre's pyramid complex overlaid with the plan from Porter and Moss 1932, map I.

Acknowledgements and References

Conolly, J. and Lake, M. 2006. Geographical Information Systems in Archaeology. Cambridge.

Porter, B, and Moss, R. 1932, Topographical Bibliography of Ancient Egyptian Hieroglyphics, Texts, Reliefs and Paintings III: Memphis 1. Abu Rawash to Abusir. Oxford.

Maps and images throughout this blog post were created using ArcGIS® software by Esri. ArcGIS® and ArcMap™ are the intellectual property of Esri and are used herein under license. Copyright © Esri. All rights reserved. For more information about Esri® software, please visit http://www.esri.com.

All the satellite imagery used is ArcGIS World Imagery. Sources: Esri, DigitalGlobe, GeoEye, i-cubed, USDA FSA, USGS, AEX, Getmapping, Aerogrid, IGN, IGP, swisstopo, and the GIS User Community.

Featured

Errors, inaccuracies, resolution and RMSE: Georeferencing a difficult map of Abu Rawash’s pyramid and cemeteries

In a previous post I introduced the georeferencing work I was doing, with a video of me georeferencing Porter and Moss’ map of the Cemetery F mastaba field at Abu Rawash. Here I delve into the process a little more, with help from a further video, which shows me georeferencing a more difficult map of Abu Rawash using ArcGIS basemap World Imagery.

The map I am georeferencing in those videos is map I in Porter and Moss’ (1932) Topographical Bibliography volume IIIi. It shows the entire area of Abu Rawash from the pyramid of Dejedefre, to the north-west cemetery in Wadi Qaren and the village of Abu Rawash on the edge of the cultivation.

Map of Abu Rawash showing the pyramid to the south-west, and various cemeteries in the desert around.
Map I from Porter and Moss’ 1932, Volume IIIi, of the Abu Rawash area.

Accuracy and precision.

When working through the georeferencing process, its important to carefully consider the precision and accuracy you are aiming for, taking into account likely distortions in the image to be georeferenced; the resolution and accuracy of the data (the satellite imagery) you are georeferencing with; and the ultimate function of the georeferenced image.

All data, even GCP collected on site with differential GPS, have some level of error in them. What matters is that they are sufficiently accurate and precise for the task you have in mind. Accuracy and precision are also different. Accuracy refers to whether something is correct. In other words, is the map you are georeferencing in the right place? Precision is easiest to think of as resolution or the level of detail. Something can be very precise but very inaccurate, or something can be very accurate and very imprecise. It is accurate to say I live in the United Kingdom, but it is not very precise. It is precise to say I live in N7 6RY (A postcode in Finsbury Park, London) but that is not accurate – I do not live there any more.

The accuracy of the georeferenced image can be affected by drafting or composing errors in the original map, or by distortions introduced during printing or digitising. Such inaccuracies and distortions affect how well the map can be overlaid on the satellite imagery. The satellite imagery can also contain distortions. Gross and obvious distortions, like the kinks in the Deir el-Bahri temples I published in a previous post and distortions due to the angle of the satellite to the ground affect the georeferencing process.

As with accuracy, the precision of the georeferenced image is affected by the precision of the map we are georeferencing, and the resolution of the satellite imagery. If the map is sketchily drawn or details are missing this may make it more difficult to locate precisely. The resolution of the satellite imagery (or any other ground control data) also affects the precision of the georeferenced image. Georeferencing requires that we line up the map with the same features in the satellite image. The more precisely a feature appears in the satellite image, the more precisely we can locate the map we are georeferencing. The pixel size (resolution) of the satellite imagery therefore places considerable limits upon the precision of the georeferenced image.

Locating the archaeological features at Abu Rawash

It feels like it ought to be easy to identify the pyramid of Abu Rawash and associated cemeteries. After all, pyramids are not known for their discretion, particularly once they’ve been excavated. While the pyramid of Abu Rawash is pretty clear in the satellite imagery, the huge amount of quarrying and agricultural and housing development in the Abu Rawash area made it difficult to identify the geographical and archaeological features in Porter and Moss’ 1932 map. The Survey of Egypt 1:25,000 scale map of 1942 shows how little development had taken place 80 years ago.

Map of the Abu Rawash area from 1942 showing minimal modern development in the desert.
Abu Rawash in 1942 (Survey of Egypt 1:25,000 scale map of Kirdasa).

In contrast the modern satellite image shows the pyramid and cemeteries as islands of archaeological landscape in a highly developed area.

Image of the Abu Rawash area showing considerable quarrying and development around the pyramid and cemeteries.
The area of Abu Rawash today, note the considerable development and quarrying in the area.

This made it more difficult to relate Porter and Moss’ map to the satellite image, although the preservation of the pyramid and the cemeteries did help (see from 0.35 minutes in the video).

Scaling the map for georeferencing

Once we identify the area, we need to scale the map to fit that area in the satellite imagery. You can see me undertaking this task from 1.10 minutes in the video. During the process, I discovered an inaccuracy in Porter and Moss’ map – the pyramid of Djedefre had clearly been drawn at a different scale to the rest of the image. Drafting and composing inaccuracies of this type are frustrating but do occur with historic imagery. Other sources of inaccuracies include distortions introduced during the scaling of maps for publication and when publications are scanned or photographed to generate digital images. Photography is particularly problematic as the camera lens needs to be parallel to the image to avoid distortion, but even scanning can produce minor inaccuracies. Dealing with these inaccuracies often means adjusting the georeferencing, splitting an image or ignoring part of the map during the georeferencing process. In this case I chose to georeference the map, while ignoring the pyramid and subsequently cropped out the pyramid and georeferenced it separately.

Adding control points (GCP)

Once Porter and Moss’ map had been scaled to approximately the right scale, it was aligned more precisely to the satellite imagery using ground control points (GCP). These appear in ArcGIS as ‘Links’, and operate essentially as pins. You select a point in the map and then select the same point in the satellite image and ‘pin’ them together. You can see me undertake this process from 7.20 minutes in the video. Ideally, it would be possible to locate these links very precisely in each set of data, but in this example, we are constrained by the contents of Porter and Moss’ map, which does not include many clear points that can be related precisely to points in the satellite imagery. The resolution of the satellite imagery is also a factor. The ArcGIS Basemap World Imagery uses a variety of satellite imagery sources, but the highest resolution of any commercial satellite imagery is currently c. 30cm and much of the imagery is likely to have a resolution of c. 40-50cm or more. This means that the pixels of the satellite image represent 40-50cm on the ground. Any feature smaller than that is invisible, and features that are only slightly larger are difficult to identify. Another effect of the satellite imagery resolution is that when we zoom in close the satellite imagery appears blurry, and a point becomes more difficult to locate than when zoomed out (you can see the effect of this from 8.45 in the video).

Root Mean Square Error (RMSE)

Once we have added four links in ArcGIS, we can open the link table and see a RMSE for the entire map in the top box and the residuals for each point in the right column of the table. Turning off links or adding new links will alter the position of the map and the RMSE accordingly (from 10.30 in the video). The RMSE represents ArcGIS’ calculation of the fit between the actual and desired link positions (Conolly and Lake 2006, 82-83). In simple terms ArcGIS uses the first three links to estimate where it thinks any further links should be. It then calculates the residual for each point as the difference between where you placed a link and where the map ended up based on the other links that have already been placed. The RMSE is the product of all the residuals. Although RMSE is useful, it’s important to recognise that it is reliant upon the accuracy and precision of the map and the satellite imagery. If there are inaccuracies in either, they will increase the RMSE. It is also reliant upon the locations and positioning of the points you choose. The old adage of ‘junk in, junk out’ definitely applies and it is entirely possible to have a low RMSE and a very inaccurate and imprecisely positioned map. So while you can reduce your RMSE by removing links with high residuals and adding new links, it is sometimes better to accept a higher RMSE and keep an important link, recognising that the higher RMSE is due to inaccuracies in the map. Alternatively, it may be necessary to chose which ground control points you believe are more accurate and only link to them.

We aim for an error of less than 1:3000 so for an original image at a scale of 1:15000 an RMSE of under 5 (i.e. 15000/3000) is ideal (Conolly and Lake 2006, 82-83). Ideally we would use the scale given in the original image, but Porter and Moss do not include scale information so we have to work with the scale we established during georeferencing. When I scaled this map I settled on a scale of 1:9000, so any RMSE under 3m would be very acceptable. Here our RMSE is slightly above 3m, which is not unreasonable given the inaccuracies in the map and the difficulty of locating very precise control points due to the resolution of the satellite imagery and changes to the landscape. I subsequently repeated the georeferencing and obtained an RMSE of 2.88, but reducing the RMSE by a large amount is not always possible depending on the scale and accuracy of the map, and the resolution of the satellite imagery. The map of Cemetery F, for example was at a scale of just over 1:500, meaning its RMSE should be 0.16m or under, but I was only able to get it to 0.3m. Nevertheless, under the circumstances that is acceptable because of the resolution of the satellite imagery, which makes it impossible to place a point more precisely than within 0.3m. This is compounded by the imprecise edges of certain archaeological features in the satellite imagery, such as the mastabas of Cemetery F or the satellite pyramid of Djedefre, and any inaccuracies or distortions in the maps. In such cases it is important to be aware of known inaccuracies and distortions in the map and satellite imagery or you can be driven to distraction trying to get inaccurately positioned features to line up.

Ideally, if the RMSE is too high and cannot be reduced, we would seek an alternative source of data, but such data does not exist for some of these sites. In those cases it is much better to have a slightly less than ideally georeferenced map, than none at all. It is also important to be aware of the purpose of your georeferenced map. In this case the relatively modest aim was to locate archaeological features to within 10m, which is achievable with the accuracy of the maps and the resolution of the satellite imagery.

Overall I was satisfied with the georeferencing of the Abu Rawash map. It was a very difficult map to georeference; hard to locate due to the changes to the landscape; difficult to scale due to the inaccuracy in the pyramid; and difficult to find enough precise features to use as GCP links . Nevertheless, the final georeferenced version gives useful insight into the archaeological landscape. With careful thought and reference to the underlying satellite image, it will be possible to locate any relevant archaeological features during the rest of the project.

A map of the Abu Rawash area, overlying a satellite image. The pyramid is clearly at the wrong scale and angle compared to the rest of the image.
Final georeferenced version of Porter and Moss’ 1932 Volume, IIIi, map I of Abu Rawash.

Acknowledgements and References

Conolly, J. and Lake, M. 2006. Geographical Information Systems in Archaeology. Cambridge.

Porter, B, and Moss, R. 1932, Topographical Bibliography of Ancient Egyptian Hieroglyphics, Texts, Reliefs and Paintings III: Memphis 1. Abu Rawash to Abusir. Oxford.

Maps and images throughout this blog post were created using ArcGIS® software by Esri. ArcGIS® and ArcMap™ are the intellectual property of Esri and are used herein under license. Copyright © Esri. All rights reserved. For more information about Esri® software, please visit http://www.esri.com.

All the satellite imagery used is ArcGIS World Imagery. Sources: Esri, DigitalGlobe, GeoEye, i-cubed, USDA FSA, USGS, AEX, Getmapping, Aerogrid, IGN, IGP, swisstopo, and the GIS User Community.

Featured

Kinky Temples: Satellite imagery ‘fails’!

High-resolution Google Earth imagery is a great resource and one widely used by archaeologists the world over. But with tens of thousands of individual satellite images of most of the planet there is bound to be the odd error. During a recent project I was looking in detail at satellite images of the Theban Necropolis, only to discover that Deir el-Bahri appeared to be suffering from some form of spatial anomaly. In the satellite image, the north-west end of the temples of Hatshepsut and Montuhotep display a distinctive kink, like melted caramel.

Screenshot taken on 18 December 2020 of ESRI ArcGIS® basemap satellite imagery, showing the temples of Deir el-Bahri . The north-west end of the temples and the cliff behind have been warped into an improbable kink.

I can’t determine the precise cause of this kink in the satellite image without more information. It may be something in the projection of the image in Google Earth. Accurately projecting images of a 3d, ellipsoidal earth onto the flat plane of a computer screen using a coordinate system that covers the entire planet is mathematically complex and inevitably leads to compromises and errors. But given that this kink is within a single satellite image, it is more likely to be the product of a georeferencing or georectification error. A satellite image that is projected in ArcGIS, Google Earth or any other GIS, contains information about the global coordinates of the image that allow it to be precisely located in the correct position within whatever coordinate system the GIS is using. These global coordinates allow the image to be ‘warped’ such that it more accurately presents the surface of the earth as it appears. This is sometimes called ‘rubber-sheeting’, which conveys the process very picturesquely. The global coordinates for georectification are calculated from the satellite ephemeris (the information about where the satellite was and how it was positioned when the imagery was recorded) together with other relevant information, including a digital elevation model. If there is an error in the data, the image can be twisted and warped in an improbable and inaccurate way. And so we have kinky temples – Hathor would surely approve!

Detail of the previous image showing the warping of the north-west end of the temples of Deir el-Bahri

Joking aside, this is an important reminder that although incredibly useful and typically highly accurate, satellite imagery can and does contain errors. In this case the inaccuracy is large and obvious, affecting an incredibly famous and well-surveyed site. The most novice researcher would be unlikely to miss such an error or believe that the temples truly bend in this improbable fashion. But even in this case, the warp is much more difficult to discern in the cliffs behind than in the rectilinear temples. Smaller-scale errors, in less well known areas, with fewer structures can be much harder to spot. In satellite imagery, as in everything else, it pays to be vigilant. Never assume that the ‘data’ is infallible.

Acknowledgements

The imagery presented in this blog post was created using ArcGIS® software by Esri. ArcGIS® and ArcMap™ are the intellectual property of Esri and are used herein under license. Copyright © Esri. All rights reserved. For more information about Esri® software, please visit http://www.esri.com

Global Xplorer: Satellite remote sensing, looting and crowd-sourcing.

On 30 January of this year, Sarah Parcak, winner of the 2016 TED Prize launched the Global Xplorer platform she has created with her prize money. Like the many archaeological crowd-sourcing  projects on the Micropasts website, Global Xplorer is a crowd-sourcing platform allowing members of the public to take part in archaeological satellite remote sensing from their laptop, phone or tablet.

The heady combination of Sarah Parcak, National Geographic’s own ‘Space Archaeologist’, a TED prize, archaeology and satellite technology has prompted a number of media outlets to report on Global Xplorer, from Forbes to The Guardian. All this publicity means that interested members of the public will soon be working through the imagery on the platform.

But the thought of anyone with a computer logging on to archaeological sites has not been universally welcomed. This recent article on the Cairo Scene website, which has been publicised on social media by the Egyptian Cultural Heritage Organisation, argues that Global Xplorer could be used by looters to identify targets, making the situation worse in countries where protection for antiquities is limited. As someone who works with satellite imagery, I decided to test out Global Xplorer and see if these concerns are justified and provide general review of platform.

Global Xplorer is a straightforward platform. There are several pages of information about the project with videos from Sarah Parcak. Archaeological and cultural details of the countries covered can be found under ‘Expedition’, and there are also areas for donations and FAQs. So far only data from Peru has been included on the platform.

Before you can begin working on satellite imagery you need to register, giving details such as name, email address and a password you generate. Then there is a tutorial explaining briefly what natural and archaeological features look like in satellite imagery and how to identify evidence of looting and avoid false positives. Once you’ve finished the tutorial you begin working on the tiles.

The first goal for users of Global Xplorer is to identify evidence of looting. You won’t be creating a map or identifying archaeological features, but locating traces of antiquities theft for further investigation. Following the tutorial you’ll be shown a 100x100m tile of satellite imagery. The area you need to examine is outlined in white, with a little bit of additional imagery greyed out around the edges to provide some extra context. Depending on whether you can see any evidence of looting or not, you click either the ‘Looting’ or ‘No looting’ buttons and the next tile loads. As you can see I appear to have ended up with a bit of amazon rainforest:

global_xplorer-interface
The Global Xplorer interface.

It’s pretty clear that it would be almost impossible for tomb robbers or antiquities thieves to make use of Global Xplorer to further their nefarious activities. In addition to the off-putting effect of registration,  the user only sees a very small area at any given time, and it is entirely divorced from any geographic context. The FAQs confirm my assumption that the tiles have no coordinates or other geographically identifying data, and the tiles you are shown appear in a randomised order so you couldn’t even associate what you’ve seen in the previous tile with what you see in subsequent ones. Within the tiles there’s very little information to assist you in identifying the location. Even if a pristine archaeological site appeared inside that white square, at best you might be able to discern it was in the rainforest, in a field or in the desert and next to a building, road or river, but the sheer number of possible locations for each tile is immense. In fact the only way looters would be able to recognise an archaeological site is if they had already been there and were familiar with the terrain, and then we can hardly blame Global Xplorer for the looting.

On the other hand Global Xplorer is quite good fun. It works on your phone or tablet, so you can cover a couple of tiles while waiting for the bus/train/plane. It’s very easy and straightforward and quite relaxing in a strange way. While currently the goal is to identify looting, the ‘Current Campaign’ menu at the bottom of image above suggests that users will move on to identifying ‘Encroachment’ and ‘Discovery’ as the programme rolls out.

So if you have a few minutes and fancy trying it out, its straightforward to learn and easy to use, and you certainly don’t need to worry about antiquities thieves learning anything useful.