Tag: Overpass

Categories
Guides

Fun with Wales’ map data: a tutorial using Overpass queries and OpenStreetmap

Map: places in Wales that have ‘llan’ in their names

Would you like to get castles, rivers, post boxes, or cycleways in Wales from a map?

How about investigating place names in Welsh in your local area?

How about getting other features in Wales and further afield, as open data from a map?

This blog post will show you how to get open data from OpenStreetMap, with a particular emphasis on Welsh-language data.

It is intended as a fun introduction, not as a comprehensive reference guide. No previous experience is necessary.

We will be passing queries to the Overpass API, and it’s easy to get started. The queries can be run from your web browser in Overpass Turbo, which is one seriously cool app. Other than that your curiosity is the only prerequisite!

Introductory concepts

Feel free to skip this section if you want to head to the practical bit straightaway.

OpenStreetMap is a global map which has been built by thousands of people. It uses a wiki-like approach to mapping – anybody can edit and re-use the content. Because it’s all open data, you can use it however you want in your own learning, work, and leisure.

There is a huge amount of Welsh-language data in OpenStreetMap.

It’s independent of proprietary mapping providers, allowing you freedom to work with the data in your own projects.

The underlying code is also freedom-respecting software and open source. As the Mapio Cymru project we have built a showcase map which shows Welsh-language names for features including places, roads, rivers, and so on.

How to run an Overpass query

The quickest way to try Overpass queries is to visit the Overpass Turbo website.

The screen will be divided into an editor panel and a map/data viewer panel. Now do this:

  1. Write (or paste!) a query into the editor.
  2. Click the Run button.
  3. The results are shown in the data viewer.
  4. Within the data viewer you can select Map tab or the Data tab.

You’ll be following these same steps every time you run a query.

Towns query

Here is a simple query you can use. First drag the map and zoom until it shows an area you want to investigate, e.g. a part of Wales. Then follow the above steps using this query.

node["place"="town"]({{bbox}});
out;

Bingo, you should now see towns plotted on the map area you’ve selected. Congratulations on accomplishing your first Overpass query!

The data

Select the Data tab in the data viewer to see the data. It will be in the default format, which is XML.

Here’s a portion of the XML data you’ll see for the results of the above query, for two towns:

<node id="8997358" lat="51.5912466" lon="-2.7517629">
  <tag k="name" v="Caldicot"/>
  <tag k="name:cy" v="Cil-y-coed"/>
  <tag k="place" v="town"/>
  <tag k="population" v="11200"/>
  <tag k="postal_code" v="NP26 4"/>
  <tag k="wikidata" v="Q722585"/>
  <tag k="wikipedia" v="en:Caldicot, Monmouthshire"/>
</node>

<node id="21413062" lat="51.8591257" lon="-4.3115907">
  <tag k="is_in" v="Wales"/>
  <tag k="name" v="Carmarthen"/>
  <tag k="name:br" v="Caerfyrddin"/>
  <tag k="name:cy" v="Caerfyrddin"/>
  <tag k="name:en" v="Carmarthen"/>
  <tag k="name:ja" v="カーマーゼン"/>
  <tag k="name:la" v="Moridunum"/>
  <tag k="name:ru" v="Кармартен"/>
  <tag k="place" v="town"/>
  <tag k="population" v="14185"/>
  <tag k="population:date" v="2011"/>
  <tag k="source" v="NPE"/>
  <tag k="source:population" v="Census"/>
  <tag k="wikidata" v="Q835835"/>
</node>

As you can see, the name:cy tag has the town’s name in Welsh. There are equivalent tags for other languages. There’s also a tag called name without a language code, here’s the definition of the name key.

In general name:cy will provide the name in Welsh for anything on the map – if it’s been submitted.

The other data in the examples above should be fairly self-explanatory, and include latitude and longitude, Wikidata item identifier, and other things.

Note that OpenStreetMap is always a work in progress. You’ll see pretty good data for many queries although some others will display gaps. (You can edit/add place names on the map, and other features and their tags.)

Change your Overpass Turbo map to Mapio Cymru

Within Overpass Turbo your underlying map will probably be the main OpenStreetMap. This is OK but it won’t always display all names in Welsh.

You can change it to the Mapio Cymru map server, like this:

  1. Select Settings menu
  2. Select Map
  3. In the Tile-Server box put: //openstreetmap.cymru/osm_tiles/{z}/{x}/{y}.png

Please note that when you click on map pins any links will still go to the main OpenStreetMap.

Farms, cities, and other places

You can take the query above and modify it:

node["place"="farm"]({{bbox}});
out;

Spot the difference between this query and the one above. Alternatively use one of the possible key values for place. For example you can use “village”, “city”, “island” and so on.

Your bounding box

In general:

  • If your query refers to a bbox (bounding box) the query will run on the visible map, the portion of the map you’ve selected.
  • You can also reduce the width of the map: drag its edge to reduce its size, and increase the size of the editor.
  • If your query has a lot of results, there may be too much data to plot on the Overpass Turbo map in your browser. Try zooming in to reduce the size of the bounding box.

Towns in Wales only

No matter how much you move the bounding box it’s not possible to get all of Wales, and Wales only. Our query needs to change.

This time, click the Wizard button and type ‘towns in Wales’ then click Build Query. When I ran it it suggested ‘town in Wales’ then gave the following query, and yours will be similar or the same.

/*
This has been generated by the overpass-turbo wizard.
The original search was:
“town in wales”
*/
[out:json][timeout:25];
// fetch area “wales” to search in
{{geocodeArea:wales}}->.searchArea;
// gather results
(
  // query part for: “town”
  node["place"="town"](area.searchArea);
  way["place"="town"](area.searchArea);
  relation["place"="town"](area.searchArea);
);
// print results
out body;
>;
out skel qt;

Where possible the Wizard will take the English you type and give a query in Overpass query language. As far as I know the Wizard is only available in English at the moment.

searchArea above is a variable containing our geocode area for Wales. It is set for the life of the query. We don’t have to call it searchArea, we can call it almost anything – as long as there’s no clash with other reserved terms.

The above query contains comments which have no effect on the query. There are two styles:

/* comment within slash star delimiters */

// comment between double slash and end of line

Llan place names

As well as Llanelwy this will return Rhosllannerchrugog in the results – and so on. It’s a case-insensitive search.

[out:json][timeout:50];
(
  node["name"~"Llan",i][place]({{bbox}});
);
out center;

This is a narrower search for Llan with a capital L.

[out:json][timeout:50];
(
  node["name"~"Llan"][place]({{bbox}});
);
out center;

Here’s a search that includes the tags name a name:cy for a comprehensive map which includes places which currently lack a name:cy tag and names like Llanandras (Presteigne) and Llanllieni (Leominster) (diolch/thanks for your replies via Twitter!).

(
node({{bbox}})["name:cy"~"Llan"][place];
node({{bbox}})["name"~"Llan"][place];
);
out;

This will give all places in Wales with Llan in the name. It gives data only – in Overpass Turbo the map tab will be blank. You can use the CSV results data in a project, e.g. in a spreadsheet.

[out:csv("name:cy", "name", ::lat, ::lon, "place", ::id; true; ",")][timeout:50];
{{geocodeArea:wales}}->.searchArea;
(
node["name"~"Llan"][place](area.searchArea);
node["name:cy"~"Llan"][place](area.searchArea);
);
out;

You could modify one of the above for ‘Aber’, ‘Caer’, ‘Tre’ and so on.

Castles in any area

Now try this query.

[out:json][timeout:25];
// gather results
(
  // query part for: “castle”
  node["historic"="castle"]({{bbox}});
  way["historic"="castle"]({{bbox}});
  relation["historic"="castle"]({{bbox}});
);
// print results
out body;
>;
out skel qt;

This is OK but how about all the castles in Wales only? Use this:

[out:json][timeout:25];
{{geocodeArea:wales}}->.searchArea;
// gather results
(
  // query part for: “castle”
  node["historic"="castle"](area.searchArea);
  way["historic"="castle"](area.searchArea);
  relation["historic"="castle"](area.searchArea);
);
// print results
out body;
>;
out skel qt;

Here are some others to try. In each case you should edit the three statements above to cover all nodes, ways and relations in the search. Let’s look up the definitions of those in a jiffy…

"natural"="peak"

"site_type"="megalith"

"historic:civilization"="ancient_roman"

"amenity"="bicycle parking"

"amenity"="recycling"

"amenity"="bus station"

The last one will identify, among others, the National Express coach station in Cardiff – currently the only bus station in the city.

Elements of OpenStreetMap

There are millions of possible Overpass queries.

You can play around with basic queries without having a comprehensive understanding of OpenStreetMap. The wizard may help.

Sooner or later though you might want more context to help you write that special query for your own interest. This portion from the documentation on elements has some vital definitions will help:

Elements are the basic components of OpenStreetMap’s conceptual data model of the physical world. They consist of

  • nodes (defining points in space),
  • ways (defining linear features and area boundaries), and
  • relations (which are sometimes used to explain how other elements work together).

All of the above can have one or more associated tags (which describe the meaning of a particular element).

If you want to see some examples of nodes, use this query.

In Overpass Turbo this will only work for small bounding boxes, because the amounts of data are relatively large.

Show the Wales Coastal Path

This is a simple query that only shows one relation – the northern part of the Wales Coastal Path.

relation(1850847);>;out;

This shows the entire Wales Coastal Path. (Because this is stored as a relation of relations, the query uses a ‘recurse down relations’ operator >> to display the relations, ways, and nodes within the overall relation. Here’s more info on the recurse down relations operator.)

[out:json][timeout:25];
rel(1820890);
(._;>>;);
out;

Hiking routes

Taken from the Overpass API examples. You probably need to zoom into a bounding box.

[bbox:{{bbox}}];
(
  relation[route=hiking][network~"^.wn$"];
  way(r);
  >;
);
out;

{{style:
way
{ color:green; fill-color:green; }

relation[network=lwn] way
{ color:blue; fill-color:cyan; }

relation[network=iwn] way
{ color:red; fill-color:red; }

relation[network=nwn] way
{ color:green; fill-color:green; }

relation[network=rwn] way
{ color:yellow; fill-color:yellow; }

}}

Rectangular buildings in Wales that are taller than they are wide

This one’s adapted from the examples.

[out:json][timeout:25];
{{geocodeArea:wales}}->.searchArea;
// Find rectangular buildings that are taller then they are wide
(
  // Compare the height to the average length of a side
  way["building"]["height"](area.searchArea)(if:
    count_members() < 6 && is_closed() &&
    number(t["height"]) > length() / 4);
  // Assume a floor is 3 m tall
  way["building"]["building:levels"](area.searchArea)(if:
    count_members() < 6 && is_closed() &&
    number(t["building:levels"]) * 3 > length() / 4);
);

// Print results
out body;
>;
out skel qt;

What’s next? Edit the map

If you notice any deficiencies in the data then you can edit the map. Welcome to the open data mapping community!

Overpass can be used deliberately to look for opportunities to improve the map.

Overpass Turbo in Welsh?

Overpass Turbo’s interface is available in a few languages but it doesn’t offer Welsh as an interface language yet. If you’d like to contribute to the translation head to its Transifex project.

Write your own queries

Start with the Overpass Turbo examples and Overpass API examples provided by the OSM community.

You can even delve into the user manual for Overpass.

Using Python instead of Overpass Turbo

If you can use Python you can run Overpass queries in your code using this simple wrapper instead of the Overpass Turbo web interface. Write an app and wow us!

Alternatively check out these other methods of querying Overpass via code.

In any case Overpass Turbo is handy for perfecting your queries.