Hello, everyone. It's good to be back. It's been a while. This is my first time giving
a talk here. I'm really pleased to be here. My name's Joe. I am a coder. I work in London
for local government. I work a lot with geospatial data, and I am a Python programmer. Have we
got any Python coders in today? Anyone using Jupyter? Cool. Right. So let's go. So in lockdown
in 21, we had a census in England and Wales, and the data is coming now. Most of the data,
all of the data, sorry, is spatial data. So we want to look at this on a map. Why? Most
of the data is geospatial. In local government, everything that we do generally happens somewhere,
whether it's collecting a bin, looking after young people, looking after old people, cleaning
the streets. We always have to think about where this is happening. Apparently, 60% of
all data is geospatial data. So I spent a lot of my time making maps in terms of data
of this. Now, I'm going to be focusing on one part of the census data set today, and
that's the east end of London in an area called Tower Hamlets. This may be familiar to some
people if you've ever seen places like Columbia Road, Bethnal Green, Canary Wharf. These are
all parts of the east end of London, and this is the main area I'm going to be talking
about. So where is Tower Hamlets in London? So what you can see here is a very small area.
It's 20 square kilometers, but this is quite a special area because in the whole of England
and Wales, it has the highest population density. It has the most people packed into a small
area. It also has the fastest growing population, so it's becoming more and more dense. So
in terms of providing services for residents, we need to have a big think about where all
the people are and how they fit in. Now, when we make maps, the first thing we usually do
is we make a coropleth map. However, the data set for population density in our area, and
I do apologize, I couldn't fit it all on screen. It doesn't appear very well as a coropleth.
The reason is because the data set is not very evenly distributed. There is, as we will
see, some areas with extremely high population density. So over here you've got Whitechapel.
We have very high population density in Whitechapel. Over here we have a new development which used
to be industrial land. Again, very, very high density developments, big, big towers full
of people. And then we also have, just to the south of the financial sector, some areas
of very high population density with a lot of people packed into a small place. But in
terms of the data viz, this map doesn't really help very much. So the coropleth data viz
didn't work for us. So we began to think, what else can we try? And we checked the data
distribution, and sure enough, we've got some serious outliers. This is why the coropleth
map didn't work very well for us. So what did we do next? We tried to log-transform
the data. And yeah, you can see, you know, this area here. You can begin to see the density
there. There's quite a few large developments with a lot of people squeezed in. Whitechapel,
you don't see so much happening there. But you do see, just to the south of the financial
sector, high density of population. The areas with low density, this is where all the banks
are. So obviously, there's no people living in there. This is an old dock near to the
Tower of London. There's no people living there. There's some very nice pubs, though.
If you ever find yourself in that area, the Dickens Inn is excellent. I can recommend
that to everybody. And then up here in the north, we have Victoria Park, which is where
the East End borders with Hackney. And obviously, there's no people there, at least having their
address registered there. Log-transform data looks better on a coropleth map. However,
you can see the legend. You lose the data. So you can try to fix the legend. But we want
to write as little code as we possibly can. We don't want to keep fixing legends and things
like that. So we began to think about other ways to visualize our data set. So what did
we do? I am a Python coder, but there's a really nice package in R called Cartogram.
And this is a technique called a density equalization algorithm that basically turns your data set
into a Voronoi first, and then it rescales the polygons from the Voronoi relative to
an attribute of the data. This technique is quite popular. There's a wonderful geographer
called Danny Dooling, who has an amazing website called World Mapper, which I strongly recommend
you have a look at. And they do things like showing poverty, inequality, food pressure
all around the world. And they size the geographies relative to the attributes of the geospatial
data. So this is a great technique. There is one issue here, though, is that if you
want to overlay different layers, then it becomes difficult. And also, the map does
look a little bit unfamiliar as well. But it does show particularly where you have like
clustering, where you have a number of census areas, and I'm going to say a little bit more
about census areas, where you have a few together that have high data attribute value, then
they all get bigger together. So what we can see here is just to the south of the financial
sector, you can see there's a lot of worker bees all crammed into this place, and then
it increases the volume on the map. So it's a nice data vis, but still we have a small
challenge if we want to add more data over the top. And also, it's a bit unfamiliar for
people that don't use cartograms. So this is a map made using Data Rapper. It's a very
nice website, and they have something called a symbol plot. And what this does is it just
basically shows little mountains, little peaks, that show the value of the data attribute
that you're interested in at the place where that data is happening. And so again, we can
see over here, you've got Whitechapel, lots of people packed in there. Just to the south
of the financial sector, lots of people packed in there. The new developments here by the
river in Blackwell, and here by the river in the old industrial zone. So this is quite
interesting. It gives us some context, and it gives us the data. I really like this data
vis, but it's Data Rapper, so it's not FOS, and it's not Python, and I like to use Python.
So it was great, but it helped, but it didn't do everything that we needed it to do. The
other thing that you will notice, and I'll try to explain this briefly, is that we have
one really high value here. And there's a reason for this. It's an outlier, because
actually it's this value here. It's an outlier, because, and the reason why it's an outlier
is because the actual census area is really, really small. And the thing about the people
who produce the census data is that they have to create census areas using roughly 100 to
600 people. Generally speaking, it's about 300 people, but they have to make it all fit
together like a big jigsaw puzzle. So sometimes, you know, it's hard for them to make it work
really well. So in this case, this census area with really high density is actually
just one building. And so it's not a particularly big building, but everyone squeezed in there.
So yes, so the data is quite hard to work with, but it is interesting. So when I was
working with Data Rapper, I really liked it, and it did remind me of when I was young and
I was reading Lord of the Rings books, I used to really like the map at the front of all
these mountains, showing the misty mountains in those books. And so I was thinking, I could
probably make a mountain with Python. How hard can it be? It turns out it's really easy.
This is the essence of the library. It's just one function. You take a point on a map, you
turn that point into a line. The line has a start point, which is just a couple of points
of longitude, a tiny little bit of longitude to the west of your point. Then you convert
your point to a latitude, which is kind of like a proxy for the height of the mountain,
using some kind of algorithm that you choose. In my case, I'm just like using a range. So
I take the minimum and maximum value of the input range, which is a separate function
here. And range one is essentially the minimum population density and the maximum. And then
I convert that to latitude values. And then the third point on the line is just a little
bit of longitude to the east of my point. And then you use that to create a small triangle,
really easy, really easy, and a lot of fun as well.
So this is what I made with Python. And it's very similar to the data wrapper map, but
I was going for like a kind of hand drawn kind of a look to make it look like something
from Lord of the Rings. And, you know, it's the same thing. You've got Whitechapel here.
You've got the financial sector here, and so on and so on. So that was fun. But, you know,
population density, we were just talking about the reasons why it's a messy data set. There's
one place in Chelsea, which has a population density of two million people per square kilometer.
So this is a very difficult data set to represent using any tools available. So, you know, it's
interesting. The other thing about Kensington and Chelsea is this is where Grenfell Tower
is, if anybody knows about that story. This is where it happened.
So let's try some other data sets to see if they're really messy. This is people that
live in one bedroom homes. So this is tiny little flats, you know, filled with people.
And so you can see all the worker bees for the financial sector. A lot of those are living
in one bedroom flats. And actually, the new builds. This is a very new development here.
And this is a very new development here. So it looks like people who are building homes
now are building a lot of one bedroom homes. Two bedroom homes. Generally, everything is
kind of the same. Nothing really jumps out here. Three bedroom homes. What you can start
to see with three bedroom homes is that, yeah, it's generally even. But actually, in this
area here, which is a bow, which is near the bow bow's church, which is used to decide
if someone's a traditional East End cockney or not. That's kind of this area, really.
So the cockneys seem to have three bedroom homes, generally. And then four or more. And
what you see here is in the areas where the financial workers live, there's still quite
a lot of four bedroom homes. But in some of these new build areas, there's very, very
few relative to the rest of the area. So let's look at another slightly more famous area.
This is Westminster in central London. And so you can see this is where Hyde Park is.
There's no one living there. Again, this is the population density dataset. And then you've
got an open street map based map just to help with orientation. And then in a future version
of the module, I think I might do some more stuff with open street map. And then if you
look at some of the outer London areas, and this is where I live, you can see like areas
of urban density, but you can also see some very suburban areas where the population density
is lower. This is like where most people are living in houses, basically. And you can also
see green space. So we're nearly finished. I just want to give a massive shout to NB
Dev. It's really good if you use Jupiter. Just check it out. Number one, if you're trying
to do version control on Jupiter notebooks, it helps you with any clashes, any merge conflicts
because it removes the metadata in the JSON that sometimes causes conflicts. If you have
a team of people working on the same notebook, this is a real lifesaver. And also it just
bakes in good practice. So it means that your code gets shared on GitHub really easily.
It helps you or encourages you at least to write good documentation for your team and
the community. It also encourages you to write good tests. And it enables you to publish
modules. So big shout to them. I'd also like to thank Jarek, who has produced a wonderful
PWA for FOSSTEM called Sejourner OX. Do check it out. It's a really good way of looking
at the schedule for FOSSTEM and you can watch the videos with Sejourner OX. And also Ed,
who's going to be giving a really cool talk on OSM and Wikidata. And finally, I'd like
to thank all the council coders everywhere. Thanks for having me.