My Thirty Years of Spatial Disease Modelling
CC Attribution - NonCommercial - NoDerivatives 4.0 International:
You are free to use, copy, distribute and transmit the work or content in unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Rowlingson, Barry
A review of “Open” in spatial epidemiology over the past thirty years. Nearly thirty years ago I quit my Physics PhD and began to drag GIS, statistics and epidemiology kicking and screaming together, which I’ve been doing ever since. From a world with no WWW, no Linux, and no Open Source, we’ve come to a place where politically-motivated groups attack disease modellers on github over code used to inform government COVID-19 pandemic policy. The talk will introduce some ideas in spatial statistics, epidemiology, and disease surveillance. It will show how the landscape has changed and how Open Source and Open Data are now influencing the debate around disease surveillance in the COVID-19 era. I’ll outline how new technologies for reproducible research can bring about more openness and accountability for disease outbreak epidemiology.
 In a previous life, I was a particle physicist at Lancaster University and I worked on B quark physics in the ALF detector, which is this thing here, and that was in the tunnel where the Large Hadron Collider now is. And in that work, I ended up on a whole load of papers with about 250 other people on things like neutrino species. And then the 90s hit and my PhD funding ran out as these things do. And physics was on the same corridor as the geography department. And the geography department had this Northwest Regional Research Lab, which was full of these computers doing GIS. And that was just down the corridor from where we worked. And this was funded by the UK Research Councils. And I knew the geographers because they were just down the corridor and we used to play cricket with the geographers. So we had these physics versus geography cricket matches. So using the time-honored English method of giving jobs to your cricketing pals, I found myself becoming a researcher in GIS. And that meant I went almost overnight from looking at the smallest, most weekly interacting particles in the universe to looking at perhaps the most impactful events in a human life. This is a map of 1980s leukemia cases in Humbaside. Each of those dots is a leukemia case, or rather each of those triangles is a leukemia case, whereas previously every dot was a neutrino interacting with a detector underground in Switzerland somewhere. And I had a bit of an epiphany at that. And I never look back. And I never got my PhD. Research project that employed me on this was driven by concerns that came up in the 80s about apparent cancer clusters caused by things like power cables or nuclear sites. And there were reports and papers all about those that came out in the 80s. And there was a reaction to that. And the project had two primary investigators who are now great friends, Tony Gatrell, a geographer who had all the GIS expertise, and Peter Diggle, who was a spatial statistician who had all the statistics expertise. And these two got together and got this project going. Now in the 1990s, GIS was very different. The leading GIS were mostly command line driven, and they were great for doing things like maintaining your spatial data in a nice database and doing some standard spatial operations like buffering and intersecting. And well, they could make lovely wall art cartographic charts, but they were lacking in statistical analysis. You could probably do simple tabulations using your database component, but you couldn't do anything too sophisticated. So our plan was to put some sophisticated spatial statistics functions into GIS. Now spatial statistics, I kind of define it as the analysis and modeling of random processes where location matters. And note that it doesn't have to be geospatial. A lot of spatial statistics is done with patterns on microscope slides, for example. And it doesn't have to be two dimensional. It can be one dimensional data where the position on that one dimension matters, or it can be three dimensional data, or even more dimensions if you've got it. The branch of spatial statistics that we were most interested in because of the diseases was point pattern analysis. Now this is where the locations themselves are generated by some interesting process. So for example, the locations of trees in a natural forest, why are the trees there and not there? Or perhaps the positions of cells in a tissue sample are a real big question about patterns of disease. Why are these people here getting a disease and these people not? Now statistics in the 90s, you were kind of in one of two camps. You were either using and abusing the sort of main statistics package. Most of these still survive today. Or you were writing your own programs, which meant writing lots of Fortran and to a certain extent things like C, APL and this. And there was increasing use of a thing called S and the new S environment for statistical data analysis and graphics. And you were kind of in one of these two camps. You were either doing the standard stuff in new settings, or you were writing your own code and developing your new methodologies. And we were looking at how to bring the statistics into the GIS. And there was a lot of discussion about the right way to integrate the functionality between your GIS and your Fortran code. Did you have a loose coupling with separate processes via what we now call an API? Or did you integrate your code tightly into the GIS so it appeared as native GIS functions? I was never convinced this was a useful discussion about this. It went on for some years about this, papers in the conferences and what not. And we ended up taking this diagram and basically flipping it inside out so that we added GIS and spatial statistics to the statistics package, which was S plus. This was a proprietary implementation of a non open source language. And we added our code into that and gave it some simple GIS functions. And this became a paper in 1993, which documented splanks, which stood for spatial point pattern analysis code in S plus, but also has Lancaster hidden in it. And this paper was 10 pages of text and examples. And after that came 19 pages purely of the function documentation. Most of the paper in the journal was the function documentation, the manual pages for the code that we'd written. This wasn't really uncommon at the time. The very next article in that volume of the journal was this one called C programs for controlling poor and bulk volume compressibility experiments. And this was actually three pages of explanatory text and a nice flow chart. And what then followed was 13 pages of C code. Okay. And this is when journals were not electronic. At the time, most people saw this on paper. And if they wanted to use this code, they had to type it in. Now, the author very nicely gave his address. You could probably write to him to get his email because no one seemed to give their emails out in these journals. There's no phone number as well. So you couldn't find him. You could go and knock on his front door. And I thought, well, I wonder what I wonder. Just the other day, I thought, I wonder what this place looks like. 149 11th Street, Del Mar, California. So I went to Google Street View and look at that. Amazing. You could have knocked on that door and said, can I have your C code, please? And then you could have gone for a dip in the Pacific. A wonderful, wonderful spot. Nice place. Anyway, there's some other differences between this and modern publications. There's no mention in this of licensing. There's no sample data. There's no unit tests. There's no example runs. And it also seems that the journal has got the copyright on the code as well. And again, this was common back in the 90s. This was how things were done. What we did with Splanks is we at least made it, we offered it out to distribute it on floppy disk for a small fee of 60 pounds. I don't know what happened to the cash. I think it's resting in someone's account. And in case you've never seen one for a while, this is what a floppy disk looks like. I couldn't find a floppy disk I wanted, an image of a floppy disk on Unsplash or anywhere I can get free images. So I looked in my desk drawer yesterday as I was writing this slide. And I found a box. I found a box of floppy drives, floppy disks, right? And I thought, which one, which one shall I use? And I picked this one because I thought, okay, I'll show a Microsoft DOS disk from 1990s. And when I looked at it closely, I noticed it actually had Splanks penciled in on it. This was an old Splanks disk from about then. And I haven't got a floppy drive to actually tell, but this has actually got a copy of Splanks on it at the moment. But it was really weird. This happened yesterday. I'll point out, if you paid 60 quid for a disk, we didn't just recycle Microsoft disks. You did get a brand new disk back in 1993 with Splanks on it. This distribution really only went on for a couple of years because other delivery mechanisms became available. Once email addresses were more ubiquitous, people could be emailed data and programs in encoded formats, or they could connect using FTP. And then eventually the shiny new world of the worldwide web came in, and everyone was using that to exchange everything. So what did we have for spatial epidemiology back then for disease analysis? There were the existing well-known statistical methods, which were based on things like likelihood and Monte Carlo randomness. There were new statistical methods coming in with things like Bayesian techniques and Markov chain Monte Carlo, which meant you could analyze very different problems. And there were also a whole swath of what I think of as geographical methods, things like the local indicators of spatial alter correlation, the Moran, Zyker, Fission, and other characters like that. And there was also the geographical analysis machine, things that were coming in from Stan Openshore and the team in Leeds. And these were looked at with some interest as techniques for looking at disease clusters. Openshore's geographical analysis machine got a lot of interest back then, partly because it was very graphical and you could watch it draw these circles of where it thought the clusters were. And geographers got very excited by this, statisticians less so because it was testing so many thousands of hypotheses. And looking at it now from an open source point of view, I thought, let's go and see if I can find the Openshore GAM code. I discovered that it was never distributed because it was not easily run and its dependency on a supercomputer severely restricted its usefulness. I did a computation yesterday that if Openshore's GAM took a month to run on a supercomputer back then, it would run in maybe an hour or so on a modern CPU like you've got on your desk in front of you now and if you converted the code around the CPU, you could run it in the blink of an eye, I think. So I don't think supercomputer reliance today is an excuse for not distributing your code. So in summary, in the 90s to sort of turn of millennium, epidemiological analysis was constrained because the data was small and rare and hard to get. Software was specialist and in many cases very slow to do anything useful. And the methods that we had were either sort of standard and not quite appropriate or they were developing and not really finely tuned to what we were trying to do. That's where we were around about the 90s and the turn of the millennium. We will remember what happened then. The millennium bug hit and everything crashed. No. Okay, I'll step us through now. I'm going to whiz through 20 years of history. Okay. And this is a bit like... Five minutes to go, Barry. Five minutes. I think I have 20 minutes on my clock. Okay. I've had 10. Okay. Anyway, I'm going to whiz through all these years of history with a couple of little points and then we get to 2020. Okay. If you want an idea of the applications of spatial statistics in health today, you can go to our group webpage and look at the projects we have on. And none of this could be achieved without the advances in open data, open software, and communications that we now have. For open data, we now have easy access to census data and census derived data. We can get NHS health data, which for disease is mostly rates or counts in small areas rather than point locations because of anonymity. We also have big data and the tools to deal with it. We have APIs for accessing the latest versions of the data we want when we need it. And there's also increasingly huge amounts of crowdsource geographic information. We have the hardware to deal with all this now, but we don't have to own it. We can use cloud computing to just rent the computers that we need. And we can even rent databases as software as a service and take our data to that and serve it to our applications. There's a growing increase in the understanding that research software is important and an understanding that publicly funded software should be open licensed for the public good. There's a recognition that science has to be reproducible to be science and that software is part of that reproducibility pipeline. And that means that closed source software cannot be part of science. This is very much driven by the Software Sustainability Institute in this country. That idea that science has to be open source worries me with things like Google Earth Engine. A lot of scientists using this now, they're unconcerned that tomorrow Google could pull this consign this to their graveyard. If you look at killedbygoogle.com, they have a very sad graveyard of all the projects that Google have killed, including a lot of geospatial projects like Mapmaker and Fluved Trends and Fusion Tables which stored a lot of spatial data at the time. So be aware of what Google may give and take away. So let's talk about coronavirus. We have to. We have to. The government has their emergency response organised by SAGE at the top level which fan out into the subgroups including SPYM which is the group tasked with modelling. Now I know people on SPYM, we have people on SPYM in our group and they have access to the line listing of COVID-19 cases, individual case data. That can't be made public for any of these reasons. But they've had a big fuss about transparency with the modelling software that they've been using, particularly Neil Ferguson's group Imperial who the newspapers, the press discovered that software has bugs and they piled in with comments like it was bugs in software of meant that we've had to close down the country. Well, that's ridiculous. We have to do this anyway. And what's the significance of the bugs in software? We didn't really know because the code was unpublished. Now eventually a version of the code was made available and it was ripped to shreds by people who didn't really understand what research was all about and what research computing had to deal with. Neil has tried to defend this and I'm not sure any of the reasons that were kind of purported for him not publishing the software were really valid. I think it was just not published because there was no need to publish it. No one was actually using it. I think it was a piece of academic code. It had the potential to affect policy but not until this pandemic did it actually do that and then it became important that this code could be seen to be correct because of the influence it had on so many lives. Several trillion pounds if you believe some commentators. Now the government was a bit better with its openness for the data. The first sort of dashboards for COVID-19 data had no link to the data and if you wanted to get the data, the case counts and the death counts, you had to build a scraper to get the data from the tables and the maps and the website requests and you could do that and some people did that, several people did that independently and these things were open sourced and they were used and people were building dashboards. Then the government produced a version 2 which actually had a link that you could click which would then construct the data and then download it but that again couldn't be scripted very easily. That was all done inside the JavaScript and eventually the third version gave you a static link that you could click and get a CSV of cases and that enables people to build all these dashboards and there are a heck of a lot of dashboards out there doing things like this and we are very interested in modeling, predicting the data and looking at areas of the country that are sort of perhaps should be further locked down or less locked down and to that extent the thing I've been working on recently has been a map of looking at the probability that the reproduction rate, whether the epidemic is increasing or decreasing, the probability that that rate is over or under 1 and I've tried to do this in conjunction with the Lancaster-Spy-M committee members in a fully open and reproducible way and the way I've done that is that all the code is in a GitLab repository and I've leveraged the continuous integration process which is often used by developers in order to run tests on their code every time you make a change to the code, use that to run an analysis process. So the continuous integration process which runs off GitLab runner pulls the latest data from the UK website, fits the model of the reproduction rate, writes a CSV and a GeoJSON file for the data so we can plot maps and charts and this is pushed to a GitLab page repository so you've then got a live updated web page driven by an automatic process that's also open source through a repository where all the software methods are open source and this is almost fully reproducible, there are one or two things in there that can't really be done and the sort of rush and panic of all these dashboards and methods brings me back to something that Professor Diggle said very early in the epidemic which was that the best time to work on pandemic modelling is when there isn't a pandemic on and I think he added but nobody wants to do that. Well I mean I want to do that, I want to be doing this in 2021 when there isn't a pandemic on and I hope to see you all perhaps in person when there isn't a pandemic next year. So everyone I want, everyone to stay safe and flatten the curve, I want to see all blue in this chart in this map, this is where R is definitely under one and this was a few days ago, this is actually live and real, that's what I want to see, I want to see a nice blue map and I want to see everyone back at FOSPAG UK 2021 somewhere next year. Thank you very much. Wonderful, thank you very much Barry, that was a great presentation and it was great to see how you've kind of utilised some of the Travis CI stuff for analysis rather than code testing as well, I've not seen that before so that's a really fascinating implementation. If you do have any questions for Barry please do continue to send them through the chat and I'll relay them. I've got one, a kind of quite general one, you mentioned in passing about the kind of pressure, issues of having open source software and many academics also using things like Google Earth Engine and the risk that Google could pull Google Earth Engine, that's a great comment. What would you say to someone who said that the reason people use Google Earth Engine is because it's so easy to use and Google have kind of transformed the ease of use of geospatial stuff and they might say that the open source stuff is quite hard to use whereas Google Earth Engine is much easier. I think if you're an academic doing science, I know it's hard to take the road less travelled but the more people that use the open source solutions, the more people are there to develop and improve those solutions and it's a mutual win. I can't improve Google Earth Engine, I can improve QGIS, I can improve GRAS GIS if I want to if I was more of a C++ programme and I think the idea that science should be open and reproducible and public is absolutely fundamental and people don't even realise perhaps that using proprietary software is in my opinion taking them out of the realm of science because we've been doing it for so long, we were doing it in 1993, many people are still doing it now, if we'd taken this path more strongly 20 years ago we'd been a different place now maybe Google Earth Engine or something like it would be an open source piece of infrastructure sitting on top of an abstract cloud system that wasn't tied to Google or anyone else but I can dream, I'm an academic, I have that freedom. That's great and it's definitely easier to use the systems we have now than it was 20 years ago I would say from what I've seen. We've got some more questions, so from Andy Murdoch, what do you think can be done to harmonise data better globally, COVID-19 and sharing the difficulties and also the politics in what is released? Yeah, I think it's hard enough to harmonise data within administrative units. The previous talk that I went to about the Romanian dashboard, the Romanian government apparently stopped publishing the data at a small level and then went back to it and how you would compare that data against the way the UK has published its data or the way that the US, the definition of a case has changed in the UK. I think you need local knowledge so you need local people, I'm not sure. It's a big effort to do this globally and I think Johns Hopkins have done a lot of work on this. The Johns Hopkins dashboard has brought together a lot of global data and I think that's, maybe there are some lessons that can be learnt from that. Great, thank you. Another question from Mils Dencek, apologies about the pronunciation. Barry mentioned earlier that on spatial modelling doesn't need to mean geospatial. We'd love to know. There's been some notable, interesting creative uses you've seen for this. You talked about microscope samples, so it'd be great to hear a bit more about that. Yeah, well a very recent one that I've been working on with a colleague was modelling kidney, the structure of kidneys because the different types of cell in the kidney, the human kidney and the way the blood and the plasma in the blood flows through a kidney and the way that they've sort of extracted, eventually urine or whatever the product of the kidney cells is. And we've been looking at how the 3D structure of the cells and how all the cells kind of relate to each other with the idea of building a 3D model, a sort of stochastic 3D model of how a kidney, a small section of kidney works in 3 dimensions. That's very new. But one of the main classic examples of spatial statistics in the textbooks is looking at the patterns of cells in a rabbit's eye and trying to figure out whether the two different cells have developed independently or whether they develop in pairs and that kind of brings back to how the retina develops in the developing animal. And back then actually data like that was more common because it was much easier to get consent from a dead rabbit than it was from the people in the leukemia study in Humbuside that I kind of started this talk of.
 In a previous life, I was a particle physicist at Lancaster University and I worked on B  quark physics in the ALF detector, which is this thing here, and that was in the tunnel  where the Large Hadron Collider now is.  And in that work, I ended up on a whole load of papers with about 250 other people on things  like neutrino species.  And then the 90s hit and my PhD funding ran out as these things do.  And physics was on the same corridor as the geography department.  And the geography department had this Northwest Regional Research Lab, which was full of these  computers doing GIS.  And that was just down the corridor from where we worked.  And this was funded by the UK Research Councils.  And I knew the geographers because they were just down the corridor and we used to play  cricket with the geographers.  So we had these physics versus geography cricket matches.  So using the time-honored English method of giving jobs to your cricketing pals, I found  myself becoming a researcher in GIS.  And that meant I went almost overnight from looking at the smallest, most weekly interacting  particles in the universe to looking at perhaps the most impactful events in a human life.  This is a map of 1980s leukemia cases in Humbaside.  Each of those dots is a leukemia case, or rather each of those triangles is a leukemia  case, whereas previously every dot was a neutrino interacting with a detector underground in  Switzerland somewhere.  And I had a bit of an epiphany at that.  And I never look back.  And I never got my PhD.  Research project that employed me on this was driven by concerns that came up in the  80s about apparent cancer clusters caused by things like power cables or nuclear sites.  And there were reports and papers all about those that came out in the 80s.  And there was a reaction to that.  And the project had two primary investigators who are now great friends, Tony Gatrell, a  geographer who had all the GIS expertise, and Peter Diggle, who was a spatial statistician  who had all the statistics expertise.  And these two got together and got this project going.  Now in the 1990s, GIS was very different.  The leading GIS were mostly command line driven, and they were great for doing things like  maintaining your spatial data in a nice database and doing some standard spatial operations  like buffering and intersecting.  And well, they could make lovely wall art cartographic charts, but they were lacking  in statistical analysis.  You could probably do simple tabulations using your database component, but you couldn't  do anything too sophisticated.  So our plan was to put some sophisticated spatial statistics functions into GIS.  Now spatial statistics, I kind of define it as the analysis and modeling of random processes  where location matters.  And note that it doesn't have to be geospatial.  A lot of spatial statistics is done with patterns on microscope slides, for example.  And it doesn't have to be two dimensional.  It can be one dimensional data where the position on that one dimension matters, or it can be  three dimensional data, or even more dimensions if you've got it.  The branch of spatial statistics that we were most interested in because of the diseases  was point pattern analysis.  Now this is where the locations themselves are generated by some interesting process.  So for example, the locations of trees in a natural forest, why are the trees there and  not there?  Or perhaps the positions of cells in a tissue sample are a real big question about patterns  of disease.  Why are these people here getting a disease and these people not?  Now statistics in the 90s, you were kind of in one of two camps.  You were either using and abusing the sort of main statistics package.  Most of these still survive today.  Or you were writing your own programs, which meant writing lots of Fortran and to a certain  extent things like C, APL and this.  And there was increasing use of a thing called S and the new S environment for statistical  data analysis and graphics.  And you were kind of in one of these two camps.  You were either doing the standard stuff in new settings, or you were writing your own  code and developing your new methodologies.  And we were looking at how to bring the statistics into the GIS.  And there was a lot of discussion about the right way to integrate the functionality between  your GIS and your Fortran code.  Did you have a loose coupling with separate processes via what we now call an API?  Or did you integrate your code tightly into the GIS so it appeared as native GIS functions?  I was never convinced this was a useful discussion about this.  It went on for some years about this, papers in the conferences and what not.  And we ended up taking this diagram and basically flipping it inside out so that we added GIS  and spatial statistics to the statistics package, which was S plus.  This was a proprietary implementation of a non open source language.  And we added our code into that and gave it some simple GIS functions.  And this became a paper in 1993, which documented splanks, which stood for spatial point pattern  analysis code in S plus, but also has Lancaster hidden in it.  And this paper was 10 pages of text and examples.  And after that came 19 pages purely of the function documentation.  Most of the paper in the journal was the function documentation, the manual pages for the code  that we'd written.  This wasn't really uncommon at the time.  The very next article in that volume of the journal was this one called C programs for  controlling poor and bulk volume compressibility experiments.  And this was actually three pages of explanatory text and a nice flow chart.  And what then followed was 13 pages of C code.  Okay.  And this is when journals were not electronic.  At the time, most people saw this on paper.  And if they wanted to use this code, they had to type it in.  Now, the author very nicely gave his address.  You could probably write to him to get his email because no one seemed to give their emails  out in these journals.  There's no phone number as well.  So you couldn't find him.  You could go and knock on his front door.  And I thought, well, I wonder what I wonder.  Just the other day, I thought, I wonder what this place looks like.  149 11th Street, Del Mar, California.  So I went to Google Street View and look at that.  Amazing.  You could have knocked on that door and said, can I have your C code, please?  And then you could have gone for a dip in the Pacific.  A wonderful, wonderful spot.  Nice place.  Anyway, there's some other differences between this and modern publications.  There's no mention in this of licensing.  There's no sample data.  There's no unit tests.  There's no example runs.  And it also seems that the journal has got the copyright on the code as well.  And again, this was common back in the 90s.  This was how things were done.  What we did with Splanks is we at least made it, we offered it out to distribute it on  floppy disk for a small fee of 60 pounds.  I don't know what happened to the cash.  I think it's resting in someone's account.  And in case you've never seen one for a while, this is what a floppy disk looks like.  I couldn't find a floppy disk I wanted, an image of a floppy disk on Unsplash or anywhere  I can get free images.  So I looked in my desk drawer yesterday as I was writing this slide.  And I found a box.  I found a box of floppy drives, floppy disks, right?  And I thought, which one, which one shall I use?  And I picked this one because I thought, okay, I'll show a Microsoft DOS disk from 1990s.  And when I looked at it closely, I noticed it actually had Splanks penciled in on it.  This was an old Splanks disk from about then.  And I haven't got a floppy drive to actually tell, but this has actually got a copy of  Splanks on it at the moment.  But it was really weird.  This happened yesterday.  I'll point out, if you paid 60 quid for a disk, we didn't just recycle Microsoft disks.  You did get a brand new disk back in 1993 with Splanks on it.  This distribution really only went on for a couple of years because other delivery mechanisms  became available.  Once email addresses were more ubiquitous, people could be emailed data and programs  in encoded formats, or they could connect using FTP.  And then eventually the shiny new world of the worldwide web came in, and everyone was  using that to exchange everything.  So what did we have for spatial epidemiology back then for disease analysis?  There were the existing well-known statistical methods, which were based on things like  likelihood and Monte Carlo randomness.  There were new statistical methods coming in with things like Bayesian techniques and  Markov chain Monte Carlo, which meant you could analyze very different problems.  And there were also a whole swath of what I think of as geographical methods, things  like the local indicators of spatial alter correlation, the Moran, Zyker, Fission, and  other characters like that.  And there was also the geographical analysis machine, things that were coming in from Stan  Openshore and the team in Leeds.  And these were looked at with some interest as techniques for looking at disease clusters.  Openshore's geographical analysis machine got a lot of interest back then, partly because  it was very graphical and you could watch it draw these circles of where it thought  the clusters were.  And geographers got very excited by this, statisticians less so because it was testing  so many thousands of hypotheses.  And looking at it now from an open source point of view, I thought, let's go and see  if I can find the Openshore GAM code.  I discovered that it was never distributed because it was not easily run and its dependency  on a supercomputer severely restricted its usefulness.  I did a computation yesterday that if Openshore's GAM took a month to run on a supercomputer  back then, it would run in maybe an hour or so on a modern CPU like you've got on your  desk in front of you now and if you converted the code around the CPU, you could run it  in the blink of an eye, I think.  So I don't think supercomputer reliance today is an excuse for not distributing your code.  So in summary, in the 90s to sort of turn of millennium, epidemiological analysis was  constrained because the data was small and rare and hard to get.  Software was specialist and in many cases very slow to do anything useful.  And the methods that we had were either sort of standard and not quite appropriate or they  were developing and not really finely tuned to what we were trying to do.  That's where we were around about the 90s and the turn of the millennium.  We will remember what happened then.  The millennium bug hit and everything crashed.  No.  Okay, I'll step us through now.  I'm going to whiz through 20 years of history.  Okay.  And this is a bit like...  Five minutes to go, Barry.  Five minutes.  I think I have 20 minutes on my clock.  Okay.  I've had 10.  Okay.  Anyway, I'm going to whiz through all these years of history with a couple of little points  and then we get to 2020.  Okay.  If you want an idea of the applications of spatial statistics in health today, you can  go to our group webpage and look at the projects we have on.  And none of this could be achieved without the advances in open data, open software,  and communications that we now have.  For open data, we now have easy access to census data and census derived data.  We can get NHS health data, which for disease is mostly rates or counts in small areas rather  than point locations because of anonymity.  We also have big data and the tools to deal with it.  We have APIs for accessing the latest versions of the data we want when we need it.  And there's also increasingly huge amounts of crowdsource geographic information.  We have the hardware to deal with all this now, but we don't have to own it.  We can use cloud computing to just rent the computers that we need.  And we can even rent databases as software as a service and take our data to that and  serve it to our applications.  There's a growing increase in the understanding that research software is important and an  understanding that publicly funded software should be open licensed for the public good.  There's a recognition that science has to be reproducible to be science and that software  is part of that reproducibility pipeline.  And that means that closed source software cannot be part of science.  This is very much driven by the Software Sustainability Institute in this country.  That idea that science has to be open source worries me with things like Google Earth Engine.  A lot of scientists using this now, they're unconcerned that tomorrow Google could pull  this consign this to their graveyard.  If you look at killedbygoogle.com, they have a very sad graveyard of all the projects that  Google have killed, including a lot of geospatial projects like Mapmaker and Fluved Trends and  Fusion Tables which stored a lot of spatial data at the time.  So be aware of what Google may give and take away.  So let's talk about coronavirus.  We have to.  We have to.  The government has their emergency response organised by SAGE at the top level which fan  out into the subgroups including SPYM which is the group tasked with modelling.  Now I know people on SPYM, we have people on SPYM in our group and they have access to  the line listing of COVID-19 cases, individual case data.  That can't be made public for any of these reasons.  But they've had a big fuss about transparency with the modelling software that they've been  using, particularly Neil Ferguson's group Imperial who the newspapers, the press discovered  that software has bugs and they piled in with comments like it was bugs in software of meant  that we've had to close down the country.  Well, that's ridiculous.  We have to do this anyway.  And what's the significance of the bugs in software?  We didn't really know because the code was unpublished.  Now eventually a version of the code was made available and it was ripped to shreds by people  who didn't really understand what research was all about and what research computing  had to deal with.  Neil has tried to defend this and I'm not sure any of the reasons that were kind of  purported for him not publishing the software were really valid.  I think it was just not published because there was no need to publish it.  No one was actually using it.  I think it was a piece of academic code.  It had the potential to affect policy but not until this pandemic did it actually do  that and then it became important that this code could be seen to be correct because of  the influence it had on so many lives.  Several trillion pounds if you believe some commentators.  Now the government was a bit better with its openness for the data.  The first sort of dashboards for COVID-19 data had no link to the data and if you wanted  to get the data, the case counts and the death counts, you had to build a scraper to get  the data from the tables and the maps and the website requests and you could do that  and some people did that, several people did that independently and these things were open  sourced and they were used and people were building dashboards.  Then the government produced a version 2 which actually had a link that you could click which  would then construct the data and then download it but that again couldn't be scripted very  easily.  That was all done inside the JavaScript and eventually the third version gave you a static  link that you could click and get a CSV of cases and that enables people to build all  these dashboards and there are a heck of a lot of dashboards out there doing things like  this and we are very interested in modeling, predicting the data and looking at areas of  the country that are sort of perhaps should be further locked down or less locked down  and to that extent the thing I've been working on recently has been a map of looking at the  probability that the reproduction rate, whether the epidemic is increasing or decreasing,  the probability that that rate is over or under 1 and I've tried to do this in conjunction  with the Lancaster-Spy-M committee members in a fully open and reproducible way and  the way I've done that is that all the code is in a GitLab repository and I've leveraged  the continuous integration process which is often used by developers in order to run tests  on their code every time you make a change to the code, use that to run an analysis process.  So the continuous integration process which runs off GitLab runner pulls the latest data  from the UK website, fits the model of the reproduction rate, writes a CSV and a GeoJSON  file for the data so we can plot maps and charts and this is pushed to a GitLab page  repository so you've then got a live updated web page driven by an automatic process that's  also open source through a repository where all the software methods are open source and  this is almost fully reproducible, there are one or two things in there that can't  really be done and the sort of rush and panic of all these dashboards and methods brings  me back to something that Professor Diggle said very early in the epidemic which was  that the best time to work on pandemic modelling is when there isn't a pandemic on and I think  he added but nobody wants to do that.  Well I mean I want to do that, I want to be doing this in 2021 when there isn't a pandemic  on and I hope to see you all perhaps in person when there isn't a pandemic next year.  So everyone I want, everyone to stay safe and flatten the curve, I want to see all blue  in this chart in this map, this is where R is definitely under one and this was a few  days ago, this is actually live and real, that's what I want to see, I want to see a nice blue  map and I want to see everyone back at FOSPAG UK 2021 somewhere next year.  Thank you very much.  Wonderful, thank you very much Barry, that was a great presentation and it was great  to see how you've kind of utilised some of the Travis CI stuff for analysis rather than  code testing as well, I've not seen that before so that's a really fascinating implementation.  If you do have any questions for Barry please do continue to send them through the chat  and I'll relay them.  I've got one, a kind of quite general one, you mentioned in passing about the kind of  pressure, issues of having open source software and many academics also using things like  Google Earth Engine and the risk that Google could pull Google Earth Engine, that's a great  comment.  What would you say to someone who said that the reason people use Google Earth Engine  is because it's so easy to use and Google have kind of transformed the ease of use of  geospatial stuff and they might say that the open source stuff is quite hard to use  whereas Google Earth Engine is much easier.  I think if you're an academic doing science, I know it's hard to take the road less travelled  but the more people that use the open source solutions, the more people are there to develop  and improve those solutions and it's a mutual win.  I can't improve Google Earth Engine, I can improve QGIS, I can improve GRAS GIS if I  want to if I was more of a C++ programme and I think the idea that science should be open  and reproducible and public is absolutely fundamental and people don't even realise perhaps that  using proprietary software is in my opinion taking them out of the realm of science because  we've been doing it for so long, we were doing it in 1993, many people are still doing it  now, if we'd taken this path more strongly 20 years ago we'd been a different place  now maybe Google Earth Engine or something like it would be an open source piece of infrastructure  sitting on top of an abstract cloud system that wasn't tied to Google or anyone else  but I can dream, I'm an academic, I have that freedom.  That's great and it's definitely easier to use the systems we have now than it was 20  years ago I would say from what I've seen.  We've got some more questions, so from Andy Murdoch, what do you think can be done to  harmonise data better globally, COVID-19 and sharing the difficulties and also the politics  in what is released?  Yeah, I think it's hard enough to harmonise data within administrative units.  The previous talk that I went to about the Romanian dashboard, the Romanian government  apparently stopped publishing the data at a small level and then went back to it and  how you would compare that data against the way the UK has published its data or the way  that the US, the definition of a case has changed in the UK.  I think you need local knowledge so you need local people, I'm not sure.  It's a big effort to do this globally and I think Johns Hopkins have done a lot of work  on this.  The Johns Hopkins dashboard has brought together a lot of global data and I think that's, maybe  there are some lessons that can be learnt from that.  Great, thank you.  Another question from Mils Dencek, apologies about the pronunciation.  Barry mentioned earlier that on spatial modelling doesn't need to mean geospatial.  We'd love to know.  There's been some notable, interesting creative uses you've seen for this.  You talked about microscope samples, so it'd be great to hear a bit more about that.  Yeah, well a very recent one that I've been working on with a colleague was modelling  kidney, the structure of kidneys because the different types of cell in the kidney, the  human kidney and the way the blood and the plasma in the blood flows through a kidney  and the way that they've sort of extracted, eventually urine or whatever the product of  the kidney cells is.  And we've been looking at how the 3D structure of the cells and how all the cells kind of  relate to each other with the idea of building a 3D model, a sort of stochastic 3D model  of how a kidney, a small section of kidney works in 3 dimensions.  That's very new.  But one of the main classic examples of spatial statistics in the textbooks is looking at  the patterns of cells in a rabbit's eye and trying to figure out whether the two different  cells have developed independently or whether they develop in pairs and that kind of brings  back to how the retina develops in the developing animal.  And back then actually data like that was more common because it was much easier to get  consent from a dead rabbit than it was from the people in the leukemia study in Humbuside  that I kind of started this talk of.
FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_0_107.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_108_191.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_192_1463.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_1464_1871.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_1872_2723.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_2724_3167.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_3168_3695.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_3696_4739.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_4740_5519.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_5520_6371.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_6372_7355.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_7356_9371.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_9372_9815.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_9816_10091.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_10092_10259.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_10260_10655.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_10656_13559.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_13560_14327.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_14328_14867.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_14868_17843.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_17844_18671.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_18672_19139.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_19140_19451.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_19452_20819.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_20820_21539.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_21540_22355.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_22356_22463.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_22464_23135.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_23136_25127.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_25128_26303.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_26304_27047.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_27048_29207.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_29208_29543.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_29544_30095.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_30096_30143.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_30144_30527.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_30528_30587.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_30588_39347.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_39348_39395.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_39396_39491.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_39492_39575.png FOSS4GUKOnline_2020_Sentinel_Rowlingson_5a2DiDnUTfM_39576_39611.png