Snippet - Drones in research #2 - Making drone data open for scientific research
CC Attribution 4.0 International:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Guru, Siddeswara
Rise of drones in the Australian research space - 27-07-17 - Drones in research #2
 Thank you, Nassima, for the introduction and giving the opportunity to talk about this topic. So as Nassima said, I am Sudeshwara Guru. I work for TURN. So today I'm talking about the making drone data open for scientific research. So this is basically the work in progress, you know, what we are exploring and what are the thought process behind that. So just a quick introduction when we say drone. So what is drone? It's nothing but it's aerial vehicle, you know, it may be controlled by the on-ground remote control or onboard computer. So drone, you know, it has got different names, you know, commonly used as a UAVs or the RPAS. I liked RPAS partly because, you know, it's a gender neutral name rather than unmanned. And so predominantly, you know, if it's a rotary wing, it's predominantly called as a drone. And then even the big wings, fixed wings, you know, aerial vehicles are also in this category. Some are pretty big, which is as big as the aircraft and then which over this sky and then gather data. And the rotary wings, the smaller one, you know, it runs in a controlled environment, very fine scale resolution and then runs predominantly in the battery and then get the data. So one of the main thing about the drone is, you know, it become quite cheap to run it. So that's why it's become quite popular in the various applications. So some of the applications is, you know, drone technology was started in the surveillance and then now currently, you know, it's a favorite of the applications in the agriculture and widely in the environmental space as well. I would call it as a scientific research. And then it's not used in the disaster recovery. So especially, you know, it's very difficult to send a human being. So the drones are quite popularly used. So in turn, basically, we use drone in the remote sensing capability aspects of the thing. In this context, you know, we use drone basically to derive fractional cover to measure vegetation species composition, to map vegetation characteristics and then to do some, you know, stock take, the pastoral stock take, and then even to do the survey, flora, fauna survey, kangaroo survey, etc. So a lot of data will be derived from the measurement that is done in the drone. So if you look at, you know, there's a favorite of the vegetation is on partly because, you know, if you take a satellite remote sensing data, the vegetation composition, I think most of the two-third of the time, you know, it has got the cloud cover over the earth. So a lot of data is measured. So drone is a really good technology which runs in the underneath the cloud, below the cloud and take a, you know, good final resolution pictures and then do the, and the researchers can do the analysis. So in this context, you know, the common sensors used in drone, so in drone, so basically drone is a platform and then the sensors are, you know, attached to the drone. So in our case, you know, is a fairly popular things used are the multispectral, hyperspectral, LiDAR, video and the thermal infrared sensors. So from a data management perspective, you know, it's basically the, it's a favorite of a challenge, partly because it's dynamic, both space and by space and time. It's keep on moving at both the spatial and temporal scale. And then the, because of its ability to capture finer scale information, you know, it's a massive amount of data is collected in that one. And the, we should have a capability to basically ingest that and then the process that data and make that data available to the users. And the other part is, you know, there's a favorite of the commercial companies who work in the drone industry and even at the research aspects of the thing, you know, there's always a partnership with the commercial entity where they will run the drone and then, you know, collect the data and give it to the researchers, especially there are consultants, especially in the mining community, in the mining area, etc. And the other thing you should be aware that, you know, you need to, you need a permit to do this from a CASA to operate this and then you should be an operator. You should have a license to operate this as well. And with all this, you know, the identifying a data owner is important. I'll come to that later why it is important. And if you look at the, so for example, to make this data, you know, openly accessible, you know, just, you know, take a bit of a fair principle. So these are the four aspects of the principle, you know, we just see how the drone data will fit in sort of the thing. So if we take the first one, you know, data is adequately described, searchable, and should have an identifier. So just to give a bit of a perspective on the drone, you know, the data is basically the flight plans and then the, you know, the files of the, you know, flight paths and the associated field data generally, especially, you know, in our case. And then the raw data files from the measurement it tooks and then the files and then the log from the flight. And once it's the process, then the, all the derived products as well. And we should also provide the auxiliary files like a QAQC files from the processing. So these are the different files, you know, that is a part of a data publication aspect. So if you look at, you know, all these are the, you know, it's a related, interrelated thing. So all this data should be made available and then the, especially for the provenance aspects of the thing, these, all these data should be made discoverable and then that should be accessible from a user perspective. And then to make this, you know, data, you know, searchable and then the, you know, in file. So basically, you know, there should be a standard to describe this data. So we, so, you know, we use ISO standards. We're not sure whether the, you know, for example, ISO 19115 completely describe everything. Or maybe we may have to, we may have to provide a customized profile of that ISO standard. And then if you make it available as part of a catalog then, you know, it's a discoverable. And then once we put it in a catalog, it would have an identifier and then that's, that looks fine. And then the next aspect is the, you know, the, one of the principles is the data retrieval using open protocol. So if you look at the, the instruments or the sensors that are used in Trone, you know, it provides by default, provides a, as a raw data, you know, from a different file format. And then even at the publication level, if you look at the, you know, the open standard, you know, in the open data policy, they say that, you know, all the file format should be of the open format. So the file format, you know, generally in the, you know, depending on the which of the instruments, you know, it may be a give, geotube, KML, shape file, or LAS file, especially in a, in a point cloud aspect, aspect of the thing. So with, with such a kind of a veracity of the, you know, file format, then the tools must be available as well to translate or manipulate even the analyze the data. And then the, you know, even at the, you know, sometimes it may be worthwhile even to provide a program, you know, the R program, Python program where, you know, they should, they can access this data and then, you know, so that they can run the analysis aspects of the thing if it's too confusing. So for example, if somebody don't know, don't have a clue about, you know, how to access the LAS file, then probably it's worthwhile to provide a program so that, you know, it's embedded, so that, you know, that is embedded in the, embedded in the R program so that they can start writing the program. So that is one of the things, sort of thing. And the next one probably is the, the, the data use vocabulary and qualified reference to other metadata. And then, so we use a, you know, a fair bit of a domain specific vocabulary, like a GCMD is quite popular, especially keyword search first. And then, you know, because the, because the drone and then you have a technology as, as, as comp so fast, probably there may be, you know, it may be in when some of the terminology, you know, incorporated into the vocabulary so that it's, it's accurately represented. With, as I was explaining the, the different data types and, you know, when each of the data type, each of the, you know, files should be referenced and then it should be made as a link and then hopefully as a persistent identifier and then all those should be a queryable as well. And then the, the final, you know, principle is basically data, metadata in domain relevant community standards with clear data usage license. So we use an ISO standard, you know, so either one on five or one three nine in the domain, which may fit well as I said before that, you know, maybe we may have to create a profile to accurately represent the, describe the data. And a lot of the standard is fine. If you want to fit the human actionable, you know, discovery query and access, that means that the human go and click couple of things. Probably we may have to look at the, you know, machine actionable, machine actions accessible, you know, actionable discovery and exploration sort of the thing where, you know, it's a machine to machine query when, then that may be a issue, partly because of the, so much of the interrelationship with the files and etc. It's also depending on the, you know, the, the what kind of file. If we are looking at the source file, then definitely that will be a issue. And then the, talk about the data usage, you know, so what it says is, you know, you should provide as an open license thing, even though, you know, we may say, we provide the all over data and the creative commons by attribution, the identifying the owner is a key, partly because, you know, even at the creative common attribution, the copyright subsist with the data, and then you need to identify who wants the copyright. So for example, if you are, if you are a, if you are a researcher who is using a consultant to collect the data. So technically, if you look at that aspect, the person who collected data is the owner of that data, unless you have a contractual obligation arrangement, make sure that the ownership is transferable to the researcher. So why this is important is that, you know, for the attribution aspect set of the thing, so that if somebody uses your data, they need to know who is attributed as a researcher, you want the attribution to go to you, not the person who just collected the data. And the thing is, if somebody is owner, they can do whatever they want with the data. The IP is with the owner, so what you want is the, as a principal investigator, you want the IP about the data. Still there are a fair bit of a challenge. One of the things, you know, VFAS is the, you know, it's just the amount of data set that is collected. We still struggle to, you know, make that data, the ingest the data into one place, and then they process everything, and then the delivery of that data. So we think that the cloud platform, you know, the managed platform to do everything would be a useful thing. And there are a few initiatives going on as well at the community level who are looking exactly at the similar problems. So one is the OGC domain working group, and the other one is the RDA interest group. So with all the aspect, you know, we want, we don't want to change a complete, you know, data management practice of the turn. So we retrofit everything to the turn data management practice. So a lot of the, whatever the data, you know, we intend to, you know, make the metadata and ISO standards, and then cataloging the geo network, and then that is harvested to the different repositories, you know, discovery portals, so that the data is discoverable, and all the data will have a clear, clear, you know, attribution statement with creative accounts attribution. And then currently we are storing, you know, data in the FTP server and then make that link available. And we are still in the process of, you know, processing a lot of data. But having said that the, the, all the raw files are there. So if somebody is really interested to use that data, they can process the data as well. And the other thing, you know, we are still working on the, you know, the effective, you know, the robust data delivery mechanism as well. So that the user just, you know, come and access the data and get the data. And one of the things, you know, we are working on the, you know, still working progress is the, you know, the portal where, you know, user can come and get the final products. So we collect, you know, we have done a campaign across, you know, love and sites across Australia. So we are working on that. They call this the field data and then we are working on the portal where, you know, all those love and sites data is accessible as well, especially at the LiDAR level. So if you look at the drone, you know, it is quite popular in the research community, you know, especially in the environmental science. There is a massive uptake of the drone technology partly because of the ease of use and then the final resolution of the data it provides. And moving ahead partly, you know, probably it is worthwhile to build an IOT platform to manage the research drone data. It is not at the institution level, at the overall research level. So this probably enable the interconnection of devices basically based on the type of sensory use and what is the application you are running. And that will enable to build a common data platform so that each of the individual does not, you know, repeat the same grappling with the same problem. And if you look at the commercial word, you know, this is already happening. There are a lot of commercial players working in this space. It may be, you know, off the shelf as well. And then there are even a lot of the open source technology already started appearing, especially the management aspects of the thing. So what we want in this one is a researcher has got a platform, they put the sensors, they collect the data, and we need a platform where that data is ingested somewhere, the processing happen, and then the product, derived product is basically available for the third party researcher to analyze that data. So at the individual level, you know, people are working, but they make everything as a pipeline and then provide that as a service to the complete community. We are still working on that one. It's still a work in progress. Thanks for the opportunity to talk about this topic.
 Thank you, Nassima, for the introduction and giving the opportunity to talk about this topic.  So as Nassima said, I am Sudeshwara Guru. I work for TURN.  So today I'm talking about the making drone data open for scientific research.  So this is basically the work in progress, you know, what we are exploring  and what are the thought process behind that.  So just a quick introduction when we say drone. So what is drone?  It's nothing but it's aerial vehicle, you know, it may be controlled by the  on-ground remote control or onboard computer.  So drone, you know, it has got different names, you know, commonly used as a UAVs or the RPAS.  I liked RPAS partly because, you know, it's a gender neutral name rather than unmanned.  And so predominantly, you know, if it's a rotary wing, it's predominantly called as a drone.  And then even the big wings, fixed wings, you know, aerial vehicles are also in this category.  Some are pretty big, which is as big as the aircraft and then which over this sky and then gather data.  And the rotary wings, the smaller one, you know, it runs in a controlled environment, very fine scale resolution  and then runs predominantly in the battery and then get the data.  So one of the main thing about the drone is, you know, it become quite cheap to run it.  So that's why it's become quite popular in the various applications.  So some of the applications is, you know, drone technology was started in the surveillance  and then now currently, you know, it's a favorite of the applications in the agriculture  and widely in the environmental space as well.  I would call it as a scientific research.  And then it's not used in the disaster recovery.  So especially, you know, it's very difficult to send a human being.  So the drones are quite popularly used.  So in turn, basically, we use drone in the remote sensing capability aspects of the thing.  In this context, you know, we use drone basically to derive fractional cover to measure vegetation species composition,  to map vegetation characteristics and then to do some, you know, stock take, the pastoral stock take,  and then even to do the survey, flora, fauna survey, kangaroo survey, etc.  So a lot of data will be derived from the measurement that is done in the drone.  So if you look at, you know, there's a favorite of the vegetation is on partly because, you know,  if you take a satellite remote sensing data, the vegetation composition,  I think most of the two-third of the time, you know, it has got the cloud cover over the earth.  So a lot of data is measured. So drone is a really good technology which runs in the underneath the cloud,  below the cloud and take a, you know, good final resolution pictures and then do the,  and the researchers can do the analysis.  So in this context, you know, the common sensors used in drone, so in drone,  so basically drone is a platform and then the sensors are, you know, attached to the drone.  So in our case, you know, is a fairly popular things used are the multispectral,  hyperspectral, LiDAR, video and the thermal infrared sensors.  So from a data management perspective, you know, it's basically the, it's a favorite of a challenge,  partly because it's dynamic, both space and by space and time.  It's keep on moving at both the spatial and temporal scale.  And then the, because of its ability to capture finer scale information, you know,  it's a massive amount of data is collected in that one.  And the, we should have a capability to basically ingest that and then the process that data  and make that data available to the users.  And the other part is, you know, there's a favorite of the commercial companies  who work in the drone industry and even at the research aspects of the thing, you know,  there's always a partnership with the commercial entity where they will run the drone  and then, you know, collect the data and give it to the researchers,  especially there are consultants, especially in the mining community, in the mining area, etc.  And the other thing you should be aware that, you know, you need to, you need a permit to do this  from a CASA to operate this and then you should be an operator.  You should have a license to operate this as well.  And with all this, you know, the identifying a data owner is important.  I'll come to that later why it is important.  And if you look at the, so for example, to make this data, you know, openly accessible, you know,  just, you know, take a bit of a fair principle.  So these are the four aspects of the principle, you know, we just see how the drone data will fit in  sort of the thing.  So if we take the first one, you know, data is adequately described, searchable,  and should have an identifier.  So just to give a bit of a perspective on the drone, you know, the data is basically the flight plans  and then the, you know, the files of the, you know, flight paths and the associated field data  generally, especially, you know, in our case.  And then the raw data files from the measurement it tooks and then the files and then the log from the flight.  And once it's the process, then the, all the derived products as well.  And we should also provide the auxiliary files like a QAQC files from the processing.  So these are the different files, you know, that is a part of a data publication aspect.  So if you look at, you know, all these are the, you know, it's a related, interrelated thing.  So all this data should be made available and then the, especially for the provenance aspects of the thing,  these, all these data should be made discoverable and then that should be accessible from a user perspective.  And then to make this, you know, data, you know, searchable and then the, you know, in file.  So basically, you know, there should be a standard to describe this data.  So we, so, you know, we use ISO standards.  We're not sure whether the, you know, for example, ISO 19115 completely describe everything.  Or maybe we may have to, we may have to provide a customized profile of that ISO standard.  And then if you make it available as part of a catalog then, you know, it's a discoverable.  And then once we put it in a catalog, it would have an identifier and then that's, that looks fine.  And then the next aspect is the, you know, the, one of the principles is the data retrieval using open protocol.  So if you look at the, the instruments or the sensors that are used in Trone, you know,  it provides by default, provides a, as a raw data, you know, from a different file format.  And then even at the publication level, if you look at the, you know, the open standard, you know,  in the open data policy, they say that, you know, all the file format should be of the open format.  So the file format, you know, generally in the, you know, depending on the which of the instruments,  you know, it may be a give, geotube, KML, shape file, or LAS file,  especially in a, in a point cloud aspect, aspect of the thing.  So with, with such a kind of a veracity of the, you know, file format, then the tools must be available as well  to translate or manipulate even the analyze the data.  And then the, you know, even at the, you know, sometimes it may be worthwhile even to provide a program,  you know, the R program, Python program where, you know, they should, they can access this data  and then, you know, so that they can run the analysis aspects of the thing if it's too confusing.  So for example, if somebody don't know, don't have a clue about, you know, how to access the LAS file,  then probably it's worthwhile to provide a program so that, you know, it's embedded,  so that, you know, that is embedded in the, embedded in the R program  so that they can start writing the program.  So that is one of the things, sort of thing.  And the next one probably is the, the, the data use vocabulary and qualified reference to other metadata.  And then, so we use a, you know, a fair bit of a domain specific vocabulary,  like a GCMD is quite popular, especially keyword search first.  And then, you know, because the, because the drone and then you have a technology as, as, as comp so fast,  probably there may be, you know, it may be in when some of the terminology, you know, incorporated into the vocabulary  so that it's, it's accurately represented.  With, as I was explaining the, the different data types and, you know, when each of the data type,  each of the, you know, files should be referenced and then it should be made as a link  and then hopefully as a persistent identifier and then all those should be a queryable as well.  And then the, the final, you know, principle is basically data, metadata in domain relevant community standards  with clear data usage license.  So we use an ISO standard, you know, so either one on five or one three nine in the domain,  which may fit well as I said before that, you know, maybe we may have to create a profile to accurately represent the,  describe the data.  And a lot of the standard is fine.  If you want to fit the human actionable, you know, discovery query and access,  that means that the human go and click couple of things.  Probably we may have to look at the, you know, machine actionable, machine actions accessible, you know, actionable discovery and exploration  sort of the thing where, you know, it's a machine to machine query when, then that may be a issue,  partly because of the, so much of the interrelationship with the files and etc.  It's also depending on the, you know, the, the what kind of file.  If we are looking at the source file, then definitely that will be a issue.  And then the, talk about the data usage, you know, so what it says is, you know,  you should provide as an open license thing, even though, you know, we may say,  we provide the all over data and the creative commons by attribution, the identifying the owner is a key, partly because,  you know, even at the creative common attribution, the copyright subsist with the data,  and then you need to identify who wants the copyright.  So for example, if you are, if you are a, if you are a researcher who is using a consultant to collect the data.  So technically, if you look at that aspect, the person who collected data is the owner of that data,  unless you have a contractual obligation arrangement, make sure that the ownership is transferable to the researcher.  So why this is important is that, you know, for the attribution aspect set of the thing,  so that if somebody uses your data, they need to know who is attributed as a researcher,  you want the attribution to go to you, not the person who just collected the data.  And the thing is, if somebody is owner, they can do whatever they want with the data.  The IP is with the owner, so what you want is the, as a principal investigator, you want the IP about the data.  Still there are a fair bit of a challenge. One of the things, you know, VFAS is the, you know, it's just the amount of data set that is collected.  We still struggle to, you know, make that data, the ingest the data into one place,  and then they process everything, and then the delivery of that data.  So we think that the cloud platform, you know, the managed platform to do everything would be a useful thing.  And there are a few initiatives going on as well at the community level who are looking exactly at the similar problems.  So one is the OGC domain working group, and the other one is the RDA interest group.  So with all the aspect, you know, we want, we don't want to change a complete, you know, data management practice of the turn.  So we retrofit everything to the turn data management practice.  So a lot of the, whatever the data, you know, we intend to, you know, make the metadata and ISO standards,  and then cataloging the geo network, and then that is harvested to the different repositories, you know, discovery portals,  so that the data is discoverable, and all the data will have a clear, clear, you know, attribution statement with creative accounts attribution.  And then currently we are storing, you know, data in the FTP server and then make that link available.  And we are still in the process of, you know, processing a lot of data.  But having said that the, the, all the raw files are there.  So if somebody is really interested to use that data, they can process the data as well.  And the other thing, you know, we are still working on the, you know, the effective, you know, the robust data delivery mechanism as well.  So that the user just, you know, come and access the data and get the data.  And one of the things, you know, we are working on the, you know, still working progress is the, you know, the portal where, you know,  user can come and get the final products.  So we collect, you know, we have done a campaign across, you know, love and sites across Australia.  So we are working on that. They call this the field data and then we are working on the portal where, you know, all those love and sites data is accessible as well,  especially at the LiDAR level.  So if you look at the drone, you know, it is quite popular in the research community, you know, especially in the environmental science.  There is a massive uptake of the drone technology partly because of the ease of use and then the final resolution of the data it provides.  And moving ahead partly, you know, probably it is worthwhile to build an IOT platform to manage the research drone data.  It is not at the institution level, at the overall research level.  So this probably enable the interconnection of devices basically based on the type of sensory use and what is the application you are running.  And that will enable to build a common data platform so that each of the individual does not, you know, repeat the same grappling with the same problem.  And if you look at the commercial word, you know, this is already happening.  There are a lot of commercial players working in this space.  It may be, you know, off the shelf as well.  And then there are even a lot of the open source technology already started appearing, especially the management aspects of the thing.  So what we want in this one is a researcher has got a platform, they put the sensors, they collect the data,  and we need a platform where that data is ingested somewhere, the processing happen,  and then the product, derived product is basically available for the third party researcher to analyze that data.  So at the individual level, you know, people are working, but they make everything as a pipeline and then provide that as a service to the complete community.  We are still working on that one. It's still a work in progress.  Thanks for the opportunity to talk about this topic.
Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_0_575.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_576_2435.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_2436_3155.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_3156_5015.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_5016_5447.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_5448_5531.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_5532_5735.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_5736_7835.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_7836_8255.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_8256_11015.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_11016_13355.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_13356_14735.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_14736_18407.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_18408_19547.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_19548_22055.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_22056_22427.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_22428_24647.png Siddeswara_Guru_-_Making_drone_data_open_for_scientific_research_qMr1XH4zBGQ_24648_24727.png