GeoServer 2.17 Updates!
CC Attribution - NonCommercial - NoDerivatives 4.0 International:
You are free to use, copy, distribute and transmit the work or content in unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Aime, Andrea
Garnett, Jody
What is new with GeoServer 2.17! A cheerful tour of new extensions, the latest research and development, improved map rendering and more! The GeoServer team shares what is new with GeoServer 2.17! GeoServer is a popular OSGeo project for publishing your geospatial data using industry standards for vector, raster and mapping. To support online presentation we are scaling back our “state of talk” to strictly focus on new features available in GeoServer 2.17. Attend this presentation for a cheerful tour of new extensions, the latest research and development, improved map rendering and more!
 Excellent. Well, welcome back everyone and thanks very much for hosting this event and facilitating Anthony. We really appreciate it. Usually for a Phosphor G conference, Andrea and myself do a state of geo server update which talks a little bit bigger picture about what's going on with the project as a whole. But it's such a busy project that stresses us out. So today for this online session, we're being a little bit more relaxed and we're just going to talk about the new fun things that are available in the latest geo server 2.17 release. So just starting off with a quick overview. We released geo server a couple times a year doing major releases of 2016 and 2017 and 2018. So for the geo server 2.17 release, it was initially released in April 2020. This is a few months late because we were changing build infrastructure and it was released alongside GeoTools 23.0. This is the first stable release of the 2017 series. Just last week, I released the first point release there to 0.17.1. How this works is geo server is going to get four more releases. Each release series lasts for a year. So the first releases are considered stable. They get a few new features are getting backported and then it goes into a maintenance mode where only bug fixes and security updates are backported. This release runs on Java 8 or Java 11. We really recommend using openJBK version 8. So the other thing that's happening here is geo server in addition to that core application has a series of optional installs or extensions. You can see some of them on the screen ahead of you. But once that plays to star next to are new for this release. So we're going to talk a little bit about the mapbox starling extension and also this resource browser tool extension. We also have a kind of an experimental R&D play area called community modules. We've got a couple of community modules that are kind of active R&D that we'd like to talk about. So flat geo buffer, GSR plugin and the OGC API plugin are new R&D projects this release. So with that intro, I'm going to dive right into the first batch of updates. And by that, I mean I'm going to hand the microphone over to Andrea. Hello. Yeah, thank you. So yeah, we got a bunch of styling improvements, especially in the mapbox style plugin. Jodi pushed it from a community to extension status, which means you will find it among the releases as a numbered module, as opposed to being a community module that you can only find in the nightly builds. The use case for this module, the mapbox style plugin is to reuse the the mapbox GL styles on the server side as well. For those cases where you want to have an hybrid architecture where you publish mapbox vector tiles, but we want also retain the ability to publish PNGs so that you have a fully standard OGC service, but also allow to use vector ties on the side. Next slide. Yeah, I'm just going to interject there. This was originally a community module started by boundless. And then my new employer GeoCat has allowed me some time to pick this up and take this R&D project and domesticate it and include it in the release. Yep. A few months ago, as part of a contract with GeoSolutions, we actually did a bunch of improvements onto the translation because, keep in mind that GeoServer at its core basically only speaks SLD. Every other language gets translated down to an object model which is strongly inspired by SLD itself. So the translation from mapbox to the subject model was lacking a few features and though was not mimicking correctly a few of the behaviors of mapbox style, we did a bunch of improvements and corrected a number of issues. Next slide. And right now, if you take the open map tiles, GL Bright, and mapbox vector, mapbox GL style, you cannot fully run it on GeoServer and produce an output which looks by all intents and purposes just like the one you would get on the client side. Of course, there are more small differences because the leveling engine is not the same but I mean, I guess that if I put you in front of this map, you wouldn't have been able to tell that it was rendered server side but you server. Next. In order to make things work, we had to make a number of map-engining improvements such as, for example, background field, as you say, symbols with labels, func-ranking, and so on. The mapbox GL work was not the only driver for improvements. There were also other improvements that were unrelated to it and we are going to have a look at them one by one. So next. This is derived from mapbox GL. In mapbox GL, you can set the background color. In SLD, you cannot because typically the background color comes from the get map request. However, we added an extension to the SLD. So it's a vendor extension for GeoServer that allows you to specify an SLD background element to specify a color or a fill. Like, for example, you could have a PNG with the waves to represent the water as the background and that would also work. Next. Another thing that we noticed is that when mapbox GL styles works with icons with labels, either both the icon and the label show up or none of them does. In GeoServer, typically you represent an icon with a label below it as a point symbolizer and a text symbolizer. But being two independent elements means that one shows up and the other doesn't some time. So we already have this ability to put a background image below a label to do text, do, road label shields, but it was always centering the shield behind the label in the same position and we wanted to actually push the label below. So we added a new vendor option, Graphic Placement, that allows to offset the symbol and the label independently, getting this result of having the bike and the label below it, but making it so that they either appear both of them or none of them. And this is shown in the next slide. So you can see the giant store Notre Dame on the Ildeperie and as I zoom in there are other labels that start showing up, which interfere with the bike shop and the bike shop label disappears and as a result also the point. You might want or not want this behavior, the thing is now it's possible before it wasn't. Next, another thing that this one has been contributed by the community, it was not done by a core developer. In a number of maps you want to try very hard to label a certain polygon, especially if it's a cadastral maps or something where the labels really need to be present. So with this new vendor option you can set a target minimum size for the font, which means G-SER will start trying to label the polygon with the normal size and if it fails because the label is too big or if there is a conflict, it's going to try to shrink it a bit more and a bit more and then a bit more until it reaches the target minimum size. And this of course increases the number of labels that appear on the map. Next, another thing that we did, this one was not related with MaposGL, was this ability to drape a symbol along the line. It was already possible but the thing is it was just repeating the symbol along the line, it was not trying to make it continuous. So you were getting all these interruptions at bands. Right now with this new vendor option Mark along line, you can actually force the renderer to connect the various repeated symbols to generate a continuous symbology. Moving on. Okay, so right. Along with the mapbox styles, we have this new ability to read the mapbox vector tiles stored in an mbtiles file. So an mbtiles is just a SQLite tile storage with two simple tables. Please, those are the mandatory ones. And you can actually stuff vector tiles into it and typically the metadata table contains the description of the vector tiles such as which layers there are and what attributes they have. So G-SR now can read one of these mbtiles data sources and publish it as a WMS layer. So next slide. It was interesting to develop this because data is styled but when we do WMS, data is no more tiled. So the use case for this store is not to get the vector tiles and render them server side just because you can. But because the target use case is to take these vector tiles which were already generated and then paint WMS maps in another coordinate reference system or just send back PNGs and so on. So the funny thing is that in vector tiles, typically data is split and when you draw the borders around the polygons, you would get the disconnects and the like or even when rendering thick lines. So typically there is a buffer in each tile that we have to handle and we modified the render so that it leverages the buffer to generate a continuous symbology like it's shown in the image on the right. And if you want to learn more about this buffering thing, there's a link at the bottom that explains it. Next slide. So right now you can literally take two things out of openmap tiles. You can take the mbtiles vector tiles container and I have this in the screenshot, the planet from some time ago which is 55 gigabytes. So vector tiles for the entire planet at I think 14 or 15 zoom levels and then you can add on top of it the openmap tiles, Mabox GL styles and have G-server render them as PNGs. The interesting bit is in whatever projection you want because that was really the driver. You provided the common case as MBTs directly in the client but then being able to switch to whatever other projection and render PNGs instead because well you don't want to have 54 gigabytes worth of storage times the number of projections you want to support, at least not in the use case of the sponsor of this feature. Next slide. Yeah, to my surprise, WMS Cascading got a number of improvements. So with WMS Cascading is this ability to connect it to another WMS server and basically proxy it and add some extra functionality on top of it such as for example read the data in another read the maps in one projection or re-project them in another or act as a tunnel for some security reasons and the like. Well WMS Cascading got a bunch of improvements during this year such as choosing the Cascading format, limiting the scale range at which the Cascading is performed and limiting the bounding box, the area under which the Cascading is performed because there are some servers out there which literally start throwing exceptions outside of the bounding box they declare which is not totally correct but there it is. Next slide. Okay, so this actually goes beyond map rendering. So this is the JSON-LD community module. It's about producing outputs. So typically in WFS you can go and ask for features in GML but also in GeoJSON. JSON-LD is an alternative JSON format which has some semantic bases and some context information that provide, it could provide a schema and it provides a definition for each element because typically what you miss in JSON is the definition of what each attribute means. So in GeoServe we now have a JSON-LD community module that can be driven by templates so that you can build the exact shape of JSON-LD you want out of your data source. Be it a simple features data source such as a shapefile or a port.js table or a complex one such as an app schema mapping. Next slide. This continues to show the mapping file. As you can see the mapping file is the JSON-LD file itself with a few interpolations here and there to pick the elements that you want out of the original data source. And yeah, you can write a template file for each data set that you want to publish via JSON-LD. Next. Okay, so some news and updates about services. Go ahead. About five minutes, Andrea. All right. So we have this new OJC API community module that you can download. It contains a features API implementation for these RESTful services that OJC has been developing, but also an implementation of the Tiles API which is an equivalent of WMTS and something new like the Styles API which allows you to fetch styles from the server and if you have a write access even to add styles on the server which is pretty handy if you want to fetch vector tiles and then figure out, well, okay, how am I going to render them? Well, you can go to the Styles service and say, okay, which styles are compatible with this feature collection and get the Mabox GL styles or SLDs because, well, SLD can be processed on the client side as well and do the rendering on the client side. Next. It's also pretty interesting to have this new FlatGeoBuff community module which is an alternative, let's say, to GML and GeoJSON. It's a performant binary encoding for geographic data based on flat buffers. If you look at the comparison table, you will find that it's generally smaller than GeoJSON and GML and quite a bit faster. So it's a very efficient alternative to those two formats and it's also used on the server side for other reasons such as the temporary storage for processing chains, for example. Next. We had some improvements in GeoWabCache. Let's go ahead. Next. So one is a faster startup. Here we are talking about a GeoWabCache integrated with GeoServer. Loading the tile layers has been, let's say, a problem when you had many layers. Gabriel Roldan here has done some tests with one million layers in a single GeoServer, which is pretty amazing in my opinion. And we have the timings of the GeoServer startup and the GeoWabCache startup. As you can see, over time, besides some hiccups along the road, the startup time improved, especially the time to read the tile definition for GWC improved quite a bit over time. And so if you have a fast machine now, you can literally start a GeoServer with one million layers in like seven minutes. The tile layers page was also a pain point. Listen, the tile layers was slow. We improved it quite a bit in 2017. Next. And we were talking about storage of tiles. So GeoCache by default stores tiles on the disk using a disk structure, directory structure, which is a bit strange, let's say, but it optimizes the number of files per directory so that certain file systems such as NTFS are not getting killed by it. However, now you can also set XYZ or TMS layout and then seed your tile cache and then throw it S3 directly or onto Apache and serve it as a static cache. Next. Also very important, we exposed a number of controls to make GeoWabCache more tolerant to tile failures during seeding. Before this, GeoWabCache was just ending the seeding thread at the first tile failure that happened. Now you can control how many failures it's going to tolerate before giving up. So it's going to make seeding much more robust. Next. Okay, Johnny. I think I'm going to take over this section talking a little bit about some of the configuration improvements that have been made. Just start out with a really nice creature comfort. Now when you're working on a workspace or a layer or a layer group, there's a new tab for the security settings. So you can actually control who has access right there without having to duck over to the correct security configuration screen. And this just makes working with GeoServer a lot more efficient as an administrator. So it sounds like a small thing, but it just makes GeoServer more of a pleasure to use. So I really appreciate that. And this is another kind of really fun creature comfort. There's an option now if you dig into the other settings, you can put a little checkbox on called show time of creation and modification in the admin lists. And then when you go through the rest of GeoServer, there's two new columns, date created and date modified. And you can use that to sort to see what layers have been modified, you know, most recently. And this is just really handy if you're working on a couple layers, because you can see what you're working on rather than having to duck into the third screen and all of that. So it's just nice creature comforts for people who set up GeoServer frequently or set up many layers. This makes it easier to use. Another new addition here is the ability to add custom dimensions for vector layers. So we're used to the vector layers working, supporting for things like time and elevation. But now you can also map custom dimension ranges. This used to be something that was only supported for raster data. And with this release of GeoServer, you can now do that for vector data as well. Excellent. That appears to be all. I thought I was going to talk about the resource browser, but I don't see that anywhere, Andrea. I'm sorry. Let me fire the questions at you. And then if you want to talk about that, if anybody, I'm sure people are interested, so they can stick around. But we've got a few questions. So I'm just going to start straight off. So first of all, the comment from Mark, who we all saw presenting earlier, who says the Matbox style stuff is really cool. So thumbs up from Mark from that one. A question from PrettyMoi, who says, does GeoServer support Blobstore AWS S3 to store raster layers out of the box? Yes. Yes. Wait. So for tiles, yes. Tiles can be stored on the file system S3 Azure Blobs and SQLite. As Geotifs, if we are talking about Cloud Optimize Geotifs, there is a plugin that allows to publish single Geotifs files, but it doesn't work along. It doesn't play well with the image more like. And I think recently, yeah, you mentioned the Azure Blobs as well. Okay. So this is one from Paul, which is kind of UK specific. He says, I wonder if we'll get a method of using WMS Cascade with the OS that is an Ordnance Survey Maps API to allow people to effectively have a local cache and limit hits on the payable service. So I am taking advantage of WMS Cascade to kind of cut your costs on hits from a commercial API. You may not be able to answer that one, but that's just throwing that out, I guess. If it's a standard WMS server, it can be done already. And yes, you can cache the tiles as long as the contract that you have with Ordnance Survey allows for that, which is something that you should check. If it's a custom API, then code needs to be written as possible. Okay. Yeah, that makes sense. I'm sure OS won't be listening. So Paul can look into that. Another couple of questions from Mark, who says, firstly, will we get GeoServer 3.x anytime soon? Probably not. Will there just be 2.x, 2.x plus 1, etc.? So when we switched from 1x to 2x, there were two very big changes. The user interface looked completely different, and the storage of the configuration of GeoServer was also completely different. There was an automatic upgrade mechanism that made it transparent, but I mean, the change was really big. And I'm not sure we are going to have a change of this magnitude anytime soon. So yeah, we chained major version numbers when the configuration on disk does a major re-factor, and we don't see the need to do that right now. Our existing structure is being, has proven quite flexible. So not in the foreseeable future, and that's a good thing, that the project is being stable. No change for the second change. Another one from Mark. So Mark says, how well is GeoServer to be running cloud environments? And related to that, are there any plans for an official optimized docker version of GeoServer? And then he says, even Helm charts and so on. So we did talk to various companies that are publishing GeoServer docker images after Phosphor G last year, and we had a meeting with them, encouraging them to combine forces. But there hasn't been any appetite, there hasn't been any volunteers to go ahead and publish an official docker image. We do have docker image hosting on the OSTO repository now. The project's going to be using that for site automation, but we don't have any volunteer that's interested in working on this right now. So we're not against it, we just don't have anyone who's for it. Yeah, there's also the thing that GeoServer has a lot of variety with all the extensions and plugin, and when we surveyed how the docker images were built, each one had their own little set of plugins that they couldn't do without and that others wouldn't have, would have wanted to have in their docker image instead. Okay, so Paul's come back on the OS Maps API and says he thinks permissions are fine. So I guess that may be something for continued discussion on the OSTO UK mailing list for anybody that's interested in that. And the last question I've got so far is from David that's picked up on something that was discussed in the previous session. David says, what's the what's the optimum raster format for very large images in 2.17? So it says, for example, should he convert his ECWs to yeah, 2.17, should he convert his ECWs to GeoTIF? Yeah, so ECW is a great format. It's it compresses very well, it can be lossless, it's fast. However, it has one significant Achilles heel, you have to pay to use it on the server side. Plugins to use it on the desktop are free, but whenever you use it on a server to publish, then you have to pay. And the first thing that you want to do is to go and check with Hexagon, I think nowadays, about the license cost of doing that. And then if you want, there are the GDAL plugins for GeoServer that include the ECW support. To be honest, I don't remember having a GeoSolutions customer using ECW in the last 80 to 10 years. So it's uncommon. Typically, people take the ECWs and they convert them into GeoTIFs with JPEG compression, keeping the compression low to avoid artifacts. Okay, and a follow up on the earlier discussion about blob storage from Pritamai. So do you see any performance issues if you use a blob store for Raster Layer Storage? So to store tiles, you mean? I would know. No, that's the answer. For GeoTIFs instead? I'm going to unmute. Yes, for the GeoTIF. Okay, so there is one layout. GeoTIF is a very rich format. There is one layout, which is Cloud Optimized GeoTIF, which is optimized for storing the TIF in an S3 or blob storage in general, which allows you to do very efficient range requests and only get to the part of the file that you want. Right now, we don't have a plugin that leverages this layout. The S3 GeoTIF plugin that I was talking about before basically slices the file in 5 megabytes block and uses some local caching to save it so you can literally throw whatever TIF layout in S3. Given there is this local caching, the performance is not bad unless you start hitting the network a lot. The direction that we are going is to have a plugin that actually uses the cog file structure to avoid the caching and still do optimal file transfers. As long as you have your blob storage and your server close to each other, like in AWS in the same region, it should be just fine at that point. But we are not there yet. And one more, if I may, one more question for me. The one million layers you said, if you have one million layers in your GeoTIF, your server takes only seven minutes to load. We are using quite an old version which is taking an hour or so to load the one million. So I hope if I just upgrade to 2.17, it should improve a lot. Just the loading time. Right. Yeah. There's also another option which is to use the JDBC config plugin, which doesn't load the configuration from the database. It loads it basically on a per request basis. I tested it recently. The startup time is basically constant no matter how many layers you have. However, even with the last improvements, like I tested a month ago, you are going to take a heavy hit on the single get map, like the overhead is. Especially when you have many layers. Yeah, we have more than one million. Okay. No, in that case, I would keep them on the file system and switch to G7217. Okay. Okay. Interject with a slide I lost somehow. There's a new tool here called resource browser. This allows you to look at the contents of your data directory. You can upload your icons and fonts. You can actually even edit small configuration files like those templates we saw earlier, or control flow for properties as this example. And so this just gives our web administration program, some of the functionality that was previously only available through the rest API. And so that's an optional install, and it shows up in the tools menu. So just a last feature that somehow escaped my slides. Sorry about that. Yeah, pretty handy when you are working against a remote to do server and for any reason you don't have SSH or SFTP access to the machine. Yeah, you know, do file transfers directly. That's great. So I'm just going to give you a couple of other questions and I think we need to wrap up. It's already running over time. So obviously there's a lot of interest in this. And these are the last two questions I've got in any case. So, firstly, Denilson says, can I may I use raster in geo package format. And the second question about rasters from Sheilash is, what is the best way to publish raster stored in Postgres with geo server. So raster in geo package and raster in Postgres. Okay, so in geo package, the core geo package allows to store PNG and JPEG tiles into into the geo package and that's supported and it's working fine. We are actually, my company is publishing 500 gigabytes geo package stuff to with the JPEG and PNGs which covers the entire planet. It's a Sentinel to a cloud less composite. However, there is also an official extension to support raster data in terms of digital elevation models or, you know, whatever is not an image. And that one is not supported yet with some development it could be just a matter of throwing resources at that development. In terms of reading data from raster data from post GIS, there are two plugins, they are both unsupported at this point. They are the image moves like JDBC store, which has lost his maintainer years ago and we just moved it down to from extension to community. And there is a new dedicated post GIS raster plugin, which also has been donated but doesn't have a maintainer. More or less the issue with both is that they are not driving business and it's difficult to keep them up if we never use them during working hours. So those are your two options but as a, if you go there I would suggest looking for some sort of support along the way, because there is no one officially maintaining them. Yeah, it doesn't necessarily have to be driven by business if your organization uses this functionality. We just need someone to take to do the Karen feeding of these modules and answer questions on the user list and so on. We try to be honest with folks if a component is not being supported and we mark it as such. Right. Before demoting the image moves like JDBC to from extension to community I actually asked asked on the on various mailing list to for a new maintainer but no one showed up. So you can see that you're more than welcome and we can push the module back up to support that land to official extension.
 Excellent. Well, welcome back everyone and thanks very much for hosting this event and  facilitating Anthony. We really appreciate it. Usually for a Phosphor G conference,  Andrea and myself do a state of geo server update which talks a little bit bigger picture about  what's going on with the project as a whole. But it's such a busy project that stresses us out.  So today for this online session, we're being a little bit more relaxed and we're just going to  talk about the new fun things that are available in the latest geo server 2.17 release. So just  starting off with a quick overview. We released geo server a couple times a year doing major  releases of 2016 and 2017 and 2018. So for the geo server 2.17 release, it was initially released  in April 2020. This is a few months late because we were changing build infrastructure and it was  released alongside GeoTools 23.0. This is the first stable release of the 2017 series. Just last week,  I released the first point release there to 0.17.1. How this works is geo server is going to get four  more releases. Each release series lasts for a year. So the first releases are considered stable.  They get a few new features are getting backported and then it goes into a maintenance mode where  only bug fixes and security updates are backported. This release runs on Java 8 or Java 11. We really  recommend using openJBK version 8. So the other thing that's happening here is geo server in  addition to that core application has a series of optional installs or extensions. You can see  some of them on the screen ahead of you. But once that plays to star next to are new for this release.  So we're going to talk a little bit about the mapbox starling extension and also this resource  browser tool extension. We also have a kind of an experimental R&D play area called community modules.  We've got a couple of community modules that are kind of active R&D that we'd like to talk about.  So flat geo buffer, GSR plugin and the OGC API plugin are new R&D projects this release.  So with that intro, I'm going to dive right into the first batch of updates. And by that,  I mean I'm going to hand the microphone over to Andrea. Hello. Yeah, thank you. So yeah, we got a  bunch of styling improvements, especially in the mapbox style plugin. Jodi pushed it from a community  to extension status, which means you will find it among the releases as a numbered module, as opposed  to being a community module that you can only find in the nightly builds. The use case for this module,  the mapbox style plugin is to reuse the the mapbox GL styles on the server side as well. For those  cases where you want to have an hybrid architecture where you publish mapbox vector tiles, but we  want also retain the ability to publish PNGs so that you have a fully standard OGC service,  but also allow to use vector ties on the side. Next slide. Yeah, I'm just going to interject there.  This was originally a community module started by boundless. And then my new employer GeoCat has  allowed me some time to pick this up and take this R&D project and domesticate it and include it in the release.  Yep. A few months ago, as part of a contract with GeoSolutions, we actually did a bunch of  improvements onto the translation because, keep in mind that GeoServer at its core basically only  speaks SLD. Every other language gets translated down to an object model which is  strongly inspired by SLD itself. So the translation from mapbox to the subject model was lacking a  few features and though was not mimicking correctly a few of the behaviors of mapbox style, we  did a bunch of improvements and corrected a number of issues. Next slide. And right now,  if you take the open map tiles, GL Bright, and mapbox vector, mapbox GL style, you cannot fully run it  on GeoServer and produce an output which looks by all intents and purposes just like the one you  would get on the client side. Of course, there are more small differences because the leveling  engine is not the same but I mean, I guess that if I put you in front of this map, you wouldn't  have been able to tell that it was rendered server side but you server. Next. In order to make things  work, we had to make a number of map-engining improvements such as, for example, background  field, as you say, symbols with labels, func-ranking, and so on. The mapbox GL work was not the only  driver for improvements. There were also other improvements that were unrelated to it and we  are going to have a look at them one by one. So next. This is derived from mapbox GL. In mapbox  GL, you can set the background color. In SLD, you cannot because typically the background color  comes from the get map request. However, we added an extension to the SLD. So it's a vendor  extension for GeoServer that allows you to specify an SLD background element to specify a color or  a fill. Like, for example, you could have a PNG with the waves to represent the water as the  background and that would also work. Next. Another thing that we noticed is that when mapbox GL  styles works with icons with labels, either both the icon and the label show up or none of them  does. In GeoServer, typically you represent an icon with a label below it as a point symbolizer and  a text symbolizer. But being two independent elements means that one shows up and the other  doesn't some time. So we already have this ability to put a background image below a label to do  text, do, road label shields, but it was always centering the shield behind the label  in the same position and we wanted to actually push the label below. So we added a new  vendor option, Graphic Placement, that allows to offset the symbol and the label independently,  getting this result of having the bike and the label below it, but making it so that they  either appear both of them or none of them. And this is shown in the next slide. So you can see  the giant store Notre Dame on the Ildeperie and as I zoom in there are other labels that start  showing up, which interfere with the bike shop and the bike shop label disappears and as a result  also the point. You might want or not want this behavior, the thing is now it's possible before  it wasn't. Next, another thing that this one has been contributed by the community, it was not done  by a core developer. In a number of maps you want to try very hard to label a certain polygon,  especially if it's a cadastral maps or something where the labels really need to be present.  So with this new vendor option you can set a target minimum size for the font,  which means G-SER will start trying to label the polygon with the normal size and if it fails  because the label is too big or if there is a conflict, it's going to try to shrink it a bit  more and a bit more and then a bit more until it reaches the target minimum size. And this of  course increases the number of labels that appear on the map. Next, another thing that we did,  this one was not related with MaposGL, was this ability to drape a symbol along the line.  It was already possible but the thing is it was just repeating the symbol along the line,  it was not trying to make it continuous. So you were getting all these interruptions  at bands. Right now with this new vendor option Mark along line, you can actually force the renderer  to connect the various repeated symbols to generate a continuous symbology. Moving on.  Okay, so right. Along with the mapbox styles, we have this new ability to read the mapbox  vector tiles stored in an mbtiles file. So an mbtiles is just a SQLite tile storage with  two simple tables. Please, those are the mandatory ones. And you can actually stuff vector  tiles into it and typically the metadata table contains the description of the vector tiles  such as which layers there are and what attributes they have. So G-SR now can read one of these  mbtiles data sources and publish it as a WMS layer. So next slide. It was interesting to  develop this because data is styled but when we do WMS, data is no more tiled. So the use case  for this store is not to get the vector tiles and render them server side just because you can.  But because the target use case is to take these vector tiles which were already generated  and then paint WMS maps in another coordinate reference system or just send back PNGs and so on.  So the funny thing is that in vector tiles, typically data is split and when you draw the borders  around the polygons, you would get the disconnects and the like or even when rendering thick lines.  So typically there is a buffer in each tile that we have to handle and we modified the render  so that it leverages the buffer to generate a continuous symbology like it's shown in the  image on the right. And if you want to learn more about this buffering thing, there's a link at the  bottom that explains it. Next slide. So right now you can literally take two things out of  openmap tiles. You can take the mbtiles vector tiles container and I have this in the screenshot,  the planet from some time ago which is 55 gigabytes. So vector tiles for the entire planet  at I think 14 or 15 zoom levels and then you can add on top of it the openmap tiles,  Mabox GL styles and have G-server render them as PNGs. The interesting bit is in whatever  projection you want because that was really the driver. You provided the common case as  MBTs directly in the client but then being able to switch to whatever other projection  and render PNGs instead because well you don't want to have 54 gigabytes worth of storage times  the number of projections you want to support, at least not in the use case of the sponsor of  this feature. Next slide. Yeah, to my surprise, WMS Cascading got a number of improvements.  So with WMS Cascading is this ability to connect it to another WMS server and basically  proxy it and add some extra functionality on top of it such as for example read the data in  another read the maps in one projection or re-project them in another or act as a tunnel  for some security reasons and the like. Well WMS Cascading got a bunch of improvements during this  year such as choosing the Cascading format, limiting the scale range at which the Cascading is  performed and limiting the bounding box, the area under which the Cascading is performed  because there are some servers out there which literally start throwing exceptions outside of  the bounding box they declare which is not totally correct but there it is. Next slide.  Okay, so this actually goes beyond map rendering. So this is the JSON-LD community module. It's  about producing outputs. So typically in WFS you can go and ask for features in GML but also in  GeoJSON. JSON-LD is an alternative JSON format which has some semantic bases and some context  information that provide, it could provide a schema and it provides a definition for each  element because typically what you miss in JSON is the definition of what each attribute means.  So in GeoServe we now have a JSON-LD community module that can be driven by templates so that  you can build the exact shape of JSON-LD you want out of your data source. Be it a simple  features data source such as a shapefile or a port.js table or a complex one such as an  app schema mapping. Next slide. This continues to show the mapping file. As you can see the mapping  file is the JSON-LD file itself with a few interpolations here and there to pick the elements  that you want out of the original data source. And yeah, you can write a template file for each  data set that you want to publish via JSON-LD. Next. Okay, so some news and updates about services.  Go ahead. About five minutes, Andrea. All right. So we have this new OJC API community module that  you can download. It contains a features API implementation for these RESTful services that  OJC has been developing, but also an implementation of the Tiles API which is an equivalent of WMTS  and something new like the Styles API which allows you to fetch styles from the server and if you  have a write access even to add styles on the server which is pretty handy if you want to  fetch vector tiles and then figure out, well, okay, how am I going to render them? Well, you can go to  the Styles service and say, okay, which styles are compatible with this feature collection  and get the Mabox GL styles or SLDs because, well, SLD can be processed on the client side as well  and do the rendering on the client side. Next. It's also pretty interesting to have this new  FlatGeoBuff community module which is an alternative, let's say, to GML and GeoJSON. It's a performant  binary encoding for geographic data based on flat buffers. If you look at the comparison table,  you will find that it's generally smaller than GeoJSON and GML and quite a bit faster. So it's  a very efficient alternative to those two formats and it's also used on the server side for other  reasons such as the temporary storage for processing chains, for example. Next. We had some improvements  in GeoWabCache. Let's go ahead. Next. So one is a faster startup. Here we are talking about a  GeoWabCache integrated with GeoServer. Loading the tile layers has been, let's say, a problem  when you had many layers. Gabriel Roldan here has done some tests with one million layers in a single  GeoServer, which is pretty amazing in my opinion. And we have the timings of the GeoServer startup  and the GeoWabCache startup. As you can see, over time, besides some hiccups along the road,  the startup time improved, especially the time to read the tile definition for GWC improved quite  a bit over time. And so if you have a fast machine now, you can literally start a GeoServer with one  million layers in like seven minutes. The tile layers page was also a pain point. Listen, the  tile layers was slow. We improved it quite a bit in 2017. Next. And we were talking about storage  of tiles. So GeoCache by default stores tiles on the disk using a disk structure,  directory structure, which is a bit strange, let's say, but it optimizes the number of files per  directory so that certain file systems such as NTFS are not getting killed by it. However,  now you can also set XYZ or TMS layout and then seed your tile cache and then throw it S3 directly  or onto Apache and serve it as a static cache. Next. Also very important, we exposed a number  of controls to make GeoWabCache more tolerant to tile failures during seeding. Before this,  GeoWabCache was just ending the seeding thread at the first tile failure that happened. Now you  can control how many failures it's going to tolerate before giving up. So it's going to make  seeding much more robust. Next. Okay, Johnny. I think I'm going to take over this section talking  a little bit about some of the configuration improvements that have been made. Just start  out with a really nice creature comfort. Now when you're working on a workspace or a layer or a layer  group, there's a new tab for the security settings. So you can actually control who has access right  there without having to duck over to the correct security configuration screen. And this just makes  working with GeoServer a lot more efficient as an administrator. So it sounds like a small thing,  but it just makes GeoServer more of a pleasure to use. So I really appreciate that. And this is  another kind of really fun creature comfort. There's an option now if you dig into the other  settings, you can put a little checkbox on called show time of creation and modification  in the admin lists. And then when you go through the rest of GeoServer, there's two new columns,  date created and date modified. And you can use that to sort to see what layers have been modified,  you know, most recently. And this is just really handy if you're working on a couple layers,  because you can see what you're working on rather than having to duck into the third screen and all  of that. So it's just nice creature comforts for people who set up GeoServer frequently  or set up many layers. This makes it easier to use. Another new addition here is the ability to  add custom dimensions for vector layers. So we're used to the vector layers working,  supporting for things like time and elevation. But now you can also map  custom dimension ranges. This used to be something that was only supported for raster data. And with  this release of GeoServer, you can now do that for vector data as well. Excellent.  That appears to be all. I thought I was going to talk about the resource browser, but I don't see that  anywhere, Andrea. I'm sorry. Let me fire the questions at you. And then if you want to talk  about that, if anybody, I'm sure people are interested, so they can stick around. But we've  got a few questions. So I'm just going to start straight off. So first of all, the comment from  Mark, who we all saw presenting earlier, who says the Matbox style stuff is really cool. So  thumbs up from Mark from that one. A question from PrettyMoi, who says,  does GeoServer support Blobstore AWS S3 to store raster layers out of the box? Yes.  Yes. Wait. So for tiles, yes. Tiles can be stored on the file system S3 Azure Blobs and  SQLite. As Geotifs, if we are talking about Cloud Optimize Geotifs, there is a plugin that allows  to publish single Geotifs files, but it doesn't work along. It doesn't play well with the image  more like. And I think recently, yeah, you mentioned the Azure Blobs as well.  Okay. So this is one from Paul, which is kind of UK specific.  He says, I wonder if we'll get a method of using WMS Cascade with the OS that is an  Ordnance Survey Maps API to allow people to effectively have a local cache and limit hits  on the payable service. So I am taking advantage of WMS Cascade to kind of cut your costs on hits  from a commercial API. You may not be able to answer that one, but that's just throwing that out, I guess.  If it's a standard WMS server, it can be done already. And yes, you can cache the tiles as long  as the contract that you have with Ordnance Survey allows for that, which is something that you  should check. If it's a custom API, then code needs to be written as possible.  Okay. Yeah, that makes sense. I'm sure OS won't be listening. So Paul can look into that.  Another couple of questions from Mark, who says, firstly, will we get  GeoServer 3.x anytime soon? Probably not.  Will there just be 2.x, 2.x plus 1, etc.?  So when we switched from 1x to 2x, there were two very big changes. The user interface looked  completely different, and the storage of the configuration of GeoServer was also completely  different. There was an automatic upgrade mechanism that made it transparent, but I mean, the change  was really big. And I'm not sure we are going to have a change of this magnitude anytime soon.  So yeah, we chained major version numbers when the configuration on disk does a  major re-factor, and we don't see the need to do that right now. Our existing structure is being,  has proven quite flexible. So not in the foreseeable future, and that's a good thing,  that the project is being stable. No change for the second change.  Another one from Mark. So Mark says, how well is GeoServer to be running cloud environments?  And related to that, are there any plans for an official optimized docker version of GeoServer?  And then he says, even Helm charts and so on. So we did talk to various companies that are  publishing GeoServer docker images after Phosphor G last year, and we had a meeting with them,  encouraging them to combine forces. But there hasn't been any appetite, there hasn't been any  volunteers to go ahead and publish an official docker image. We do have docker image hosting on  the OSTO repository now. The project's going to be using that for site automation, but we don't  have any volunteer that's interested in working on this right now. So we're not against it,  we just don't have anyone who's for it. Yeah, there's also the thing that GeoServer has a  lot of variety with all the extensions and plugin, and when we surveyed how the docker images were  built, each one had their own little set of plugins that they couldn't do without and that  others wouldn't have, would have wanted to have in their docker image instead.  Okay, so Paul's come back on the OS Maps API and says he thinks permissions are fine. So I guess  that may be something for continued discussion on the OSTO UK mailing list for anybody that's  interested in that. And the last question I've got so far is from David that's picked up on  something that was discussed in the previous session. David says, what's the what's the  optimum raster format for very large images in 2.17? So it says, for example, should he convert his  ECWs to yeah, 2.17, should he convert his ECWs to GeoTIF? Yeah, so ECW is a great format. It's  it compresses very well, it can be lossless, it's fast. However, it has one significant Achilles  heel, you have to pay to use it on the server side. Plugins to use it on the desktop are free,  but whenever you use it on a server to publish, then you have to pay. And the first thing that you  want to do is to go and check with Hexagon, I think nowadays, about the license cost of doing that.  And then if you want, there are the GDAL plugins for GeoServer that include the ECW support.  To be honest, I don't remember having a GeoSolutions customer using ECW in the last  80 to 10 years. So it's uncommon. Typically, people take the ECWs and they convert them into  GeoTIFs with JPEG compression, keeping the compression low to avoid artifacts.  Okay, and a follow up on the earlier discussion about blob storage from Pritamai. So do you see any  performance issues if you use a blob store for Raster Layer Storage? So to store tiles, you mean?  I would know. No, that's the answer.  For GeoTIFs instead?  I'm going to unmute. Yes, for the GeoTIF.  Okay, so there is one layout. GeoTIF is a very rich format. There is one layout, which is Cloud Optimized GeoTIF,  which is optimized for storing the TIF in an S3 or blob storage in general, which allows you to do very efficient  range requests and only get to the part of the file that you want. Right now, we don't have a plugin that  leverages this layout. The S3 GeoTIF plugin that I was talking about before basically slices the file in 5 megabytes block  and uses some local caching to save it so you can literally throw whatever TIF layout in S3. Given there is this local caching,  the performance is not bad unless you start hitting the network a lot. The direction that we are going is to have a plugin that  actually uses the cog file structure to avoid the caching and still do optimal file transfers. As long as you have your blob storage  and your server close to each other, like in AWS in the same region, it should be just fine at that point. But we are not there yet.  And one more, if I may, one more question for me. The one million layers you said, if you have one million layers in your GeoTIF,  your server takes only seven minutes to load. We are using quite an old version which is taking an hour or so to load the one million.  So I hope if I just upgrade to 2.17, it should improve a lot. Just the loading time.  Right. Yeah. There's also another option which is to use the JDBC config plugin, which doesn't load the configuration from the database.  It loads it basically on a per request basis. I tested it recently. The startup time is basically constant no matter how many layers you have.  However, even with the last improvements, like I tested a month ago, you are going to take a heavy hit on the single get map, like the overhead is.  Especially when you have many layers.  Yeah, we have more than one million.  Okay. No, in that case, I would keep them on the file system and switch to G7217.  Okay.  Okay.  Interject with a slide I lost somehow. There's a new tool here called resource browser. This allows you to look at the contents of your data directory.  You can upload your icons and fonts. You can actually even edit small configuration files like those templates we saw earlier, or control flow for properties as this example.  And so this just gives our web administration program, some of the functionality that was previously only available through the rest API.  And so that's an optional install, and it shows up in the tools menu. So just a last feature that somehow escaped my slides. Sorry about that.  Yeah, pretty handy when you are working against a remote to do server and for any reason you don't have SSH or SFTP access to the machine.  Yeah, you know, do file transfers directly.  That's great. So I'm just going to give you a couple of other questions and I think we need to wrap up. It's already running over time. So obviously there's a lot of interest in this.  And these are the last two questions I've got in any case. So, firstly, Denilson says, can I may I use raster in geo package format. And the second question about rasters from Sheilash is, what is the best way to publish raster stored in Postgres with geo server.  So raster in geo package and raster in Postgres.  Okay, so in geo package, the core geo package allows to store PNG and JPEG tiles into into the geo package and that's supported and it's working fine.  We are actually, my company is publishing 500 gigabytes geo package stuff to with the JPEG and PNGs which covers the entire planet. It's a Sentinel to a cloud less composite.  However, there is also an official extension to support raster data in terms of digital elevation models or, you know, whatever is not an image. And that one is not supported yet with some development it could be just a matter of throwing resources at that development.  In terms of reading data from raster data from post GIS, there are two plugins, they are both unsupported at this point. They are the image moves like JDBC store, which has lost his maintainer years ago and we just moved it down to from extension to community.  And there is a new dedicated post GIS raster plugin, which also has been donated but doesn't have a maintainer.  More or less the issue with both is that they are not driving business and it's difficult to keep them up if we never use them during working hours.  So those are your two options but as a, if you go there I would suggest looking for some sort of support along the way, because there is no one officially maintaining them.  Yeah, it doesn't necessarily have to be driven by business if your organization uses this functionality.  We just need someone to take to do the Karen feeding of these modules and answer questions on the user list and so on. We try to be honest with folks if a component is not being supported and we mark it as such.  Right.  Before demoting the image moves like JDBC to from extension to community I actually asked asked on the on various mailing list to for a new maintainer but no one showed up.  So you can see that you're more than welcome and we can push the module back up to support that land to official extension.
FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_0_107.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_108_4343.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_4344_4655.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_4656_6335.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_6336_7547.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_7548_10115.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_10116_11975.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_11976_15047.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_15048_15095.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_15096_15179.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_15180_16415.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_16416_20159.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_20160_21803.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_21804_23711.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_23712_27095.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_27096_27143.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_27144_28379.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_28380_28859.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_28860_31631.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_31632_33503.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_33504_34487.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_34488_36551.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_36552_36791.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_36792_36875.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_36876_36995.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_36996_37079.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_37080_37223.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_37224_37271.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_37272_37343.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_37344_37403.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_37404_37955.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_37956_38195.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_38196_40715.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_40716_40763.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_40764_41291.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_41292_48563.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_48564_48623.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_48624_49079.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_49080_49235.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_49236_51887.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_51888_54215.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_54216_54323.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_54324_54371.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_54372_54431.png FOSS4GUKOnline_2020_Mercator_Aime_3gSPERPmoNU_2_54432_54476.png