Eclipse Loves LSP - Achieving More with Less
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Kurtakov, Alexander
Eclipse and Language Server Protocol - what, why, how and most important visible results for users thanks to it. Brief description of how it all started. Followed by examples of the success stories benefiting almost every Eclipse user by getting better tools while minimizing the burden on Eclipse developers.
 Let's start with why Eclipse loves LSP. So I'm not going to demo anything. This would be done in the next sessions. What I'm going to speak about is kind of strategy we have at Eclipse and my team pushes heavily for it. So far I would call it a strategy as we already executed multiple steps of it and people seems to love it and no one is opposed. So since that we are in the right direction. I'm Alexander Kurtakov. I worked at the desktop IDE team at Red Hat and over the years been involved in multiple languages plug-in in Eclipse. Now let's first see why should we look at the LSP. I was, well, I really copied from the main LSP page these nice tables and in the past we were really, really implementing things over and over again and it wasn't really working. But I think with making Java developers as most Eclipse plug-in developers are either good C parser or shell script parser, it's not an easy task and at the same time it happens to be that using the language your IDE is written in making it even harder to achieve certain tasks. So what the LSP provides us is a separate process which intercepts all the textual data and gives the IDE the content it needs. So what we get with the LSP right now is people really knowledgeable in the given language, with the language server themselves and people knowledgeable in the IDE itself provide a good way to embed language servers in general. It's a really common separation of responsibilities which unfortunately in the IDE space we were not doing. There are multiple reasons why this was not happening but the most important was kind of not invented here and we suffered in Eclipse land a lot from that for years. Finally, I think that things are starting to normalize in this area. What's more important is that thanks to language servers we start to see a lot of collaborations between IDE. In the past like 10 or 12 years ago Eclipse and NetBeans we were collaborating on getting the IDE running on Linux distributions and how to actually make it start there. These were different tasks but now we see a lot more collaboration even if indirect. NetBeans use certain language server, they face an issue, they report it, it got fixed, the Eclipse users get it for free and vice versa. So how we see the whole LSP usage in Eclipse going. Few years ago we added something that's called generic editor in Eclipse platform. This was totally missing, we had some basic blocks but pretty much everyone was starting with copying existing editor. There were many copies of the Java editor from which other editors originated even the CDT editor was initially copied one and so were the ones in the dynamic language is plugging. This plug-in gives you no API, it gives you only extension points to which you hook. It's really important to get it that way so we have the freedom to change the internal implementation as needed because when we started it we were not really sure what additional needs will be and over the last couple of years there were few additional extension points added and no one wants to get an interface growing more and more and people implementing it becoming incompatible suddenly. This was mainly done by Mika out there so if you have any question about it, that's the right person. The other main block is LSP4J, it's an implementation of the language server protocol. It's pure Java implementation and what's important is it's done in a way that you can use it also to implement servers not only clients. That actually gives a lot more interest in LSP4J as we already see a number of implementations based on it. I forgot to mention that the generic editor, you might look at it as a monocoque or eclipse ID, it serves similar needs. How these things are bind together? We have an LSP4E project, it's not in the eclipse platform but overall same people drive it on both sides on the top and create the merge between them. What does LSP4E? Through LSP4J to hook into the extensions provided by generic editor. These are the main points which we see as a need for implementing it proper on an ID. These are the points which we as a platform developers should provide and make it really, really easy for people to have support for their preferred language. That picture wouldn't be full without these two small blocks. One is TM4E, it's a textmate interpreter which allows you to use textmate grammars to do highlighting in eclipse. It saves a lot of time again so getting pure highlighting for your language is pretty much a single step. The more important part is the debug adapter protocol. Let's call it a sister protocol to LSP4J. What it allows is to not integrate with given debugger but use a unified protocol how to communicate with the burgers. This gives us easy implementation for the burgers. I might be wrong, the debug adapter protocol is not yet as popular as the LSP1 but it would be way more important over time as coming with a good debugging story is actually quite harder than getting the editor correct. So with these things I'm moving to some of the success stories we got and pretty much whoever used Eclipse in the last years and was relying on web tools know the sad story of JavaScript editor there and the HTML editor was not that great also. It was great if you were stuck to technology 10 years ago if you were using something modern it was failing in a spectacular way sometimes. What we achieved here is unifying a lot of the random plugins people needed to collect in order to do a modern web development as a single stop and what's more important is we did that without knowing not enough to even write more than Hello World let's say in TypeScript. None of us ever developed anything in TypeScript but by reusing the language server this was easy enough. Why I call it a success story in addition to all these features users get from community point of view we saw a number of people just jumping in and being enthusiastic and helping how to get the whole story running and most of the people they were not seasoned Eclipse plugin developers but it was easy to come in and help with the integration cost they knew the technology they wanted to integrate. That's why doing the IDE site from the IDE people and letting the people knowing the technology deal with it is so efficient. Let's say that everyone had some interactions with web development in a way but the other one was more of an adventure because learning crust I would say it's not as straightforward as I would like it to be but what came here is we faced the exactly same benefit we have seen with the wild web developer. We got the plugin running few missteps with getting the Rust OS integrated. Now we got the Rust people fully dealing and even leading the project with the need from Eclipse platform developers being to just hook up some new addition that came on the IDE site. So what's needed more or less is to create one integration for your language server, hook the text made grammar if you have one, if you don't want to write your high-witing code and if you want to provide really, really nice way for users to start give them a wizard like in the corrosion one to use a template so they get some of the boilerplate generated including the build system. But these are not the only cases. There are a number of other projects. What I'm listing here are just projects that are at Eclipse.org or should be soon but in the bigger community we see others like Apache Camo providing some integration based on language server and also the SpringTools being language server based tooling more and more. But this one has kind of a sweet spot for us. It has the C-sharp integration, the.NET core in Eclipse. When we started it, it was the very first effort to do a LSP4E based project in Eclipse and it was based on.NET core because VS Code was really advertising a lot and the language server for.NET was kind of most separated one and easy to integrate. I have to admit we kind of not managed to get as many people interested as I would like but this might be due to some difficulties with getting the framework running and some limitations coming legally. So this one, it's still there if someone is interested. I'm offering all the support to take it over so contact me or Mikael. We will help as much as we can to get someone taking it in. So far it will probably stay as it is and also that one, it's a shell script editor. Whoever tried to parse a shell script knows that this is one of the biggest adventures, pretty much everything can happen so coming with a grammar is not possible I would say and ShellWox is the third generation of Eclipse plugins for shell script I do. So first we were trying to do everything on our own, it worked to some degree then we based it on.NET, it was a common tooling for dynamic languages. That was a move in the right direction but there were way too many additional steps needed and I would say that what we see with LSP4E is this separation finally done in a proper way and we have Dart board, plugin to integrate Dart in Eclipse. What I'm mentioning also is Ruby SolarGraph, we had Ruby plugin at Eclipse but it was stuck in the Ruby 2 time frame so that's one case where we admitted okay we're not doing good enough job so we dropped this plugin, it's no longer in the simultaneous release and so on and I strongly recommend people to go to the marketplace and install such LSP based plugin from marketplace. Now as every last story JDTLS language server for Java based on top of Eclipse compiler and other goodies that were coming from Eclipse ID for Java developers. Now first I would say I was not enthusiastic about it, it was like okay we are going to make something for the benefit of other ID's so they can embed all the goodness we are doing, what's the point in all that work. Actually it appears that JDT managed to get quite a few additional contributors thanks to the work that was going on JDTLS. We had to restructure a bit the code base for JDT and that restructure allowed people to come easier as let's say our core and UI separation was kind of unclear and nowadays we even saw few patches coming from VS Code guys to get fixes they need and it being used in the cloud ID's it allows to get people working on JDT full time so that work, that language server happened to be kind of the most important work in JDT in terms of community building in the last couple of years, my opinion only. There will be probably others not agreeing as all the JVM work is also important but it's a technical work so we should make this separation. This is soon to come project which integrates, well it doesn't integrate, it's a language server for XML, XSD, DTD and all the XML technologies. What is nice about it is that it's an extensible one so we are looking into ways to replace a POM XML editor in the Maven plugin with a language server based one so we get all the XML smartness directly from the Lemmings plugin but we extend it additionally with Maven specific completions and so on. That would allow how to complete for group ID's, artifact ID's, this is all work in progress but at least the work has started and we see some of these features coming in, let's not say the March release but for June there should be a preview in a way. Seems like I'm almost out of my time so if you have any questions, yes? The question is which providers we are missing for highlighting so the approach we took is to not provide highlighting by default. I personally have everything I need right now. What is important here is that I'm looking for a bit more full story like not only highlighting but more of a full language server based editor for given language so people get at least some error checking and auto completion in addition. Now this is something I have been thinking of but it never, I never managed to find some information about it. It is, if anyone knows of a text made registry in a way so some kind of plugin can be made based on such a registry and Eclipse ID can auto fetch from this registry, text made grammar so highlighting can be added on demand. We are moving away from text made that's the idea right? That's what we are trying to do. At least in language server world. The syntax highlighting is proposed and I'm hoping it will add and if the studio is spawning it it seems. Okay I was not aware of that part. Sorry. If the protocol changes and provides some new value we want we will integrate it but on the syntax highlighting from the full perspective we are not pushing for more syntax highlighting because it's not actually the top of the value we can produce at the moment. So yeah we are following. At the beginning when we started this integration we were more involved in the protocol because it was new and we wanted it to fit with our needs but actually the community is smart and doing job correctly so things go their way without our intervention so we know we just follow what's in it and just monitoring because we totally disagree to voice before it's too late but in general everything is fine and we just implement things when they are there. Thank you.
 Let's start with why Eclipse loves LSP.  So I'm not going to demo anything.  This would be done in the next sessions.  What I'm going to speak about is kind of strategy we have at Eclipse and my team pushes heavily  for it.  So far I would call it a strategy as we already executed multiple steps of it and people seems  to love it and no one is opposed.  So since that we are in the right direction.  I'm Alexander Kurtakov.  I worked at the desktop IDE team at Red Hat and over the years been involved in multiple  languages plug-in in Eclipse.  Now let's first see why should we look at the LSP.  I was, well, I really copied from the main LSP page these nice tables and in the past  we were really, really implementing things over and over again and it wasn't really working.  But I think with making Java developers as most Eclipse plug-in developers are either  good C parser or shell script parser, it's not an easy task and at the same time it happens  to be that using the language your IDE is written in making it even harder to achieve certain  tasks.  So what the LSP provides us is a separate process which intercepts all the textual data and  gives the IDE the content it needs.  So what we get with the LSP right now is people really knowledgeable in the given language,  with the language server themselves and people knowledgeable in the IDE itself provide a good  way to embed language servers in general.  It's a really common separation of responsibilities which unfortunately in the IDE space we were  not doing.  There are multiple reasons why this was not happening but the most important was kind  of not invented here and we suffered in Eclipse land a lot from that for years.  Finally, I think that things are starting to normalize in this area.  What's more important is that thanks to language servers we start to see a lot of collaborations  between IDE.  In the past like 10 or 12 years ago Eclipse and NetBeans we were collaborating on getting  the IDE running on Linux distributions and how to actually make it start there.  These were different tasks but now we see a lot more collaboration even if indirect.  NetBeans use certain language server, they face an issue, they report it, it got fixed,  the Eclipse users get it for free and vice versa.  So how we see the whole LSP usage in Eclipse going.  Few years ago we added something that's called generic editor in Eclipse platform.  This was totally missing, we had some basic blocks but pretty much everyone was starting  with copying existing editor.  There were many copies of the Java editor from which other editors originated even the  CDT editor was initially copied one and so were the ones in the dynamic language is  plugging.  This plug-in gives you no API, it gives you only extension points to which you hook.  It's really important to get it that way so we have the freedom to change the internal  implementation as needed because when we started it we were not really sure what additional  needs will be and over the last couple of years there were few additional extension  points added and no one wants to get an interface growing more and more and people implementing  it becoming incompatible suddenly.  This was mainly done by Mika out there so if you have any question about it, that's  the right person.  The other main block is LSP4J, it's an implementation of the language server protocol.  It's pure Java implementation and what's important is it's done in a way that you can use it  also to implement servers not only clients.  That actually gives a lot more interest in LSP4J as we already see a number of implementations  based on it.  I forgot to mention that the generic editor, you might look at it as a monocoque or eclipse  ID, it serves similar needs.  How these things are bind together?  We have an LSP4E project, it's not in the eclipse platform but overall same people drive  it on both sides on the top and create the merge between them.  What does LSP4E?  Through LSP4J to hook into the extensions provided by generic editor.  These are the main points which we see as a need for implementing it proper on an ID.  These are the points which we as a platform developers should provide and make it really,  really easy for people to have support for their preferred language.  That picture wouldn't be full without these two small blocks.  One is TM4E, it's a textmate interpreter which allows you to use textmate grammars to do  highlighting in eclipse.  It saves a lot of time again so getting pure highlighting for your language is pretty much  a single step.  The more important part is the debug adapter protocol.  Let's call it a sister protocol to LSP4J.  What it allows is to not integrate with given debugger but use a unified protocol how to  communicate with the burgers.  This gives us easy implementation for the burgers.  I might be wrong, the debug adapter protocol is not yet as popular as the LSP1 but it would  be way more important over time as coming with a good debugging story is actually quite  harder than getting the editor correct.  So with these things I'm moving to some of the success stories we got and pretty much  whoever used Eclipse in the last years and was relying on web tools know the sad story  of JavaScript editor there and the HTML editor was not that great also.  It was great if you were stuck to technology 10 years ago if you were using something modern  it was failing in a spectacular way sometimes.  What we achieved here is unifying a lot of the random plugins people needed to collect  in order to do a modern web development as a single stop and what's more important is  we did that without knowing not enough to even write more than Hello World let's say in TypeScript.  None of us ever developed anything in TypeScript but by reusing the language server this was  easy enough.  Why I call it a success story in addition to all these features users get from community  point of view we saw a number of people just jumping in and being enthusiastic and helping  how to get the whole story running and most of the people they were not seasoned Eclipse  plugin developers but it was easy to come in and help with the integration cost they  knew the technology they wanted to integrate.  That's why doing the IDE site from the IDE people and letting the people knowing the  technology deal with it is so efficient.  Let's say that everyone had some interactions with web development in a way but the other  one was more of an adventure because learning crust I would say it's not as straightforward  as I would like it to be but what came here is we faced the exactly same benefit we have  seen with the wild web developer.  We got the plugin running few missteps with getting the Rust OS integrated.  Now we got the Rust people fully dealing and even leading the project with the need from  Eclipse platform developers being to just hook up some new addition that came on the  IDE site.  So what's needed more or less is to create one integration for your language server,  hook the text made grammar if you have one, if you don't want to write your high-witing  code and if you want to provide really, really nice way for users to start give them a wizard  like in the corrosion one to use a template so they get some of the boilerplate generated  including the build system.  But these are not the only cases.  There are a number of other projects.  What I'm listing here are just projects that are at Eclipse.org or should be soon but in  the bigger community we see others like Apache Camo providing some integration based on language  server and also the SpringTools being language server based tooling more and more.  But this one has kind of a sweet spot for us.  It has the C-sharp integration, the.NET core in Eclipse.  When we started it, it was the very first effort to do a LSP4E based project in Eclipse  and it was based on.NET core because VS Code was really advertising a lot and the language  server for.NET was kind of most separated one and easy to integrate.  I have to admit we kind of not managed to get as many people interested as I would like  but this might be due to some difficulties with getting the framework running and some  limitations coming legally.  So this one, it's still there if someone is interested.  I'm offering all the support to take it over so contact me or Mikael.  We will help as much as we can to get someone taking it in.  So far it will probably stay as it is and also that one, it's a shell script editor.  Whoever tried to parse a shell script knows that this is one of the biggest adventures,  pretty much everything can happen so coming with a grammar is not possible I would say  and ShellWox is the third generation of Eclipse plugins for shell script I do.  So first we were trying to do everything on our own, it worked to some degree then we  based it on.NET, it was a common tooling for dynamic languages.  That was a move in the right direction but there were way too many additional steps needed  and I would say that what we see with LSP4E is this separation finally done in a proper  way and we have Dart board, plugin to integrate Dart in Eclipse.  What I'm mentioning also is Ruby SolarGraph, we had Ruby plugin at Eclipse but it was  stuck in the Ruby 2 time frame so that's one case where we admitted okay we're not  doing good enough job so we dropped this plugin, it's no longer in the simultaneous release  and so on and I strongly recommend people to go to the marketplace and install such  LSP based plugin from marketplace.  Now as every last story JDTLS language server for Java based on top of Eclipse compiler and  other goodies that were coming from Eclipse ID for Java developers.  Now first I would say I was not enthusiastic about it, it was like okay we are going to  make something for the benefit of other ID's so they can embed all the goodness we are  doing, what's the point in all that work.  Actually it appears that JDT managed to get quite a few additional contributors thanks  to the work that was going on JDTLS.  We had to restructure a bit the code base for JDT and that restructure allowed people  to come easier as let's say our core and UI separation was kind of unclear and nowadays  we even saw few patches coming from VS Code guys to get fixes they need and it being used  in the cloud ID's it allows to get people working on JDT full time so that work, that  language server happened to be kind of the most important work in JDT in terms of community  building in the last couple of years, my opinion only.  There will be probably others not agreeing as all the JVM work is also important but  it's a technical work so we should make this separation.  This is soon to come project which integrates, well it doesn't integrate, it's a language  server for XML, XSD, DTD and all the XML technologies.  What is nice about it is that it's an extensible one so we are looking into ways to replace  a POM XML editor in the Maven plugin with a language server based one so we get all  the XML smartness directly from the Lemmings plugin but we extend it additionally with  Maven specific completions and so on.  That would allow how to complete for group ID's, artifact ID's, this is all work in progress  but at least the work has started and we see some of these features coming in, let's  not say the March release but for June there should be a preview in a way.  Seems like I'm almost out of my time so if you have any questions, yes?  The question is which providers we are missing for highlighting so the approach we took is  to not provide highlighting by default.  I personally have everything I need right now.  What is important here is that I'm looking for a bit more full story like not only highlighting  but more of a full language server based editor for given language so people get at least  some error checking and auto completion in addition.  Now this is something I have been thinking of but it never, I never managed to find some  information about it.  It is, if anyone knows of a text made registry in a way so some kind of plugin can be made  based on such a registry and Eclipse ID can auto fetch from this registry, text made grammar  so highlighting can be added on demand.  We are moving away from text made that's the idea right?  That's what we are trying to do.  At least in language server world.  The syntax highlighting is proposed and I'm hoping it will add and if the studio is spawning  it it seems.  Okay I was not aware of that part.  Sorry.  If the protocol changes and provides some new value we want we will integrate it but  on the syntax highlighting from the full perspective we are not pushing for more syntax highlighting  because it's not actually the top of the value we can produce at the moment.  So yeah we are following.  At the beginning when we started this integration we were more involved in the protocol because  it was new and we wanted it to fit with our needs but actually the community is smart and  doing job correctly so things go their way without our intervention so we know we just  follow what's in it and just monitoring because we totally disagree to voice before it's too  late but in general everything is fine and we just implement things when they are there.  Thank you.
eclipselsp1_0_143.png eclipselsp1_144_1403.png eclipselsp1_1404_5951.png eclipselsp1_5952_8447.png eclipselsp1_8448_9743.png eclipselsp1_9744_11543.png eclipselsp1_11544_14135.png eclipselsp1_14136_18035.png eclipselsp1_18036_21227.png eclipselsp1_21228_23183.png eclipselsp1_23184_24851.png eclipselsp1_24852_25103.png eclipselsp1_25104_26027.png eclipselsp1_26028_29207.png eclipselsp1_29208_31259.png eclipselsp1_31260_32099.png eclipselsp1_32100_33011.png eclipselsp1_33012_33071.png eclipselsp1_33072_33119.png eclipselsp1_33120_33191.png eclipselsp1_33192_33239.png eclipselsp1_33240_33287.png eclipselsp1_33288_33335.png eclipselsp1_33336_33395.png eclipselsp1_33396_33455.png eclipselsp1_33456_33515.png eclipselsp1_33516_33563.png eclipselsp1_33564_33611.png eclipselsp1_33612_33695.png eclipselsp1_33696_33779.png eclipselsp1_33780_33827.png eclipselsp1_33828_33899.png eclipselsp1_33900_34079.png eclipselsp1_34080_35783.png eclipselsp1_35784_35901.png