Editing Simulation
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Tümer, Mert
Performance measurements are really challenging when it comes to real life situations. One of them was a user experience while typing on a document. This presentation covers that how we can achieve reliable and repeatable performance tests by implementing clever tools. These tools allow us to find out what happens in the background when a lot of people are typing rapidly and what we can optimize out of it.
 you Hello, my name is Matt. I work at Collaboration Productivity as a consultant software engineer for three years. Today I'm going to talk about one of our recent works, which is about editing simulation. So what is it? Imagine you are having a document session with other people in it. And everybody is editing a part of the document. One would expect a smooth experience with this. We created this tool to simulate this experience, number of people in the document, each in different part of it and just writing some text. What is written is not important, of course, but what happens when we do it is what we care about. Also, we want to think that everybody is a really fast typist because bad things happens when you are fast, which this is all about. The tool allows us to adjust various things such as the typing speed, duration and number of people. Also, it can give us some statistics as well to measure the performance and the user experience. Of course, this tool is not enough by itself, which I'm going to mention in the following slides. So why do we need this exactly? The main reason is that it is very difficult and time-consuming to measure the performance without proper tools, especially when you have a client server application, possibly even more difficult to have a solid idea for our case, which is the collaborative editing. In such cases, we have problems like who would record the performance and what will others do while recording, what is the specs of the computer we use for recording or how can we make sure what happens when we do something and not to mention the concurrency with others or typing speeds of individuals. So the results would not be reliable because we may not be able to repeat the exact same steps as a software would do. Also, the hardware would be different for everybody. Therefore, we need a repeatable test environment with known hardware and reliable numbers because to see the real problem, the process should be exact and so the results. We would like to see these reliable results, especially when we improve something and we would like to tell the world that like, hey, we improved 10% this or that or we are 50% faster now. All of these can be achieved with an automated tool. Before I move on to a real example, which I have discovered, it was like many other issues because LibreOffice was not really meant to be used by multiple people in the same document. But before that, let's talk about how you can use this tool in case you would want to get involved with our project. The tool runs on the terminal. You just CD into the online directory slash browser and type make per test. It is enough to start the session with the predetermined parameters. Of course, you can change them however you like, but unfortunately, they are not documented just yet. But you can easily check them inside the make file. I mentioned that the tool was not enough by itself. You preferably need to use other tools along with it, such as Linux Perf and Hotspot. Linux Perf can record the CPU stats based on a process and Hotspot can display these recordings with very beautiful flame grabs, which I am going to show you in the following slides. If you want to get involved in terms of development, the project consists of only two files. One is Bootstrap.js. It starts the test, runs the collaboration online in the background and spawns the child views. Also, it controls the lifecycle of them as they are separate processes. But the fun stuff happens in Bootstrap.js actually. It represents a child view and each of them enters the document, jumps into a different location. Also, each has a different zoom level because we want everything to be rendered for each user at the same time and we don't want our caching mechanism to interfere. A kind of a torture test, really. And they start typing with the given interval of time and the duration. The test starts by default 100ms between each key event. But we can adjust it to smaller numbers because a real keyboard measure can be fast as 20ms with typing each letter. Only technology that is used is Node.js with the project. If you want to get involved and know JavaScript, you can easily do it and improve it even further. To summarize, as promised, I'm going to show you a real example that I dealt with in the next slide. Also, worth mentioning that we discovered various other problems with this tool. They were related to the sidebar, text composition, networking, rendering and many more. The issue I discovered made us gain approximately 30% performance in CPU time. It really showed us what we can do with this tool and we made a lot of other improvements after that. All thanks to these wonderful people that I mentioned in the slide. The beauty of this tool is that we can run it as many times as we want and check the results. And by adjusting the parameters, each time you can discover something new. Also, we plan to add more document types and scenarios in the future to test other kinds of things related to performance. There are a lot of possibilities that we can apply and it doesn't need to be just about typing. So we have a flame graph here. It goes up even beyond this. It is recorded with this tool and it displays what happens and the amount of time for a specific function that is called each time someone different types of letter. The function here at the bottom is named setView, but I'm not sure if it will be visible in the video. But I said earlier that LibreOffice is not meant to be used by more than one person, but I like. Interesting enough, it actually supports multiple views and you can see it for yourself on the top toolbar under Window and New Window option. It creates another window with a new view or a cursor, you may say. I don't know the reason for this actually, because why do we need a new window for the same document with a single keyboard and mouse? But that feature is what our product is based on today. So coming back to the problem, we create a new window or view in the background for each user and this setView function changes these views as the name suggests. According to the user who sent the key event. All of other functions above are called after setView. As you can see, there are a lot of other functions are involved for such a small change. I discovered that the problem was at the LibreOffice core site. Good knows that it was a very easy fix and this piece of code left for a unit test to run correctly. But it was not needed for our end. Usually we don't notice this performance issues with single user and this kind of code pieces can be left because naturally no one considers the multiple view editing case. After the fix, the result is as seen much, much better. All of the other pile of functions are gone from the graph. So that is it for me. Thank you so much for listening and I wish everybody to have a wonderful conference.
 you  Hello, my name is Matt. I work at Collaboration Productivity as a consultant software engineer for three years.  Today I'm going to talk about one of our recent works, which is about editing simulation.  So what is it?  Imagine you are having a document session with other people in it.  And everybody is editing a part of the document.  One would expect a smooth experience with this.  We created this tool to simulate this experience, number of people in the document, each in different part of it and just writing some text.  What is written is not important, of course, but what happens when we do it is what we care about.  Also, we want to think that everybody is a really fast typist because bad things happens when you are fast, which this is all about.  The tool allows us to adjust various things such as the typing speed, duration and number of people.  Also, it can give us some statistics as well to measure the performance and the user experience.  Of course, this tool is not enough by itself, which I'm going to mention in the following slides.  So why do we need this exactly?  The main reason is that it is very difficult and time-consuming to measure the performance without proper tools,  especially when you have a client server application, possibly even more difficult to have a solid idea for our case, which is the collaborative editing.  In such cases, we have problems like who would record the performance and what will others do while recording,  what is the specs of the computer we use for recording or how can we make sure what happens when we do something  and not to mention the concurrency with others or typing speeds of individuals.  So the results would not be reliable because we may not be able to repeat the exact same steps as a software would do.  Also, the hardware would be different for everybody.  Therefore, we need a repeatable test environment with known hardware and reliable numbers  because to see the real problem, the process should be exact and so the results.  We would like to see these reliable results, especially when we improve something and we would like to tell the world that  like, hey, we improved 10% this or that or we are 50% faster now.  All of these can be achieved with an automated tool.  Before I move on to a real example, which I have discovered, it was like many other issues  because LibreOffice was not really meant to be used by multiple people in the same document.  But before that, let's talk about how you can use this tool in case you would want to get involved with our project.  The tool runs on the terminal. You just CD into the online directory slash browser and type make per test.  It is enough to start the session with the predetermined parameters.  Of course, you can change them however you like, but unfortunately, they are not documented just yet.  But you can easily check them inside the make file.  I mentioned that the tool was not enough by itself.  You preferably need to use other tools along with it, such as Linux Perf and Hotspot.  Linux Perf can record the CPU stats based on a process and Hotspot can display these recordings  with very beautiful flame grabs, which I am going to show you in the following slides.  If you want to get involved in terms of development, the project consists of only two files.  One is Bootstrap.js. It starts the test, runs the collaboration online in the background and spawns the child views.  Also, it controls the lifecycle of them as they are separate processes.  But the fun stuff happens in Bootstrap.js actually.  It represents a child view and each of them enters the document, jumps into a different location.  Also, each has a different zoom level because we want everything to be rendered for each user at the same time  and we don't want our caching mechanism to interfere.  A kind of a torture test, really.  And they start typing with the given interval of time and the duration.  The test starts by default 100ms between each key event.  But we can adjust it to smaller numbers because a real keyboard measure can be fast as 20ms with typing each letter.  Only technology that is used is Node.js with the project.  If you want to get involved and know JavaScript, you can easily do it and improve it even further.  To summarize, as promised, I'm going to show you a real example that I dealt with in the next slide.  Also, worth mentioning that we discovered various other problems with this tool.  They were related to the sidebar, text composition, networking, rendering and many more.  The issue I discovered made us gain approximately 30% performance in CPU time.  It really showed us what we can do with this tool and we made a lot of other improvements after that.  All thanks to these wonderful people that I mentioned in the slide.  The beauty of this tool is that we can run it as many times as we want and check the results.  And by adjusting the parameters, each time you can discover something new.  Also, we plan to add more document types and scenarios in the future to test other kinds of things related to performance.  There are a lot of possibilities that we can apply and it doesn't need to be just about typing.  So we have a flame graph here.  It goes up even beyond this.  It is recorded with this tool and it displays what happens and the amount of time for a specific function that is called each time someone different types of letter.  The function here at the bottom is named setView, but I'm not sure if it will be visible in the video.  But I said earlier that LibreOffice is not meant to be used by more than one person, but I like.  Interesting enough, it actually supports multiple views and you can see it for yourself on the top toolbar under Window and New Window option.  It creates another window with a new view or a cursor, you may say.  I don't know the reason for this actually, because why do we need a new window for the same document with a single keyboard and mouse?  But that feature is what our product is based on today.  So coming back to the problem, we create a new window or view in the background for each user and this setView function changes these views as the name suggests.  According to the user who sent the key event.  All of other functions above are called after setView.  As you can see, there are a lot of other functions are involved for such a small change.  I discovered that the problem was at the LibreOffice core site.  Good knows that it was a very easy fix and this piece of code left for a unit test to run correctly.  But it was not needed for our end.  Usually we don't notice this performance issues with single user and this kind of code pieces can be left because naturally no one considers the multiple view editing case.  After the fix, the result is as seen much, much better.  All of the other pile of functions are gone from the graph.  So that is it for me.  Thank you so much for listening and I wish everybody to have a wonderful conference.
lotech_editingsimulation_0_179.png lotech_editingsimulation_180_935.png lotech_editingsimulation_936_1295.png lotech_editingsimulation_1296_3203.png lotech_editingsimulation_3204_3275.png lotech_editingsimulation_3276_5819.png lotech_editingsimulation_5820_6371.png lotech_editingsimulation_6372_7799.png lotech_editingsimulation_7800_9179.png lotech_editingsimulation_9180_10067.png lotech_editingsimulation_10068_11783.png lotech_editingsimulation_11784_11927.png lotech_editingsimulation_11928_11975.png lotech_editingsimulation_11976_14195.png lotech_editingsimulation_14196_14999.png lotech_editingsimulation_15000_15251.png lotech_editingsimulation_15252_15467.png lotech_editingsimulation_15468_15614.png