Rawgraphs: An open visualization framework for open outputs
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Uboldi, Giorgio
RAWGraphs is an open source web application for the creation of static data visualisations that are designed to be further modified. Originally conceived for graphic designers to provide a series of tasks not available with other tools, it evolved into a platform widely used in research and data journalism contexts that provides simple ways to map data dimensions onto visual variables. It presents a chart-based approach to data visualisation: each visual model is an independent module exposing different visual variables that can be used to map data dimensions. Consequently, users can create complex data visualisations. Finally, the tool is meant to produce outputs that are open, that is, not subjected to proprietary solutions, which can be further edited. Thanks to an intuitive user interface and experience drafting visualizations become an easy task, enabling the user to produce the visualizations not only as a mere output but also as a tool within the research process. Last year we launched a successful crowdfunding campaign to raise funds for the redesign and development of a new version of RAWGraphs, that will be released in the first months of 2021. The new version is written from scratch with the aim to make the tool more flexible for customisation and to create an active community also on the development side. The talk aims at presenting how RAWGraphs has been used in research context and our strategies to keep the project free, open source, economically sustainable and independent.
 Hello everyone and thank you very much for giving us the opportunity to talk about our project today. I'm Giorgio and I am part of the team of people who design, develop and maintain ROGRAPH. The team is composed by Density Design, our search lab at Politecnico di Milano that is also been the birthplace of the project, initially conceived and developed by Giorgio Caviglia. Then there is Calibro, a design studio founded by Mie Matteo Atzi after our experience at Density Design. And then we have in Magic a development studio focused on open source web application, mobile apps and data management systems. The project is seven years old and today we would like to briefly talk about the reasons why we created this project, in which context and how it's used. And we will also talk about how the project is evolving and give you a preview of the upcoming new version that we will release publicly in a few weeks. So I think that some of you may be familiar with ROGRAPH but I would like to start from the beginning. And first of all, what is ROGRAPH? ROGRAPH is an open source data visualization tool and framework built with the goal of making the visual exploration and representation of complex data easy. It was conceived mostly for designers and it helps creating semi-finished vector visualizations that are easy to refine and aided with other softwares. The tool is freely available at ROGRAPH.io and I would like to show you a very quick demo on how it works and the steps that you need to do in order to produce a visualization with ROGRAPH. So first of all, you need to take a data set from any spreadsheet software, you copy and you paste it inside ROGRAPH. Then you can select a layout within the charts built on top of the 3JS that are available. Some of them are common charts like scatter plots or bar charts while others are a bit more difficult to reproduce with other softwares like alluvial diagrams or horizon charts. After this, the user has to map the dimension of the data set with the visual variables of the selected charts through an easy to use drag and drop interface. For example, for a scatter plot, the visual variables you can define are the x and y axis, the area and the color of the bubbles and the labels. As soon as the data set dimensions are mapped, you see the visualization appear below and you have the possibility to change some options and finally download the visualization as a raster image or as an open vector file. The tool, as I said, was originally created as an internal tool and a site project-attended design, our research lab within the communication design department at Polytechnico di Milano. As designers and researchers focused on the visualization of data and complex phenomena, we were always looking for new tools and technologies that could help us during our daily activities at the lab and for producing these kind of outputs. In our very iterative process of exploration and visualization, we always use different tools and approaches. At the time, there were not so many tools available like now for producing complex and non-conventional charts. Some of them were either expensive or others required development skills and a lot of time. So we thought that it was a gap and we decided to make our own tool. Here, you can see one of the very, very first versions of the tool, a simple interface, no frameworks, just JavaScripts and HTML and a few selection of charts that were made with the 3JS. After a while, we started working on more functionalities and we improved the user interface. We added more charts and further details. We added more charts and further functionalities. And while we were developing it, we understood that what we were doing could have been useful also for other researchers and other people with the same needs. So since we were always been fascinated and also thankful to the open source community, we found natural to publish the code and the tool with an open license to allow everyone to use it, collaborate and improve it. Designers, students, journalists, historians, scientists, activists started including row graphs in their toolkits and they also started sharing their work done with row graphs with us. Here at this page of our website, you can see some of the examples that were made with row graphs. Another positive note and a positive feedback is that the tool has also became widely used in the tactical activities and many resources, tutorials and presentations were shared online. So from 2013 to 2019, apart from some sponsorships, we kept on working on row graphs as a site project and basically for free. We added the charts and features according to our needs without following a clear roadmap of development. So we got to a point where we realized that if we wanted to keep on working on row graphs and if you wanted to keep row graphs still updated, we had to invest more time and resources into it. So this is why in 2019, we decided that it was time to find a sustainable way to maintain the project. The goal was to fairly compensate the people that worked on it and also expand our team, dedicate the right amount of time and resources and skills in it. So in order to accomplish this, we launched a crowdfunding campaign on Indiegogo to raise funds to support the design and development of a brand new version of row graphs. And we went for the crowdfunding option because we thought it was an interesting way to get in touch with our pretty wide user base and also to continuing being dependent. The most important thing for us was to keep the tool 100% free and open to everyone, staying somehow true to the original spirit of the project. And after a couple of months, we launched our campaign, we likely reached the goal and we got general sponsorship from academic institution, research lab and private companies. We use row graphs regularly. We received more than 400 donations from 40 countries and since we reached the goal, we kept the campaign open for the whole year. Here are some of the sponsors that have generously supported us. So after the campaign, we dedicated a full year to design and develop what we promised in the campaign. And in September, we released the biggest update of row graphs to the main supporters of the crowdfunding campaign as a perk for being so generous with us and as a way also to get feedback and insights about how the new version is received. So let's get into the details of this new version and see some previews of what will be released to everyone in the coming weeks. So first of all, we have to clarify the fact that this is not just a simple update of the software, but is a completely refactored version from the core, the row graphs library, to the interface. The three things that are at the center of the project, so the library, the visual models and the interface have been divided in three interlinked repositories released under Apache 2. The very first period of development was spent on defining and implement a brand new library, as I said. So the new library is written in ES6 and makes use of updated releases of third-party libraries like this 3JS. Our goal is to have a more robust and flexible library to better handle the data input and ease the implementation of new features and new charts. And thanks to the fact that the row graphs library is independent from the row graphs application, it will be possible to use it in any web application created with HTML and JavaScript to bring the simplicity, the data mapping concepts and the visual models of row graphs to any other project. The library that will be available as well in the coming weeks as a free and open source library. The library that will be available as well in the coming weeks with the app will be published in a separate and dedicated repository with documentation about how to use it and how to integrate it in other projects. After the library, a lot of effort has been also put in refactoring already existing charts and adding new ones based on the request of our users. It's a mix of conventional and unconventional charts that allows different types of data exploration and data visualizations. The app and the interface of the app was completely redesigned, keeping in mind the design principles that we used from the first version. So a simple one-page app divided in different sections, each one dedicated to one particular aspect of the process. So now I would like to present the new features of row graphs and give you a preview of the new app that will be released in the coming weeks. So the first part is related to the loading data process. So as in row graphs one, you can do input different kind of data. You can paste it from a table, you can load a CSV, a CSV, a JSON, or you can load a file from a URL. What we included now is a completely new part that is related to the possibility to specify data types and to change the data parsing options. Then when we go to the section about selecting the chart, we can see that we expanded the collection and we presented and organized it in a better way. The users can filter charts based on the type and they will have access to the source code and to the learning guides. In the future, we will also add a possibility to load custom charts in an easy way. Then if we scroll down, we go to the mapping section. The mapping process is very similar to row graphs one in terms of interaction, but we introduced the possibility to choose also the way you aggregate data. Since in the previous version, the only way was the addition. So for some charts, we also included, as you can see maybe in the preview above, the possibility to create series to automatize the work of producing a lot of charts. Then if we scroll down, we can see the preview of the charts and we can see that we added a lot of options that were not present in the first version of row. So even if row graphs is still focused on producing, let's say, semi-finished visualization, we felt the need to add more control over the output. And thanks to the new library, we were able to introduce more options. According to the type of chart, the user is able to manipulate the artwork options, the legends, edit the color scales, and other options related to each chart type. Here there are some examples of the different layouts and all the different options that you can change. And then we get to the export section and is where we can see one of the biggest updates of the project. So as in row graphs one, you could export the raster format, so PNG and JPEGs, or open vector format that can be then refined with other software or can be included in web pages as SVG. What we included that is completely new is the possibility to export projects and reopen it again in other sessions. The project is a file that includes the data, the mapping options, and the charts options. So you can go back to the app, open the project, and start from where you left. So at the moment of this presentation, row graphs 2 is available only to our sponsors and to the supporters of the campaign. But by the end of the month, it will be publicly available to everyone and the repositories as well. For us, it will be a very important moment to collect feedback and publish tutorials, guides, and documentation and work on improving functionalities, fixing bugs that we will encounter and involve also the community around the project. After this phase, we will also think about other ways to keep the project financially sustainable through other types of sponsorships and donations. And that's all for today, and we would like to hear your feedback and questions about our project.
 Hello everyone and thank you very much for giving us the opportunity to talk about our  project today. I'm Giorgio and I am part of the team of people who design, develop and  maintain ROGRAPH. The team is composed by Density Design, our search lab at Politecnico di Milano  that is also been the birthplace of the project, initially conceived and developed by Giorgio  Caviglia. Then there is Calibro, a design studio founded by Mie Matteo Atzi after our experience  at Density Design. And then we have in Magic a development studio focused on open source  web application, mobile apps and data management systems. The project is seven years old and  today we would like to briefly talk about the reasons why we created this project, in which  context and how it's used. And we will also talk about how the project is evolving and give you  a preview of the upcoming new version that we will release publicly in a few weeks. So I think  that some of you may be familiar with ROGRAPH but I would like to start from the beginning. And  first of all, what is ROGRAPH? ROGRAPH is an open source data visualization tool and framework  built with the goal of making the visual exploration and representation of complex data  easy. It was conceived mostly for designers and it helps creating semi-finished vector  visualizations that are easy to refine and aided with other softwares. The tool is freely  available at ROGRAPH.io and I would like to show you a very quick demo on how it works and the steps  that you need to do in order to produce a visualization with ROGRAPH. So first of all,  you need to take a data set from any spreadsheet software, you copy and you paste it inside  ROGRAPH. Then you can select a layout within the charts built on top of the 3JS that are  available. Some of them are common charts like scatter plots or bar charts while others are a  bit more difficult to reproduce with other softwares like alluvial diagrams or horizon charts.  After this, the user has to map the dimension of the data set with the visual variables of the  selected charts through an easy to use drag and drop interface. For example, for a scatter plot,  the visual variables you can define are the x and y axis, the area and the color of the bubbles  and the labels. As soon as the data set dimensions are mapped, you see the visualization appear below  and you have the possibility to change some options and finally download the visualization  as a raster image or as an open vector file. The tool, as I said, was originally created as an  internal tool and a site project-attended design, our research lab within the communication design  department at Polytechnico di Milano. As designers and researchers focused on the visualization of  data and complex phenomena, we were always looking for new tools and technologies that could help us  during our daily activities at the lab and for producing these kind of outputs.  In our very iterative process of exploration and visualization, we always use different tools and  approaches. At the time, there were not so many tools available like now for producing complex  and non-conventional charts. Some of them were either expensive or others required development  skills and a lot of time. So we thought that it was a gap and we decided to make our own tool.  Here, you can see one of the very, very first versions of the tool, a simple interface, no  frameworks, just JavaScripts and HTML and a few selection of charts that were made with the 3JS.  After a while, we started working on more functionalities and we improved the user  interface. We added more charts and further details. We added more charts and further  functionalities. And while we were developing it, we understood that what we were doing could have  been useful also for other researchers and other people with the same needs. So since we were always  been fascinated and also thankful to the open source community, we found natural to publish the  code and the tool with an open license to allow everyone to use it, collaborate and improve it.  Designers, students, journalists, historians, scientists, activists started including row graphs  in their toolkits and they also started sharing their work done with row graphs with us. Here at  this page of our website, you can see some of the examples that were made with row graphs.  Another positive note and a positive feedback is that the tool has also became widely used  in the tactical activities and many resources, tutorials and presentations were shared online.  So from 2013 to 2019, apart from some sponsorships, we kept on working on row graphs as a site project  and basically for free. We added the charts and features according to our needs without  following a clear roadmap of development. So we got to a point where we realized that if we wanted  to keep on working on row graphs and if you wanted to keep row graphs still updated, we had to invest  more time and resources into it. So this is why in 2019, we decided that it was time to find a  sustainable way to maintain the project. The goal was to fairly compensate the people that worked  on it and also expand our team, dedicate the right amount of time and resources  and skills in it. So in order to accomplish this, we launched a crowdfunding campaign on Indiegogo  to raise funds to support the design and development of a brand new version of row graphs.  And we went for the crowdfunding option because we thought it was an interesting way to get in  touch with our pretty wide user base and also to continuing being dependent. The most important  thing for us was to keep the tool 100% free and open to everyone, staying somehow true to the  original spirit of the project. And after a couple of months, we launched our campaign, we likely  reached the goal and we got general sponsorship from academic institution, research lab and private  companies. We use row graphs regularly. We received more than 400 donations from 40 countries and  since we reached the goal, we kept the campaign open for the whole year.  Here are some of the sponsors that have generously supported us.  So after the campaign, we dedicated a full year to design and develop what we promised in the  campaign. And in September, we released the biggest update of row graphs to the main supporters  of the crowdfunding campaign as a perk for being so generous with us and as a way also to get feedback  and insights about how the new version is received. So let's get into the details of this new version  and see some previews of what will be released to everyone in the coming weeks.  So first of all, we have to clarify the fact that this is not just a simple update of the software,  but is a completely refactored version from the core, the row graphs library, to the interface.  The three things that are at the center of the project, so the library, the visual models and  the interface have been divided in three interlinked repositories released under Apache 2.  The very first period of development was spent on defining and implement a brand new library,  as I said. So the new library is written in ES6 and makes use of updated releases of third-party  libraries like this 3JS. Our goal is to have a more robust and flexible library to better handle the  data input and ease the implementation of new features and new charts. And thanks to the fact  that the row graphs library is independent from the row graphs application, it will be possible  to use it in any web application created with HTML and JavaScript to bring the simplicity,  the data mapping concepts and the visual models of row graphs to any other project.  The library that will be available as well in the coming weeks as a free and open source library.  The library that will be available as well in the coming weeks with the app  will be published in a separate and dedicated repository with documentation about how to use it  and how to integrate it in other projects. After the library, a lot of effort has been also put in  refactoring already existing charts and adding new ones based on the request of our users.  It's a mix of conventional and unconventional charts that allows different types of data exploration  and data visualizations. The app and the interface of the app was completely redesigned,  keeping in mind the design principles that we used from the first version. So a simple one-page app  divided in different sections, each one dedicated to one particular aspect of the process. So now  I would like to present the new features of row graphs and give you a preview of the new app that  will be released in the coming weeks. So the first part is related to the loading data process. So  as in row graphs one, you can do input different kind of data. You can paste it from a table,  you can load a CSV, a CSV, a JSON, or you can load a file from a URL. What we included now  is a completely new part that is related to the possibility to specify data types and to  change the data parsing options. Then when we go to the section about selecting the chart,  we can see that we expanded the collection and we presented and organized it in a better way.  The users can filter charts based on the type and they will have access to the source code  and to the learning guides. In the future, we will also add a possibility to load custom charts in an  easy way. Then if we scroll down, we go to the mapping section. The mapping process is very  similar to row graphs one in terms of interaction, but we introduced the possibility to choose also  the way you aggregate data. Since in the previous version, the only way was the addition. So for  some charts, we also included, as you can see maybe in the preview above, the possibility to  create series to automatize the work of producing a lot of charts. Then if we scroll down, we can see  the preview of the charts and we can see that we added a lot of options that were not present in the  first version of row. So even if row graphs is still focused on producing, let's say,  semi-finished visualization, we felt the need to add more control over the output. And thanks to the  new library, we were able to introduce more options. According to the type of chart, the user is able  to manipulate the artwork options, the legends, edit the color scales, and other options related to  each chart type. Here there are some examples of the different layouts and all the different options  that you can change. And then we get to the export section and is where we can see one of the biggest  updates of the project. So as in row graphs one, you could export the raster format, so PNG and JPEGs,  or open vector format that can be then refined with other software or can be included in web pages  as SVG. What we included that is completely new is the possibility to export projects  and reopen it again in other sessions. The project is a file that includes the data,  the mapping options, and the charts options. So you can go back to the app, open the project,  and start from where you left. So at the moment of this presentation, row graphs 2 is available only  to our sponsors and to the supporters of the campaign. But by the end of the month, it will be  publicly available to everyone and the repositories as well. For us, it will be a very important moment  to collect feedback and publish tutorials, guides, and documentation and work on improving functionalities,  fixing bugs that we will encounter and involve also the community around the project.  After this phase, we will also think about other ways to keep the project financially  sustainable through other types of sponsorships and donations. And that's all for today,  and we would like to hear your feedback and questions about our project.
open_research_rawgraphs_0_3251.png open_research_rawgraphs_3252_3395.png open_research_rawgraphs_3396_3503.png open_research_rawgraphs_3504_3551.png open_research_rawgraphs_3552_4055.png open_research_rawgraphs_4056_4307.png open_research_rawgraphs_4308_4367.png open_research_rawgraphs_4368_4475.png open_research_rawgraphs_4476_4691.png open_research_rawgraphs_4692_5003.png open_research_rawgraphs_5004_6743.png open_research_rawgraphs_6744_6959.png open_research_rawgraphs_6960_7163.png open_research_rawgraphs_7164_7367.png open_research_rawgraphs_7368_7475.png open_research_rawgraphs_7476_18635.png open_research_rawgraphs_18636_18983.png open_research_rawgraphs_18984_19103.png open_research_rawgraphs_19104_19295.png open_research_rawgraphs_19296_19391.png open_research_rawgraphs_19392_19595.png open_research_rawgraphs_19596_22881.png