Here Come The Notebooks

Post written by

Noah Gift

Founder of Pragmatic AI Labs. Lecturer and consultant at UC Davis Graduate School of Management’s MSBA program. Author of Pragmatic AI.

About five years ago in business school, almost every MBA course used Excel. Now, as someone who teaches at business school, I've seen firsthand how almost every class uses some type of notebook technology. Outside of the classroom, businesses are rapidly adopting notebook-based technologies that either replace or augment traditional spreadsheets. There are two main flavors of notebook technology: Jupyter Notebooks and R Markdown.

Of the two notebook technologies, Jupyter has been on the faster tear as of late. Google has versions of Jupyter via Colab notebooks and DataLab. Amazon Web Services (AWS) uses Jupyter technology in Sagemaker, Kaggle uses Jupyter technology to host its data science competitions, and companies like Databricks, which is a managed Spark provider, also use Jupyter. Back in 2015, Project Jupyter received $6 million in funding via grants from the Gordon and Betty Moore Foundation as well as the Helmsley Charitable Trust. These funds have been funneled into upgrading a core technology that is in use by all major could providers, including AWS, Google and Azure. Each cloud provider adds there custom twist (in the case of Google, it is GCP integration, and in the case of Azure, it is F# and R integration).

Why are all of these companies using Jupyter? The biggest reason is that it just works. Another reason is that Python has become the standard for doing data science and machine learning.

Of the flavors of Jupyter Notebook, Colab notebooks, developed by Google, is one the newest and more compelling versions. It capitalizes on the existing Google docs ecosystem by adding the ability to open, edit, share and run Jupyter notebooks. If you want to see what this is like, I have several Colab notebooks that cover the basics of machine learning with Python that you can explore. If this is your first experience with either Jupyter or Colab notebooks, your jaw might drop when you see what it can do.

William Gibson had a great quote: "The future is already here -- it's just not very evenly distributed." This is certainly the case with notebook technology. It has already disrupted the data world, and it is probably already in your company, whether you personally use it or not. One specific way it has interrupted the data world is that it is the default platform for doing data science projects. Essentially, if you use a cloud and do data science, there is a very good chance you will be using Jupyter technology. Second, some of the inherent limitations of traditional spreadsheets, like dealing with large data, disappear with the ability to write code that talks to cloud database technology. And, if there is a vendor providing a service in the data science ecosystem, it is assumed there is an example in Jupyter and a workflow in Jupyter.

One of the powerful side effects of this disruption is going to be how it puts machine learning and data science directly into the hands of business professionals. Many disruptions have an effect of displacing workers, but this disruption is very empowering. Within a few years (or less), it wouldn't be a stretch to say that interactive data science notebooks will be as pervasive or more pervasive than traditional spreadsheets.

As I mentioned earlier, there is another flavor of notebooks called R Markdown, which is a markdown-based notebook technology used heavily by the R community. There are many compelling features of R Markdown, including the ability to output to many formats like interactive Shiny dashboards, PDF and even Microsoft Word. Shiny is an interactive dashboard technology that is created using only the R language and is becoming a popular choice for interactive data visualization.

In the case of both of these predominant notebook technologies, they are authored via Markdown, a simple and lightweight markup language that is also heavily used in Github. The advantage of using Markdown is that it takes out the tricky step of needing to master HTML, CSS and Javascript to author rich documents. Additionally, Github even supports rending out Jupyter notebooks as web pages natively. Now that Microsoft has agreed to purchase Github, you can see that the major cloud vendors are heavily backing markdown and notebook technology, including Jupyter.

One of the best ways to understand notebooks is to use them. There are several ways to get started using notebooks in the workplace. Perhaps the easiest way to get started in your job is to create a shared Google Colab notebook. Many business professionals are familiar with using a shared Google Document, and a similar style can be applied to a share Colab notebook. First, open up a Colab notebook starting from the welcome page, change it, then share it with a co-worker or two.

In about 30 seconds, you can be using the same workflow as the majority of data scientists around the world.


Founder of Pragmatic AI Labs. Adjunct professor at UC Davis MSBA and Northwestern Graduate Data Science program. Author of Pragmatic AI.