New Plot Types in Tablesaw



NumericColumn x = baseball.nCol("Record");


Although they’re from different libraries, the Tablesaw API is similar:

Table baseball = Table.createFromCsv("data/baseball.csv");







Finally, we have a BoxPlot.




I hope you’ll find Tablesaw useful for your data analytics work.


Tablesaw gets Graphic

Today we introduced the first elements of what will be Tablesaw’s support for exploratory data visualization in pure Java. As Tablesaw expands its scope to integrate statistical and machine learning capabilities, this kind of visualization will be critical.tornadosThis slightly ghostly US map image was created by as a simple scatter plot of the starting latitude and longitude for every US tornado between 1950 and 2014. The code below loads the data, filters out missing records, and renders the plot:

Table tornado = Table.createFromCsv("data/tornadoes_1950-2014.csv");

tornado = tornado.selectWhere(
    both(column("Start Lat").isGreaterThan(0f),
         column("Scale").isGreaterThanOrEqualTo(0)));"US Tornados 1950-2014",
    tornado.numericColumn("Start Lon"),
    tornado.numericColumn("Start Lat"));

These plots provide visual feedback to the analyst while she’s working. They’re for discovery, rather than for presentation, and ease of use is stressed over beauty. Behind the scenes, the charts are created with Tim Molter’s awesome XChart library:

The following chart is taken from a baseball data set. It shows how to split a table on the values of one or more columns, producing a series for each group. In this case, we color the mark differently if the team made the playoffs. winsByYear

Here’s the code:

Table baseball = Table.createFromCsv("data/baseball.csv");"Regular season wins by year",

A chart that looks like a scatter plot and works like a histogram is a Quantile Plot. The plot below presents the distribution of public opinion poll ratings for one US president.


This chart was build using the Quantile class:

String title = "Quantiles: George W. Bush (Feb. 2001 - Feb. 2004)";, bush.numericColumn("approval"));

Further down the line, I expect to add JavaScript plot support based on D3. These plots will be focused more on presentation, especially Web-based presentation, as Tablesaw becomes a complete platform for data science.


New: Load data from any RDBMS

Ted Codd

As of today, you can easily import into Tablesaw from any data source with a JDBC driver. Meaning, pretty much every relational database. Meaning, we are now fully compliant with the 1970s, and with Ted Codd, who I’ve stolen many ideas from. Now I’m repaying Ted by putting his photo in this post.

Thank you, Ted.

To use this feature, you write standard Java/JDBC client code, execute a query, and pass the returned ResultSet into a static create() method on Table.  There’s a simple example below.

So bring on your databases.


String DB_URL = "jdbc:derby:CoffeeDB;create=true";
Connection conn = DriverManager.getConnection(DB_URL);

Table customer = null; 
try (Statement stmt = conn.createStatement()) {
  String sql = "SELECT * FROM Customer";
  try (ResultSet results = stmt.executeQuery(sql)) {
    customer = Table.create(results, "Customer");


RAM eats Big Data

My work on Tablesaw is focused on making as many data-munging jobs doable on a single machine as possible. It’s a tall order, but I’ll be getting a lot of help from hardware trends.

KDNuggets recently posted poll results that show that most analytics don’t require “Big Data” tools. The poll asked data scientists about the largest data sets they work with, and found that the largest were often not so large.

In another post based on that poll, they note that:


In other words, most people can do their work on a laptop.


The more interesting finding was that RAM is growing faster than data. By their estimate, RAM is growing at 50% per year, while the trend for the largest data sets is increasing at 20% per year.

If your laptop is too small, you can probably get your work done faster, easier, and cheaper by leasing a server on the cloud. This is basically the findings of  Nobody ever got fired for using Hadoop on a Cluster out of Microsoft Research, which discusses the cost tradeoffs of using distributed “big data” tools like Spark and Hadoop. Their summary:


A post from the FastML blog quotes another Microsoft Researcher Paul Mineiro:

Since this is my day job, I’m of course paranoid that the need for distributed learning is diminishing as individual computing nodes… become increasingly powerful.

When he wrote that, Mineiro was taking notes at a talk by Stanford prof. Jure Leskovic. Leskovic is co-author of the text Mining of Massive Datasets, so he understands large-scale data crunching. What he said was:

Bottom line: get your own 1TB RAM server
Jure Leskovic’s take on the best way to mine large datasets.


Pretty soon, you can have one, too. Amazon has dropped hints that EC2 instances with 2 TB of RAM are coming soon. Once you have one, you can make the most of it by using a RAM optimized data manipulation tool. This is, of course, the idea behind Tablesaw.