Open Source Software is Important for Modern Science

A Call for Open Crowdsourcing Platforms

Open Source Spot










Locally crowdsourced weather notifications to improve model accuracy

The weather is sometimes extremely localized: it might be snowing where you are, but two blocks away it might be sunny. This level of granularity is rarely captured by weather forecasts, since obtaining such high-resolution data and making forecasts at this micro scale is extremely difficult.

I’m hoping to solve this problem with pressureNET, and we’ve just shipped a new feature that I’m really excited about. We’ve improved our current conditions feature by adding a new widget to quickly submit current conditions as well as a notification if someone nearby reports a condition.

Nearby snowWeather over North AmericaNearby snow notification

You’ll get an alert about the nearby weather and will be prompted to report what’s it like outside where you are. We’ll combine this data with our live stream of atmospheric pressure data to create very high-resolution, personalized weather forecasts that are validated against reality. This will take us some time, so for now, enjoy looking at the map and watching the weather change block-by-block in your city (if enough people contribute)!

Soon we’ll add a feature to show animations of these conditions over time – it’ll be really cool to watch storms roll across cities, and this data will help validate the forecasts we build.

pressureNET 4.0 Beta: testing new methods of crowdsourcing weather data






Imagining Success for Today’s Startups

Imagine 20 to 30 years out and pretend your favourite startups today have been wildly successful. What kind of world will they have created? Google seems to be on a quest to develop AI and provide it to everyone for free. SpaceX is aiming for a thriving Martian colony by using rapidly and completely reusable rockets and spaceships. Tesla is working towards most ground transportation being fast, electric and solar. Planetary Resources will mine asteroids for rocket fuel to build deep space gas stations, and for precious metals to return to Earth for manufacturing.

If each of these endeavours is successful, the 2030s are going to be an incredible time: commonplace AI, spaceships, renewable energy and cheap, clean transportation. What kind of society will we be? Will those technological changes shape humanity as a group?

I took a pretty narrow focus in picking the examples above; what startups do you imagine 20 years out, and what happens when they succeed?

Crowdsourced Weather Forecasting with Google Glass

Google Glass creates more opportunities for live crowdsourced weather data collection. An app built for Glass could make use of the accelerometer and camera to automatically take a photo of the sky when the wearer is outside and tilts their head upward. The data would be collected and processed and added to existing weather data collection efforts. Algorithms could automatically detect weather features such as sky colour  cloud types, precipitation type and amount, maybe even wind speed. If this app were installed by a significant number of Glass wearers, the amount of data could be significant.

From the Developer Guides it doesn’t seem this app is possible with the Explorer Edition. However, I would expect that the right APIs would be enabled in the near future, potentially for the initial public release.

In their initial video for Project Glass “One Day…”, Google shows the trigger of looking at the sky to display weather information. Perhaps they are already thinking about snapping a photo at the same time, but it hasn’t been built yet as far as I can see.

Since Glass is an Android device, it will be very simple to integrate with pressureNET. Our data collection and analysis system is already in place at Cumulonimbus, so we could provide the Glass wearer with additional data to improve their personal forecasts.

If Google would like to pass me some Glass I’d love to build a prototype. I promise not to name it SkyNET.

SpaceX’s Grasshopper Test Altitudes Graphed

SpaceX is experimenting with building a completely and rapidly reusable rocket called Grasshopper. There have been five tests so far, each to a higher altitude, and the most recent test was a very large leap above the previous. Here’s a quick graph:SpaceXGrasshopperTests

It’s very tempting to draw out an extrapolated exponential curve into the coming months and years. What faults could hold the altitudes down in the short term? Perhaps a large slowdown after reaching LEO, or even for middle-atmosphere tests? Are they on track to land robots on Mars by ~2018? I think they are. I imagine a future Dragon packed with instruments, sensors, food, water and fuel sitting on Mars in 2020, waiting for a human mission in the early 2020s.

Of course, anyone who lands on Mars will have to be careful of planetary-scale duststorms. I’ll try to scale pressureNET out to Mars to help out.

Living in the Future: Startup Ideas

As of yesterday I’m working full time on Cumulonimbus, my weather data startup that’s seen fantastic growth and media attention recently. We’re collecting about 300,000 atmospheric pressure measurements per day from Android devices with pressureNET and intend to use the data to improve short term severe weather forecasts. I’m currently focused on Cumulonimbus (product development, growth, and looking for funding!) but I have a lot of ideas for future startups. What do you think?

1. Autonomous air delivery

Build a fleet of small solar-powered airplanes and helicopters that will make long-distance air deliveries autonomously. Monetize by offering lower rates for package delivery; a lower cost structure should exist for the delivery network if the transportation uses no fuel and has little human involvement.

This is a very difficult problem but fast progress is being made in the autonomous flight of small vehicles. The initial work for this startup should probably start in 2014 and it should go to market in 2017-2018. At that time, much of the lower-level functions and engineering problems will be solved and the business can focus on optimizing. There are already some existing projects in this space; very interesting ones are Tacocopter and Blizzard (pdf). Recently a Star Trek Into Darkness promotion involved networked, coordinated hovering quadrocopters forming an emblem in the sky.

I see two ways to get this off the ground:

The first is to test an expensive, beautiful small package delivery in a single area. Make the service very sleek and charge a high price at first, then refine the technology, and expand reach and scale to a global package delivery service. The initial package delivery could be a document courier: to send a small document, request a pickup from a quadrocopter and it will notify you when to come outside for document attachment. Attach the package and it will fly to its destination, where the package drop could include a dramatic parachute from a small height, or a direct package landing in the recipient’s hands from feet above. Most vehicles should be able to land in order for calm deliveries and package attachment.

The second way to get this started is to deliver medicine and lightweight aid to remote locations. Set up an office in a big city of a developing country and fly to remote areas with medicine or other lightweight packages like e-readers. Expand to neighboring countries targeting global reach. The initial approach of sending small aid packages to remote areas is exciting because this is normally accomplished with significant human involvement which is often dangerous. Dropping aid from small autonomous flying robots could immensely reduce the costs currently associated with remote aid distribution.

2. Autonomous flying atmosphere sensors

Build the same fleet of aircraft I just described but pack them with weather sensors and send them away from people. This fleet would gather and send live atmosphere data from over the oceans and remote land areas. This would be an excellent extension to pressureNET and it may even be in Cumulonimbus’ future.  However, it is a very different undertaking than a smartphone network so we’re not working on it just yet. This should also be done parallel and integrated with the autonomous air delivery system described above. There are many paths to monetization, such as early and accurate warning systems for tropical storms. Additionally, the data could be made completely open, so that other innovations could arise from researchers and engineers everywhere.

I think that in the future it may be possible for extremely large swarms of autonomous, solar powered airplanes and helicopters may be sent to forming storms to alter their formation and gently nudge them away from populated areas or towards wind farms built for tropical storm speed winds.

3. Improve Collaborative Document Editing

Compete with Google Drive. Drive is close to my ideal productivity suite but it has major flaws that annoy me on a daily basis. Primarily, I have issues with the current offline model and the poor state of mobile document editing. Is Google actively improving and working on these issues? Or will years pass where Google rides on their excellent real-time collaboration features and makes few changes? I’d also like to experiment with building a git UI into document editing. A large flaw in current Drive workflow is the primitive revision history view and no real branching mechanism.

I also think that there may be a very important hardware decision to make in this process. Form factors are changing rapidly and it may be the case that a certain form factor is found to be the best. I’m writing this blog post on my Xoom tablet, with the wireless bluetooth keyboard, using Google Drive on Android. I’m distinctly aware of the software annoyances and what I want to fix, but hardware-wise I’m not sure where to begin. Having a separate screen and keyboard is pretty fantastic and allows for a lot of flexibility, but sure it’s not the best possible solution to mobile document editing from a hardware perspective. What else can be improved? Are those improvements specific to writing and editing documents or can they be ported elsewhere?

4. Galactic Exploration Game

I’ve wanted to play a very specific game for long time but I’ve never been satisfied with anything that’s been made. It looks like I’ll have to make it myself:

Build a multiplayer online game about space exploration. The game scales from single player to an entire galaxy of inhabited star systems played by millions of people.

Each player begins on an Earth-like planet as an alien intelligence. Initially most players are separated a few star systems apart. Players grow their civilization, starting on a single planet and expanding into rest of their system if they choose. Gameplay would involve creating a growing civilization by good management, learning about the cosmos, and examining the morality of species expansion among other things. This exploration continues into star systems nearby the origin planet by building fast ships, at which point civilizations will interact as they encounter each other. Civilizations may interact far sooner, in fact, if they choose to build powerful telescopes and communicate by radio early in the game. Each player would experience a First Contact moment – if they’re listening.

I expect to see a pattern in the livelihood of civilizations when they choose to explore compared to when they do not. Additionally, poor resource management will result in unstable civilizations who will have to make careful decisions in order to survive. To take this further, I’d envision that a player ought to only play once: if their civilization is wiped out, they’re not welcome in that galaxy any more. Increased connection with reality is a bonus: early-stage gameplay could model optimistic predictions about human reality in the near future. For example, poor decisions may lead to complex and out-of-control global climate change; asteroid mining projects may be the most likely route for players to take in starting their space exploration; unexpected events occur especially for the under-prepared civilizations.

Galactic simulation takes place on servers running open source galaxy simulation code. Star systems are procedurally generated and created on-the-fly. The game could be set in the Milky Way Galaxy but significantly far from Earth. This would allow for the procedural generation of early planets to meld with real science data about extrasolar planets, though only if the game grew sufficiently large as to intersect the areas that we have mapped in detail.

This game must be free and open source to build a community and allow for organic growth. This could be monetized by charging for next-level graphics and simulations. The core game and logic will be free and open source and free to play and modify forever. MVP with minimal graphics and build them out from there, always charging a premium to access the next-level graphical appearance.

5. Mobile IDE

I’m really excited about Android IDE (AIDE) and LightTable because of their fantastic ideas and excellent execution. Those projects make me optimistic about the future of the user experience of software development but there are lots of ideas left unexplored so far. Similar to the ideas above about collaborative document editing, I wonder if there will be a convergence of form factors towards something we agree as ideal? What would this look like, and what would it mean for the software?

What kind of code editing can be done in an extremely mobile environment? There are certainly some coding shortcuts that we can create to enable faster mobile development. In the near future, smart watches, glasses, etc will be gaining traction and combining these with minimal artificial intelligence constructs, I think that the software development experience could be radically altered to focus more on hard problems and less on typing code.

There’s also Google’s new language Go. I recently attended an introductory talk on Go at Google in Montreal and I’m now very interested. Since the language is so new it would be excellent to begin testing and prototyping mobile-focused IDEs for Go. Focus must also include collaborative code editing, perhaps with new additions to a git-like ecosystem for real-time tracking of code.




Binary Search in Haskell

I have just written a function in Haskell that will perform a binary search on a list of integers. Pass it your list, the value you’re searching for, default low and high (0 and n-1) and it will return for you the position of your value. I’m sure there are plenty of experienced Haskell coders that will tell me there’s a better way, but I haven’t heard from one yet.

binsearch :: [Int] -> Int -> Int -> Int -> Int -- list, value, low, high, return int
binsearch xs value low high
   | high < low       = -1
   | xs!!mid > value  = binsearch xs value low (mid-1)
   | xs!!mid < value  = binsearch xs value (mid+1) high
   | otherwise        = mid
   mid = low + ((high - low) `div` 2)