Huffpost Technology
The Blog


Otto Scharmer Headshot

From Big Data to Deep Data

Posted: Updated:

Over the past ten months I have chaired and co-facilitated MIT's IDEAS China program--a ten month innovation journey for a group of 30 or so senior Chinese business leaders. This year the IDEAS China program enrolled executives of a major state-owned Chinese bank. One goal of this team was to reinvent the future of their organization in the face of big data and other related disruptive changes, which provided me with a little more exposure to that aspect of the world economy. For example, Jack Ma, the visionary founder of Alibaba, says that "In five years, we anticipate that the human era will move from the information technology era to the data technology era."

But what does it mean to be in an era of "data technology" and "big data"? Until today, it has often meant that big companies like Google, Amazon, Facebook, and Apple--the same companies that we used to love and now increasingly begin to question, mistrust or fear--take your data without asking and sell it to other companies without your knowledge (until you notice the targeted Web commercials that appear on your screen). I find it interesting that people's initially very positive view of these American big data empires has been shifting first in Europe, but now also in many other parts of the world, including North America. Edward Snowden made all of us more sensitive to the misuse of big data. But that's just the surface issue. The real problem is on a deeper level.



Deep data is used to make people and communities see themselves. Deep data functions like a mirror: it makes you see yourself--both as an individual and as a community. Over the past twenty years of my professional life I have been helping teams and organizations go through processes of profound innovation and transformative change across sectors and cultures. The one thing that I have learned from all these projects is that the key to transformative change is to make the system see itself. That's why deep data matters. It matters to the future of our institutions, our societies, and our planet.





The journey from science 1.0 to 2.0 is a journey of bending the beam of scientific observation back onto the observing self--both individually and collectively.

At the end of our closing meeting last week, the senior leaders of the Chinese state-owned bank reflected on their own journey of the past ten months. Every one of them reported a profound shift in how they think and operate. Here are two exemplary statements:



In essence, what the IDEAS participants described was a transformation of
• thinking: from downloading old patterns to thinking creatively
• conversing: from debate to generative dialogue

Over the past months, while staying in their jobs, the participants split into four teams that each tried to prototype some new way of operating in order to explore future opportunities. What struck me was that each team ended up developing a new platform of cross-organizational collaboration that used data as a tool for transforming the way their stakeholders communicate. All of their prototypes are still in an early stage. But one lesson that was mentioned by the teams repeatedly was the importance of shifting their mindset from me to we, from ego to eco.

The question that their efforts have left me with is this: On a societal level, what types of deep data infrastructures might facilitate this 'bending of the beam of observation' back onto the observer on the level of entire eco-systems?

For example, today we use GDP to measure economic progress. GDP is an excellent measure of surface data. But what would the equivalent deep data tool be for measuring real economic progress in a community? I believe that it would include a new indicator system that is grounded in real outcomes (e.g., life expectancy), and in the wellbeing of individuals and their communities (e.g., quality of life). Last year we--the Presencing Institute, with the GIZ Global Leadership Academy (German Ministry for Development Cooperation) and the Gross National Happiness Centre in Bhutan-- launched the Global Wellbeing Lab, The lab links leaders from government, business, and civil society around the world who are pioneering new indicators and deep data tools that help communities and eco-systems to see themselves, in order to sense, and prototype new ways of operating.

Where are you seeing the seeds of such new indicator systems or deep data tools today? What can be learned from these first examples? What would deep data mean for your self? What are the real sources of well-being and happiness in your own life and work and what metrics could help you to see and sense your own developmental path in a more meaningful way? How can we co-pioneer the shift from big data to deep data in business, society and self?

From Our Partners