Making data more actionable

In recent years, organizations have sought out more sources of data so they can best optimize customer experience in a growingly complex, multi-platform digital world. However, many organizations are drastically underutilizing their data due to internal challenges that limit the efficacious use of these datasets. If a marketer is truly aiming to tell a great story with their data, there are three major steps they must look to take.

1. Organize Data Silos

Data silo-ing occurs when companies, deliberately or incidentally, make certain datasets available only to a small subset of teams or individuals: such moves may often be necessary, but a recent Harvard Business Review survey found that “the best-in-class companies—those with strong financial performance and competitive customer experiences—are more likely to have removed those silos than…other organizations.” As a common example of how data silo-ing may occur, marketers within a company may end up using one tool to examine data, while data analysts end up using a completely different tool. This may mean that the two teams are looking at completely different data, or they may face the subtler (but no less pernicious) problem of examining those data from two limited, non-holistic viewpoints.

2. Visualize Data Effectively

Data visualization tools allow analytics teams to take complicated datasets and synthesize their narrative into a few easily consumable major points. However, many data visualizations confuse instead of clarify a narrative. Good data visualizations should avoid obvious misinterpretations of data (they don’t manipulate scale or draw spurious comparisons) and have a clean, uncluttered aesthetic.

3. Share Data in Real Time

The proliferation of data has unlocked a world of potential for many organizations, but the sheer scope and scale of that data also presents new challenges. Namely, companies that are receiving new data every day often accidentally rely on old data sets that may not accurately encapsulate the challenges they are facing on that day. When companies build real-time data into their stories and their visualizations, they are then able to fully capture a customer’s journey and make better decisions to optimize and improve those journeys.

For nearly all marketers, even a slight improvement in data organization can greatly improve the experiences of their customers. By thoughtfully improving data streams from their back-end collection and organization to the ultimate synthesis and visualization of said datasets, marketers will be able to provide a better experience for all of their customers.

Driving Value from Data

Organizations are increasingly relying on data to drive value and many disruptive businesses have done so very successfully. Whether they are monetizing the data they collect directly like Facebook and Google or using the data to create new products like self driving cars or better TV shows data is playing an ever more important role in businesses across the board.

Marketing has been one of the beneficiaries of increasing amounts of data in todays digital world, gaining a lot more visibility of the consumer. Marketers need data across all stages of the buying cycle before they come up with a strategy or a marketing campaign. They need data related to actions, trends and behavior to name a few. So it’s no surprise that marketers are often seen as professionals whose strategies are driven by data.

The data in question is typically accumulated in a number of ways. Marketers can tap into first party data in the company’s databases, look into results of past campaigns, purchase third party data, work with partners and gather new data through surveys and focus groups. The applications responsible for creating these data sets include the likes of web analytics (such as Google Analytics or Adobe Analytics) and CRM systems (like Salesforce or Microsoft Dynamics) to enterprise application data warehouses, data-lakes, and DMPs (for example BlueKai, Krux). The data is then analyzed and the results put to work in the decision making process.

With a growing number of data sources and silos marketers have had to recognize they are ill equipped to collect, process and analyze the shear volume of structured and unstructured information in this new data driven age. Whether internally or externally sourced the expertise within these marketing teams has evolved to encompass technology, data collection, processing and analysis.  There’s significant upside though and a marketing organization with a data driven culture and well put-together data science team can have a far reaching impact across the business. Data engineers and data scientists are just two of the new roles appearing in marketing to help deliver this value.

A data engineer will help collect, sift and connect customer data across numerous structured and unstructured data sources. They will be very comfortable handling large complex data sets and the infrastructure needed to ingest, house and model the data.  They will have familiarity with ETL tools and data modeling, Apache Spark, data warehouses and NoSQL / SQL database systems.

The data scientist is the one tasked with extracting the value from the data. It’s a tough role to fill and good ones are increasingly tough to find. (McKinsey predicts demand for data scientists will soar 50% by 2018). To be effective a data scientist needs domain expertise, should be adept at articulating insights, creating visualizations and storytelling. They operate much like scientists creating testable hypotheses, then executing tests on data sets that can be modeled and prototyped. Once new models and prototypes have been validated they must then be “productized” or “operationalized” and deployed to can drive efficiency in day to day operations. 

Data Analyst, Engineer, Scientist

It’s no secret that big data is, well, getting bigger. As more companies recognize the vital importance of having a robust data intelligence practice, opportunities within the field have exploded. The McKinsey Global Institute has estimated that, by 2018, the U.S could have 1.5 million positions in the world of data that go unfilled due to a lack of adequate training.

This raises an important question that many are too afraid to ask: what exactly is the difference between data analysts, data scientists, and data engineers? While the three positions often have overlapping duties, they are distinct jobs with distinct skillsets and workloads.

Data Analysts

Data analysts are, in essence, junior data scientists. While data scientists will often be responsible for the development of data algorithms and higher-level decisions about how to engage in data analysis, data analysts are largely in charge of the nitty-gritty of data analysis. They will be the figurative on-the-ground employees of the data world, ensuring that a company is working with accurate, well-scraped data. They will then use the tools given to them to engage in data analysis of those datasets.

This isn’t to say, however, that data analysts don’t need a well-rounded toolkit of skills in order to excel at their job. Data analysts should be competent at programming and data visualization in addition to their core statistics skills, and since they will be responsible for ensuring the fidelity and accuracy of massive datasets, attention to detail is also a must.

Data Scientists

The difference between data scientists and data analysts is largely one of degree, not of kind. This is often literally true: data scientists will regularly have an advanced degree in a quantitative field (e.g. a Ph. D. in computer science or physics), and will also have a job that is a degree of magnitude more complex than that of a data analyst. Data scientists are the ones who give form to the analysis that they, along with data analysts, will conduct: they tackle open-ended questions and attempt to find important trends within datasets.

The skillset required of a data scientist is much broader than that that required of an analyst. While a data analyst may simply have an understanding of several programming languages, data scientists have to work with a wide swath of computerized data tools. They may be required to have a familiarity with data programming languages (e.g. R), database programs (e.g. MySQL, NoSQL programs), mapping software (e.g. D3.js, Tableau), and far too many other toolsets to list here.

Data Engineers

While similar in name to data analysts and data scientists, data engineers have a set of duties more distinct from those of the other two positions discussed. Data engineers are basically software engineers, responsible for the construction of the pipeline that funnels messy raw datasets into easily usable and well-organized databases and applications. Data engineers, in essence, construct the infrastructure upon which data analysts and data scientists conduct their analyses.

Data engineers, require a deep skillset in the construction and efficient maintenance of databases. They will use SQL languages such as MySQL and NoSQL database structures like MongoDB, work to both query and build out API’s, and look at ways to create streamlined databases with human-fault-tolerant pipelines. By building a data structure that continuously integrates with a variety of tools, data engineers can make the lives of the rest of the data team vastly easier.

In Conclusion...

Ultimately, data analysts, data scientists, and data engineers all have a vital role to play on any data team. Data engineers set the stage by building the infrastructure that stores and maintains all the data, and data scientists then use that infrastructure make the top-level decisions which guide data analysis. Finally, data analysts take the ball over the goal line by ensuring that the analysis itself goes smoothly. By understanding the different strengths and capabilities of all three positions, a company can maximize the value of its data.


Fraud has well and truly made it’s way into the advertising world and it’s not only our banking details that are targets. Malware is tiggering multiple events through our advertising ecosystems without discriminating between targets in the supply chain: performance, attribution, reach & frequency. Whatever your KPIs none are immune and marketing budgets are getting drained across the board. 

Yes it’s a big problem

ANA and Whiteops 2015 study puts the adfraud problem at between $250K and $42M annually for each of the 49 study participants, the average is $10M. Estimates for 2015 put global loses at $6.3B and $7.2B for 2016 across the industry. WoW! Crime clearly does pay and by the looks of things marketing budgets of the Fortune 500 are footing the bill. As if we didn’t have enough to worry about. Now our budgets are literally being stolen by criminals. 

No one is immune

The unfortunate reality is that it’s not going to be one of those problems that can get fixed and forgotten about. Current advertising and marketing technology and the systems in place that are designed to prevent this kind of fraudulent activity are in a veritable arms race against a very sophisticated enemy. Some tell tale signs are easier to spot than others but no one appears to be immune and everyone is a victim to varying degrees.  

It’s not easy to spot

The cyber criminals are using many and varied techniques to evade detection. Replaying human behavior, copying cookies, spoofing IP addresses to name a few, that we know about. This problem won’t be solved with a single solution. Blacklisting and whitelisting approaches are woefully inadequate and data science will only go part of the way. 

How do marketers even begin to address this?

It starts with transparency.  

  1. Don’t assume it isn’t happening in your campaigns. Acknowledge that everyone with a marketing budget is a target, the size of the target depends on the size of the budget and you need to take proactive steps towards resolution or the problem will only get bigger.
  2. Set expectations with vendors in your media supply chain that you will only pay for non-bot and viewable traffic. Introduce specific terms and conditions into your insertion orders and terms and conditions such as that suggested by Reed Smith (the ANA’s outside legal counsel).
  3. Request granular transparency from your suppliers. If you have a  good data strategy in place already then this is already being taken care of. Ensure you’re gathering the data captured outside your company at the event not aggregate level.
  4. Analyze the data. With all the data coming inhouse your data scientists and experts are in an ideal position to interpret the data for fraud, viewability, performance, attribution, geography etc.