In a recent post, we started scoping our executive level dashboards and reporting project by mapping out who the primary consumers of the data will be, what their top priorities / challenges are, which data we need and what we are trying to measure. It might seem like we are ready to start evaluating vendors and building it out the project, but we still have a few more requirements to gather.
What data can we exclude?
With our initial focus around sales analytics, the secondary data we would want to include (NetProspex, Marketo and ToutApp) all integrates fairly seamlessly with the Salesforce so it won't require as much effort on the data prep side. If we pivot over to our marketing function however, things get a bit murkier. On the low end this could mean a dozen or so data sources. But what about our social channels, Google Ads, etc, as well as various spreadsheets. In more and more instances, particularly for a team managing multiple brands or channels, the number of potential data sources can easily shoot into the dozens.
Although knowing what data we should include is important, what data can we exclude? Unlike the data lake philosophy (Forbes: Why Data Lakes Are Evil,) when we are creating operational level reporting, its important focus on creating value, not to overcomplicating our project with additional data sources that don't actually yield additional value.
Who's going to manage it?
Just as critical to the project as what and how; who’s going to be managing it? What skills do we have out our disposal and how many hours can we allocate for the initial setup as well as ongoing maintenance and change requests? Will this project be managed by IT, our marketing analytics team, or both? Perhaps IT will manage data warehousing and data integration and the analyst will focus on capturing end user requirements and creating the dashboards and reports. Depending on who's involved, the functionality of the tools and the languages used will vary. As mentioned in a recent CMS Wire post Buy and Build Your Way to a Modern Business Analytics Platform, its important to take an analytical inventory of what skills we have as well as what tools and resources we already have we may be able to take advantage of.
As we covered in our recent NLP blog, there are a lot of cool use cases for text / sentiment analysis. One recent instance we found really interesting came out of our May presentation at SeaTUG (Seattle Tableau User Group.) As part of our presentation / demo we decided to find out what some of the local Tableau users could do with trial access to Keboola; below we’ll highlight what Hong Zhu and a group of students from the University of Washington were able to accomplish with Keboola + Tableau for a class final project!
What class was this for and why did you want to do this for a final project?
We are a group of students at the University of Washington’s department of Human Centered Design and Engineering. For our class project for HCDE 511 – Information Visualization, we made an interactive tool to visualize music data from Last FM. We chose the topic of music because all 4 of us are music lovers.
Initially, the project was driven by our interest in having an international perspective on the popularity vs. obscurity of artists and tracks. However, after interviewing a number of target users, we learned that most of them were not interested in rankings in other countries. In fact, most of them were not interested in the ranking of artists/tracks at all. Instead, our target users were interested in having more individualized information and robust search functions, in order to quickly find the right music that is tailored to one’s taste, mood, and occasion. Therefore, we re-focused our efforts on parsing out the implicit attributes, such as genre and sentiment, from the 50 most-used tags of each track. That was when Keboola and its NLP plug-in came into play and became instrumental in the success of this project.
The age old conflict. IT needs centralization, governance, standards and control; on the other side of the coin? Business units need the ability to move fast and try new things. How can we get lines of business access to the data they need to for projects so they can spend their time focused on discovering new insights? Typically they get stuck in a bottleneck of IT requests or spending 80% of their time doing data integration and preparation. Neither group seems particularly excited to do it, and I don’t blame them. For the analyst it increases the complexity of their tasks and seriously raises the technical knowledge requirements. For IT, it’s a major distraction from their main purpose in life, an extra thing to do. Self serve BI is trying to destroy the backlogged “report factories,” only to replace them with “data stores,” which are sadly even less equipped for the job at hand. Either way, the result is a painfully inefficient process, straining both ends of the value chain in any company that embarks on the data driven journey.
The Bi-Modal BI Answer?
An organization's ability to effectively extract value from data and analytics while maintaining a well governed source of truth is the difference between competitive advantage or sunken costs and missed opportunities. How can we create an environment that provides the agile data access needed by the business users while still maintaining sound data governance? Gartner has referred to a Bi-modal IT strategy. A big challenge with Bi-modal IT is that it pushes IT management to divide their efforts between ITs traditional focus and a more business focused agile methodology.
The DBA and Analyst Divide
Another major challenge in data access comes from the separation between DBAs and business users. Although the technical side may have the necessary expertise to implement ETL projects, they often lack the business domain expertise needed to make the correct assumptions around context and how the data is regarded. With so many projects competing for resources, we shouldn’t have to task a DBA on all of them. Back to the flip side of the coin, data analysts and scientists want the right data for their tools of choice and they want it fast. Even though there is growing set of data integration tools that allows individual business units to create and maintain their own data projects, this typically requires a lot of manual data modeling and can lead to siloed data or inconsistent metrics.
Instead of controlling all of BI, IT can enable the business to develop their analytics without sacrificing control and governance standards. So how can we get the right data in the hands of people who understand and need it in a timely manner?
Having access to the right data in a clean and accessible format is the first step (or series of steps) leading up to actually extracting business value from your data. As much as 80% of the time spent on data science projects involves data integration and preparation. Once we get there, the real fun begins. With the continued focus on big data and analytics to drive competitive advantage, data science has been spending a lot of time in the headlines. (Can we fit a few more buzzwords into one sentence?)
Let’s take a look at a few data science apps available on our platform and how they can help us into our data monetization efforts.
One of the most popular algorithms is market basket analysis. It provides the power behind things like Amazon’s product recommendation engine and identifies that if someone buys product A, they are likely to buy product B. More specifically, it’s not identifying products placed next to each other on the site that get bought together, rather products that aren’t placed next to each, This can be useful in improving in-store and on site customer experience, target marketing and even the placement of content items on media sites.
Anomaly detection refers to identifying specific events that don’t conform to the expected pattern from the data. This could take the form of fraud detection, identifying medical problems or even detecting subtle change in consumer buying behaviors. If we look at the last example, this could help us in identifying new buying trends early and taking advantage. Using the example of an eCommerce company, you could identify anomalies in carts created per minute, a high number of carts abandons, an odd shift in orders per minute or a significant variance in any other number of metrics.
Guest post by Kevin Smith
For a product owner, one of the biggest fears is that the product you're about to launch won't get the necessary adoption to achieve success. This might happen for a variety of reasons— two of the most common are a lack of fit to the customers' needs and confusing design (it's just too hard to use!).
To combat the possibility of failure, many product owners have adopted the "agile" approach to building products that have enough functionality to meet to minimum needs, but are still lean enough to facilitate easy change.
As a data product builder — someone building customer-facing analytics that will be part of a product — the needs are no different but achieving agility can be a real challenge. Sure, every analytics platform provider you might consider claims that they can connect to any data, anywhere, but this leaves a lot of wiggle room. Can you really connect to anything? How easy is it? How hard is it to change later? What about [insert new technology on the horizon here] that I just heard about? If you want to build an agile data product, you've got a tough road ahead... as I found out.
Recently I started working on the analytics strategy for a small start-up firm focused on providing services to large enterprises. As they delivered their services, they wanted to show the results in an analytics dashboard instead of the traditional PowerPoint presentation. It would be more timely, easier to deliver, and could be an on-going source of revenue after an engagement was completed. As I spoke with the team, a few goals surfaced:
- They wanted to buy an analytics platform rather than build from scratch. The team realized that they would be better off developing the methodology that would differentiate them from the competition instead of creating the deep functionality already provided by most analytics platforms.
- The system had to be cost-effective both to set-up and to operate. As a start-up, there simply wasn't the cashflow available for costly analytics platforms that required extensive professional services to get started. The product had to be flexible and "configurable" by non-Engineers. With little to no budget for an Engineering staff, the team wanted a BI platform that could be configured easily as customer needs changed.
- Up and running quickly. This company had customers ready to go and needed a solution quickly. It would be essential to get a solution in front of the customers NOW, rather than try to migrate them to a new way of operating once the dashboards were ready. Changes would certainly be needed post-launch, but this was accepted as part of the product strategy.
None of this seemed to be impossible. I've worked on many data products with similar goals and constraints. Product teams always want to have a platform that's cost-effective, doesn't strain the technical capabilities of the organization, is flexible, and is launched sooner rather than later. It was only after a few more conversations that the problem arose: uncertain data sources.
Most data-driven products work like this: you've got a workflow application such as a help desk application or an ordering system that generates data into a database that you control. You know what data is flowing out of the workflow application and therefore, you understand the data that is available for your analytics. You connect to your database, transform the data into an analytics-ready state, then display the information in analytics on a dashboard. The situation here was different. As a services company, this business had to operate in a technology environment dictated by the customer. Some customers might use Salesforce, some might use Sugar CRM. Still others might use Zoho or one of the myriad other CRM platforms available. Although the team would structure the dashboards and analytics based on their best practices and unique methodology, the data driving the analytics product would differ greatly from customer to customer.
As we examined in part 1 of our Data Monetization blog series, the first step to increasing revenue with data is identifying who the analytics will be surfaced to, what their top priorities are, what questions we need to ask and which data sources we need to include. For this blog, let’s take a look at what tools we will need to bring it all together.
With our initial example of a VP of Sales dashboard, fortunately the secondary data sources (NetProspex, Marketo and HubSpot Signals) all integrate fairly seamlessly with the Salesforce CRM. This should allow for some fairly straightforward analytics built on top of all the data we’ve aggregated. If we pivot over to our CMO dashboard, things get a bit murkier.
Although our Marketo instance easily integrates with Salesforce, the sheer volume of data sources that can provide insight to our marketing activity makes this project a much more daunting ask. What about our social channels, Adobe Omniture, Google Ads, LinkedIn Ads, Facebook Ads, SEO as well as various spreadsheets. In more and more instances, especially for a team managing multiple brands / channels, this number can easily shoot into the dozens.
When a company thinks about monetizing data, the things that come to mind are increasing revenue, identifying operational inefficiencies or creating a new revenue stream. It’s important to keep in mind that these are the results of an effective strategy but can't be the only goal of the project. In this blog series, we will exam these avenues with a focus on the added value that ultimately leads to monetization. For this blog, lets look at it from the perspective of creating executive level dashboards at a B2B software company.
Who will be consuming the data and what do they care about?
Before we jump into the data itself, take a step back and understand who the analytics will be surfaced to and what their challenges are. Make profiles with their top priorities, pain points and the questions they will be asking. One way to get started is to make a persona priority matrix listing the top three to five challenges for each (ex. below.)
Once the matrix is laid out, you can begin mapping specific questions to each priority. What answers might help a VP of Sales increase the effectiveness of the sales team and ultimately revenue?
What do our highest velocity deals look like (vertical, company size, who’s involved)?
What do our largest deals look like?
Where do our deals typically get stuck in the sales process?
What activities and actions are our best reps performing?
Data can be vast and overwhelming, so understanding the different types helps to simplify what kind of numbers we are looking for. Even with the treasure trove of data most organizations have in-house, there are tons of additional data sets that can be included in a project to add valuable context and create even deeper insights. It’s important to keep in mind what type of data it is, when and where it was created, what else was going on in the world when this data was created, and so forth. Using the example of a restaurant, let’s look at some different types of data and how they could impact an analytics project.
Numerical data is something that is measurable and always expressed in numerical form. For example, the number of diners attending a particular restaurant over the course of a month or the number of appetizers sold during a dinner service. This can be segmented into two sub-categories.
Discrete data represent items that can be counted and is listed as an exact number and take on possible values that can be listed out. The list of possible values may be fixed (also called finite); or it may go from 0, 1, 2, on to infinity (making it countably infinite). For example:
Number of diners that ate at the restaurant on a particular day (you can’t have half a diner.)
Amount of beverages sold each week.
How many employees were staffed at the restaurant on a day.
Continuous data represent measurements; their possible values cannot be counted and can only be described using intervals on the real number line. For example, the exact amount of vodka left in the bottle would be continuous data from 0 mL to 750 mL, represented by the interval [0, 750], inclusive. Other examples:
Pounds of steak sold during dinner service
The high temperature in the city on a particular day
How many ounces of wine was poured in a given week
You should be able to do most mathematical operations on numerical data as well as list in ascending/descending order and display in fractions.
Its that time of year again and there are so many gift options to choose from. Be it hover-boards (that may explode,) drones or Star Wars’ own BB-8 remote control droid, there’s been quite a boom in tech gadgets this year. At Keboola we love all things data, so to get you in the holiday spirit, we wanted to share some cool gift ideas that use data to make your life easier (or at least a bit more interesting.)
Similar to the gadget seen in the Progressive commercials, the Automatic Adapter is basically a fitness app for your vehicle. It provides a full report on behavior through an app or a web interface regarding where you’ve been, driving behavior and even tag routes for business travel expenses.
The Economist Intelligence report Big data evolution: forging new corporate capabilities for the long term published earlier this year provided insight into big data projects from 550 executives across the globe. When asked what their company’s most significant challenges are related to big data initiatives, maintaining data quality, collecting and managing vast amounts of data and ensuring good data governance were 3 of the top 4 (data security and privacy was number 3.) Data availability and extracting value were actually near the bottom. This is a bit surprising as ensuring good data quality and governance is critical to getting the most value from your data project.
Maintaining data quality
Having the right data and accurate data is instrumental in the success of a big data project. Depending on the focus, data doesn’t always have to be 100% accurate to provide business benefit, numbers that are 98% confident is enough to give you insight into your business. That being said, with the sheer volume and sources available for a big data project, this is a big challenge. The first issue is ensuring that the original system of record is accurate (the sales rep updated Salesforce correctly, the person filled out the webform accurately, and so forth) as the data needs to be cleaned before integration. I’ve personally worked through CRM data projects; doing cleanup and de-duping can take a lot of resources. Once this is completed, procedures for regularly auditing the data should be put in place. With the ultimate goal of creating a single source of truth, understanding where the data came from and what happened to it is also a top priority. Tracking and understanding data lineage will help identify issues or anomalies within the project.
Collecting and managing vast amounts of data
Before the results of a big data project can be realized, processes and systems need to be put into place to bring these disparate sources together. With data living in databases, cloud sources, spreadsheets and the like, bringing all the disparate sources together into a database or trying to fuse incompatible sources can be complex. Typically, this process consists of using a data warehouse + ETL tool or custom solution to cobble everything together. Another option is to create a networked database that pulls in all the data directly, this route also requires a lot of resources. One of the challenges with these methods is the amount of expertise, development and resources required. This spans from database administration to expertise in using an ETL tool. It doesn’t end there unfortunately; this is an ongoing process that will require regular attention.
Ensuring good data governance
In a nutshell, data governance is the policies, procedures and standards an organization applies to its data assets. Ensuring good data governance requires an organization to have cross-functional agreement, documentation and execution. This needs to be a collaborative effort between executives, line of business managers and IT. These programs will vary based on their focusbut will all involve creating rules, resolving conflicts and providing ongoing services. Verifications should be put into place that confirm the standards are being met across the organization.
Having a successful big data project requires a combination of planning, people, collaboration, technology and focus to realize maximum business value. At Keboola, we focus on optimizing data quality and integration in our goal to provide organizations with a platform to truly collaborate on their data assets. If you’re interested in learning more you can check out a few of our customer stories.