Your Next Enterprise Data Fabric
Modern data management help teams of knowledge workers harness the data from AI and automation.
Imagine you’re a chief data officer at a Fortune 1000 company. You manage thousands of databases. You support 10,000 knowledge workers who use three different business intelligence tools. Your data landscape is evolving and messy. At times, it resembles your grandpa’s attic. To find the data you’re looking for requires hours of searching.
In the past, it took weeks or months to provide new data views to your customers. You bought data virtualization tools to fix the problem. Now, you create data views quickly, securely, with good performance. Life is good.
Not so fast! Demands for data are increasing and changing. With increasing automation, you’re getting requests for real-time data. With the increasing use of AI, you’re getting requests to incorporate data science models.
Your next data management fabric will be different in five ways. Here’s how.
One: Virtualize Data
Today’s data fabrics are complex, distributed, and fast-changing. Yet data management techniques from 30 years ago are still commonly used that centralize data. Centralization worked well when IT was the center of the universe. But today, it’s common for a business to have thousands of competent knowledge workers that demand self-service access to data. Centralized “systems of record” don't work well for knowledge-worker-driven business.
Data Virtualization helps data engineers quickly provide virtual views of dozens or hundreds of data sources without moving it. Data virtualization engines combine, prepare, and provision data in memory so that it looks like a single logical database. Security and access are controlled in one place.
Data virtualization helps democratize data for knowledge workers. It’s the first element of your next enterprise data fabric.
Two: Infuse Artificial Intelligence in Your Data Fabric
Every enterprise is trying to do more with artificial intelligence. Data science teams demand more and more data. They uncover new algorithms. Business partners clammer to use those algorithms in their decision-making and the services they provide customers. Data science teams carry out their research with a dizzying array of tools. As they uncover interesting algorithms, what do you do? Some organizations take algorithms and reimplement them for performance, scale, and quality reasons.
New “model operationalization” tools are emerging that help firms more easily deploy AI algorithms. Model operationalization, or Model Ops, is like an airport-taxi system for algorithms. Data scientists place algorithms in a queue awaiting takeoff. Data engineers can review algorithms, ask questions, and check how they perform and scale. Then, when they choose a model to deploy, they can do so with a click of a button and deploy an “AI scoring service” right into their virtualization fabric. From there, the algorithms take flight.
For example, knowledge workers using business intelligence tools like Tableau, Power BI, or Spotfire can ask questions with predictive signals inside, like this:
"Show me what customer X bought in the past and which products they're likely to buy next.”
The scoring service, embedded in the virtualization fabric, generates predictions as the knowledge worker explores. Putting algorithms “behind the button” of visual analytics helps democratize AI and turn math into inputs that accelerate human insight.
Three: Data Virtualization for Automation
Every business today is becoming more automated. Industrial equipment automates manual jobs. Logistics knowledge workers use real-time weather forecasts to anticipate and act before it’s too late. Retailers react to social media sentiment and chatbots in real-time.
All this analysis requires data in motion. Every day, new sources of information stream into the enterprise: from the internet, embedded sensors on devices, drones, and from social media, just to name a few.
But most data virtualization tools are designed for data at rest. Consider this request:
“Show me the maintenance history for machine 123 and its up-to-the-second sensor readings.”
The request for up-to-the-second sensor readings is a tricky one. It requires access to data that, in some cases, can change hundreds of times a second. Some IT teams tackle this challenge by storing moving data in a database. But analyzing moving data with a database is like watching a Formula One race with only still photography. Context, motion, rate of change are all lost. This is what I call a “too late architecture.” By the time moving data available, opportunities are gone; it’s too late.
Streaming data virtualization connects data in motion to business systems. It converts each business event into a row in a table, in real-time. Streaming data virtualization helps knowledge workers any data in motion: sensor readings, drone data, weather forecasts, and more.
Four: Data-as-a-Service
Software services are the store front of modern digital business. Application programming interfaces, or APIs, allows electronic access to that store front.
But if you're a data engineer, should you just slap APIs on your data services? Sure, you can throw a REST interface on your data, but should you? Who’s calling it? How many times? Will the business charge a fee to access the API? How much?
Data as a Service make it easy to create APIs from data. And they integrate with API Management tools that turn APIs into a managed service.
Data as a Service makes it easy and safe to create, manage and optimize digital store fronts.
Five: A Culture of Data Curation
Good habits forge good health.
In 2020, Panera Bread pulled off an incredible pandemic pivot. They flipped their business model on its head in just ten days. Now, in addition to selling prepared food, they sell milk, yogurt, tomatoes, avocados. Panera leveraged their team of 50,000 and a network of 2,000 stores to deliver items in 40 minutes through Panera’s delivery network of 10,000 drivers, locations with pick-up, and on Grubhub.
Panera’s transformation was made possible by a culture of data curation. Faced with increasing data, new complexity, and manual processes, Panera CIO John Meister established a vision called “One Panera.” One Panera instilled a habit of data curation throughout the company. He placed responsibility for data into the hands of his business partners. One Panera democratizes data instead of centralizing it.
Panera’s good data habits made their pandemic pivot possible. Their 50,000 employees and over 2,000 locations have one view of data their menu, items, and inventory.
Panera Director of Enterprise Services Noel Nitecki provides a glimpse of how Panera manages their data as a team sport. Their Menu Master Data Space provides a single view of 135,000 prices for items, including price tiers and categories. Nitecki said, “At Panera, our business partners take responsibility for their data. So when it came to changing our business model, data was an afterthought for us. We were free to execute instead of worrying about our data.”
Like brushing your teeth, tools help form good habits. Tools that help form a culture of data curation include:
Tools that make it easy for knowledge workers to curate and validate business data
Tools that make it easy to extract metadata from databases
Tools that make it easy to incorporate AI models created by data scientists
Tools that help knowledge workers identify and fix quality issues
These tools, when integrated with a data fabric, make data a team sport and can lead to Panera Bread-level business agility.
Who will be the Tesla of Data?
Data is the oil of digital business. Automation, AI and a modern service-centric digital business models are the future of digital business. This vision of an intelligent data fabric is like the technology-aware experience that Tesla gives drivers. Informed by technology, accessible by all, infused with artificial intelligence. Who will be the Tesla of enterprise data?
This is part two of The Future of Analytics.
