Dynamic Learning 101
Adjust machine learning models based on real-time inputs like a child learning to walk.
Traditional machine learning uses historical data to uncover patterns and predictive signals. When conditions are stable, this works well. But today’s world is increasingly volatile and hyper-connected. Pandemics, extreme weather, and tweetstorms can alter decisions instantly.
In volatile conditions, looking-only-at-the-past learning can fall short. Google chief decision scientist Cassie Kozyrkov put it this way: “during an extreme shock, your historical data sources may become obsolete. Then it doesn’t matter how good your information was yesterday. You need new information.”
Dynamic learning, an emerging field in analytics (see recent patent filing), helps meet this challenge. It adapts like a child learning to walk: incrementally, with every step. Here’s how it works and the new analytical applications it helps make possible.
Dynamic learning learns like a child learns to walk: incrementally, with every step.
A Day in the Life of Dynamic Learning
Imagine you run a retail operation. You want to promote timely products to customers as they enter your store. Near the entrance, your manager displays umbrellas when it’s raining and Halloween candy in October. But you want to do more. You have this historical and real-time data available:
Historical data: prior spend, loyalty status, product categorization, seasonality, brand score by product
Real-time data: weather forecast, product stock levels, prices (discounts), social sentiment, last customer interaction.
Dynamic learning can learn from which factors are likely to influence buying decisions today, in real-time. Below, we visualize which factors are predictive of buying behavior. This visual analytical dashboard shows the factors determined solely based on history in yellow, at left. The dynamic model is in green, at right.
Both models start with the same assumptions, drawn from history. It shows that product scarcity, customer tenure, and product category strongly predict what the customer will buy. Press play, and you’ll see the dynamic learning model start with the same assumptions but adapt to each new piece of real-time data. Imagine this engine at work billions of times a day as new real-time data comes in. Over time, the learning model shows that prior spending, social proof, and brand value are increasing in importance.
Direct-to-Consumer Analytics
Who, exactly, is responsible for the learning here? The algorithm? No. The data scientist? Not really.
One of the challenges of scaling AI is that data scientists often act as an insight middleman. They design tests, gather data, interpret findings, measure probabilities, then “present” insights to domain experts.
Dynamic learning disintermediates insight middleman. I think of it as direct-to-consumer data science. Knowledge teams can all consume dashboards that show what’s going on. Do customers feel Spring has sprung early? Is that why customers with more discretionary income (prior spend) buying outdoor and exercise clothing earlier than expected?
By exposing its assumptions, dynamic learning encourages analytics teamwork and collaboration.
The Business Implications of Dynamic Learning
Our dynamic learning retail example reveals insights as the model learns. But what should we do about it? The first step, of course, is to act! In this scenario, we may adjust product placement in the store, run an email campaign, or engage on social media.
The next step is to measure. The analytics dashboard below illustrates another advantage of direct-to-consumer dynamic learning. We can monitor key results like the rate of offer acceptance, gross, and cumulative profit over time. In this simulation, the historical and dynamic models start together. Over time, the historical model decays; the dynamic model adjusts and keeps acceptance rates high.
Buyers and store managers can drill down. In this view, the color and size of each dot show the importance of each factor. Michelle Lacy from Bayer calls this “putting the algorithm behind the button.”
Algorithmic Bias Awareness
Transparency is not only good for sales decisions. It helps manage algorithm bias too. The direct-to-consumer model of dynamic learning can help risk managers see the assumptions the model makes. The examples below show the factors that affect insurance policy risk.
In the beginning, the risk factors look like a giant hairball, at right.
With clever visual design, risk managers can easily explore the risk factors identified by dynamic learning. Below, we filter to reveal that claims per year, claim type, and claim cost impact risk the most. A human analyst can filter, explore, dissect, and better understand those factors as they change in seconds.
Forensics don’t need to happen in real-time. The dynamic learning engine can stream model changes in real-time, as they’re observed, for future exploration.
The Applications of Dynamic Learning
Dynamic learning can help for any application where real-time data is available. For example, in fraud detection, fraudsters regularly adjust how they operate. Dynamic learning can spot newly suspicious patterns as they occur and flag them to data scientists and fraud investigators. Continuous learning is essential.
In the enterprise data quality market, dynamic learning can be applied to automatically detect inconsistent data formats, patterns, and changes to related variables on the fly. Cleansing can be adjusted to be more or less aggressive depending on those changing factors, or remediation steps and business processes can be started to flag to human governance staff based on shifting patterns.
Automated high-tech manufacturing is susceptible to problems due to suboptimal machine configuration, wear, and operator error. For highly automated systems, problems can manifest in real-time, in unpredictable ways. Dynamic learning learns from streaming IoT sensor data, maintenance history, and streaming machine vision to flag problems before affecting production.
In the era of global pandemics and global warming, supply chains are more susceptible to sudden disruption than ever before. Dynamic learning can correlate the location of cargo ships and the weather to anticipate problems before they occur.
Read analytics with moving data for more on real-time analytics.
Dynamic Learning for an Adaptive Business
Traditional looking-only-at-the-past machine learning is here to stay. But will dynamic learning be “machine learning, 2.0?” When business conditions are stable, predictions based on past behavior deliver massive value. But for volatile environments, dynamic learning can improve insights and make new applications possible.
This article was co-written by Dr. Tom Hill and Mark Palmer. The details of dynamic learning are patent-pending and beyond the scope of this post, but you can read the filing here, “Algorithmic learning engine for dynamically generating predictive analytics from high volume, high velocity streaming data.”
Contact the authors to discuss or subscribe below for updates.
