By Dave DeFusco
A team of Katz School researchers has solved one of the biggest headaches in modern machine learning鈥攈ow to make AI models that can adapt to new information without needing to start over from scratch鈥攂y developing a deep learning system that updates itself using new information about the world without retraining, and does so in a way that鈥檚 easier to understand.
Dr. David Li, director of the Katz School鈥檚 M.S. in Data Analytics and Visualization, and Ruixin Chen, a student in the M.S. in Artificial Intelligence, presented their work, 鈥淎daptive Deep Learning with Batch Feature Re-Engineering and Differential Dynamical Systems,鈥 in March at IEEE SoutheastCon鈥攁 major technology conference in Charlotte, North Carolina.
鈥淎t its heart, our new system shows how to build AI that keeps up with real-world changes, like shifting patterns in infectious diseases, evolving customer behaviors or new financial trends,鈥 said Dr. Li, senior author of the study. 鈥淢ost deep learning models are trained once on a big set of data and then expected to make predictions. But real-world data doesn鈥檛 sit still.鈥
Imagine training a model to predict the spread of a disease based on old data. What happens when the disease mutates, or people鈥檚 behaviors change? The model gets worse and worse over time. To fix it, the whole system usually needs to be retrained, which takes a lot of time, money and computing power, and it might still miss the mark because modelers are chasing a moving target.
鈥淓ven models that try to combine machine learning with the old-school physics of systems modeling, like differential equations, often struggle,鈥 said Chen, lead author of the study. 鈥淭hey are either too rigid, too complicated, too slow or too specific to one problem.鈥
Dr. Li and Chen鈥檚 work answers: What if we could create a smarter way for AI to learn from new information right as it comes in, without constantly retraining? Their idea rests on the following two breakthroughs.
Batch Feature Re-Engineering: When training a deep learning model, data is usually fed in little chunks called 鈥渂atches.鈥 In traditional training, these batches treat every sample as if it鈥檚 just like the others. The researchers decided to embed 鈥渃lass information鈥濃攌nowledge about how categories of data are behaving right now鈥攊nto each batch. For example, if a sudden wave of new infection cases is happening, the model is told about this trend in real time, without having to retrain the whole system.
The second piece of the puzzle, Differential Dynamical Systems, uses the math behind systems that change over time, known as differential equations, to guide the learning process. Instead of only relying on past data to predict the future, the model simulates possible future changes using equations that mirror how systems naturally evolve, whether that鈥檚 how a disease spreads or how stock prices fluctuate. By merging deep learning with dynamic system simulation, the model becomes more flexible and accurate at predicting what鈥檚 next, even when patterns change unexpectedly.
This approach solves some of the most stubborn problems in machine learning today:
- No constant retraining needed: The model adapts on the fly as new data comes in.
- Better performance over time: Because it鈥檚 always adjusting, the model doesn鈥檛 get stale or outdated as quickly.
- More understandable: By tying the learning process to real-world mechanisms, like how diseases spread, it鈥檚 easier for scientists and policymakers to trust what the model is telling them.
鈥淥ur paper shows this system in action by modeling infectious disease outbreaks,鈥 said Chen. 鈥淎s the infection patterns changed over time, the adaptive model stayed accurate much longer than traditional models, even as new waves and variants appeared.鈥
The work by Dr. Li and Chen lays the groundwork for a new kind of AI鈥攐ne that evolves naturally as the world changes and without needing constant human intervention. This could be a game-changer in many fields: predicting economic shifts, managing supply chains, tracking environmental changes or even personalizing healthcare treatments as patient conditions evolve.
鈥淏y teaching AI to understand change itself, we鈥檙e making it more human-like in its ability to adapt,鈥 said Dr Li. 鈥淭his is essential for building systems that can operate reliably in the real world, not just in a lab.鈥
