One reason deep learning exploded over the past decade was the provision of programming languages that might automate the maths — college-level calculus — that is required to coach each latest model. Neural networks are trained by tuning their parameters to try to maximise a rating that could be rapidly calculated for training data. The equations used to regulate the parameters in each tuning step was once derived painstakingly by hand. Deep learning platforms use a way called automatic differentiation to calculate the adjustments mechanically. This allowed researchers to rapidly explore an enormous space of models, and find those that actually worked, without having to know the underlying math.
But what about problems like climate modeling, or financial planning, where the underlying scenarios are fundamentally uncertain? For these problems, calculus alone will not be enough — you furthermore mght need probability theory. The “rating” is not any longer only a deterministic function of the parameters. As an alternative, it’s defined by a stochastic model that makes random selections to model unknowns. Should you try to make use of deep learning platforms on these problems, they’ll easily give the flawed answer. To repair this problem, MIT researchers developed ADEV, which extends automatic differentiation to handle models that make random selections. This brings the advantages of AI programming to a much wider class of problems, enabling rapid experimentation with models that may reason about uncertain situations.
Lead creator and MIT electrical engineering and computer science PhD student Alex Lew says he hopes people will probably be less wary of using probabilistic models now that there’s a tool to mechanically differentiate them. “The necessity to derive low-variance, unbiased gradient estimators by hand can result in a perception that probabilistic models are trickier or more finicky to work with than deterministic ones. But probability is an incredibly great tool for modeling the world. My hope is that by providing a framework for constructing these estimators mechanically, ADEV will make it more attractive to experiment with probabilistic models, possibly enabling latest discoveries and advances in AI and beyond.”
Sasa Misailovic, an associate professor on the University of Illinois at Urbana-Champaign who was not involved on this research, adds: “Because the probabilistic programming paradigm is emerging to resolve various problems in science and engineering, questions arise on how we are able to make efficient software implementations built on solid mathematical principles. ADEV presents such a foundation for modular and compositional probabilistic inference with derivatives. ADEV brings the advantages of probabilistic programming — automated math and more scalable inference algorithms — to a much wider range of problems where the goal will not be simply to infer what might be true but to make your mind up what motion to take next.”
Along with climate modeling and financial modeling, ADEV may be used for operations research — for instance, simulating customer queues for call centers to reduce expected wait times, by simulating the wait processes and evaluating the standard of outcomes — or for tuning the algorithm that a robot uses to know physical objects. Co-author Mathieu Huot says he’s excited to see ADEV “used as a design space for novel low-variance estimators, a key challenge in probabilistic computations.”
The research, awarded the SIGPLAN Distinguished Paper award at POPL 2023, is co-authored by Vikash Mansighka, who leads MIT’s Probabilistic Computing Project within the Department of Brain and Cognitive Sciences and the Computer Science and Artificial Intelligence Laboratory, and helps lead the MIT Quest for Intelligence, in addition to Mathieu Huot and Sam Staton, each at Oxford University. Huot adds, “ADEV gives a unified framework for reasoning about the ever-present problem of estimating gradients unbiasedly, in a clean, elegant and compositional way.” The research was supported by the National Science Foundation, the DARPA Machine Common Sense program, and a philanthropic gift from the Siegel Family Foundation.
“Lots of our most controversial decisions — from climate policy to the tax code — boil right down to decision-making under uncertainty. ADEV makes it easier to experiment with latest ways to resolve these problems, by automating among the hardest math,” says Mansinghka. “For any problem that we are able to model using a probabilistic program, we now have latest, automated ways to tune the parameters to attempt to create outcomes that we wish, and avoid outcomes that we do not.”