Newton's Lantern: A Reinforcement Learning Framework for Finetuning AC Power Flow Warm Start Models

Authors: Shourya Bose, Helgi Hilmarsson, Dhruv Suri

arXiv · PDF

Summary

Newton's Lantern fine-tunes neural warm-start models for AC power flow using reinforcement learning with iteration count as the reward, addressing a problem supervised learning struggles with: heavily loaded cases near voltage collapse. The authors prove that warm-start error direction matters more than magnitude, and that supervised regression's guarantees vanish near singularities in the power-flow Jacobian. Their RL pipeline (group relative policy optimization + learned reward model) converges on every test case while achieving the lowest mean iteration count across three power-grid benchmarks.

Main takeaways:

• Neural warm starts for power-flow solvers fail near voltage collapse because supervised learning ignores error direction
• Iteration-count lower bound depends on warm-start error direction, not magnitude; bound becomes vacuous near Jacobian singularities
• RL fine-tuning with iteration count as reward fixes the problem where supervised learning fails
• Newton's Lantern converges on 100% of test snapshots across IEEE 118-bus, GOC 500-bus, and GOC 2000-bus grids
• Only method to achieve both perfect convergence and lowest mean iteration count

Relevance

Not related to my persona/assistant work—this is domain-specific power-grid optimization. Included because it demonstrates using RL to fix a supervised-learning failure mode, which could be conceptually relevant if I ever need to fine-tune models where standard loss functions miss important behavioral objectives.

Threat model

Potential threat/caveat for clean result "Training a [ZLT] persona-marker into Qwen-2.5-7B doesn't increase system-prompt attention at the marker timestep — base Qwen on identical tokens attends the same way (LOW confidence)": this item discusses failure, benchmark.

Abstract

arXiv:2605.11102v1 Announce Type: new Abstract: Neural warm starts can sharply reduce the number of Newton-Raphson iterations required to solve the AC power flow problem, but existing supervised approaches generalize poorly on heavily loaded instances near voltage collapse. We prove a lower bound on the Newton-Raphson iteration count that depends on the direction of the warm start error rather than on its magnitude, and show as a corollary that the bound becomes vacuous as the smallest singular value of the power-flow Jacobian shrinks, identifying the failure mode of supervised regression near the saddle-node bifurcation. Motivated by this analysis, we introduce Newton's Lantern, a finetuning pipeline that combines group relative policy optimization with a learned reward model trained on perturbations of the base model's predictions, using the iteration count itself as the supervisory signal. Across IEEE 118-bus, GOC 500-bus, and GOC 2000-bus benchmarks, Newton's Lantern is the only method that converges on every test snapshot while attaining the smallest mean iteration count.