Model-based Reinforcement Learning with Active Learning for Efficient Electrical Power Converter Design

Default ARPA-E Project Image

Yorktown Heights, New York
Project Term:
04/27/2020 - 12/31/2022

Critical Need:

The DIFFERENTIATE program seeks to leverage the emerging artificial intelligence (AI) revolution to help resolve the energy and environmental challenges of our time. The program aims to speed energy innovation by incorporating machine learning (ML) into the energy technology development process. A core part of AI, ML is the study of computer algorithms that improve automatically through experience. This approach is expected to facilitate a rapid transition to lower-carbon-footprint energy sources and systems. To organize the proposed efforts, the program uses a simplified engineering design process framework to identify six general mathematical optimization problems common to many engineering design processes. It then conceptualizes several ML tools that could help engineers execute and solve these problems in a manner that dramatically accelerates the pace of energy innovation.

Project Innovation + Advantages:

IBM Research will develop a reinforcement learning (RL)-based electrical power converter design tool. Such converters are widely used and critically important in many applications. Designing a specific converter is a lengthy and expensive process that involves multiple manual steps—selecting and configuring the correct components and topologies; evaluating the design performance via simulations; and iteratively optimizing the design while satisfying resource, technology, and cost constraints. In this project, the design problem will be formulated as mixed integer optimization to be intelligently and automatically solved using an RL-based optimizer. Because the physics-based simulation models used for design are computationally expensive and time-consuming, IBM’s framework will use surrogate models with similar fidelity but lower computation cost and query the physics-based models only infrequently via intelligent strategies. The proposed RL-enhanced automated design can explore the design space much more effectively, decreasing development time and cost without compromising performance.

Potential Impact:

DIFFERENTIATE aims to enhance the productivity of energy engineers in helping them to develop next-generation energy technologies. If successful, DIFFERENTIATE will yield the following benefits in ARPA-E mission areas:


Seek U.S. technological competitive advantage by leading the development of machine-learning enhanced engineering design tools.


Use these tools to solve our most challenging energy and environmental problems by facilitating an economically-attractive transition to lower carbon-footprint energy sources and systems.


Reap the economic productivity benefits associated with the commercial adoption of the resulting higher-value energy technologies and associated products.


ARPA-E Program Director:
Dr. David Tew
Project Contact:
Xin Zhang
Press and General Inquiries Email:
Project Contact Email:

Related Projects

Release Date: