Skip to main content

AI (Deep Reinforcement Learning) for Strategic Bidding in Energy Markets

Primary supervisor

Hao Wang

The world’s energy markets are transforming, and more renewable energy is integrated into the electric energy market. The intermittent renewable supply leads to unexpected demand-supply mismatches and results in highly fluctuating energy prices. Energy arbitrage aims to strategically operate energy devices to leverage the temporal price spread to smooth out the price differences in the market, which also generates some revenue. The ancillary energy market provides frequency regulation services to ensure power system security in the face of limited dispatchability of the renewable supply. It is often difficult to forecast prices in the energy market and challenging to develop a bidding strategy for arbitrage, given the market's complex behavior.

This project aim to design effective bidding strategies to leverage historical market data and secure market operation. Deep reinforcement learning is a category of artificial intelligence that learns the best actions through a series of trials and errors similar to humans. Deep reinforcement learning's unique features make it an ideal technology for dynamic and stochastic environments, such as energy markets. But most existing studies focused on either a single market using reinforcement learning or multiple markets using optimisation methods given market prices, leaving a research gap of how to bid in multiple markets in real-time optimally.

#sustainability

Student cohort

Double Semester

Aim/outline

1) Model the energy market and bidding behaviors; 

2) Develop a deep reinforcement learning algorithm for strategic bidding in energy markets to maximise the expected profit;

3) Train and test the developed strategy on real-world data and compare with baseline methods.

URLs/references

Some recent papers from the team.

1) Hao Wang, B. Zhang, Energy storage arbitrage in real-time markets via reinforcement learning, IEEE Power & Energy Society General Meeting (PESGM), 2018.

2) Y Cao, Hao Wang, D Li, G Zhang, Smart Online Charging Algorithm for Electric Vehicles via Customized Actor-Critic Learning, IEEE Internet of Things Journal, 9(1), pp. 684-694, 2022.

3) M. Anwar*, Changlong, Wang, Frits de Nijs, Hao Wang, Proximal Policy Optimization Based Reinforcement Learning for Joint Bidding in Energy and Frequency Regulation Markets, IEEE Power & Energy Society General Meeting (PESGM), 2022.

(*Thesis students.)

Required knowledge

Programming in Python (required), Markov decision process and/or reinforcement learning (preferred).