Adaptive hybrid reinforcement learning for enhanced cooling system efficiency and energy management

In order to achieve carbon neutrality, reducing energy consumption and enhancing system efficiency are paramount. Building cooling systems are significant energy consumers and present a prime opportunity for energy optimization. Many researchers have addressed this issue by utilizing either standalone model-free or model-based reinforcement learning (RL) control to manage operational parameters. However, these approaches may require a lengthy learning process to adapt to specific conditions, such as occupant schedules, leading to decreased adaptiveness when faced with unseen situations. Thus, this study investigated the effectiveness of combining model-based and model-free approaches, especially Monte Carlo Tree Search (MCTS) and deep Q-Learning algorithms (DQL), by allowing MCTS to explore future scenarios and inform DQL's decision-making, leading to improved adaptability and more efficient energy management. This control algorithm was developed, then implemented in the EnergyPlus programming environment. This framework was used for training, leveraging experimental data, and verifying the building simulation model. The results show that EnergyPlus validation successfully achieved a satisfied coefficient of variation (root mean square error), CV (RMSE). Consequently, the control algorithm was trained accordingly. The combined model effectively reduced energy usage during normal operations and adapted well to sudden changes in load profiles by generating control signals in response to those changes. It demonstrated higher adaptability, making them more powerful compared to the standalone DQL model, and achieved a reduction in energy consumption of up to 5-10%. This study underscored the importance of innovative approaches in addressing contemporary energy challenges and supported the ongoing transition towards more sustainable energy practices.