Deep reinforcement learning for optimizing the thermoacoustic core in a supercritical CO2 thermoacoustic engine

Thermoacoustic engines (TAEs) are promising energy conversion technologies due to their absence of moving parts, flexibility, and environmental friendliness. The driver of such an engine is the thermoacoustic core (TAC). In this study, we propose a framework that integrates CFD simulations, a surrogate model based on an artificial neural network (ANN), and deep reinforcement learning (DRL) to optimize the channel shape in the TAC of a supercritical CO2 TAE. CFD simulations generate a dataset for the surrogate model. The surrogate model demonstrates exceptional generalization capability (R2=0.992) and computational efficiency (within 3.8 ms per prediction), enabling fast reward evaluation during the DRL optimization. The TD3 algorithm is employed to explore the continuous design space. The optimized channel achieves a pressure amplitude of 0.663MPa, an 8.51% improvement compared to the original straight channel, which can be attributed to the enhanced heat transfer matching between the hot heat exchanger and the ambient one. This study demonstrates the potential of combining ANN-based surrogate models with DRL for optimizing thermoacoustic devices. The proposed framework is adaptable for optimizing other thermal systems and casts light on integrating artificial intelligence with physical modeling for engineering optimization.