基于深度Q神经网络DQN)的空调冷却水系统无模型优化
摘要：在建筑空调水系统的优化控制领域，基于模型的控制方法得到了广泛的研究和验证。但基于模型的控制很大程度上依赖于精确的系统性能模型和足够的传感器，而这对于某些建筑来说是很难获得的。针对这一问题，本文提出了一种基于深度Q神经网络(DQN)的空调冷却水系统无模型优化方法，该方法以室外空气湿球温度、系统冷负荷及冷水机组开启状态为状态，以冷却塔风机和水泵的频率为动作，以系统性能系数(COP)为奖励。根据实际系统的实测数据进行建模，在模拟环境中使用基于粒子群优化算法的模型优化方法、基于Q值(Q learning)优化的强化学习方法和基于DQN的无模型优化方法进行实验，结果表明基于DQN的无模型优化方法的优化效果最好，有7.68%的平均COP提升与7.15%的节能率，在复杂系统下拥有较好的节能效果。
关键词：无模型优化深度Q神经网络冷却水系统优化控制能耗
尊敬的用户，本篇文章需要2元，点击支付交费后阅读
限时优惠福利：领取VIP会员
全年期刊、VIP视频免费!
全年期刊、VIP视频免费!
参考文献[1] 曲凯阳，胡德祥，王连吉，等.空调冷却水系统最优节能控制策略[J].暖通空调，2010,40(4):110- 114,109.
[2] 陆琼文，刘飘.冷却水系统设计温差节能性分析[J].暖通空调，2022,52(4):13- 17,103.
[3] 杨露露，卢军，唐红琴，等.空调冷却水系统节能控制策略[J].暖通空调，2013,43(4):97- 99.
[4] KANG W L,YOON Y,LEE J H,et al.In-situ application of an ANN algorithm for optimized chilled and condenser water temperatures set-point during cooling operation[J].Energy and buildings,2021,233:110666.
[5] ZHU N,SHAN K,WANG S W,et al.An optimal control strategy with enhanced robustness for air-conditioning systems considering model and measurement uncertainties[J].Energy and buildings,2013,67:540- 550.
[6] MNIH V,KAVUKCUOGLU K,SILVER D,et al.Human-level control through deep reinforcement learning[J].Nature,2015,518(7540):529- 533.
[7] QIU S N,LI Z H,LI Z W,et al.Model-free control method based on reinforcement learning for building cooling water systems:validation by measured data-based simulation[J].Energy and buildings,2020,218:110055.
[8] DU Y,ZANDI H,KOTEVSKA O,et al.Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning[J].Applied energy,2021,281:116117.
[9] JIANG Z H,RISBECK M J,RAMAMURTI V,et al.Building HVAC control with reinforcement learning for reduction of energy cost and demand charge[J].Energy and buildings,2021,239:110833.
[10] DU Y,LI F X,MUNK J,et al.Multi-task deep reinforcement learning for intelligent multi-zone residential HVAC control[J].Electric power systems research,2021,192:106959.
[11] MNIH V,KAVUKCUOGLU K,SILVER D,et al.Playing atari with deep reinforcement learning[J].Computer science,2013(12):1312.5602.
[12] FRIEDMAN J H.Greedy function approximation:a gradient boosting machine[J].The annals of statistics,2001,29(5):1189- 1232.
[13] KE G,MENG Q,FINLEY T,et al.LightGBM:a highly efficient gradient boosting decision tree[C]//Neural Information Processing Systems,2017:3149- 3157.
[2] 陆琼文，刘飘.冷却水系统设计温差节能性分析[J].暖通空调，2022,52(4):13- 17,103.
[3] 杨露露，卢军，唐红琴，等.空调冷却水系统节能控制策略[J].暖通空调，2013,43(4):97- 99.
[4] KANG W L,YOON Y,LEE J H,et al.In-situ application of an ANN algorithm for optimized chilled and condenser water temperatures set-point during cooling operation[J].Energy and buildings,2021,233:110666.
[5] ZHU N,SHAN K,WANG S W,et al.An optimal control strategy with enhanced robustness for air-conditioning systems considering model and measurement uncertainties[J].Energy and buildings,2013,67:540- 550.
[6] MNIH V,KAVUKCUOGLU K,SILVER D,et al.Human-level control through deep reinforcement learning[J].Nature,2015,518(7540):529- 533.
[7] QIU S N,LI Z H,LI Z W,et al.Model-free control method based on reinforcement learning for building cooling water systems:validation by measured data-based simulation[J].Energy and buildings,2020,218:110055.
[8] DU Y,ZANDI H,KOTEVSKA O,et al.Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning[J].Applied energy,2021,281:116117.
[9] JIANG Z H,RISBECK M J,RAMAMURTI V,et al.Building HVAC control with reinforcement learning for reduction of energy cost and demand charge[J].Energy and buildings,2021,239:110833.
[10] DU Y,LI F X,MUNK J,et al.Multi-task deep reinforcement learning for intelligent multi-zone residential HVAC control[J].Electric power systems research,2021,192:106959.
[11] MNIH V,KAVUKCUOGLU K,SILVER D,et al.Playing atari with deep reinforcement learning[J].Computer science,2013(12):1312.5602.
[12] FRIEDMAN J H.Greedy function approximation:a gradient boosting machine[J].The annals of statistics,2001,29(5):1189- 1232.
[13] KE G,MENG Q,FINLEY T,et al.LightGBM:a highly efficient gradient boosting decision tree[C]//Neural Information Processing Systems,2017:3149- 3157.
Model-free optimization for air conditioning cooling water systems based on deep Q network (DQN)
Abstract: Model-based control methods have been widely investigated and validated in the domain of optimal control for building air conditioning water systems. However, the performance of model-based control highly depends on accurate system models and enough sensors, which are difficult to obtain in some buildings. To overcome this problem, a model-free optimization method for air conditioning cooling water system based on deep Q network(DQN) is proposed. The wet bulb temperature of outdoor air, system cooling load and chiller on/off states are taken as the states, the frequencies of cooling tower fans and cooling water pumps are taken as the actions, and the reward is the system COP. In the simulation environment built by the measured data of an actual system, the model optimization method based on particle swarm optimization, the reinforcement learning method based on Q-value(Q learning) optimization and the model-free optimization method based on DQN are used to conduct experiment. The results show that the model-free optimization method based on DQN has the best optimization effect with 7.68% average COP improvement and 7.15% energy saving rate, which has a better energy saving effect in complex systems.
Keywords: model-free optimization; deep Q network(DQN); cooling water system; optimal control; energy consumption;
559
0
0