DOI: https://doi.org/10.61435/ijred.2026.62121
View PDF Download fulltext

Source-grid-load-storage multi-terminal collaborative control technology for photovoltaic distribution network based on DQN

State Grid Gansu Electric Power Company Jiayuguan Power Supply Company, Jiayuguan 735100, China

Received: 16 Dec 2025; Revised: 17 Aug 2026; Accepted: 19 Oct 2026; Published: 1 Nov 2026.
Editor(s): H Hadiyanto
Open Access Copyright (c) 2026 The Author(s). Published by Centre of Biomass and Renewable Energy (CBIORE)
Creative Commons License This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Citation Format:
Abstract

As the photovoltaic penetration rate continues to increase, the dynamic characteristics of the multi-dimensional coupling of source, grid, load and storage in the distribution network have become increasingly significant. Traditional control methods often face response lag and regulation instability in this multi-time-varying scenario. Therefore, the study proposes a source-grid-load-storage multi-terminal collaborative control strategy for photovoltaic distribution network. A centralized and distributed two-level collaborative architecture is established to enable deep reinforcement learning agents to learn the optimal power regulation strategy. The multi-time scale control of voltage regulator taps, photovoltaic, energy storage systems, and static reactive power compensation devices is coordinated to achieve voltage optimization and efficient energy allocation. A comparative test was conducted using the IEEE 33-node system as the simulation platform. The results showed that the proposed method could increase the peak photovoltaic output by about 20%-25% on the power generation side, and the average power generation utilization rate reached 93%. The load transfer rate increased to 26.2%, and the load balance dropped to 0.21. In terms of voltage stability, the system fluctuation was controlled within 0.15 p.u., and the steady-state error was less than 0.01 p.u. In typical operating scenarios, deep reinforcement learning control increased the energy storage utilization rate to 86.3%-88.9% and reduced the network loss rate to 3.1%-3.5%. The results show that this method is significantly better than the comparison strategy in terms of multi-source collaboration and dynamic stability, and provides an efficient and intelligent optimization solution for the safe and economical operation of high-penetration photovoltaic distribution networks.

Keywords: Photovoltaic distribution network; Deep reinforcement learning; Source-grid-load-storage collaborative control; Voltage control; Energy management

Article Metrics:

Article Info
Section: Original Research Article
Language : EN
Recent articles
Enhance EV station charge and discharge based on deep learning forecasting method incorporating renewable energy sources and multi-objective optimization algorithm Source-grid-load-storage multi-terminal collaborative control technology for photovoltaic distribution network based on DQN Experimental and machine learning-based electro-thermal performance enhancement of PV/T renewable energy systems with advanced cooling and power optimization techniques for sustainable buildings More recent articles
Most cited articles
Simulation of biogas utilization effect on the economic efficiency and greenhouse gas emission: a case study in Isfahan, Iran Thermal Energy Optimization of Building Integrated Semi-Transparent Photovoltaic Thermal Systems Design and Development of Prototype Cylindrical Parabolic Solar Collector for Water Heating Application Numerical Performance Analyses of Different Airfoils for Use in Wind Turbines Multi-Criteria Decision Making (MCDM) Approach for Selecting Solar Plants Site and Technology: A Review More cited articles
  1. Amer, A. A., Shaban, K. B., & Massoud, A. M. (2023). DRL-HEMS: Deep reinforcement learning agent for demand response in home energy management systems considering customers and operators perspectives. IEEE Transactions on Smart Grid, 14(1), 239–250. https://doi.org/10.1109/TSG.2022.3198401
  2. Bosisio, A., Soldan, F., Pisani, M., Bionda, E., Belloni, F., & Morotti, A. (2025). A Q-learning algorithm for optimizing on-load tap changer operation and voltage control in distribution networks with high integration of renewable energy sources. Journal of Modern Power Systems and Clean Energy, 13(6), 2063–2073. https://doi.org/10.35833/MPCE.2024.000528
  3. Chaturvedi, S., Bui, V. H., Su, W., & Wang, M. (2024). Reinforcement learning-based integrated control to improve the efficiency of DC microgrids. IEEE Transactions on Smart Grid, 15(1), 149–159. https://doi.org/10.1109/TSG.2023.3286801
  4. Dang, Y., Xu, J., Yang, F., Jiang, C., & Li, D. (2025). Meta reinforcement learning based adaptive and interpretable energy storage control meets dynamic scenarios. IEEE Transactions on Sustainable Energy, 16(4), 2560–2572. https://doi.org/10.1109/TSTE.2025.3555002
  5. Doostizadeh, M., Jalili, H., & Babaee, A. (2024). A decentralized framework for economic-resilient operation of networked microgrids, considering demand response program. IET Renewable Power Generation, 18(8), 1428–1454. https://doi.org/10.1049/rpg2.12984
  6. Eichelbeck, M., Markgraf, H., & Althoff, M. (2026). CommonPower: A framework for safe data-driven smart grid control. IEEE Transactions on Smart Grid, 17(1), 71–82. https://doi.org/10.1109/TSG.2025.3616402
  7. Gao, H., Jiang, S., Li, Z., Wang, R., Liu, Y., & Liu, J. (2024). A two-stage multi-agent deep reinforcement learning method for urban distribution network reconfiguration considering switch contribution. IEEE Transactions on Power Systems, 39(6), 7064–7076. https://doi.org/10.1109/TPWRS.2024.3371093
  8. Ge, L., Li, J., Hou, L., & Lai, J. (2025). Autonomous voltage regulation for smart distribution network with high-proportion PVs: A graph meta-reinforcement learning approach. IEEE Transactions on Sustainable Energy, 16(4), 2768–2781. https://doi.org/10.1109/TSTE.2025.3566469
  9. Guan, Y., Ma, W., Che, L., & Shahidehpour, M. (2025). Model-based safe reinforcement learning for active distribution network scheduling. IEEE Transactions on Smart Grid, 16(3), 2375–2388. https://doi.org/10.1109/TSG.2025.3547843
  10. Hong, L., Wu, M., Wang, Y., Yan, M., Chen, Z., & Yan, Z. (2024). MADRL-based DSO-customer coordinated bi-level Volt/VAR optimization method for power distribution networks. IEEE Transactions on Sustainable Energy, 15(3), 1834–1846. https://doi.org/10.1109/TSTE.2024.3380605
  11. Hou, S., Fu, A., Salazar Duque, E. M., Palensky, P., Chen, Q., & Vergara, P. P. (2025). DistFlow safe reinforcement learning algorithm for voltage magnitude regulation in distribution networks. Journal of Modern Power Systems and Clean Energy, 13(1), 300–311. https://doi.org/10.35833/MPCE.2024.000253
  12. Irmak, E., Kabalcı, E., & Calpbinici, A. (2024). Event-triggered distributed secondary control for enhancing efficiency, reliability and communication in island mode DC microgrids. IET Renewable Power Generation, 18(1), 78–94. https://doi.org/10.1049/rpg2.12897
  13. Juma, S. A., Ayeng’o, S. P., & Kimambo, C. Z. M. (2024). A review of control strategies for optimized microgrid operations. IET Renewable Power Generation, 18(14), 2785–2818. https://doi.org/10.1049/rpg2.13056
  14. Kaloti, S., & Chowdhury, B. (2025). Dual agent framework for scheduling networked microgrids using DRL to improve resilience. IEEE Transactions on Sustainable Energy, 16(4), 2989–3002. https://doi.org/10.1109/TSTE.2025.3576153
  15. Lee, S., Seon, J., Sun, Y. G., Kim, S. H., Kyeong, C., Kim, D. I., & Kim, J. Y. (2024). Novel architecture of energy management systems based on deep reinforcement learning in microgrid. IEEE Transactions on Smart Grid, 15(2), 1646–1658. https://doi.org/10.1109/TSG.2023.3317096
  16. Locher, L., Stai, E., & Hug, G. (2026). A safe combined reinforcement learning and model predictive control scheme for utility-level battery control in distribution grids. IEEE Transactions on Smart Grid, 17(1), 144–160. https://doi.org/10.1109/TSG.2025.3617966
  17. Manojkumar, R., Kumar, C., Ganguly, S., & Catalão, J. P. S. (2024). Net load forecast error compensation for peak shaving in a grid-connected PV storage system. IEEE Transactions on Power Systems, 39(2), 4372–4381. https://doi.org/10.1109/TPWRS.2023.3317309
  18. Meng, Q., Hussain, S., Luo, F., Wang, Z., & Jin, X. (2025). An online reinforcement learning-based energy management strategy for microgrids with centralized control. IEEE Transactions on Industry Applications, 61(1), 1501–1510. https://doi.org/10.1109/TIA.2024.3430264
  19. Mohsenzadeh-Yazdi, H., Kebriaei, H., & Aminifar, F. (2024). Multi-agent reinforcement learning in a new transactive energy mechanism. IET Generation, Transmission & Distribution, 18(18), 2943–2955. https://doi.org/10.1049/gtd2.13244
  20. Mohamed, M., & Tsuji, T. (2023). Battery scheduling control of a microgrid trading with utility grid using deep reinforcement learning. IEEJ Transactions on Electrical and Electronic Engineering, 18(5), 665–677. https://doi.org/10.1002/tee.23768
  21. Mu, C., Liu, Z., Yan, J., Jia, H., & Zhang, X. (2024). Graph multi-agent reinforcement learning for inverter-based active voltage control. IEEE Transactions on Smart Grid, 15(2), 1399–1409. https://doi.org/10.1109/TSG.2023.3298807
  22. Mukherjee, S., Hossain, R. R., Mohiuddin, S. M., Liu, Y., Du, W., Adetola, V., Jinsiwale, R. A., Huang, Q., Yin, T., & Singhal, A. (2025). Resilient control of networked microgrids using vertical federated reinforcement learning: Designs and real-time test-bed validations. IEEE Transactions on Smart Grid, 16(2), 1897–1910. https://doi.org/10.1109/TSG.2024.3466768
  23. Nematshahi, S., Shi, D., Wang, F., Yan, B., & Nair, A. (2023). Deep reinforcement learning based voltage control revisited. IET Generation, Transmission & Distribution, 17(21), 4826–4835. https://doi.org/10.1049/gtd2.13001
  24. Peng, H., Liao, K., Yang, J., Pang, B., & He, Z. (2025). Deep reinforcement learning based multi-timescale Volt/Var control in distribution networks considering network reconfiguration. IEEE Transactions on Sustainable Energy, 16(4), 2948–2958. https://doi.org/10.1109/TSTE.2025.3574806
  25. Rousseau, J., Cai, H., Heer, P., Orehounig, K., & Hug, G. (2026). Uncertainty-aware flexibility of HVAC systems in buildings: From quantification to provision. IEEE Transactions on Smart Grid, 17(1), 269–282. https://doi.org/10.1109/TSG.2025.3611455
  26. Shahbazian, S., Kharrati, S., & Rastgou, A. (2024). Developing a cooperative approach under normal and contingency conditions for generation expansion planning of microgrids. IET Renewable Power Generation, 18(13), 2065–2079. https://doi.org/10.1049/rpg2.13034
  27. Shalaby, A. A., Abdeltawab, H., & Mohamed, Y. A. R. I. (2025). Safe deep reinforcement learning for resilient self-proactive distribution grids against wildfires. IEEE Transactions on Smart Grid, 16(6), 5065–5082. https://doi.org/10.1109/TSG.2025.3602849
  28. Stanojev, O., Guo, Y., & Hug, G. (2025). Online feedback droop scheduling in distribution grids for frequency and local voltage control. IEEE Transactions on Power Systems, 40(5), 4105–4118. https://doi.org/10.1109/TPWRS.2025.3528752
  29. Sun, Z., & Lu, T. (2024). Collaborative operation optimization of distribution system and virtual power plants using multi-agent deep reinforcement learning with parameter-sharing mechanism. IET Generation, Transmission & Distribution, 18(1), 39–49. https://doi.org/10.1049/gtd2.13037
  30. Vu, L., Vu, T., Vu, T. L., & Srivastava, A. (2024). Multi-agent deep reinforcement learning for distributed load restoration. IEEE Transactions on Smart Grid, 15(2), 1749–1760. https://doi.org/10.1109/TSG.2023.3310893
  31. Wang, R., Bi, X., & Bu, S. (2024). Real-time coordination of dynamic network reconfiguration and Volt-VAR control in active distribution network: A graph-aware deep reinforcement learning approach. IEEE Transactions on Smart Grid, 15(3), 3288–3302. https://doi.org/10.1109/TSG.2023.3324474
  32. Wang, Y., Hu, W., Cao, D., Zhao, P., Abulanwar, S., Chen, Z., & Blaabjerg, F. (2025). Local distribution voltage control using large-scale coordinated PV inverters: A novel multi-agent deep reinforcement learning-based approach. IEEE Transactions on Smart Grid, 16(3), 2683–2686. https://doi.org/10.1109/TSG.2025.3533958
  33. Xu, T., Chen, T., Gao, C., Song, M., Wang, Y., & Yuan, H. (2024). Distributed flexible resource regulation strategy for residential communities based on deep reinforcement learning. IET Generation, Transmission & Distribution, 18(21), 3378–3391. https://doi.org/10.1049/gtd2.13284
  34. Yan, R., Xing, Q., & Xu, Y. (2024). Multi-agent safe graph reinforcement learning for PV inverters-based real-time decentralized Volt/Var control in zoned distribution networks. IEEE Transactions on Smart Grid, 15(1), 299–311. https://doi.org/10.1109/TSG.2023.3277087
  35. Yan, R., & Xu, Y. (2025). Multi-objective and multi-agent deep reinforcement learning for real-time decentralized Volt/VAR control of distribution networks considering PV inverter lifetime. IEEE Transactions on Power Systems, 40(2), 1558–1569. https://doi.org/10.1109/TPWRS.2024.3452154
  36. Yang, X., Liu, H., & Wu, W. (2024). Attention-enhanced multi-agent reinforcement learning against observation perturbations for distributed Volt-VAR control. IEEE Transactions on Smart Grid, 15(6), 5761–5772. https://doi.org/10.1109/TSG.2024.3423700
  37. Ye, Y., Papadaskalopoulos, D. A., Yuan, Q., Tang, Y., & Strbac, G. (2023b). Multi-agent deep reinforcement learning for coordinated energy trading and flexibility services provision in local electricity markets. IEEE Transactions on Smart Grid, 14(2), 1541–1554. https://doi.org/10.1109/TSG.2022.3149266
  38. Ye, Y., Wang, H., Chen, P., Tang, Y., & Strbac, G. (2023a). Safe deep reinforcement learning for microgrid energy management in distribution networks with leveraged spatial-temporal perception. IEEE Transactions on Smart Grid, 14(5), 3759–3775. https://doi.org/10.1109/TSG.2023.3243170
  39. Yu, P., Zhang, H., & Song, Y. (2024). Adaptive tie-line power smoothing with renewable generation based on risk-aware reinforcement learning. IEEE Transactions on Power Systems, 39(6), 6819–6832. https://doi.org/10.1109/TPWRS.2024.3379513
  40. Zhang, J., Sang, L., Xu, Y., & Sun, H. (2024). Networked multiagent-based safe reinforcement learning for low-carbon demand management in distribution networks. IEEE Transactions on Sustainable Energy, 15(3), 1528–1545. https://doi.org/10.1109/TSTE.2024.3355123

Last update:

No citation recorded.

Last update: 2026-11-01 14:48:27

No citation recorded.