Parameter Optimization of Battery Energy Storage System Considering Degradation Using Reinforcement Learning

Muhammad Dzaky Ashidqi; Silviana  Windasari; Rahmat Rahmat; Probokusumo Probokusumo

doi:10.22146/jnteti.v15i1.24882

Muhammad Dzaky Ashidqi Department of Electrical Engineering, Faculty of Engineering, Universitas Sains Indonesia, Bekasi, Jawa Barat 17530, Indonesia
Silviana Windasari Department of Electrical Engineering, Faculty of Engineering, Universitas Sains Indonesia, Bekasi, Jawa Barat 17530, Indonesia
Rahmat Rahmat Department of Mechanical Engineering, Faculty of Engineering, Universitas Sains Indonesia, Bekasi, Jawa Barat 17530, Indonesia
Probokusumo Probokusumo Department of Industrial Engineering, Faculty of Engineering, Universitas Sains Indonesia, Bekasi, Jawa Barat 17530, Indonesia

DOI: https://doi.org/10.22146/jnteti.v15i1.24882

Keywords: Battery Degradation, Battery Energy Storage System (BESS), Parameter Optimization, Reinforcement Learning

Abstract

Accurate and sustainable operation of battery energy storage systems (BESS) is critical for supporting renewable energy integration, ensuring both short-term reliability and long-term asset preservation. This study proposed a reinforcement learning (RL)-based scheduling framework designed to minimize power mismatch while mitigating degradation in lithium-ion batteries. The framework dynamically adapted to fluctuations in photovoltaic generation and residential load, enabling real-time decision-making. The performance was evaluated over a 30-day horizon using three indicators: average power mismatch, cumulative capacity loss, and system stability index (SSI). Results demonstrated that the proposed method achieved near-perfect load balance with an average mismatch of only 0.002 kW, while cumulative degradation remained limited to 0.22% and SSI was maintained at 0.96, reflecting high operational stability. The estimated daily degradation rate of 0.0073% corresponded to an annual capacity loss of approximately 2.7%, significantly lower than the 5–6% typically observed in uncontrolled cycling scenarios. Comparative analysis with simulated annealing (SA) and multi-objective genetic algorithm (MOGA) highlighted the balanced performance of the RL method. While MOGA eliminated mismatch at the expense of excessive degradation (0.60%) and simulated annealing reduced degradation but suffers from high mismatch (0.012 kW), the RL framework delivered the most balanced trade-off across all metrics. These findings confirm the potential of RL as a practical and sustainable strategy for PV–BESS integration, providing both technical resilience and extended battery lifetime.

References

N. Mlilo, J. Brown, and T. Ahfock, “Impact of intermittent renewable energy generation penetration on the power system networks – A review,” Technol. Econ. Smart Grids Sustain. Energy, vol. 6, pp. 1–19, Dec. 2021, doi: 10.1007/s40866-021-00123-w.

S. Motta et al., “A cost-benefit analysis of large-scale battery energy storage systems for frequency markets,” in IET Conf. Proc., 2021, pp. 3130–3134, doi: 10.1049/icp.2021.1470.

C. Zhao, P.B. Andersen, C. Træholt, and S. Hashemi, “Grid-connected battery energy storage system: A review on application and integration,” Renew. Sustain. Energy Rev., vol. 182, pp. 1–19, Aug. 2023, doi: 10.1016/j.rser.2023.113400.

H. Wigger et al., “Quantitative assessment of sector coupling battery energy storage systems and their contribution to the resilience of the power system,” Int. J. Energy Res., vol. 2024, no. 1, pp. 1–20, Dec. 2024, doi: 10.1155/2024/8812115.

M. Sangsoda, A. Prasertnoi, and J. Kluabwang, “A comprehensive review of battery energy storage systems for grid stability and renewable integration: A focus on lithium-ion’s grid batteries in Thailand,” in 2024 IEEE Int. Smart Cities Conf. (ISC2), 2024, pp. 1–6, doi: 10.1109/ISC260477.2024.11004256.

Y. Gu and T.C. Green, “Power system stability with a high penetration of inverter-based resources,” Proc. IEEE, 2023, pp. 832–853, doi: 10.1109/JPROC.2022.3179826.

C. Ghanjati and S. Tnani, “Optimal sizing and energy management of a stand-alone photovoltaic/pumped storage hydropower/battery hybrid system using genetic algorithm for reducing cost and increasing reliability,” Energy Environ., vol. 34, no. 6, pp. 2186–2203, Sep. 2023, doi: 10.1177/0958305X221110529.

Y. Yang, S. Bremner, C. Menictas, and M. Kay, “Modelling and optimal energy management for battery energy storage systems in renewable energy systems: A review,” Renew. Sustain. Energy Rev., vol. 167, pp. 1–27, Oct. 2022, doi: 10.1016/j.rser.2022.112671.

T. Rahman and T. Alharbi, “Exploring lithium-ion battery degradation: A concise review of critical factors, impacts, data-driven degradation estimation techniques, and sustainable directions for energy storage systems,” Batteries, vol. 10, no. 7, pp. 1–32, Jul. 2024, doi: 10.3390/batteries10070220.

G. Zhang et al., “Investigation the degradation mechanisms of lithium-ion batteries under low-temperature high-rate cycling,” ACS Appl. Energy Mater., vol. 5, no. 5, pp. 6462–6471, May 2022, doi: 10.1021/acsaem.2c00957.

J. Schmitt et al., “Impedance change and capacity fade of lithium nickel manganese cobalt oxide-based batteries during calendar aging,” J. Power Sources, vol. 353, pp. 183–194, Jun. 2017, doi: 10.1016/j.jpowsour.2017.03.090.

B. Sood et al., “Lithium-ion battery degradation mechanisms and failure analysis methodology,” in Int. Symp. Test. Fail. Anal., 2012, pp. 239–249, doi: 10.31399/asm.cp.istfa2012p0239.

Aravindhanathan G et al., “Analysis of cyclic degradation of Li-ion batteries,” in 2024 IEEE 11th Power India Int. Conf. (PIICON), 2024, pp. 1–6, doi: 10.1109/PIICON63519.2024.10995090.

S. Nazaralizadeh, P. Banerjee, A.K. Srivastava, and P. Famouri, “Battery energy storage systems: A review of energy management systems and health metrics,” Energies, vol. 17, no. 5, pp. 1–21, Mar. 2024, doi: 10.3390/en17051250.

M. Dubarry and A. Devie, “Battery durability and reliability under electric utility grid operations: Representative usage aging and calendar aging,” J. Energy Storage, vol. 18, pp. 185–195, Aug. 2018, doi: 10.1016/j.est.2018.04.004.

M.D. Ashidqi, A.I. Cahyadi, and A. Ataka, “Capacity loss modeling of Li-ion battery using lightweight neural network considering equivalent circuit model,” in 2023 Int. Conf. Technol. Policy Energy Elect. Power (ICT-PEP), 2023, pp. 133–138, doi: 10.1109/ICT-PEP60152.2023.10351143.

L.A. Wong et al., “Optimal placement and sizing of battery energy storage system for losses reduction using whale optimization algorithm,” J. Energy Storage, vol. 26, pp. 1–13, Dec. 2019, doi: 10.1016/j.est.2019.100892.

B. Cortés-Caicedo, L.F. Grisales-Noreña, O.D. Montoya, and R.I. Bolaños, “Optimization of BESS placement, technology selection, and operation in microgrids for minimizing energy losses and CO2 emissions: A hybrid approach,” J. Energy Storage, vol. 73, pp. 1–15, Dec. 2023, doi: 10.1016/j.est.2023.108975.

P. Parczyk and R. Burduk, “Optimisation of battery energy storage systems capacity for purpose to reduce energy cost,” in Comput. Inf. Syst. Ind. Manag. (CISIM 2024), 2024, pp. 414–427, doi: 10.1007/978-3-031-71115-2_29.

V. Saxena, N. Kumar, and U. Nangia, “Computation and optimization of BESS in the modeling of renewable energy based framework,” Arch. Comput. Methods Eng., vol. 31, no. 4, pp. 2385–2416, May 2024, doi: 10.1007/s11831-023-10046-7.

C. Jiang et al., “Optimal operation strategy for integrated energy systems based on improved grey wolf optimization algorithm,” in 2024 3rd Int. Conf. Power Syst. Elect. Technol. (PSET), 2024, pp. 720–724, doi: 10.1109/PSET62496.2024.10808556.

Y. Zhi, D. Gao, G. Wei, and X. Yang, “Simultaneous sizing and scheduling optimization for farmhouse PV-battery systems with a multi-structured power system model,” J. Energy Storage, vol. 104, pp. 1–12, Dec. 2024, doi: 10.1016/j.est.2024.114564.

R.G. Mohamed, A.A. Hassan, and S.H.E.A. Aleem, “Techno-economical-environmental approach using multi-objective-based Pareto frontier optimization for hybrid photovoltaic/diesel/battery system considering different battery types,” J. Energy Storage, vol. 115, pp. 1–21, Apr. 2025, doi: 10.1016/j.est.2025.115979.

S.P. Pimentel et al., “A systematic literature review on Li-Ion BESSs integrated with photovoltaic systems for power supply to auxiliary services in high-voltage power stations,” Energies, vol. 18, no. 13, pp. 1–38, Jul. 2025, doi: 10.3390/en18133544.

J. He et al., “A data-driven reinforcement learning enabled battery fast charging optimization using real-world experimental data,” IEEE Trans. Ind. Electron., vol. 72, no. 1, pp. 430–438, Jan. 2025, doi: 10.1109/TIE.2024.3398687.