作者：陈剑龙1, 刘俊峰1, 王振刚2, 曾君2, 洪晓斌3
作者单位：1. 华南理工大学自动化科学与工程学院, 广东 广州 510640;
2. 华南理工大学电力学院, 广东 广州 510640;
3. 华南理工大学机械与汽车工程学院, 广东 广州 510640
关键词：混合动力船舶;无人船;能量管理;深度强化学习;深度Q网络
摘要：
无人船混合动力系统的能量管理是提高无人船能源供应稳定性和提升无人船续航里程的关键因素。该文针对无人船运行特点，设计一种光柴储混合动力系统，并提出一种基于深度强化学习的无人船混合动力系统能量管理策略。该文首先分析光柴储混合动力无人船的动力系统结构和运行模式，引入深度强化学习理论，将结合深度学习与强化学习的深度Q网络算法应用于混合动力无人船的智能能量管理方法。仿真结果表明混合动力船舶相较柴电动力船舶在成本、节能和环保等方面具有优势，所提出的能量管理策略具备自学习能力，能合理有效地自主应对突发场景，从而有效支撑无人船技术的发展。
Energy management strategy of hybrid ship based on deep reinforcement learning
CHEN Jianlong1, LIU Junfeng1, WANG Zhen'gang2, ZENG Jun2, HONG Xiaobin3
1. School of Automation Science and Engineering, South China University of Technology, Guangzhou 510640, China;
2. School of Electric Power, South China University of Technology, Guangzhou 510640, China;
3. School of Mechanical and Automotive Engineering, South China University of Technology, Guangzhou 510640, China
Abstract: The unmanned surface vehicles (USVs) has become a trend of marine technology. A deep reinforcement learning (DRL) based strategy was proposed for energy management of hybrid USVs. Firstly, a hybrid power system structure of hybrid USVs with diesel engine, photovoltaic cell and energy storage is demonstrated. Secondly, deep Q network is introduced and applied to energy management of hybrid USVs. Finally, the energy management performances in two different scenes were shown and the effectiveness of the algorithm was verified by simulation.
Keywords: hybrid ship;USVs;energy management;deep reinforcement learning;deep Q network
2020, 46(2):9-15 收稿日期: 2019-05-31;收到修改稿日期: 2019-07-20
基金项目: 广东省科技计划项目（2018B010109005，2019B151502057）；广东省自然资源厅科技项目（GDoE[2019]A13）；广州市科技计划项目（201802020009）
作者简介: 陈剑龙(1994-),男,福建泉州市人,硕士研究生,专业方向为微电网能量管理、机器学习
参考文献
[1] 周洪光, 马爱民, 夏朗. 无人水面航行器发展[J]. 国防科技, 2009, 30(6):17-20
[2] 熊亚洲, 张晓杰, 冯海涛, 等. 一种面向多任务应用的无人水面艇[J]. 船舶工程, 2012(1):16-19
[3] YAN R J, PANG S, SUN H B, et al. Development and missions of unmanned surface vehicle[J]. Journal of Marine Science & Application, 2010, 9(4):451-457
[4] DUNBABIN M, GRINHAM A. Experimental evaluation of an autonomous surface vehicle for water quality and greenhouse gas emission monitoring[C]//IEEE International Conference on Robotics & Automation. IEEE, 2010.
[5] 张益敏, 陈俐, 朱剑昀. 混合动力船舶动力装置及能量管理研究综述[J]. 舰船科学技术, 2018(5):1-5
[6] BASSAM A M, PHILLIPS A B, TURNOCK S R, et al. Development of a multi-scheme energy management strategy for a hybrid fuel cell driven passenger ship[J]. International Journal of Hydrogen Energy, 2016, 42(1)
[7] 俞万能, 廖卫强, 杨荣峰, 等. 基于太阳能锂电池及柴油发电机组的多能源(光柴储)船舶微网能量控制系统研发[J]. 中国造船, 2017, 58(1):170-176
[8] 秦锋, 施伟锋, 张威. 能量管理策略在混合动力船舶上的应用[J]. 船电技术, 2017, 37(7):32-36
[9] SOLEYMANI M, YOOSOFI A, KANDID M. Sizing and energy management of a medium hybrid electric boat[J]. Journal of Marine Science & Technology, 2015, 20(4):739-751
[10] STONE P, OPILA D F, PARK H, et al. Shipboard power management using constrained nonlinear model predictive control[C]//Electric Ship Technologies Symposium. IEEE, 2015:1-7.
[11] KANELLOS F D. Optimal power management with GHG emissions limitation in all-electric ship power systems comprising energy storage systems[J]. IEEE Transactions on Power Systems, 2013, 29(1):330-339
[12] PANDAY A, BANSAL H O. Energy management strategy for hybrid electric vehicles using genetic algorithm[J]. Journal of Renewable & Sustainable Energy, 2016, 8(1):741-646
[13] ERDINC O, VURAL B, UZUNOGLU M. A wavelet-fuzzy logic based energy management strategy for a fuel cell/battery/ultra-capacitor hybrid vehicular power system[J]. Journal of Power Sources, 2009, 194(1):369-380
[14] YI L M, PARK J, CHEN Z, et al. Intelligent hybrid vehicle power control-part I:machine learning of optimal vehicle power[J]. IEEE Transactions on Vehicular Technology, 2012, 61(8):3519-3530
[15] WATKINS C J C H. Learning from delayed rewards[D]. London:University of Cambridge, 1989.
[16] RIEDMILLER M. Neural fitted Q iteration-first experiences with a data efficient neural reinforcement learning method[C]//European Conference on Machine Learning. Springer-Verlag, 2005:317-328.
[17] 高迪驹, 沈爱弟, 褚建新, 等. 混合动力船舶的能量管理与控制策略[J]. 上海海事大学学报, 2015, 36(1):70-74
