Mohammad Etemad (Dalhousie University), Nader Zare (Dalhousie University), Mahtab Sarvmaili (Dalhousie University), Amilcar Soares (Institute for Big Data Analytics), Bruno Brandoli Machado (Institute for Big Data Analytics) and Stan Matwin (Dalhousie Unbiversity).

Unmanned Surface Vehicles (USVs) is an exciting topic that essentially deploys an algorithm to safely and efficiently perform a mission. Although reinforcement learning is a well-known approach to model such task, instability and divergence may occur when combining off-policy and function approximation. In this work, we used deep reinforcement learning combining Q-learning with a neural representation to avoid instability. Our methodology uses deep q-learning and combined it with a rolling wave planning approach in agile methodology. Our method contains two critical parts in order to perform missions in an unknown environment. The first addresses a path planner that is responsible for generating a potential effective path to a destination without considering the details of the movement. The latter, a decision-making module that is responsible for short-term decisions on avoiding obstacles during the USV exploitation in the near future steps with the context of the value function. Simulations were performed using two algorithms: vanilla vessel navigator (VVN) as a baseline and an improved one to the vessel navigator with a planner and local view (VNPLV). Experimental results show that the proposed method enhanced the performance of VVN by 55.31\% on average for long-distance missions. Our model successfully demonstrated obstacle avoidance by means of deep reinforcement leaning using planning adaptive paths in unknown environments.

свернуть