Distributed Deep Reinforcement Learning for Fighting Forest Fires with a Network of Aerial Robots

Ravi N. Haksar, Mac Schwager

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018

Abstract

This paper proposes a distributed deep reinforcement learning (RL) based strategy for a team of Unmanned Aerial Vehicles (UAVs) to autonomously ﬁght forest ﬁres. We ﬁrst model the forest ﬁre as a Markov decision process (MDP) with a factored structure. We consider optimally controlling the forest ﬁre without agents using dynamic programming, and show any exact solution and many approximate solutions are computationally intractable. Given the problem complexity, we consider a deep RL approach in which each agent learns a policy requiring only local information. We show with Monte Carlo simulations that the deep RL policy outperforms a handtuned heuristic, and scales well for various forest sizes and different numbers of UAVs as well as variations in model parameters. Experimental demonstrations with mobile robots ﬁghting a simulated forest ﬁre in the Robotarium at the Georgia Institute of Technology are also presented.

BibTeX

@inproceedings{haksar_distributed_2018,
  address = {Madrid, Spain},
  title = {Distributed {Deep} {Reinforcement} {Learning} for {Fighting} {Forest} {Fires} with a {Network} of {Aerial} {Robots}},
  isbn = {978-1-5386-8094-0},
  url = {https://ieeexplore.ieee.org/document/8593539/},
  abstract = {This paper proposes a distributed deep reinforcement learning (RL) based strategy for a team of Unmanned Aerial Vehicles (UAVs) to autonomously ﬁght forest ﬁres. We ﬁrst model the forest ﬁre as a Markov decision process (MDP) with a factored structure. We consider optimally controlling the forest ﬁre without agents using dynamic programming, and show any exact solution and many approximate solutions are computationally intractable. Given the problem complexity, we consider a deep RL approach in which each agent learns a policy requiring only local information. We show with Monte Carlo simulations that the deep RL policy outperforms a handtuned heuristic, and scales well for various forest sizes and different numbers of UAVs as well as variations in model parameters. Experimental demonstrations with mobile robots ﬁghting a simulated forest ﬁre in the Robotarium at the Georgia Institute of Technology are also presented.},
  language = {en},
  urldate = {2020-09-15},
  booktitle = {2018 {IEEE}/{RSJ} {International} {Conference} on {Intelligent} {Robots} and {Systems} ({IROS})},
  publisher = {IEEE},
  author = {Haksar, Ravi N. and Schwager, Mac},
  month = oct,
  year = {2018},
  keywords = {reinforcement learning, multi-robot, aerial},
  pages = {1067--1074},
  month_numeric = {10}
}