Year
2024
Season
Spring
Paper Type
Master's Thesis
College
College of Computing, Engineering & Construction
Degree Name
Master of Science in Electrical Engineering (MSEE)
Department
Engineering
NACO controlled Corporate Body
University of North Florida. School of Engineering
First Advisor
O. Patrick Kreidl, Ph.D.
Second Advisor
Ayan Dutta, Ph.D.
Third Advisor
John Nuszkowski, Ph.D.
Fourth Advisor
Alan Harris, Ph.D.
Fifth Advisor
William Klostermeyer, Ph.D.
Department Chair
Alan Harris, Ph.D.
Abstract
Previous literature demonstrates that autonomous UAVs (unmanned aerial vehicles) have the po- tential to be utilized for wildfire surveillance. This advanced technology empowers firefighters by providing them with critical information, thereby facilitating more informed decision-making processes. This thesis applies deep Q-learning techniques to the problem of control policy design under the objective that the UAVs collectively identify the maximum number of locations that are under fire, assuming the UAVs can share their observations. The prohibitively large state space underlying the control policy motivates a neural network approximation, but prior work used only convolutional layers to extract spatial fire information from the current observations. This work utilizes a network that also incorporates a recurrent module to capture temporal information from the history of observations. Experiments involving two simulated fixed-wing aircraft demonstrate that the proposed technique uses approximately 20 times less memory than the approach of prior work, while achieving comparable results in locating the fires. A secondary contribution of the experiment is the inclusion of a more realistic physics-based wildfire propagation model, in con- trast to the simplified probabilistic fire propagation models assumed in prior work. This work relies on modeling fire propagation and fuel reaction rates as a pair of partial differential equations with respect to time. A wildfire simulation with a stronger temporal dependency emphasized the importance of using a neural network architecture that can effectively integrate information over time.
Suggested Citation
Cantor, Jeremy A., "A memory efficient deep recurrent Q-learning approach for autonomous wildfire surveillance" (2024). UNF Graduate Theses and Dissertations. 1251.
https://digitalcommons.unf.edu/etd/1251
Included in
Artificial Intelligence and Robotics Commons, Other Computer Engineering Commons, Robotics Commons