TY - JOUR
T1 - Reinforcement learning for control
T2 - Performance, stability, and deep approximators
AU - Buşoniu, Lucian
AU - de Bruin, Tim
AU - Tolić, Domagoj
AU - Kober, Jens
AU - Palunko, Ivana
N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
PY - 2018
Y1 - 2018
N2 - Reinforcement learning (RL) offers powerful algorithms to search for optimal controllers of systems with nonlinear, possibly stochastic dynamics that are unknown or highly uncertain. This review mainly covers artificial-intelligence approaches to RL, from the viewpoint of the control engineer. We explain how approximate representations of the solution make RL feasible for problems with continuous states and control actions. Stability is a central concern in control, and we argue that while the control-theoretic RL subfield called adaptive dynamic programming is dedicated to it, stability of RL largely remains an open question. We also cover in detail the case where deep neural networks are used for approximation, leading to the field of deep RL, which has shown great success in recent years. With the control practitioner in mind, we outline opportunities and pitfalls of deep RL; and we close the survey with an outlook that – among other things – points out some avenues for bridging the gap between control and artificial-intelligence RL techniques.
AB - Reinforcement learning (RL) offers powerful algorithms to search for optimal controllers of systems with nonlinear, possibly stochastic dynamics that are unknown or highly uncertain. This review mainly covers artificial-intelligence approaches to RL, from the viewpoint of the control engineer. We explain how approximate representations of the solution make RL feasible for problems with continuous states and control actions. Stability is a central concern in control, and we argue that while the control-theoretic RL subfield called adaptive dynamic programming is dedicated to it, stability of RL largely remains an open question. We also cover in detail the case where deep neural networks are used for approximation, leading to the field of deep RL, which has shown great success in recent years. With the control practitioner in mind, we outline opportunities and pitfalls of deep RL; and we close the survey with an outlook that – among other things – points out some avenues for bridging the gap between control and artificial-intelligence RL techniques.
KW - Adaptive dynamic programming
KW - Deep learning
KW - Function approximation
KW - Optimal control
KW - Reinforcement learning
KW - Stability
UR - http://www.scopus.com/inward/record.url?scp=85055043462&partnerID=8YFLogxK
U2 - 10.1016/j.arcontrol.2018.09.005
DO - 10.1016/j.arcontrol.2018.09.005
M3 - Review article
AN - SCOPUS:85055043462
SN - 1367-5788
VL - 46
SP - 8
EP - 28
JO - Annual Reviews in Control
JF - Annual Reviews in Control
ER -