DESIGN OF AN ONLINE ADAPTIVE CONTROLLER FOR ACTIVE DISTURBANCE REJECTION IN A FIXED WING UAV USING REINFORCEMENT LEARNING AND DIFFERENTIAL GAMES

Abstract

The challenge of coping with highly nonlinear and rapidly time-varying dynamics is a prevailing factor when designing controllers for next-generation Unmanned Aerial Vehicles (UAVs). Stochastic disturbances such as wind gusts and atmo- spheric turbulence form the biggest challenge which the control structure must be capable of minimizing, while handling the errors in \dynamic modelling". This work presents an online adaptive controller for active disturbance rejection in a xed wing UAV using reinforcement learning. The approach is based on modelling the UAV system dynamics as a two player zero-sum di erential game against nature, which will represent external disturbances a ecting the UAV such as wind. Then an online learning algorithm using reinforcement learning is devel- oped to solve the continuous-time two-player di erential game. Game theoretic methods together with SARSA, a reinforcement learning technique were used to calculate a cost function for each state action pair. A one step gradient search of the cost function was done which was implemented as the reinforcement signal in the di erential game. Back propagation technique was used to update the feed forward neural network weights in real time to compensate for the tracking errors in the heading from a commanded reference for each time step. The simulation tests carried out under various disturbances in MATLAB and X- Plane, showed satisfactory performance of the proposed method to eliminate the disturbances and maintain the UAV in the desired target path. The responses got were compared with responses from of a well tuned PID controller both in MATLAB and X-Plane platforms. The adaptive method responses were better.