倒立摆毕业设计外文翻译MultiAgent旋翼试验台控制系统设计.doc

Multi-Agent Quadrotor Testbed Control Design: Integral Sliding Mode vs. Reinforcement LearningSteven L. Waslander, Gabriel M. HoffmannPh.D. Candidate Aeronautics and Astronautics Stanford Universitystevenw, gabehstanford.eduJung Soon Jang Research Associate Aeronautics and Astronautics Stanford University jsjangstanford.eduClaire J. Tomlin Associate Professor Aeronautics and Astronautics Stanford University tomlinstanford.eduAbstractThe Stanford Testbed of Autonomous Rotorcraft for Multi-Agent Control (STARMAC) is a multi-vehicle testbed currently comprised of two quadrotors, also called X4-yers, with capacity for eight. This paper presents a comparison of control design techniques, specically for outdoor altitude control, in and above ground effect, that accommodate the unique dynamics of the aircraft. Due to the complex airow in- duced by the four interacting rotors, classical linear techniques failed to provide sufcient stability. Integral Sliding Mode and Reinforcement Learning control are presented as two design techniques for accommodating the nonlinear disturbances. The methods both result in greatly improved performance over classical control techniques.I. INTRODUCTION As rst introduced by the authors in 1,the Stanford Testbed of Autonomous Rotorcraft for Multi-Agent Control(STARMAC) is an aerial platform intended to validate novel multi-vehicle control techniques and present real-world problems for further investigation.The base vehicle for STARMAC is a four rotor aircraft with xed pitch blades, referred to as a quadrotor, or an X4-yer.They are capable of 15 minute outdoor ights in a 100m square area1.Fig. 1. One of the STARMAC quadrotors in action.There have been numerous projects involving quadrotors to date,with the rst known hover occurring in October,19222. Recent interest in the quadrotor concept has been sparked by commercial remote control versions, such as the DraganFlyer IV3. Many groups 47have seen significant success in developing autonomous quadrotor vehicles. To date,however,STARMAC is the only operational multi-vehicle quadrotor platform capable of autonomous outdoor ight, without tethers or motion guides.The rst major milestone for STARMAC was autonomous hover control,with closed loop control of attitude, altitude and position. Using inertial sensing, the attitude of the aircraft is simple to control, by applying small variations in the relative speeds of the blades. In fact, standard integral LQR techniques were applied to provide reliable attitude stability and tracking for the vehicle.Position control was also achieved with an integral LQR, with careful design in order to ensure spectral separation of the successive loops.Unfortunately, altitude control proves less straightforward. There are many factors that affect the altitude loop specifically that do not amend themselves to classical control techniques. Foremost is the highly nonlinear and destabilizing effect of four rotor downwashes interacting. In our experience, this effect becomes critical when motion is not damped by motion guides or tethers. Empirical observation during manual ight revealed a noticeable loss in thrust upon descent through the highly turbulent ow eld.Similar aerodynamic phenomenon for helicopters have been studied extensively8, but not for the quadrotor, due to its relative obscurity and complexity. Other factors that introduce disturbances into the altitude control loop include blade ex, ground effect and battery discharge dynamics. Although these effects are also present in generating attitude controlling moments, the differential nature of the control input eliminates much of the absolute thrust disturbances that complicate altitude control. Additional complications arise from the limited choice in low cost, high resolution altitude sensors. An ultrasonic ranging device9 was used, which suffers from non-Gaussian noise-false echoes and dropouts. The resulting raw data stream includes spikes and echoes that are difcult to mitigate, and most successfully handled by rejection of infeasible measurements prior to Kalman ltering.In order to accommodate this combination of noise and disturbances, two distinct approaches are adopted. Integral Sliding Mode (ISM) control1012 takes the approach that the disturbances cannot be modeled, and instead designsa control law that is guaranteed to be robust to disturbances as long as they do not exceed a certain magnitude. Model-based reinforcement learning13 creates a dynamic model based on recorded inputs and responses, without any knowledge of the underlying dynamics, and then seeks an optimal control law using an optimization technique based on the learned model. This paper presents an exposition of both methods and contrasts the techniques from both a design and implementation point of view.II. SYSTEM DESCRIPTION STARMAC consists of a eet of quadrotors and a ground station. The system communicates over a Bluetooth Class 1 network. The core of the aircraft are microcontroller circuit boards designed and assembled at Stanford, for this project. The microcontrollers run real-time control code, interface with sensors and the ground station, and supervise the system. The aircraft are capable of sensing position, attitude, and proximity to the ground. The differential GPS receiver is theTrimble Lassen LP, operating on the L1 band, providing 1Hz updates. The IMU is the MicroStrain 3DM-G, a low cost, light weight IMU that delivers 76 Hz attitude, attitude rate, and acceleration readings. The distance from the ground is found using ultrasonic ranging at 12 Hz.The ground station consists of a laptop computer, to interface with the aircraft, and a GPS receiver, to provide differential corrections. It also has a battery charger, and joysticks for control-augmented manual ight, when desired.III. QUADROTOR DYNAMICSThe derivation of the nonlinear dynamics is performed in North-East-Down (NED) inertial and body xed coordinates. Let eN , eE , eD denote the inertial axes, and xB , yB , zB denote the body axes, as dened in Figure 2. Euler angles of the body axes are , , with respect to the eN , eE and eD axes, respectively, and are referred to as roll, pitch andyaw. Let r be dened as the position vector from the inertial origin to the vehicle center of gravity (CG), and let B be dened as the angular velocity in the body frame. The current velocity direction is referred to as ev in inertial coordinates.Fig.2. Free body diagram of a quadrotor aircraft.The rotors, numbered 14, are mounted outboard on the xB,yB,xB and -yB axes,respectively, with position vectors ri with respect to the CG. Each rotor produces an aerodynamic torque, Qi , and thrust, Ti , both parallel to the rotors axis of rotation, and both used for vehicle control.Here, , where ui is the voltage applied to the motors, as determined from a load cell test. In ight, Ti can vary greatly from this approximation. The torques, Qi , are proportional to the rotor thrust, and are given by Qi = kr*Ti . Rotors 1 and 3 rotate in the opposite direction as rotors 2 and 4, so that counteracting aerodynamic torques can be used independently for yaw control. Horizontal velocity results in a moment on the rotors, Ri , about ev , and a drag force, Di , in the direction, ev.The body drag force is dened as DB , vehicle mass is m, acceleration due to gravity is g, and the inertia matrix is I R3×3 . A free body diagram is depicted in Figure 2. The total force, F, and moment, M, can be summed as,（1）(2)The full nonlinear dynamics can be described as,（3）where the total angular momentum of the rotors is assumed to be near zero, because they are counter-rotating. Near hover conditions, the contributions by rolling moment and drag can be neglected in Equations (1) and (2). Dene the total thrust as The translational motion is dened by,（4）Where R,R, and R are the rotation matrices for roll, pitch, and yaw, respectively. Applying the small angle approximation to the rotation matrices,（5）Finally, assuming total thrust approximately counteracts gravity, except in the eD axis.（6）For small angular velocities, the Euler angle accelerations are determined from Equation (3) by dropping the second order term,×I, and expanding the thrust into its four constituents. The angular equations become,（7）Where the moment arm lengthl=|ri×zB|is identical for all rotors due to symmetry. The resulting linear models can now be used for control design. IV. ESTIMATION AND CONTROL DESIGNApplying the concept of spectral separation, inner loop control of attitude and altitude is performed by commanding motor voltages, and outer loop position control is performed by commanding attitude requests for the inner loop. Accurate attitude control of the plant in Equation (7) is achieved with an integral LQR controller design to account for thrust biases. Position estimation is performed using a navigation lter that combines horizontal position and velocity information from GPS, vertical position and estimated velocity information from the ultrasonic ranger, and acceleration and angular rates from the IMU in a Kalman lter that includes bias estimates. Integral LQR techniques are applied to the horizontal components of the linear position plant described in Equation (6). The resulting hover performance is shown in Figure 6. As described above, altitude control suffers exceedingly from unmodeled dynamics. In fact, manual command of the throttle for altitude control remains a challenge for the authors to this day. Additional complications arise from the ultrasonic ranging sensor, which has frequent erroneous readings, as seen in Figure 3. To alleviate the effect of this noise, rejection of infeasible measurements is used to remove much of the non-Gaussian noise component. This is followed by altitude and altitude rate estimation by Kalman ltering, which adds lag to the estimate. This section proceeds with a derivation of two control techniques that can be used to overcome the unmodeled dynamics and the remaining noise.Fig. 3. Characteristic unprocessed ultrasonic ranging data, displaying spikes, false echoes and dropouts. Powered ight commences at 185 seconds.A. Integral Sliding Mode ControlA linear approximation to the altitude error dynamics of a quadrotor aircraft in hover is given by,（8）wherex1, x2=(rz,desrz),( rz,desrz)are the altitude error states,ui is the control input, and(·) is a bounded model of disturbances and dynamic uncertainty. It is assumed that (·) satises | where is the upper bounded norm of (·). In early attempts to stabilize this system, it was observed that LQR control was not able to address the instability and performance degradation due to (g, x). Sliding Mode Control (SMC) was adapted to provide a systematic approach to the problem of maintaining stability and consistent performance in the face of modeling imprecision and disturbances. However, until the system dynamics reach the sliding mani-fold, such nice properties of SMC are not assured. In order to provide robust control throughout the ight envelope, the Integral Sliding Mode (ISM) technique is applied. The ISM control is designed in two parts. First, a standard successive loop closure is applied to the linear plant. Second, integral sliding mode techniques are applied to guarantee disturbance rejection. Let（9）Where Kp and Kd are proportional and derivative loop gains that stabilize the linear dynamics without disturbances. For disturbance rejection, a sliding surface,s, is designed,（10）such that state trajectories are forced towards the manifold s= 0. Here,s0 is a conventional sliding mode design, Z is an additional term that enables integral control to be included, and , kR are positive constants. Based on the following Lyapunov function candidate, , the control component,ud, can be determined such that V <0, guranteeing convergence to the sliding manifold.（11）The above condition holds if z = (up+kx2) and ud can be guaranteed to satisfy,（12）Since the disturbances,(g, x), are bounded by , dene ud to be ud=s with R. Equation (11) becomes,（13）and it can be seen that |s| >0. As a result, for up and ud as above, the sliding mode condition holds when,（14）With the input derived above, the dynamics are guaranteed to evolve such that s decays to within the boundary layer, of the sliding manifold. Additionally, the system does not suffer from input chatter as conventional sliding mode controllers do, as the control law does not include a switching function along the sliding mode.V. REINFORCEMENT LEARNING CONTROLAn alternate approach is to implement a reinforcement learning controller. Much work has been done on continuous state-action space reinforcement learning methods13, 14. For this work, a nonlinear,nonparametric model of the system is rst constructed using ight data, approximating the system as a stochastic Markov process15, 16. Then a model-based reinforcement learning algorithm uses the model in policy-iteration to search for an optimal control policy that can be implemented on the embedded microprocessors. In order to model the aircraft dynamics as a stochastic Markov process, a Locally Weighted Linear Regression (LWLR) approach is used to map the current state,S(t)Rns, and input,u(t)Rnu, onto the subsequent state estimate,S(t+ 1).In this application,where V is the battery level. In the altitude loop, the input,uR, is the total motor power,u. The subsequent state mapping is the summation of the traditional LWLR estimate, using the current state and input, with the random vector,vRns, representing unmodeled noise. The value for v is drawn from the distribution of output error as determined by using a maximum likelihood estimate16 of the Gaussian noise in the LWLR estimate. Although the true distribution is not perfectly Gaussian, this model is found to be adequate. The LWLR method17 is well suited to this problem, as it ts a non-parametric curve to the local structure of the data. The scheme extends least squares by assigning weights to each training data point according to its proximity to the input value, for which the output is to be computed. The technique requires a sizable set of training data in order to reect the full dynamics of the system, which is captured from ights own under both automatic and manually controlled thrust, with the attitude states under automatic control. For m training data points, the input training samples are stored in XR(m)×(ns+nu+1), and the outputs corresponding to those inputs are stored inYRm×ns. These matrices are dened as,（15）The column of ones in X enables the inclusion of a constant offset in the solution, as used in li