Evolutionary Intelligence
pp 1–12 | Cite as
A staged approach to evolving real-world UAV controllers
Gerard David Howard
Alberto Elfes
A testbed has recently been introduced that evolves controllers for arbitrary hover-capable UAVs, with evaluations occurring directly on the robot. To prepare the testbed for real-world deployment, we investigate the effects of state-space limitations brought about by physical tethering (which prevents damage to the UAV during stochastic tuning), on the generality of the evolved controllers. We identify generalisation issues in some controllers, and propose an improved method that comprises two stages: in the first stage, controllers are evolved as normal using standard tethers, but experiments are terminated when the population displays basic flight competency. Optimisation then continues on a much less restrictive tether, effectively free-flying, and is allowed to explore a larger state-space envelope. We compare the two methods on a hover task using a real UAV, and show that more general solutions are generated in fewer generations using the two-stage approach. A secondary experiment undertakes a sensitivity analysis of the evolved controllers.
Differential evolution Evolutionary robotics Evolutionary hardware UAV control
This research received funding from the CSIRO Office of the Chief Executive for the Postdoctoral position in Evolutionary Aerial Robotics.
The author declares that they have no conflict of interest.
Appendix A: Fitness function
$$\begin{aligned}& f_{\mathrm{cycle}}=f_{\mathrm{p}}+f_{\mathrm{h}}+f_\psi + f_{\mathrm{a}}+f_{\mathrm{vh}}+f_{\mathrm{vv}}+f_{{\omega }}+f_{\mathrm{l}} \\&f_{\mathrm{p}}=f\left( \sqrt{(p_{\mathrm{nsp}}-p_{\mathrm{n}})^2+(p_{\mathrm{esp}}-p_{\mathrm{e}})^2},l_{\mathrm{p}},l_{\mathrm{pc}}\right) \\&f_{\mathrm{h}}=f(|h_{\mathrm{sp}}-h|,l_{\mathrm{h}},l_{\mathrm{hc}})\\&f_{{\psi }}=f\left( |{\mathrm{wrap}}(\psi _{\mathrm{sp}}-\psi )|,l_\psi ,l_{\psi {\mathrm{c}}}\right) \\ &f_{\mathrm{a}}=f(|\phi _{\mathrm{sp}}-\phi |,l_{\mathrm{a}},l_{\mathrm{ac}})+f(|\theta _{\mathrm{sp}}-\theta |,l_{\mathrm{a}},l_{\mathrm{ac}})\\&f_{\mathrm{vh}}=f\left( \sqrt{v_{\mathrm{n}}^2+v_{\mathrm{e}}^2},l_{\mathrm{vh}},l_{\mathrm{vhc}}\right) \\&f_{\mathrm{vv}}=f(|v_{\mathrm{v}}|,l_{\mathrm{vv}},l_{\mathrm{vvc}})\\&f_{{\omega }}=f(|p|,l_\omega ,l_{\omega {\mathrm{c}}})+f(|q|,l_\omega ,l_{\omega {\mathrm{c}}})\\&f_{\mathrm{l}}=1-\frac{{\text{number of controllers exceeding }}l_\delta \text{ limits}}{4} \\&f(e,l,l_{\mathrm{c}})= {\left\{ \begin{array}{ll} {\mathrm{max}}\{\frac{l-e}{4(l-l_{\mathrm{c}})},0\}, & \text{if } e>l_{\mathrm{c}} \\ \frac{3(l_{\mathrm{c}}-e)}{4 l_{\mathrm{c}}}+\frac{1}{4}, & \text{otherwise} \end{array}\right. } \\&{\mathrm{wrap}}(\gamma )={\mathrm{atan2}}(\sin (\gamma ),\cos (\gamma )) \end{aligned}$$
Symbol definitions: fitness function
$$\begin{aligned}& f_{\mathrm{cycle:}} \text{fitness for one control cycle}\\&f_{\mathrm{p}}/f_{\mathrm{h}}: \text{fitness for position/height tracking}\\&f_\psi : \text{fitness for yaw angle tracking}\\&f_{\mathrm{a}}: \text{fitness for pitch and roll angle tracking}\\&f_{\mathrm{vh}}/f_{\mathrm{vv}}: \text{fitness for low horizontal/vertical velocity}\\&f_{{\omega }}: \text{fitness for low pitch and roll rates} \\&f_{\mathrm{l}}: \text{fitness for staying within controller limits}\\&p_{\mathrm{n}}/p_{\mathrm{e}}: \text{north/east position}\\&p_{\mathrm{nsp}}/p_{\mathrm{esp}}: \text{north/east position setpoint}\\&h: \text{height above ground}\\&h_{\mathrm{sp}}: \text{height setpoint}\\&\phi /\theta /\psi : \text{roll/pitch/yaw angle}\\&\phi _{\mathrm{sp}}/\theta _{\mathrm{sp}}/\psi _{\mathrm{sp}}: \text{roll/pitch/yaw setpoint}\\&v_{\mathrm{n}}/v_{\mathrm{e}}/v_{\mathrm{v}}: \text{north/east/vertical velocity}\\&p/q/r: \text{roll/pitch/yaw rate}\\&l_{\mathrm{p}}: \text{position error range (0.2 m)}\\&l_{\mathrm{pc}}: \text{core position error range (0.07 m)}\\&l_{\mathrm{h}}: \text{height error range (0.2 m)}\\&l_{\mathrm{hc}}: \text{core height error range (0.03 m)}\\&l_{\psi }: \text{yaw error range}\, (30^{\circ})\\&l_{\psi c}: \text{core yaw error range}\, (10^{\circ})\\&l_{\mathrm{a}}: \text{attitude angle error range}\, (15^{\circ})\\&l_{\mathrm{ac}}: \text{core attitude angle error range}\, (3^{\circ})\\&l_{\mathrm{vh}}: \text{horizontal velocity error range (1 m/s)}\\&l_{\mathrm{vhc}}: \text{core horizontal velocity error range (0.2 m/s)}\\&l_{\mathrm{vv}}: \text{vertical velocity error range (1 m/s)}\\&l_{\mathrm{vvc}}: \text{core vertical velocity error range (0.1 m/s)}\\&l_{{\omega }}: \text{pitch and roll rate error range}\, (250^{\circ}/\text{s})\\&l_{\omega \mathrm{c}}: \text{core pitch and roll rate error range}\, (30^{\circ}/\text{s)}\\ \end{aligned}$$
Acar E, Zhang Y, Choset H, Schervish M, Costa AG, Melamud R, Lean D, Graveline A (2001) Path planning for robotic demining and development of a test platform. Int Conferen Field Serv Robot 1:161–168Google Scholar
Biswas A, Das S, Abraham A, Dasgupta S (2009) Design of fractional-order pi \(\lambda \) d \(\mu \) controllers with an improved differential evolution. Eng Appl Artif Intell 22(2):343–350CrossRefGoogle Scholar
Chiha I, Ghabi J, Liouane N (2012) Tuning pid controller with multi-objective differential evolution. In: 2012 5th international symposium on communications control and signal processing (ISCCSP), pp 1–4Google Scholar
De Nardi R, Togelius J, Holland O, Lucas S (2006) Evolution of neural networks for helicopter control: why modularity matters. In: IEEE congress on evolutionary computation, 2006—CEC 2006, pp 1799–1806Google Scholar
Degrave J, Burm M, Kindermans PJ, Dambre J, wyffels F (2015) Transfer learning of gaits on a quadrupedal robot. Adapt Behav 23(2):69–82CrossRefGoogle Scholar
Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Springer, BerlinCrossRefzbMATHGoogle Scholar
Faina AJ, Toft L, Risi S (2017) Automating the incremental evolution of controllers for physical robots. Artif Life 23(2):142–168CrossRefGoogle Scholar
Floreano D, Zufferey JC, Nicoud JD (2005) From wheels to wings with evolutionary spiking circuits. Artif life 11(1-2):121–138CrossRefGoogle Scholar
Ghiglino P, Forshaw JL, Lappas VJ (2015) Online evolutionary swarm algorithm for self-tuning unmanned flight control laws. J. Guid. Control Dyn. 38(4):772–782CrossRefGoogle Scholar
Gongora M, Passow B, Hopgood A (2009) Robustness analysis of evolutionary controller tuning using real systems. In: IEEE congress on evolutionary computation, 2009. CEC '09, pp 606–613Google Scholar
Harvey I, Husbands P, Cliff D (1994) Seeing the light: artificial evolution, real vision. School of Cognitive and Computing Sciences, University of Sussex FalmerGoogle Scholar
Heijnen H, Howard D, Kottege N (2017) A testbed that evolved hexapod controllers in hardware. In: 2017 IEEE/RSJ international conference on robotics and automation (ICRA), IEEE (in press)Google Scholar
Holland OE, Nardi RD (2008) Coevolutionary modelling of a miniature rotorcraft. In: Burgard W, Dillmann R, Plagemann C, Vahrenkamp N (eds) Intelligent autonomous systems 10 (IAS10). Univ Freiburg, Autonomous Intelligent Syst Lab, Baden Baden, Germany (loc)Google Scholar
How JP, BEHIHKE B, Frank A, Dale D, Vian J (2008) Real-time indoor autonomous vehicle test environment. IEEE Control Syst 28(2):51–64MathSciNetCrossRefzbMATHGoogle Scholar
Howard D (2017) A platform that directly evolves multirotor controllers. IEEE Trans Evolut Comput 21(6):943–955CrossRefGoogle Scholar
Howard D, Elfes A (2014) Evolving spiking networks for turbulence-tolerant quadrotor control. In: International conference on artificial life (ALIFE14), pp 431–438Google Scholar
Howard D, Merz T (2015) A platform for the direct hardware evolution of quadcopter controllers. In: 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 4614–4619Google Scholar
Howard GD (2017) On self-adaptive mutation restarts for evolutionary robotics with real rotorcraft. In: Proceedings of the 17th annual conference on genetic and evolutionary computation. ACM (in press)Google Scholar
Jakobi N, Husbands P, Harvey I (1995) Noise and the reality gap: The use of simulation in evolutionary robotics. In: Morán F, Moreno A, Merelo JJ, Chacón P (eds) Advances in artificial life. ECAL 1995. Lecture notes in computer science (Lecture notes in artificial intelligence), vol 929. Springer, Berlin, HeidelbergGoogle Scholar
Johnson W (2012) Helicopter theory. Courier Corporation, ChelmsfordGoogle Scholar
Koos S, Mouret JB, Doncieux S (2010) Crossing the reality gap in evolutionary robotics by promoting transferable controllers. In: Proceedings of the 12th annual conference on genetic and evolutionary computation, GECCO '10. ACM, New York, pp 119–126Google Scholar
Koppejan R, Whiteson S (2009) Neuroevolutionary reinforcement learning for generalized helicopter control. In: Proceedings of the 11th annual conference on genetic and evolutionary computation, GECCO '09. ACM, New York, NY, pp 145–152Google Scholar
Merz T, Rudol P, Wzorek M (2006) Control system framework for autonomous robots based on extended state machines. In: 2006 International conference on autonomic and autonomous systems, 2006. ICAS'06. IEEE, pp 14–14Google Scholar
Moravec J, Pošík P (2014) A comparative study: the effect of the perturbation vector type in the differential evolution algorithm on the accuracy of robot pose and heading estimation. Evolut. Intell. 6(3):171–191. https://doi.org/10.1007/s12065-013-0090-2 CrossRefGoogle Scholar
Nishiwaki K, Sugihara T, Kagami S, Kanehiro F, Inaba M, Inoue H (2000) Design and development of research platform for perception-action integration in humanoid robot: H6. In: Proceedings of 2000 IEEE/RSJ international conference on intelligent robots and systems 2000, (IROS 2000). IEEE, vol 3, pp 1559–1564Google Scholar
Powers C, Mellinger D, Kushleyev A, Kothmann B, Kumar V (2013) Influence of aerodynamics and proximity effects in quadrotor flight. In: Desai J, Dudek G, Khatib O, Kumar V (eds) Experimental robotics. Springer tracts in advanced robotics, vol 88. Springer, HeidelbergGoogle Scholar
Rechenberg I (1973) Evolutionsstrategie: optimierung technischer systeme nach prinzipien der biologischen evolution. Frommann-Holzboog, StuttgartGoogle Scholar
Rossi C, Eiben AE (2014) Simultaneous versus incremental learning of multiple skills by modular robots. Evolut. Intell. 7(2):119–131. https://doi.org/10.1007/s12065-014-0109-3 CrossRefGoogle Scholar
Samuele R, Varshneya R, Johnson T, Johnson A, Glassman T (2010) Progress at the starshade testbed at northrop grumman aerospace systems: comparisons with computer simulations. In: Proceedings of SPIE, vol 7731, p 773151Google Scholar
Scheper KYW, Tijmons S, de Visser CC, de Croon GCHE (2016) Behavior trees for evolutionary robotics. Artif Life 22(1):23–48CrossRefGoogle Scholar
Storn R, Price K (1997) Differential evolution—a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359MathSciNetCrossRefzbMATHGoogle Scholar
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, CambridgezbMATHGoogle Scholar
Yosinski J, Clune J, Hidalgo D, Nguyen S, Zagal J, Lipson H (2011) Evolving robot gaits in hardware: the hyperneat generative encoding vs. parameter optimization. In: Proceedings of the 20th European conference on artificial life, pp 890–897Google Scholar
© Springer-Verlag GmbH Germany, part of Springer Nature 2019
1.QCATPullenvaleAustralia
Howard, G.D. & Elfes, A. Evol. Intel. (2019). https://doi.org/10.1007/s12065-019-00242-5
Received 04 September 2018
Revised 09 May 2019
Accepted 12 May 2019