  * [ Home ](/documentation/repository/crazyflie-lib-python/master/ "Home")
  * [ installation ](/documentation/repository/crazyflie-lib-python/master/installation/ "installation")
    * [ Installation ](/documentation/repository/crazyflie-lib-python/master/installation/install/ "Installation")
    * [ USB permissions ](/documentation/repository/crazyflie-lib-python/master/installation/usb_permissions/ "USB permissions")
  * [ User guides ](/documentation/repository/crazyflie-lib-python/master/user-guides/ "User guides")
    * [ Step-by-Step: Connecting, logging and parameters ](/documentation/repository/crazyflie-lib-python/master/user-guides/sbs_connect_log_param/ "Step-by-Step: Connecting, logging and parameters")
    * [ Step-by-Step: Motion Commander ](/documentation/repository/crazyflie-lib-python/master/user-guides/sbs_motion_commander/ "Step-by-Step: Motion Commander")
    * [ Step-by-Step: Swarm Interface ](/documentation/repository/crazyflie-lib-python/master/user-guides/sbs_swarm_interface/ "Step-by-Step: Swarm Interface")
    * [ The Crazyflie Python API explanation ](/documentation/repository/crazyflie-lib-python/master/user-guides/python_api/ "The Crazyflie Python API explanation")
  * [ Functional Areas ](/documentation/repository/crazyflie-lib-python/master/functional-areas/ "Functional Areas")
    * [ Python Crazyradio Library ](/documentation/repository/crazyflie-lib-python/master/functional-areas/crazyradio_lib/ "Python Crazyradio Library")
  * [ Development ](/documentation/repository/crazyflie-lib-python/master/development/ "Development")
    * [ Debugging CRTP using Wireshark ](/documentation/repository/crazyflie-lib-python/master/development/wireshark/ "Debugging CRTP using Wireshark")
    * [ Reset EEPROM ](/documentation/repository/crazyflie-lib-python/master/development/eeprom/ "Reset EEPROM")
    * [ UART communication ](/documentation/repository/crazyflie-lib-python/master/development/uart_communication/ "UART communication")
    * [ Using Matlab with the Crazyflie API ](/documentation/repository/crazyflie-lib-python/master/development/matlab/ "Using Matlab with the Crazyflie API")
  * [ Auto-generated API Documentation ](/documentation/repository/crazyflie-lib-python/master/api/ "Auto-generated API Documentation")
    * [ API reference for CFLib ](/documentation/repository/crazyflie-lib-python/master/api/cflib/ "API reference for CFLib")


[ Go to CrazyFlie  ](/r/CrazyFlie/)


The library supports reading and writing parameters at run-time to the
firmware. This is intended to be used for data that is not continuously
changed by the firmware, like setting regulation parameters and reading out if
the power-on self-tests passed. Parameters should only be changed in the
firmware when being set from the host (cfclient or a cflib script) or during
start-up.


5. If yaw orientation tracking is also desired, the angle of the target relative to the forward direction
needs to be distinguished by fitting a rectangle with the minimum area in which the contour of
the target can be enclosed. The respective angle of this rectangle can then be found from the
corners of the rectangle, where this angle will provide the orientation of the target.
Implementing a blur before applying the threshold was initially considered, however preparatory
tests indicated that this was unnecessary as long as a morphology transformation was performed, as
is preferred for better noise reduction - in other words, a blur offered no perceivable advantages and
will only result in greater computational effort. It was also possible to apply an adaptive threshold,
but this is expected to be unnecessary given the stark contrast between the background and target.
5.4 TARGET TRACKING
Since the features of the white disk on the target will be constant for every orientation, the current
altitude of the camera can be estimated based on the area of the target in terms of pixels in the captured
frame, due to the target remaining at a constant altitude on the ground and relative altitude changes
coming from the altitude of the camera. Thus, once the area of the target is known from the target
detection, the desired altitude for the Crazyflie can be related with the found relationship from the
camera calibration in the form of Equation 16 (instead of being used for calibration with a known
altitude, the relationship will be known and the associated altitude will be calculated).
Using the pinhole camera model, the actual position of the target (which is the desired position of the
Crazyflie) can also be estimated based on the location of the centroid of the target in terms of pixels
relative to a chosen coordinate system within the captured frame. Thus, once the centroid of the target
is known from the target detection, the desired position of the Crazyflie can be estimated based on
Equation 21 for the latitude coordinate and Equation 22 for the longitude coordinate (the previously
found altitude and experimentally calibrated focal length will actually be used instead of the ratio of
the areas from the expanded theoretical model). These methods are illustrated in Figure 28.
34
x = u
z
f
= u
k
f
√
Auv
= u
r
Axy
Auv
(21)
y = v
z


        # Create a Crazyflie object without specifying any cache dirs
        self._cf = Crazyflie()


        # Callback called when the connection is established to the Crazyflie
        def connected(link_uri):
            crazyflie.log.add_config(logconf)

            if logconf.valid:
                logconf.data_received_cb.add_callback(data_received_callback)
                logconf.error_cb.add_callback(logging_error)
                logconf.start()
            else:
                print "One or more of the variables in the configuration was not found in log TOC. No logging will be possible."

        def data_received_callback(timestamp, data, logconf):
            print "[%d][%s]: %s" % (timestamp, logconf.name, data)

        def logging_error(logconf, msg):
            print "Error when logging %s" % logconf.name


logconf.add_variable('pm.vbat', 'FP16')


The control setpoints are not implemented as parameters, instead they have a
special API.


Enable here


logconf.add_variable('stateEstimate.x', 'float')


###  Variables and logging




−ψi
if W < H
−ψi − 90o otherwise
(25)
Where ψ, yaw orientation, o
; ψi
, initial yaw orientation, o
; W, width, px; and H, height, px.
90o 90o
0o 90o
0o 90o x
y
0
1
2
3
W
H
The side between 1 and 2
or 0 and 3 is labelled W.
The side between 2 and 3
or 0 and 1 is labelled H.
Orientation
0o 90o
0o 90o x
y
0
1
2
3
W
H
Orientation
The lowest corner is always labelled as 0 with the other
corners labelled in a clockwise pattern. The angle is always
measured at this lowest corner to the first edge (negatively). 0o x
y
0


A0
D2
D3
D4
D7
D8
D9
D12
D13
Digital Input Pins
Reset Button
Left Motor Connection
Battery Input 6V-9V Mini-USB
Power Switch
D5 = Left Speed
D10 = Left Direction
D6 = Right Speed
D11 = Right Direction
A6 = Rear Solder Tab
A7 = Battery Monitor
Dagu S4A EDU Controller Dagu DG02S 48:1
Geared DC Motor
DC Motor
48:1 Gearbox
Shaft
S4A
153mm
148mm
Bottom
Isometric
View Omni
Wheel
65mm
Top Isometric View
Battery Pack
Driven
Wheel
Driven
Wheel Chassis
Motors,


128 cv.drawContours (morphology, contours, -1, (255, 0, 0), 2) # cv.FILLED
129 output = np.zeros ((res_y * 2, res_x * 2, 3), dtype = "uint8")
130 output [0:res_y, 0:res_x] = frame
131 output [0:res_y, res_x:(res_x * 2)] = mode
132 output [res_y:(res_y * 2), 0:res_x] = threshold
133 output [res_y:(res_y * 2), res_x:(res_x * 2)] = morphology
134
135 # Record (optional) and display the combined output.
136 if record == "Y":
137 video.write (output)
138 cv.imshow ("Combined Output", output)
139
140 # Function to print the characteristics of the target centroid.
141 def monitor_text (centroid_x, centroid_y, altitude, orientation):
142
143 # Print the target characteristics to the shell.
144 print ("Centroid X: ", centroid_x)
145 print ("Centroid Y: ", centroid_y)
146 print ("Altitude: ", "{0:.2f}".format (altitude))
147 print ("Orientation: ", "{0:.2f}".format (orientation))
148 print ("")
149
150 # --------------------------------------------------------------------------------
151 # Crazyflie Control Functions
152 # --------------------------------------------------------------------------------
153
154 # Function to update the position estimator after the extend Kalman filter is reset.
155 def wait_for_position_estimator (crazyflie):
156
157 # Print an update to the shell for monitoring.
158 print ("Waiting for estimator to calibrate the current state.")
159
160 # Set up logging and add the variables to log for updating.
161 log_config = LogConfig (name = "Kalman Variance", period_in_ms = 500)
74
162 log_config.add_variable ("kalman.varPX", "float")
163 log_config.add_variable ("kalman.varPY", "float")
164 log_config.add_variable ("kalman.varPZ", "float")
165 var_y_history = [1000] * 10
166 var_x_history = [1000] * 10


Trial 1 574 477 495 551 772 799 824 14.7 2.38
Trial 2 583 473 490 511 767 793 820 17.2 2.79
Circular
Trial 1 947 778 794 814 470 484 497 17.6 2.64
Trial 2 833 780 793 808 471 510 539 4.98 0.75
Combined
Trial 1 663 552 576 593 - - - 14.1 2.22
Trial 2 664 564 574 596 - - - 14.5 2.29
Yaw
Trail 1 467 375 393 404 - - - 17.3 2.90
Trail 2 468 370 393 411 - - - 17.3 2.91
Overall Averages - - - - - - - 14.7 2.36
7 DISCUSSION
Using inexpensive and low-end hardware in the form of a Raspberry Pi and Pi Camera for target
detection with computer vision techniques in OpenCV, it was possible to implement position-based
visual servoing through end-point open-loop control, where a Crazyflie effectively tracked the relative
position of a target in a mirrored fashion. Specifically, as is required for the aim of the research,
the Raspberry Pi is definitely considered to have marginal computational effort capabilities, since
it only utilises a basic 1.0GHz single-core processor and 512MB of RAM, as compared to modern
hardware where common consumer devices have multiple threads and cores with processing speeds
over 1.8GHz and access to over 2GB of RAM - this is further highlighted by the cost of a Raspberry
Pi which is only around R200 [29]. The results are discussed for direct relation to the objectives.
7.1 CAMERA AND PROCESSING FACTORS
Examining the first significant finding in Table 2, the image processing tests at different resolutions are
clear and, in a sense, restrictive when considering the apparatus and computationally limited hardware
55
in general. As a consequence, it would be absolutely infeasible to use computer vision techniques to
control the Crazyflie in real-time with the default resolution of 640px by 480px in OpenCV, since a
setpoint is only generated every 263ms (frequency of only 3.80Hz) on average with minimum grayscale processing; and, with a resolution of 320px by 240px, a setpoint can be generated every 118ms
(frequency of 8.47Hz) on average with minimum grayscale processing, but this may be impractical as
it is below the recommendation for setpoints to be generated at 10Hz as a minimum limit. By reducing the resolution to 160px by 120px, it was possible to achieve a generation of setpoints every 33ms
(frequency of 30.1Hz) on average with minimum grayscale processing, while colour processing and
maximum processing were also satisfactory if desired. The relationship between the resolution and
time for each setpoint to be generated appears to be non-linear which is expected since the number
of pixels in the image is decreasing by a factor of four as the resolution is halved in each dimension, but additional in-between resolutions should be investigated to find a reliable relationship, if
this relationship is desired to be found (which was unnecessary for the aim of this research).
To compare the grayscale and colour processing, it is evident that the minimum grayscale processing
obtains a higher frame rate over the minimum colour processing with actual percentage differences
of 19.3%, 24.7%, and 25.6% (magnitude percentage differences of 2.93%, 6.17%, and 10.6%) for
resolutions of 160px by 120px, 320px by 240px, and 640px by 480px respectively. Similar results


##  cfclient


To register for callbacks the following is used:


  * Register parameter updated callbacks at any time in your application
  * Connect to your Crazyflie (this will download the parameter TOC)
  * Request updates for all the parameters
  * The library will call all the callbacks registered
  * The host can now write parameters that will be forwarded to the firmware
  * For each write all the callbacks registered for this parameter will be called back


    * [ Old products ](/products/old-products/ "Old products")
  * Buy
    * [ Buy online __ ](https://store.bitcraze.io/ "Buy online")
    * [ Local retailers ](/buy/local-retailers/ "Local retailers")
    * [ Buyers guide ](/buy/buyers-guide/ "Buyers guide")
  * Support
    * [ Getting help ](/support/getting-help/ "Getting help")
    * [ Discussions __ ](https://discussions.bitcraze.io "Discussions")
    * [ Downloads & Install ](/support/downloads/ "Downloads & Install")
    * [ FAQ ](/support/f-a-q/ "FAQ")
    * [ Troubleshooting ](/support/troubleshooting/ "Troubleshooting")
  * Documentation
    * [ Start here ](/documentation/start/ "Start here")
    * [ Tutorials ](/documentation/tutorials/ "Tutorials")
    * [ System overview ](/documentation/system/ "System overview")
    * [ Repository overview ](/documentation/repository/ "Repository overview")
    * [ Developer meetings ](/documentation/meetings/ "Developer meetings")
    * [ Video gallery ](/documentation/video-gallery/ "Video gallery")
  * Development
    * [ Development overview ](/development/development-overview/ "Development overview")
    * [ Contribute ](/development/contribute/ "Contribute")
    * [ GitHub __ ](https://github.com/bitcraze "GitHub")
    * [ Discussions __ ](https://discussions.bitcraze.io "Discussions")
    * [ Early access ](/development/early-access/ "Early access")
    * [ External projects ](/development/external-projects/ "External projects")
  * About
    * [ Bitcraze ](/about/bitcraze/ "Bitcraze")
    * [ Team ](/about/team/ "Team")
    * [ Press ](/about/press/ "Press")
    * [ Product cycle ](/about/product-cycle/ "Product cycle")
    * [ Thesis ](/about/thesis/ "Thesis")
    * [ Jobs ](/about/jobs/ "Jobs")
    * [ Contact ](/about/contact/ "Contact")
    * [ Feedback ](/about/feedback/ "Feedback")
    * [ Events ](/about/events/ "Events")
    * [ Credits ](/about/credits/ "Credits")


6.2 Image Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.3 Camera Calibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6.4 Quadrotor Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.5 Visual Servoing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.5.1 Primitive Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
6.5.2 Results Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
7.1 Camera And Processing Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
7.2 Visual Servoing Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
7.3 Other Findings And Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
iv
8 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
9 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
A Additional Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
B Control Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
C Ethics Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
D Risk Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
E BigQuad Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
E.1 End-Point Closed-Loop Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
E.2 BigQuad Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
v
LIST OF FIGURES
1 Examples of the common multirotor layouts with a general layout of a front view . . . . 1
2 Examples of end-point closed-loop control and end-point open-loop control . . . . . . . 2
3 Comparison between the methods of position-based and image-based visual servoing . . 4
4 Coordinate system relative to the quadrotor showing translation and rotation . . . . . . . 5
5 Plus-mode (+) rotor arrangement to enable translation and rotation . . . . . . . . . . . . 6
6 Cross-mode (×) rotor arrangement to enable translation and rotation . . . . . . . . . . . 7
7 Conversion between two reference frames with translation and rotation . . . . . . . . . . 8
8 Rasterisation of a real object with high and low resolutions . . . . . . . . . . . . . . . . 9
9 Simplification of a model for a pinhole camera with a lens and imaging sensor . . . . . . 9
10 Geometric relationships resulting from the pinhole camera model . . . . . . . . . . . . . 10
11 Fish-eye distortion model in a field of view and example of fish-eye distortion . . . . . . 11
12 Example of an image from a camera in original colour and with processing . . . . . . . 13
13 Arrangement for end-point open-loop control through target detection and tracking . . . 15
14 General schematic of an open-loop control system without feedback . . . . . . . . . . . 16
15 General schematic of a closed-loop control system with feedback . . . . . . . . . . . . . 16
16 Outline and interactions of the apparatus from a broad overview. . . . . . . . . . . . . . 18
17 Photographs showing the assembled Bitcraze Crazyflie 2.1 quadrotor . . . . . . . . . . . 19


M4
Figure 6: Cross-mode (×) rotor arrangement to enable the translational and rotational displacements.
For further understanding, a simplified analytical model of a quadrotor is given by Equation 1, which
describes the resultant thrust acting on the quadrotor from the thrust of each rotor, and Equation 2 to
Equation 4 for plus-mode or Equation 5 to Equation 7 for cross-mode, which respectively describe the
resultant torque acting on the quadrotor while rolling, pitching, and yawing. These relationships are
fairly basic and present an ideal model, which can be implemented using the inherent relationships
between thrust, torque, and supplied voltage for each rotor determined experimentally [11].
F = FM1 + FM2 + FM3 + FM4 −→ F − mg = mz¨ (1)
Tφ,+ = d+(FM4 − FM2) = Ixφ¨ (2)
Tθ,+ = d+(FM3 − FM1) = Iyθ¨ (3)
Tψ,+ = TM1 + TM3 − TM2 − TM4 = Izψ¨ (4)
Tφ,× = d×((FM3 + FM4) − (FM1 + FM2)) = Ixφ¨ (5)
Tθ,× = d×((FM2 + FM3) − (FM1 + FM4)) = Iyθ¨ (6)
Tψ,× = TM1 + TM3 − TM2 − TM4 = Izψ¨ (7)
Where F, resultant thrust, N; FM1, motor one thrust, N; FM2, motor two thrust, N; FM3, motor
three thrust, N; FM4, motor four thrust, N; m, mass, kg; g, gravitational acceleration, 9.807m/s2
; z¨,
acceleration along z-axis, m/s2
; Tφ,+, plus-mode roll torque, N.m; d+, plus-mode moment arm, m;
Ix, mass moment of inertia about x-axis, kg.m2
; φ¨, roll angular acceleration, rad/s2
; Tθ,+, plus-mode
pitch torque, N.m; Iy, mass moment of inertia about y-axis, kg.m2
; θ¨, pitch angular acceleration,
7
rad/s2
; Tψ,+, plus-mode yaw torque, N.m; TM1, motor one torque, N.m; TM2, motor two torque, N.m;
TM3, motor three torque, N.m; TM4, motor four torque, N.m; Iz
, mass moment of inertia about zaxis, kg.m2
; ψ¨, yaw angular acceleration, rad/s2
; Tφ,×, cross-mode roll torque, N.m; d×, cross-mode
moment arm, m; Tθ,×, cross-mode pitch torque, N.m; and Tψ,×, cross-mode yaw torque, N.m.
When considering the representation of a quadrotor or another body with six degrees of freedom in
three-dimensional space, it is possible to use a reference frame with a coordinate system either fixed
to an observer or fixed to the quadrotor [2, 6, 11]. For an observer-fixed reference frame, it is assumed
that the observer as the origin is arbitrarily on the surface of the Earth, which is seen to be flat and
stationary. For a body-fixed reference frame, the centre of gravity of the quadrotor is conditioned as
the origin, which is convenient when accounting for inertial properties. Followingly, the conversion
between the observer-fixed and quadrotor-fixed reference frame can be seen as a relative transformation which is divided into the difference in distance along each axis and then the change in rotational


Controller
Mounted Target
Internals
Figure 23: Photographs showing the Dagu S4A EDU Controller (top-left), Dagu DG02S 48:1 Geared
DC Motor (top-right), and various isometric views of the target assembly (bottom).
24
Connecting to the two driven wheels of 65mm in diameter, there are two Dagu DO02S 48:1 Geared
DC Motors (subsequently referred to as the Dagu Motors) requiring an input voltage between 3V and
6V with a no load current of 200mA, stall current of 1.5A at 3V or 3A at 6V, and maximum allowable
torque of approximately 0.078N.m [27]. The built-in gearbox reduces the motor speed based on a
ratio of 48:1, where the output angular speed can vary from 65rev/min at 3V unloaded to 190rev/min
at 6V unloaded [27]. A generic omni-wheel will also be positioned centrally for stability.
For target detection with grayscale processing, a white disk of 93mm in diameter will be used as the
marker, since it contrasts with the colour of the target and background for reliable detection and will
not change shape as the orientation of the target changes. For target detection with colour processing,
a red rectangle of 95mm by 74mm will be used as the marker, since it is unique in the environment.
The Dagu Controller and Dagu Motors are shown in Figure 23 with the complete assembly, which is
able to accomplish a suitable range of translational motion. The mass and quantity of each component
are summarised in Table 1 for a total mass of approximately 528g.
Table 1: Quantity and mass of the components used to assemble the moving target.
Quantity [unit] Unit Mass [g]
Dagu S4A EDU Controller 1 14
Dagu DG02S 48:1 Geared DC Motor 2 32
Dagu 65mm Wheels With Rubber Tyres 2 40
Generic Metal Omni-Wheel 1 36
Dagu DG0-12 Metal Chassis 1 218
Duracell 1.5V AA Battery 5 25
Other Parts (Cables, Nuts, Bolts, Target, etc) - 39
Total Assembly - 576
To separately evaluate the yaw orientation tracking, a SG90 Micro Servo Motor will be used to set the
desired orientation with a white rectangle of 140mm by 95mm as the marker, as seen in Figure 24.
This servo motor can rotate between 0o
and 180o with a torque up to 0.245N.m and mass of 14.7g,
where control is issued through a PWM signal and a voltage between 4.8V and 6V is supplied.
SG90
Controller,
Battery Pack
Target
Motion Capture Marker SG90 Micro Servo
Shaft


Scan this QR code to download the app now


rotation (right). (In this context, the rotational angles can also be referred to as Euler angles).
2.3 RASTER IMAGES
A raster image or bitmap is composed of multiple pixels with the amount of pixels in the horizontal
and vertical direction forming a rectangular grid and resolution. The process of rasterisation involves
representing a real object or vector image in the form of a raster image, where the continuous data
describing the real object or vector image is converted into discrete data for representation through
8
the raster image. However, there will always be information lost during rasterisation due to the
discretization error arising from a limit in the ability to completely render the continuous data. The
degree to which information is lost is dependent on the construction and resolution of the raster
image, which is a principal concern for lower resolutions when features reduce in quality and cannot
be resolved. The process of rasterisation and rendering a raster image is demonstrated in Figure 8.
Real Object
(Vector Image)
High Resolution
Rasterisation
Low Resolution
Rasterisation
Figure 8: Demonstration of the rasterisation of a real object with high and low resolutions. (Although
the original image of the real object is actually a raster image, its resolution is sufficiently large to
avoid raster artefacts and it serves as a view from an observer for this demonstration).
2.4 PINHOLE CAMERA MODEL
In fundamental terms, digital cameras capture raster images by measuring the amount and wavelength
or colour of light projected onto an imaging sensor with pixels (analogue cameras will not be considered) [2]. This imaging sensor is usually a charge-coupled device (CCD) or a complementary
metal-oxide semiconductor (CMOS) array [2, 6, 12]. As abridged in Figure 9, there will be rays of
light emitted from each point on the target. This light will spread and be incident on the lens of the
camera which will then focus the light to the corresponding points on the imaging sensor. So, it is
essential for the lens to be located at the exact focal length to ensure the light is focussed to a singular
point, rather than distorted or blurred discs which create an unclear image [2].
Lens
Optical
Centre
Target
Surface
Image
Sensor
Virtual
Image
Plane
Focal Length Focal Length


To create a logging configuration the following can be used:


* RESOURCES


To set a parameter you must be connected to the Crazyflie. A parameter is set
using:


  1. [ Advisor  ](/advisor)
  2. [ cfclient  ](/advisor/python/cfclient)
  3. [ functions  ](/advisor/python/cfclient/functions)
  4. [ cfclient.utils.logconfigreader.LogConfig  ](/advisor/python/cfclient/functions/cfclient.utils.logconfigreader.LogConfig)


are evident for the maximum processing with actual percentage differences of 10.1%, 6.55%, and
15.4% (magnitude percentage differences of 2.22%, 2.70%, and 5.26%) for resolutions of 160px by
120px, 320px by 240px, and 640px by 480px respectively. Furthermore, there was severely more
noise observed around the target in the frames captured with colour processing, which results in a
less accurate interpretation of the target in colour processing compared to grayscale processing.
The Pi Camera experienced no detrimental fish-eye distortion or effects from rolling shutter, and it
was calibrated to find the relationship between the virtual target area and current altitude of the camera
with the experimental trend relationships for the disk and rectangle targets expressed in Equation 27
and Equation 28 respectively. By averaging the 111px and 116px experimental focal lengths from the
disk and rectangle targets respectively, the focal length in terms of pixels is estimated to be 113.5px.
In addition, these remarkable correlations between the experimental trend relationships and with the
theoretical predictions further reinforce and validate the legitimacy of the pinhole camera model. (It
should be noted that this calibration is unique to the tested resolution of 160px by 120px, where
higher resolutions will have a greater virtual target area due to the increased number of pixels).
Unfortunately, the calibration for the rectangle target suffered greater uncertainty than the calibration
for the disk target, where the variation for the rectangle target increased from 16px to 58px for the
minimum bound and from 27px to 75px for the maximum bound, and the variation for the disk target
increased from 8px to 32px for the minimum bound and from 8px to 46px for the maximum bound
(and this was over a larger range of altitudes). The power nature of this increase (observed in Figure 37
and Figure 38) is also expected as it mimics the power nature of the calibration. The reason for the
56
greater uncertainty with the rectangle target is proposed to be due to the possibility for the orientation
to change, which, with pronounced effects from the low resolution, results in poor rasterisation along
the edges when they are diagonal compared to when they are vertical or horizontal. Conversely,
the disk target appears almost identical for all orientations, since it is continuously symmetric and
proportional and, thus, its uncertainty will be mostly independent from orientation.
7.2 VISUAL SERVOING PERFORMANCE
With the grayscale processing and calibration of the Pi Camera to use the pinhole camera model, the
real-time target detection and tracking was implemented with the Crazyflie mirroring the motion of
the target while maintaining the altitude of the Pi Camera. The take off of the Crazyflie was occasionally erratic with large variations in the latitude and longitude positions as it ascended. There was
also initial overshooting before reaching the desired altitude, where the actual percentage difference
in the overshoot averaged at 14.70% over all of the tests with a fairly consistent trend among the tests,
which is expected since this is an independent event only associated with the take off before the target
begins any motion. Nevertheless, this behaviour did not affect the operation once it had reached the
desired altitude and, accordingly, the main substance of the results were not affected.
During the target detection and tracking, the Crazyflie lagged slightly behind the target in terms of
position with an average of 0.838s for the translation tests and 0.415s for the rotation tests. A lag is
expected since each frame is being processed for 0.33s on average and then the generated setpoint
needs to be sent wirelessly over the Crazyradio to the Crazyflie which then needs to be processed
internally. The reason for the difference in lag between the translation tests and rotation tests is due to


    self.link_status_pub.publish(self.link_status)

            self.setupStabilizerLog()

            """
            Configure the logger to log accelerometer values and start recording.

            The logging variables are added one after another to the logging
            configuration. Then the configuration is used to create a log packet
            which is cached on the Crazyflie. If the log packet is None, the
            program exits. Otherwise the logging packet receives a callback when
            it receives data, which prints the data from the logging packet's
            data dictionary as logging info.
            """
            # Set accelerometer logging config
            accel_log_conf = LogConfig("Accel", 10)
            accel_log_conf.addVariable(LogVariable("acc.x", "float"))
            accel_log_conf.addVariable(LogVariable("acc.y", "float"))
            accel_log_conf.addVariable(LogVariable("acc.z", "float"))

            # Now that the connection is established, start logging
            self.accel_log = self.crazyflie.log.create_log_packet(accel_log_conf)

            if self.accel_log is not None:
                self.accel_log.dataReceived.add_callback(self.log_accel_data)
                self.accel_log.start()
            else:
                print("acc.x/y/z not found in log TOC")

            motor_log_conf = LogConfig("Motor", 10)
            motor_log_conf.addVariable(LogVariable("motor.m1", "int32_t"))
            motor_log_conf.addVariable(LogVariable("motor.m2", "int32_t"))


[ r/CrazyFlie ](/r/CrazyFlie/)


[ ![Snyk|Open Source Security Platform](/advisor/images/snyk-wordmark.svg)
](https://snyk.io/ "Open Source Security Platform") © 2024 Snyk Ltd.


        init_drivers()
           """ Search for and initialize link drivers."""


    def _disconnected(self, link_uri):
        """Callback when the Crazyflie is disconnected (called in all cases)"""
        print('Disconnected from %s' % link_uri)
        self.is_connected = False


LIST OF TABLES
1 Quantity and mass of the components used to assemble the moving target . . . . . . . . 25
2 Evaluation of the most suitable resolution for acceptable real-time performance . . . . . 41
3 Outline of the lag and position deviation of the Crazyflie for the translation tests . . . . . 52
4 Outline of the lag and angular deviation of the Crazyflie for rotation tests . . . . . . . . . 52
5 Outline of the altitude of the Crazyflie for each of the translation and rotation tests . . . . 55
6 Evaluation of the potential risks involved in the experiment . . . . . . . . . . . . . . . . 83
viii
LIST OF SYMBOLS
The general symbols used, with a description and relevant units, are as follows:
a first arbitrary variable -
Auv virtual image area px2
Axy actual area m2
b second arbitrary variable -
b(t) feedback signal -
d+ plus-mode moment arm m
d× cross-mode moment arm m
Dxy latitude and longitude position deviation m
Dψ yaw orientation deviation rad or o
e(t) error signal -
f focal length px
f(t) control signal -
F resultant thrust N
FM1 motor one thrust N
FM2 motor two thrust N
FM3 motor three thrust N
FM4 motor four thrust N
g gravitational acceleration; or grayscale value 9.807m/s2
; or -
gnew new grayscale value -
gthr grayscale threshold value -
h hue value -
hmax maximum hue value -
hmin minimum hue value -
H height px
Ix mass moment of inertia about x-axis kg.m2
Iy mass moment of inertia about y-axis kg.m2
Iz mass moment of inertia about z-axis kg.m2
k camera correlation coefficient m.px
KD derivative gain -


rW
2
(23)
v = −ui +
rH
2
(24)
Where u, new latitude coordinate, px; ui
, initial latitude coordinate, px; rW, latitude resolution, px; v,
new longitude coordinate, px; vi
, initial longitude coordinate, px; and rH, longitude resolution, px.
( rW , rH )
Desired System
Initial System
(0,rH) (rW,rH)
(0, 0) (rW, 0) ( rW ,rH )
u
v
u
v
Coordinate System
Transformation
(rW ,rH )
(rW , rH )
(rW ,rH )
Figure 29: Coordinate system transformation to convert the initial system into the desired system.
A coordinate system also needs to be established for the yaw orientation tracking. As seen in Figure 30, the direct output from OpenCV is uncertain and it is not possible to establish the orientation
35
since the target could be aligned in either of the two forward quadrants. So, relative to the rectangle
target, it is devised that the returned output will be the angle measured between 0o
and 90o
from a
horizontal projected at the lowest corner to the first edge. The orientation relative to the two forward
quadrants can then be evaluated based on the labelling of the corners and which of the edges are
labelled as the width and height. Thus, the yaw orientation can be found between 90o
and 90o using
the conditions of Equation 25 where the forward direction has a yaw orientation of 0o
.
ψ =



In order to easily use and control the Crazyflie there's a library made in
Python that gives high-level functions and hides the details. This page
contains generic information about how to use this library and the API that it
implements.


If your Crazyflie has a positioning system (Loco, flowdeck, MoCap,
Lighthouse), you can also send velocity or position setpoints, like for
instance:


##  Similar packages


0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
0 200 400 600 800 1000 1200 1400 1600 1800 2000
z Altitude [m]
0
Auv Area [px2
]
Data Point
Trend Line
Trend:
Figure 38: Altitude calibration using a rectangle as the target with edges of 140mm by 95mm. (The
uncertainty in each altitude measurement was about ±5mm which is not distinguishable on the plot).
6.4 QUADROTOR OBSERVATIONS
Considering ground effects as the increased lift and decreased drag experienced when flying close to
a fixed surface, there were no noticeable ground effects once the Crazyflie had taken off and reached
the desired altitudes above 300mm. For an aircraft not designed to normally operate with ground
effects, these ground effects can be identified by abnormal flight control when flying near the ground,
but this was not present in a perceivable form as the Crazyflie was tracking the target.
The Flow deck was able to successfully stabilise the Crazyflie and hover at a fixed position without the
need for roll or pitch trimming. However, during take off, there were instabilities where the behaviour
of the Crazyflie was partially uncontrolled and erratic, and this had to be corrected by accounting for
the slightly altered origin after take off. There were also indications of very slow and minor drift in
the yaw orientation over extended periods of time, but this was seen to be almost undetectable for
short flight times and did not affect the performance of the Crazyflie.
6.5 VISUAL SERVOING
While the tests were being performed, the primitive data was collected in the form of the threedimensional position and three-dimensional rotation of both the target and Crazyflie. To extract
meaning from these data, it is necessary to analyse it with regards to lag time in position, target
detection robustness, relative deviation fluctuations, and ability to track the target at varying speeds.
45
6.5.1 PRIMITIVE DATA
From the motion capture system, the basic three-dimensional position of the target and Crazyflie was
recorded and is shown in Figure 39 to Figure 41 for each trial in each of the rectilinear, circular, and
combined tests respectively. The position of the target is directly presented relative to its starting
position, while the position of the Crazyflie is re-evaluated relative to its initial position after taking


All callbacks are handled using the ` Caller ` class that contains the
following methods:


Check out the [ automated API documentation
](/documentation/repository/crazyflie-lib-
python/master/api/cflib/crazyflie/commander/) for the Crazyflie cflibâs
commander framework to find out what other functions you can use.


recorded?
Set up the recording
parameters including
the video codec.
Yes
No
Must the
camera operation
be checked?
Yes
No
Set the initial position
of the Crazyflie as its
origin point (0,0,0).
Reset the extended
Kalman filter session
on the Crazyflie.
Calibrate the extended
Kalman filter for the
state of the Crazyflie.
Read a frame from the
camera to initialise
the image processing.
Send the setpoint
function for the
Crazyflie to take off.
Must the
Crazyflie keep
flying?
Read the live frame
from the camera.
Yes
Apply the PCM to
find the altitude and
position of the target.
Get the area, centroid,
and orientation of the
target in pixels.
Send a hover setpoint
to the Crazyflie with


(middle), and 640px by 480px (bottom), with an overlay of the target detection (as a graphical convenience for monitoring) found through grayscale processing (left) and colour processing (right).
While the camera calibration was being completed, the current area of the target in the captured frame
was printed to the shell. Unexpectedly, it was gathered from this that the execution drastically slowed
down with a sharply dropped frame rate and lag of approximately 2s to 3s between the displayed
frame and real-time arrangement, even though the resolution remained the same. By removing the
42
commands to print to the shell for confirmation, it was concluded that printing to the shell indeed
caused these issues and is not a viable option for real-time monitoring of the camera view.
The rasterisation of the target was also considered for the disk and rectangle targets. The comparison
for a resolution of 160px by 120px is seen in Figure 35, where it was observed that the pixelated
appearance of the rectangle target was greatly altered depending upon the orientation, but there was
almost no change in the pixelated appearance of the disk target for any orientation.
Disk Target
Arbirary Orientation
Rectangle Target
Diagonal Edges
Rectangle Target
Straight Edges
Figure 35: Rasterisation at 160px by 120px for the disk (left) and rectangle (middle and right) targets.
(These images have been cropped from the appearance of the targets during the altitude calibration).
6.3 CAMERA CALIBRATION
For the camera calibration, the altitude of the camera was increased between 300mm and 780mm
corresponding to a decrease in the area of the target, with example frames captured during the process
shown in Figure 36. The recorded data is plotted in Figure 37 which allows for the trend relationship
to be evaluated and used for the Crazyflie to maintain an altitude. By manipulating Equation 16
to form Equation 26, the focal length in terms of pixels can be determined using the actual area of
7088.2mm2
for the disk target. To achieve confirmation, the rectangle target of 13300mm2 was also
used for the same calibration methodology with the recorded data plotted in Figure 38.
f =
k
p
Axy
(26)
Where f , focal length, px; k, camera correlation coefficient, m.px; and Axy, actual area, m2
.
From Equation 16 and Equation 17, it is evident that it is theoretically expected for a power relationship with a correlation coefficient and exponent of 0.5. From the recorded data, the disk calibration
resulted in a trend relationship given by Equation 27 with an exponent of 0.484. and the rectangle
calibration resulted in a trend relationship given by Equation 28 with an exponent of 0.492. With the
correlation coefficient and Equation 26, the focal length in terms of pixels is then found to be 111px


[ ![github](/advisor/images/snyk-icon-github.svg) bitcraze / crazyflie-
clients-python / lib / cfclient / ui / tabs / FlightTab.py  View on Github
![external](/advisor/images/snyk-icon-external.svg)
](https://github.com/bitcraze/crazyflie-clients-
python/blob/0b64640096567992bb6d0820dd1cb47cfe1dbc6a/lib/cfclient/ui/tabs/FlightTab.py#L246)


for the disk calibration and 116px for the rectangle calibration, which is an actual percentage difference of 4.41% and magnitude percentage difference of 0.931% using Equation 29 and Equation 30.
43
Altitude: 460mm
Area: 470px2
Altitude: 530mm
Area: 358px2
Altitude: 600mm
Area: 277px2
Altitude: 660mm
Area: 227px2
Altitude: 750mm
Area: 175px2
Figure 36: Example frames captured at 160px by 120px using grayscale processing at altitudes
between 460mm and 750mm for the camera calibration using a disk as the target with a radius of
95mm, with an overlay of the target detection (as a graphical convenience for monitoring).
z = 9.128A
−0.484
uv (27)
z = 13.47A
−0.492
uv (28)
%pa =

a − b
(a + b)/2

× 100% (29)
%pm =

log(a) − log(b)
(log(a) + log(b))/2


  * Top
  * [ Home ](/ "Home")
  * [ Research ](/portals/research/ "Research")
  * [ Education ](/portals/education/ "Education")
  * [ Development ](/portals/development/ "Development")
  * [ DIY ](/portals/diy/ "DIY")
  * [ License ](/license/ "License")
  * [ Cookie Policy ](/cookies/ "Cookie Policy")
  * [ Privacy Policy ](/privacy-policy/ "Privacy Policy")
  * [ Sign up for newsletter ](/signup/ "Signup for newsletter")
  * [ Feedback ](/about/feedback/ "feedback")


  * [ Log In / Sign Up  ](https://www.reddit.com/login/)
  * [ Advertise on Reddit  ](https://ads.reddit.com?utm_source=web3x_consumer&utm_name=user_menu_cta)
  * [ Shop Collectible Avatars  ](/avatar/shop)


18 Schematics and photographs showing the Bitcraze Crazyflie 2.1 control board . . . . . . 20
19 Interactions of the microcontrollers and sensors on the Bitcraze Crazyflie 2.1 . . . . . . 21
20 Photographs showing the Bitcraze Flow V2 expansion deck . . . . . . . . . . . . . . . . 21
21 Schematics and photographs showing the Raspberry Pi Zero and Camera Module . . . . 23
22 Photographs showing the Bitcraze Crazyradio PA . . . . . . . . . . . . . . . . . . . . . 24
23 Photographs showing the Dagu S4A EDU, Dagu DG02S Motor, and target assembly . . 24
24 Photographs showing the SG90 Micro Servo motor and yawing target assembly . . . . . 25
25 Photographs showing a Qualisys Miqus M1 and layout of the motion capture facilities . 26
26 Open-source software used for visual servoing, quadrotor control, and target control . . . 27
27 Method of calibrating the camera and determining the focal length . . . . . . . . . . . . 32
vi
28 Tracking of the target with the corresponding movement of the Crazyflie . . . . . . . . . 35
29 Coordinate system transformation to convert the initial system into the desired system . . 35
30 Explanation of the initial yaw orientation and desired coordinate system . . . . . . . . . 36
31 High-level flow diagram of the control flow for the target detection and tracking . . . . . 37
32 Paths on which the target will aim to move to test various motions . . . . . . . . . . . . 38
33 Photograph and evaluation of the fish-eye distortion in the Pi Camera . . . . . . . . . . . 40
34 Example frames captured at 160px by 120px, 320px by 240px, and 640px by 480px . . . 42
35 Rasterisation at 160px by 120px for the disk and rectangle targets . . . . . . . . . . . . 43
36 Example frames captured at 160px by 120px for the camera calibration . . . . . . . . . 44
37 Altitude calibration using a disk as the target with a radius of 95mm . . . . . . . . . . . 44
38 Altitude calibration using a rectangle as the target with edges of 140mm by 95mm . . . . 45
39 Position of the markers in the first (top) and second (bottom) trials of the rectilinear test . 46
40 Position of the markers in the first and second trials of the circular test . . . . . . . . . . 47
41 Position of the markers in the first (top) and second (bottom) trials of the combined test . 47
42 Relevant results against time throughout the first trial of the rectilinear motion test . . . . 48
43 Relevant results against time throughout the second trial of the rectilinear motion test . . 49
44 Relevant results against time throughout the first trial of the circular motion test . . . . . 49
45 Relevant results against time throughout the second trial of the circular motion test . . . 50
46 Relevant results against time throughout the first trial of the combined motion test . . . . 50
47 Relevant results against time throughout the second trial of the combined motion test . . 51
48 Relevant results against time throughout the first trial of the yaw orientation test . . . . . 53
49 Relevant results against time throughout the second trial of the yaw orientation test . . . 53
50 Deviations with adjustments for average lag in the translation and rotation tests . . . . . 54
51 Standard operating procedure in the motion capture facilities . . . . . . . . . . . . . . . 84
52 Arrangement for end-point closed-loop control through detection and tracking of a target. 85
53 Comparison between the field of view during rolling or pitching . . . . . . . . . . . . . 85
54 Schematic and photographs showing the Bitcraze BigQuad expansion deck . . . . . . . 86
55 Photographs showing the assembled larger quadrotor . . . . . . . . . . . . . . . . . . . 86
vii


Here's an example of how to use the calls.


[ ](/advisor/python)


~800mm
Rectilinear Path
Start
End
Start
End
Camera Altitude
Decreased Manually
Camera View
780mm 480mm
~1050mm
~800mm
Camera View
~660mm
~500mm
Circular Path
Combined Path
Camera View
~800mm
~600mm
Camera Altitude
Constant At 580mm
End
Start
End Start
Yaw Rotation
Camera View
~520mm
~390mm
Camera Altitude
Constant At 380mm
Start
End
End
Start
0o 180o
Figure 32: Paths on which the target will aim to move to test rectilinear (top), circular (middle),
combined (bottom-left), and yaw rotation (bottom-right) motions. (The actual movement may differ).
38
intentionally try to assist or avoid the tracking. Furthermore, it is also assumed that the environment


25mm
Top View
Figure 21: Schematics (bottom) and photographs (top) showing the top and bottom views of the
Raspberry Pi Zero W 1.1 (left) and Raspberry Pi Camera Module 2.1 (right) [22, 23].
of 6g, and features a nRF24LU1+ radio microcontroller (16MHz, 2kB SRAM, and 32kB flash) with
a 20dBm or 100mW low-noise amplifier for communication through the 2.4GHz ISM band radio at
frequencies from 2.400GHz to 2.525GHz for 125 channels [16, 24]. The communication data rates
can be set at 250kB/s, 1MB/s, or 2MB/s and the data sizes can be up to packets of 32B for low latency,
while the ideal range can exceed 1km under normal conditions with a direct line-of-sight and without
interference or obstacles [16, 24]. The minimum latency to send a packet is estimated to be about
2ms with 1ms for the USB serial communication and 1ms measured latency for the radio at 2MB/s
without any retries [25]. Photographs of the Crazyradio are seen in Figure 22.
By default, the Crazyradio operates in primary transmitter mode (PTX), while the Crazyflie operates
in primary receiver mode (PRX), where the Crazyradio sends data over the communication signal to
the Crazyflie and the Crazyflie returns an acknowledgement packet which may also contain additional
data for logging and monitoring [16]. So, for the control and communication, the high-level control
setpoints will be generated on the Raspberry Pi and then wirelessly transmitted to the Crazyflie using
the Crazyradio, where the firmware on the Crazyflie will interpret these setpoints and implement the
corresponding low-level commands to control the rotors and overall flight.
23
18mm
160mm
View
Top Bottom
View
18mm
USB Microcontroller Adjustable Hinge Antenna
Figure 22: Photographs showing the top and bottom views of the Bitcraze Crazyradio PA.
4.3 MOVING TARGET PARTS
The moving target will be a simple two-wheeled design with a Dagu S4E EDU Controller (subsequently referred to as the Dagu Controller) as the motor driver and microcontroller which is compatible with the Arduino integrated development environment (IDE) with an Arduino Pro Mini 328
5V 16MHz bootloader [26]. This controller has an ATmega328P processor at 16MHz with 2kB of
SRAM, 32kB of flash, 1kB of EEPROM, and will receive power from a battery pack with five Duracell 1.5V AA batteries with a maximum distribution of 2.5A to each motor pin [26].
ISP Header
Analog Input Pins
Right Motor Connection
A5
A4
A3
A2
A1


logconf.add_variable('pm.state', 'float')


The synchronous classes are wrappers around the asynchronous API, where the
asynchronous calls/callbacks are replaced with blocking calls. The synchronous
API does not provide the full flexibility of the asynchronous API, but is
useful when writing small scripts, for logging for instance.


Magnify icon


  * The parameter is not changed by the Crazyflie but by the client
  * The parameter is not read periodically


    def _connected(self, link_uri):
        """ This callback is called form the Crazyflie API when a Crazyflie
        has been connected and the TOCs have been downloaded."""
        print('Connected to %s' % link_uri)


  * [ ![](/advisor/images/footer-twitter-icon.svg) ](https://twitter.com/snyksec "Twitter")
  * [ ![](/advisor/images/footer-youtube-icon.svg) ](https://www.youtube.com/channel/UCh4dJzctb0NhSibjU-e2P6w "YouTube")
  * [ ![](/advisor/images/footer-facebook-icon.svg) ](https://www.facebook.com/snyksec "Facebook")
  * [ ![](/advisor/images/footer-linkedin-icon.svg) ](https://www.linkedin.com/company/snyk "LinkedIn")


##  Popular cfclient functions


scf.cf.param.add_update_callback(group='deck', name='bcLighthouse4',


maximum bounds (usually with HLS or HVS). This is typically used to generate a binary image.
• erode, dilation, or morphologyEx: Apply a morphology transformation. This is typically
used to further reduce noise, where the latter will either apply an erosion transformation (remove
white noise) followed by a dilation transformation (restore the original areas without noise) with
MORPH_OPEN or apply a dilation transformation (connect white regions) followed by an erosion
transformation (restore the original areas without disconnections) with MORPH_CLOSE.
• Laplacian, Sobel, or Scharr: Find the gradient or derivative intensities in a direction. This is
typically used to highlight edges using a high-pass filter based on feature changes.
• Canny: Perform edge detection. This is an existing algorithm with multiple stages including
noise reduction with a Gaussian filter, gradient intensity finding with the Sobel method in both
latitudinal and longitudinal directions, non-maximum suppression with the removal of pixels
which are not of the maximum value in the local neighbourhood in the gradient directions, and
hysteresis thresholding to classify pixels as edges to be kept or non-edges to be discarded. Since
a self-derived algorithm is required, this function will not be used, but it provides an initial guide.
• findContours: Find the contours (as mentioned, a contour is a continuous curve joining the
points which have the same colour or intensity along a boundary). This is typically used for basic
shape analysis and target detection with RETR_TREE to retrieve all contours and reconstruct a full
hierarchy of nested contours and CHAIN_APROX_SIMPLE to only store the end points of straight
lines for decreased processing. (The contours can be drawn with findContours).
• boundingRect: Find the top-left coordinate for a bounding rectangle and the width and height
of this bounding rectangle applied to a contour. This is typically used for monitoring by drawing
the found bounding rectangle around a contour. (There is no consideration for rotation).
• minAreaRect: Find the centroid, dimensions (height and width), and orientation between 900
and 0o
for a contour. This is typically used when the orientation of a contour is needed.
• matchTemplate: Search for and find the location of a provided template image in a larger
image. This is typically used when an almost identical match is possible, where there is no
scaling, colour, or distortion changes of the template image within the larger image.
29
By default, OpenCV uses the connected camera to capture an image at a resolution of 640px by
480px, but this can be changed along with other default parameters using set, such as the frame rate,
codec, file format, brightness, contrast, saturation, hue, gain, exposure, and white balance - it should
be noted that the camera must inherently support changing these parameters. There are also more
advanced functions available which allow for camera calibration, optical flow, augmented reality,
epipolar geometry, and various machine learning techniques, but these are mostly only applicable for
specialised applications where greater computational effort is available [13].
(It should be noted that the OpenCV Python API library requires Numpy for backend processing,
which is an open-source Python library for performing advanced calculations and matrix operations).
4.6.3 RASPBIAN
Raspbian Buster July 2019 is an operating system based on Debian 10 with the Linux Kernel 4.19.


[ ![github](/advisor/images/snyk-icon-github.svg) gtagency / crazyflie-ros /
scripts / crazyflie_node.py  View on Github  ![external](/advisor/images/snyk-
icon-external.svg) ](https://github.com/gtagency/crazyflie-
ros/blob/150da091a7b9562411318fdd0f98e71d58fba445/scripts/crazyflie_node.py#L158)


field of view of 48.8o
[21]. For the imaging sensor, the Pi Camera uses an 8Mpx Sony IMX219 with
square pixels of 1.12µm and support for photograph resolutions up to 3280px by 2464px and video
resolutions up to 1920px by 1080px with 30fr/s, 1280px by 720px at 60fr/s, or 640px by 480px at
90fr/s (or a lower resolution can be chosen, but the maximum frames rate is 90fr/s) [21]. The basic
schematics and photographs of the Raspberry Pi and Pi Camera are seen in Figure 21.
Fortunately, the Pi Camera aims to have little to no noticeable fish-eye distortion. However, the Pi
Camera does use a rolling shutter where frames are rapidly scanned vertically and, as a result, the
frames may be somewhat susceptible to various distortion effects, which are especially pronounced
when the relative speed of the target is very high and related to the magnitude of the seconds per frame
captured by the camera. As mentioned, there are methods to model and compensate for distortions
from rolling shutter, but these are expected to be unnecessary since it is not crucial for exceptional
detail and the discrepancies are expected to be very minimal at the operating speeds.
The communication between the Raspberry Pi and Crazyflie will be facilitated by a Bitcraze Crazyradio PA (subsequently referred to as Crazyradio), which is a USB Type-A radio dongle, has a mass
22
Bottom View
65mm 23.9mm
30mm
25mm
Raspberry Pi Camera Module 2.1
Top View Bottom View
Standard Connection
Lens, Image Sensor
The 150mm camera
cable converts the
standard connection at
the Pi Camera to the
mini-connection at the
Raspberry Pi Zero.
Raspberry Pi Zero W 1.1
Micro-USB Power
Micro-USB Input
Mini-HDMI Output
Micro-SD Card Slot
Cam. Mini-Connect
Single-Core CPU
40 GPIO Header
Top View
150mm
30mm


433 time_end = time.time ()
434
435 # --------------------------------------------------------------------------------
436 # Finishing Commands
437 # --------------------------------------------------------------------------------
438
439 # Terminate the capture and release the source.
440 print ("Terminating the capture and releasing the source.")
441 source.release ()
442 if record == "Y":
443 print ("The output video is ’Out.avi’ in the current directory.")
444 video.release ()
445 cv.destroyAllWindows ()
446
447 # --------------------------------------------------------------------------------
448 # End
449 # --------------------------------------------------------------------------------
C ETHICS CONSIDERATIONS
For the completion of the project, there were no ethical considerations with regards to participants,
as the requisite knowledge, information, and resources was acquired through self-derived means in
the form of published research, online resources, and analytical calculations. Furthermore, the results
of the experiment were not doctored or plagiarized in any form so that the scientific method was
thoroughly and sufficiently followed. As evaluated during the project proposal by a member of the
School Ethics Committee, there were no ethical risks and ethical clearance was not required.
81
D RISK ASSESSMENT
The following precautions must be followed throughout experimentation:
• At all times within the motion capture facilities, the operators, supervisors, and nearby bystanders must wear the required personal protective equipment (PPE) in the form of a laboratory dust
coat, steel-toed boots, and eye protection to reduce risk in the event of a crash.
• There should be at least two operators or an operator and supervisor monitoring the Crazyflie.
• The environment must be clear of obstacles not involved in the tests with no debris or loose items
in the vicinity which could be unpredictably lifted by the thrust of the Crazyflie.
• The ground control laptop must be charged or charging before testing or charging while testing
to ensure it will not power off during a test while the Crazyflie is flying.
• The Raspberry Pi must also have a reliable power supply for the duration of testing.
• Before charging or testing, it must be ensured that the battery of the Crazyflie is not punctured
or swollen. If the battery is damaged in any way, a supervisor must be notified immediately.
• There must be no interference with the connections between the ground control laptop, Raspberry Pi, and Crazyflie to ensure there will be no disconnections while the Crazyflie is flying.
• The state of the hardware should be visually checked before each test to ensure there are no
damages and the installation is still correct. This is specifically applicable to the propellers,


![](/advisor/images/snyk-icon-published.svg) Latest version published 27 days
ago


167 var_z_history = [1000] * 10
168
169 # Define a threshold for maximum and minimum variations.
170 threshold = 0.001
171
172 # Begin logging the added variables.
173 with SyncLogger (crazyflie, log_config) as logger:
174 for log_entry in logger:
175
176 # Append the current values to the stored history.
177 data = log_entry [1]
178 var_x_history.append (data ["kalman.varPX"])
179 var_x_history.pop (0)
180 var_y_history.append (data ["kalman.varPY"])
181 var_y_history.pop (0)
182 var_z_history.append (data ["kalman.varPZ"])
183 var_z_history.pop (0)
184
185 # Compare the maximum and minimum variations until within the threshold.
186 minimum_x = min (var_x_history)
187 maximum_x = max (var_x_history)
188 minimum_y = min (var_y_history)
189 maximum_y = max (var_y_history)
190 minimum_z = min (var_z_history)
191 maximum_z = max (var_z_history)
192 if (maximum_x - minimum_x) < threshold and (maximum_y - minimum_y) <
> threshold and (maximum_z - minimum_z) < threshold:
193 break
194
195 # Function to set a custom initial position.
196 def set_initial_position (crazyflie, initial_x, initial_y, initial_z, initial_yaw):
197
198 # Print an update to the shell for monitoring.
199 print ("Setting the initial position to (", initial_x, ",", initial_y, ",",
> initial_z, ") at ", initial_yaw, " radians.")
200
201 # Setting the initial position.
202 crazyflie.param.set_value ("kalman.initialX", initial_x)
203 crazyflie.param.set_value ("kalman.initialY", initial_y)
75


[ ![github](/advisor/images/snyk-icon-github.svg) gtagency / crazyflie-ros /
scripts / crazyflie_node.py  View on Github  ![external](/advisor/images/snyk-
icon-external.svg) ](https://github.com/gtagency/crazyflie-
ros/blob/150da091a7b9562411318fdd0f98e71d58fba445/scripts/crazyflie_node.py#L128)


        crazyflie.param.add_update_callback(group="group", name="name", param_updated_callback)

        def param_updated_callback(name, value):
            print "%s has value %d" % (name, value)


###  Uniform Resource Identifier (URI)


The synchronous API uses the python Context manager concept, that is the `
with ` keyword. A resource is allocated when entering a ` with ` section and
automatically released when exiting it, for instance a connection or take
off/landing of a Crazyflie.


BLE Bluetooth Low Energy
CCD Charge-Coupled Device
CPPM Chaotic Pulse Position Modulation
CPU Central Processing Unit
CMOS Complementary Metal-Oxide Semiconductor
CSI Camera Serial Interface
D Derivative
DC Direct Current
DOF Degrees Of Freedom
EEPROM Electrically Erasable Programmable Read Only Memory
ESC Electronic Speed Controller
FAST Feature from Accelerated Segment Test
FPS Frames Per Second
GPIO General Purpose Input/Output
GPS Global Positioning System
GUI Graphical User Interface
HDMI High-Definition Multimedia Interface
HLS Hue, Lightness, and Saturation
HP Hewlett-Packard
HSI Human System Interactions
HVS Hue, Value, and Saturation
I Integral
IBVS Image-Based Visual Servoing
ICUAS International Conference on Unmanned Aircraft Systems
IDE Integrated Development Environment
IEEE Institute of Electrical and Electronics Engineers
IMU Inertial Measurement Unit
I2C inter-Integrated Circuit
LAN Local Area Network
LED Light Emitting Diode
LiPo Lithium-Polymer
xii
MIMO Multiple-Input Multiple-Output
MOCAP Motion Capture
NIR Near Infra-Red
NoIR No Infra-Red
P Proportional
PCM Pinhole Camera Model
PI Proportional-Integral
PD Proportional-Derivative


class LoggingExample:
    """
    Simple logging example class that logs the Stabilizer from a supplied
    link uri and disconnects after 5s.
    """


###  Serial driver


246 print (" Average SPF:", "{0:.4f}".format (spf_avg), "seconds per frame.")
247 except ZeroDivisionError:
248 print (" ! The resolution passed for capture does not match the output.")
249 print (" ! The average frame rate values could not be determined.")
250 fps_avg = 0
251 spf_avg = 0
252
253 # ----------------------------------------------------------------------------------
254 # End
255 # ----------------------------------------------------------------------------------
For altitude calibration, the above script was modified with print(cv.contourArea(maximum))
within the target_detection function to display the target area in pixels, while maximum grayscale
processing monitored the target as it was positioned and the altitude of the camera was varied. As
mentioned, this printing to the shell dramatically slowed execution with a noticeable lag.
The final script for target detection and tracking is presented below, where further optimisations have
also been performed after the script for the resolution tests and altitude calibration to make the code
70
more efficient and robust. An optimisation which was not exploited, as it was not necessary, is the
idea to restrict the region in which to look for the target based on the detected location of the target in
the previous frame, such that even less computational effort would be required as a result.
1 # ********************************************************************************
2 # Date: 2019-18-10. Author: Edward Rycroft. Description: This program is used to
> perform visual servoing through end-point open-loop control using a Raspberry Pi
> and Pi Camera for image processing with OpenCV and a Crazyflie to mirror the
> relative position of a moving target for effective target detection and tracking.
3 # ********************************************************************************
4
5 # --------------------------------------------------------------------------------
6 # Import Files
7 # --------------------------------------------------------------------------------
8
9 # Import the necessary libraries (note the changes of names).
10 import time
11 import logging
12 import numpy as np
13 import cv2 as cv
14 import cflib.crtp
15 from cflib.crazyflie import Crazyflie
16 from cflib.crazyflie.log import LogConfig
17 from cflib.crazyflie.syncCrazyflie import SyncCrazyflie


The ` PositionHlCommander ` uses the high level commander in the Crazyflie and
is based on a global coordinate system and absolute positions. It is intended
to be used with a positioning system such as Loco, the lighthouse or a mocap
system.


###  Callbacks


90 # Locate the largest contour and find the corresponding properties. Note that
> the image moments are simply weighted averages based on the pixel intensities.
91 try:
92 maximum = max (contours, key = cv.contourArea)
93 x, y, w, h = cv.boundingRect (maximum)
94 bounds = cv.minAreaRect (maximum)
95 orientation = bounds [2]
96 if bounds [1][0] < bounds [1][1]: # width < height
97 orientation = - orientation
98 else:
99 orientation = - orientation - 90
100 box = np.int0 (cv.boxPoints (bounds))
101 cv.drawContours (frame, [box], 0, (0, 0, 255), 2)
102 except:
103 orientation = 0
104
105 # Return the relevant variables to store.
106 return frame, orientation
107
108 # Function to record and display the basic output as a video.
109 def monitor_minimum (frame, record):
110
111 # Record (optional) and display the basic output.
112 if record == "Y":
113 video.write (frame)
114 cv.imshow ("Original Output", frame)
115
116 # Function to record and display the complete output as a video.
117 def monitor_maximum (frame, mode, threshold, morphology, record, contours):
73
118
119 # Combine the processed frames into a single frame.
120 if np.size (mode) == res_x * res_y:
121 mode = cv.merge ([mode, mode, mode])
122 elif np.size (mode) == res_x * res_y * 3:
123 mode = cv.cvtColor (mode, cv.COLOR_HLS2BGR)
124 mode = cv.bitwise_and (mode, mode, mask = morphology)
125 threshold = cv.merge ([threshold, threshold, threshold])
126 morphology = cv.merge ([morphology, morphology, morphology])
127 if len (contours) != 0:


KI
integral gain -
KP proportional gain -
l lightness value -
lmax maximum lightness value -
lmin minimum lightness value -
m mass; or arbitrary virtual image distance kg; or px
ix
m˙ arbitrary virtual image velocity px/s
µ air molar mass 0.02896kg/mol
n arbitrary actual distance m
n˙ arbitrary, actual velocity m/s
p pressure; or pixel colour value Pa; or -
p0 pressure at the lower troposphere limit 101350Pa
%pa actual percentage difference %
%pm magnitude percentage difference %
φ roll rotation rad or o
φ¨ roll angular acceleration rad/s2
ψ yaw rotation or orientation rad or o
ψi
initial yaw orientation rad or o
ψq quadrotor yaw angle rad or o
ψt
target yaw angle rad or o
ψ¨ yaw angular acceleration rad/s2
rH longitude resolution px
rW latitude resolution px
r(t) reference signal -
R universal gas constant 8.315kg.m2
/(s2
.mol.K)
s saturation value -
smax maximum saturation value -
smin minimum saturation value -
t time s
T temperature K
T0 temperature at the lower troposphere limit 288.15K
TM1 motor one torque N.m
TM2 motor two torque N.m
TM3 motor three torque N.m


####  Connect


* Sports


    lg = LogConfig("Baro", 50)
            lg.addVariable(LogVariable("baro.asl", "float"))
            lg.addVariable(LogVariable("baro.aslLong", "float"))
            lg.addVariable(LogVariable("baro.aslRaw", "float"))
            lg.addVariable(LogVariable("baro.temp", "float"))

            self.log = self.helper.cf.log.create_log_packet(lg)
            if (self.log is not None):
                self.log.dataReceived.add_callback(self._baro_data_signal.emit)
                self.log.error.add_callback(self.loggingError)
                self.log.start()
            else:
                logger.warning("Could not setup logconfiguration after "
                               "connection!")
            # HOVER
            lg = LogConfig("Hover", 50)
            lg.addVariable(LogVariable("hover.target", "float"))

            self.log = self.helper.cf.log.create_log_packet(lg)
            if (self.log is not None):
                self.log.dataReceived.add_callback(self._hover_data_signal.emit)
                self.log.error.add_callback(self.loggingError)
                self.log.start()
            else:
                logger.warning("Could not setup logconfiguration after "
                               "connection!")


  * All Packages
    * [ JavaScript ](/advisor/packages/javascript/popular)
    * [ Python ](/advisor/packages/python/popular)
    * [ Go ](/advisor/packages/golang/popular)
  * Code Examples
    * [ JavaScript ](/advisor/code/javascript/popular)
    * [ Python ](/advisor/code/python/popular)
  * Categories
    * [ JavaScript - Popular ](/advisor/categories/javascript/popular)
    * [ JavaScript - Healthiest ](/advisor/categories/javascript/healthiest)
    * [ Python - Popular ](/advisor/categories/python/popular)
    * [ Python - Healthiest ](/advisor/categories/python/healthiest)
  * Developer Tools
    * [ Vulnerability DB ](https://security.snyk.io/)
    * [ Code Checker ](https://snyk.io/code-checker/)
    * [ Snyk Learn ](https://learn.snyk.io/)
    * [ Blog ](https://snyk.io/blog/)


##  Finding a Crazyflie and connecting


204 crazyflie.param.set_value ("kalman.initialZ", initial_z)
205 crazyflie.param.set_value ("kalman.initialYaw", initial_yaw)
206
207 # Function to set the initial position as the origin.
208 def set_initial_position_origin (crazyflie):
209
210 # Print an update to the shell for monitoring.
211 print ("Setting the initial position as the origin (0,0,0) at 0 radians.")
212
213 # Setting the initial position.
214 crazyflie.param.set_value ("kalman.initialX", "0")
215 crazyflie.param.set_value ("kalman.initialY", "0")
216 crazyflie.param.set_value ("kalman.initialZ", "0")
217 crazyflie.param.set_value ("kalman.initialYaw", "0")
218
219 # Function to reset the extended Kalman filer.
220 def reset_estimator (crazyflie):
221
222 # Print an update to the shell for monitoring.
223 print ("Resetting the extended Kalman filter for a clean state.")
224
225 # Reset the extended Kalman filter and update the position estimator.
226 crazyflie.param.set_value ("kalman.resetEstimation", "1")
227 time.sleep (0.1)
228 crazyflie.param.set_value ("kalman.resetEstimation", "0")
229 wait_for_position_estimator (crazyflie)
230
231 # Function to vertically take off the Crazyflie.
232 def take_off (crazyflie, altitude):
233
234 # Print an update to the shell for monitoring.
235 print ("Crazyflie is taking off to the starting altitude.")
236
237 # Define the desired time in which to move and the number of setpoints to send.
238 action_time = 1
239 setpoint_time = 0.05
240 steps = int (action_time / setpoint_time)
241
242 # Loop to send the required setpoint at the desired time intervals.
243 for i in range (steps):


it is actually more accurate to use the terms “multirotor” and “quadrotor” respectively [3]).
Propellers
Stands
Front View
Frame
Trirotor Quadrotor Hexarotor Octarotor
Figure 1: Examples of the common multirotor layouts for three (left), four (left-middle), six (middle),
and eight (right-middle) rotors with a general layout of a front view (right).
With regards to a quadrotor, visual servoing for moving target tracking is essentially the technique in
which the motion of the quadrotor is controlled by commands generated based on a passive visionbased sensor. This visual servoing may be in the form of end-point closed-loop or eye-in-hand control
with a camera mounted on-board the quadrotor to observe the relative position and motion of a target. An alternative method of visual servoing is end-point open-loop or eye-to-hand control, where
multiple external cameras are fixed in the surroundings and form a motion capture system observing
the absolute position and motion of a target. These methods are demonstrated in Figure 2.
To distinguish within visual servoing, target detection is concerned with using a captured image to
identify a designated target, while target tracking is concerned with recognising the movements of the
1
End-Point Closed-Loop Control End-Point Open-Loop Control
Quadrotor
Target Camera
Views
Camera
Quadrotor
Target Camera
View
Camera
Figure 2: Examples of end-point closed-loop control (left) and end-point open-loop control (right).
designated target. Particularly, the target is identified through image processing with computer vision
techniques based on distinct features such as colours, areas, edges, corners, and centroids. However,
this becomes more difficult with a moving target where variable factors become apparent, which can
possibly include changes in the target movement, colour pattern, and structure or geometry, with
concurrent changes in the background environment and camera tilting orientations.
1.2 MOTIVATION
Due to their growing importance and utility, quadrotors are an important part of the robotics and engineering fields of research with the incorporation of knowledge from mechanics, aviation, electronics, and computer science [4]. Additionally, the incorporation of visual servoing with quadrotors has
become highlighted when signals from global positioning systems (GPS) are unreliable or unavailable in indoor environments [5, 6]. The various methods of implementing visual servoing through
computer vision have been extensively developed and show promising results with regards to target
detection - this is especially evident in systems with end-point closed-loop control, as successfully
demonstrated by Rabah et al. [1], Dunkley [2], Karlsson [6], and Kendall et al. [7] for examples.
However, there are still challenges with regards to implementing visual servoing through end-point
open-loop control with only a single camera, such that a quadrotor is able to achieve optimal moving
target tracking in three-dimensions while maintaining fully autonomous flight without manual inputs


    def setupStabilizerLog(self):
            log_conf = LogConfig("Pitch", 10)
            log_conf.addVariable(LogVariable("stabilizer.pitch", "float"))
            log_conf.addVariable(LogVariable("stabilizer.roll", "float"))
            log_conf.addVariable(LogVariable("stabilizer.thrust", "int32_t"))
            log_conf.addVariable(LogVariable("stabilizer.yaw", "float"))
            self.pitch_log = self.crazyflie.log.create_log_packet(log_conf)

            if self.pitch_log is not None:
                self.pitch_log.dataReceived.add_callback(self.log_pitch_data)
                self.pitch_log.start()
                print("stabilizer.pitch/roll/thrust/yaw now logging")
            else:
                print("stabilizer.pitch/roll/thrust/yaw not found in log TOC")


Pinhole Camera Model Light
Rays Through Optical Centre
Aperture
=
Figure 9: Simplification of a model for a pinhole camera with a lens and imaging sensor.
For simplification while maintaining reasonable accuracy, a pinhole model can be considered for the
camera, where it is assumed that the rays of light only pass through the optical centre of the lens or, in
9
other words, the rays of light not passing through the optical centre of the lens are neglected - this is
equivalent to an infinitesimal point as the aperture [2]. With this model, each ray of light can be seen
to travel in a straight line from the target to the imaging sensor and, for final simplification, the virtual
image plane can be considered with the uninverted projection [2]. Thus, these assumption allow for
the image to be regarded as a graphically equivalent description of the observed environment without
distortions or blurring, which is fairly valid as long as the lens is accurately located at the focal length.
Using the pinhole model and considering the virtual image plane, the associated relationship between
distances in the image can be related to actual distances in three-dimensional space. This is essentially
based on geometrically similar triangles formed by the rays of light between the actual point, projection on the virtual image plane, and optical centre of the lens, as explained in Figure 10 to produce
Equation 9 [2, 6]. As a result, Equation 10 describes the latitude relationships and Equation 11 describes the longitude relationship, where the focal length is a parameter of the camera and the virtual
image coordinates can be found from the image. Thus, the pinhole model can be seen as a first-order
approximation for the mapping of the observed environment. (If the real image plane is considered,
it is either necessary to redefine the directions or use a negative sign with the relationships).
f
z
=
u
x
=
v
y
(9)
(u, v)
(x, y)
u
v
y
x
Principal
Point
z
Lens With


# Only output errors from the logging framework
logging.basicConfig(level=logging.ERROR)


logconf.add_variable('stateEstimate.y', 'float')


There's a limit of 28 chars in total and here are some examples:


    def connected(self, linkURI):
            # IMU & THRUST
            lg = LogConfig("Stabalizer", 50)
            lg.addVariable(LogVariable("stabilizer.roll", "float"))
            lg.addVariable(LogVariable("stabilizer.pitch", "float"))
            lg.addVariable(LogVariable("stabilizer.yaw", "float"))
            lg.addVariable(LogVariable("stabilizer.thrust", "uint16_t"))

            self.log = self.helper.cf.log.create_log_packet(lg)
            if (self.log is not None):
                self.log.dataReceived.add_callback(self._imu_data_signal.emit)
                self.log.error.add_callback(self.loggingError)
                self.log.start()
            else:
                logger.warning("Could not setup logconfiguration after "
                               "connection!")

            # MOTOR


[ ![github](/advisor/images/snyk-icon-github.svg) gtagency / crazyflie-ros /
scripts / crazyflie_node.py  View on Github  ![external](/advisor/images/snyk-
icon-external.svg) ](https://github.com/gtagency/crazyflie-
ros/blob/150da091a7b9562411318fdd0f98e71d58fba445/scripts/crazyflie_node.py#L142)


168 elif correct == True:
169 cv.imshow ("Image Test", frame)
170 cv.waitKey (1)
171 correct = input ("Is the capture displayed correctly? [Y/N] ")
172 if correct == "N":
173 print ("! The capture is not operating correctly.")
174 active = False
175 cv.destroyAllWindows ()
176
68
177 # ----------------------------------------------------------------------------------
178 # Main Project Loop
179 # ----------------------------------------------------------------------------------
180
181 # Begin capture of the video frames and image processing.
182 print ("Beginning capture of the video frames and image processing.")
183 print ("To terminate the capture, press [Q] in the video window.")
184 time_start = time.time ()
185 while active == True:
186
187 # Load the current frame from the source.
188 correct, frame = source.read ()
189
190 # A: No processing with only basic video capture.
191 ##record_minimum (frame, record)
192
193 # B: Minimum grayscale processing with results video capture.
194 ##frame, mode, threshold, morphology = processing_gray (frame)
195 ##frame, contours, centroid_x, centroid_y = target_detection (frame, morphology)
196 ##record_minimum (frame, record)
197
198 # C: Maximum grayscale processing with complete video capture.
199 frame, mode, threshold, morphology = processing_gray (frame)
200 frame, contours, centroid_x, centroid_y = target_detection (frame, morphology)
201 record_maximum (frame, mode, threshold, morphology, record, contours)
202
203 # D: Minimum colour processing with results video capture.
204 ##frame, mode, threshold, morphology = processing_colour (frame)
205 ##frame, contours, centroid_x, centroid_y = target_detection (frame, morphology)
206 ##record_minimum (frame, record)


f
= v
k
f
√
Auv
= v
r
Axy
Auv
(22)
Where x, actual latitude coordinate, m; u, virtual image latitude coordinate, px; z, altitude, m; f , focal
length, px; y, actual longitude coordinate, m; and v, virtual image longitude coordinate, px.
Pinhole Camera Model
Position Estimation
Camera View (Target)
Position
Mapping
u
v
Mirroring Target
Movements
(x,y)
Quadrotor (View)
x
y
Quadrotor Cross-Mode
Coordinate System
x
y
(u,v)
Figure 28: Tracking of the target with the corresponding movement of the Crazyflie, where the desired
altitude will also be varied by manually changing the relative altitude of the camera.
To achieve successful results, the Crazyflie needs to know its starting location. So, for a coordinate
system with the origin chosen to be at the centre of the image (coordinates of (u, v) = (0, 0) are
assigned to the centre pixel, which corresponds to position coordinates of (x, y) = (0, 0)), it will be
required a conversion of the initial coordinate system through manipulation with Equation 23 and
Equation 24, such that the coordinate system is moved and rotated from the default location in the
top-left corner. This coordinate system conversion is detailed in Figure 29.
u = −vi +


* Gaming


Secure your code as it's written. Use Snyk Code to scan source code in minutes
- no build needed - and fix issues immediately.


Board
(Frame)
Battery
Connection
Forward
Indicator
Rotors
Landing Stands
Stand
Rotor
Contorl
Board
Battery
Forward
Landing Stands
Rotors
Rotor
Stand
29mm
Stand
92mm
92mm
Propeller Guard
Single MOCAP Marker
Holder
Figure 17: Photographs showing various isometric views of the assembled Bitcraze Crazyflie 2.1
quadrotor, including the Bitcraze Flow V2 expansion deck mounted at the bottom viewing the ground.
The control board, seen in Figure 18 with the basic schematics and photographs, executes the control setpoints for the motion of the quadrotor with the flashable firmware and on-board sensors. The
firmware for the real-time operating system is based on the FreeRTOS kernel, which handles the
scheduling of processes and calculations for the flight control - FreeRTOS is also open-source [4, 16].
This board has diagonals of 80mm and a mass of approximately 7g using a symmetric form factor
and is equipped with a STM32F405 microcontroller as the main processor (Cortex-M4, 168MHz,
192kB SRAM, and 1MB flash), nRF51822 microcontroller for radio interfacing and power management (Cortex-M0, 32Mhz, 16kB SRAM, and 128kB flash), 2.4GHz ISM band radio with 20dBm
or 100mW low-noise amplifier, 8kB EEPROM, Bluetooth low energy (BLE) module for interfacing
through a smartphone application client, micro-USB Type-B for charging or interfacing over a wired
connection, BMP388 pressure barometer, BMI088 inertial measurement unit containing a three-axis
gyroscope and three-axis accelerometer, and JST-DS connector for connection to the battery [4, 6,
16, 17]. The processor and radio chips exchange data over an internal link protocol using UART as
a physical interface, while the processor and sensors exchange data over I2C, PWM, and SPI - these
interactions are illustrated in Figure 19 [4, 6, 16]. There is also an integrated attitude and heading


####  Product


> **Note 2** : ` get_value() ` and ` set_value() ` can not be called from
> callbacks until the Crazyflie is fully connected. Most notably they can not
> be called from the ` connected ` callback as the parameter values have not
> been downloaded yet. Use the ` fully_connected ` callback to make sure the
> system is ready for parameter use. It is OK to call ` get_value() ` and `
> set_value() ` from the ` fully_connected ` callback.


  * [ NFL  ](https://reddit.com/t/nfl/)
  * [ NBA  ](https://reddit.com/t/nba/)
  * [ Megan Anderson  ](https://reddit.com/t/megan_anderson/)
  * [ Atlanta Hawks  ](https://reddit.com/t/atlanta_hawks/)
  * [ Los Angeles Lakers  ](https://reddit.com/t/los_angeles_lakers/)
  * [ Boston Celtics  ](https://reddit.com/t/boston_celtics/)
  * [ Arsenal F.C.  ](https://reddit.com/t/arsenal_fc/)
  * [ Philadelphia 76ers  ](https://reddit.com/t/philadelphia_76ers/)
  * [ Premier League  ](https://reddit.com/t/premier_league/)
  * [ UFC  ](https://reddit.com/t/ufc/)


        roll    = 0.0
        pitch   = 0.0
        yawrate = 0
        thrust  = 10001
        crazyflie.commander.send_setpoint(roll, pitch, yawrate, thrust)


* [ About Reddit  ](https://www.redditinc.com)
* [ Advertise  ](https://ads.reddit.com?utm_source=web3x_consumer&utm_name=left_nav_cta)
* [ Help  ](https://www.reddithelp.com)
* [ Blog  ](https://redditblog.com/)
* [ Careers  ](https://www.redditinc.com/careers)
* [ Press  ](https://www.redditinc.com/press)


There are a few synchronous wrappers for selected classes that create a
synchronous API by wrapping the asynchronous classes, see the  Synchronous API
section


##  Initiating the link drivers


5V Input PWM Signal
Ground
Chassis
95mm
140mm
Figure 24: Photographs showing the SG90 Micro Servo motor and yawing target assembly.
25
4.4 MOTION CAPTURE FACILITIES
The motion capture system utilises four Qualisys Miqus M1 cameras. These cameras feature a 1Mpx
image sensor with a resolution of 1216px by 800px, high-speed frame rate up to 250fr/s, latitudinal
field of view of 58o
, longitudinal field of view of 40o
, maximum range of 10m using 16mm markers,
and low latency for real-time applications with a camera latency of 2.9ms and system latency of 5ms
(there is also an alternate mode for a latitudinal field of view of 41o
and longitudinal field of view
of 27o
) [28]. The body of the camera has dimensions of 140mm by 84mm by 84mm with 102 NIR
LEDs and infra-red strobe at 850nm [28]. Overall, the system allows for an accuracy anticipated to
be on the order of millimetres. The setup in the motion capture facilities is shown in Figure 25, where
the operating area for the quadrotor to use is approximately 2420mm by 2420mm with a height up to
about 1500mm and black mats have been placed on the floor to cover this area.
Adjustable Tripod Cantilevered Level
Ground Control Laptop Target Mats Crazyflie
Visual Servoing Parts
7200mm
Quadrotor
Area
2420mm
2420mm
7200mm
= Miqus Camera
Motion Capture
Facilities Layout
Qualisys Miqus M1
84mm
140mm
84mm
Wall
Mount


[ __ Improve this page ](https://github.com/bitcraze/crazyflie-lib-
python/edit/master/docs/user-guides/python_api.md)


![npm](/advisor/images/snyk-icon-npm.svg) npm  ![PyPI](/advisor/images/snyk-
icon-pypi.svg) PyPI  ![PyPI](/advisor/images/snyk-icon-golang.svg) Go
![Docker](/advisor/images/snyk-icon-docker.svg) Docker


Top View
Bottom View
ESCs Frame
Figure 55: Photographs showing the top isometric (left) and bottom isometric (right) views of the
larger quadrotor which was manually assembled, including the parts for visual servoing.
86


  * [ About us ](https://snyk.io/about/)
  * [ Customers ](https://snyk.io/customers/)
  * [ Jobs at Snyk ](https://snyk.io/jobs/)
  * [ Legal Terms ](https://snyk.io/policies/terms-of-service/)
  * [ Privacy ](https://snyk.io/policies/privacy/)
  * [ Press Kit ](https://snyk.io/press-kit/)
  * [ Events ](https://snyk.io/events/)
  * [ Secure by Design ](https://snyk.io/security/)


autonomous landing, where the desired location for landing is marked by the target and detected
through target detection with the Crazyflie then moving to this location to land - in essence, this is
already being performed since the Crazyflie is landing at the final location of the target.
Thus, this overall implementation serves as a fairly successful proof of concept for target detection and
tracking in three-dimensions using a single camera and hardware with marginal computational effort.
With expansion, the possible uses include indoor sports arenas, storage warehouses, manufacturing
or assembly workshops, and other applications in buildings requiring target tracking, surveillance, or
monitoring - although it is recommended for the quadrotor to frequently return to the initial origin
to recalibrate and account for any accumulation of error in position. Finally, despite a one-to-one
mapping being pursued where the Crazyflie directly mirrored the target, it is also easily adaptable
to use a different mapping, such as one-to-two, where the Crazyflie moves twice the distance of the
target, or inverse mapping, where the Crazyflie moves in the opposite direction to the target.
8 CONCLUSIONS
An investigation into position-based visual servoing through end-point open-loop control was conducted for estimation of the three-dimensional position and yaw orientation of a moving target using
58
a single camera, where a Crazyflie then tracked this position and yaw orientation autonomously. Additionally, inexpensive and low-end hardware with marginal computational effort was used in the
form of a Raspberry Pi and Pi Camera for lightweight image processing with OpenCV. From the
implementation, the following conclusions can be summarised in relation to the objectives:
• The pinhole camera model is valid and can be successfully implemented using the Pi Camera at
a resolution of 160px by 120px which results in a focal length of approximately 113.5px. There
were no major distortions experienced and compensation was not required.
• Comparing the resolutions of 160px by 120px, 320px by 240px, and 640px by 480px, it was
found that the most satisfactory resolution was 160px by 120px with the capability to generate
setpoints at a frequency of 30.1Hz. Moreover, when considering the minimum processing required, it is evident that the grayscale processing allows for an increased frame rate compared to
colour processing by an average actual percentage difference of 23.2% across the resolutions of
160px by 120px, 320px by 240px, and 640px by 480px (this equivalently corresponds with an
average actual percentage difference of 23.2% in the number of setpoints generated), while also
eliminating more background noise for a more accurate interpretation of the target.
• Through calibrating the Pi Camera based on the pinhole camera model, the three-dimensional
position of the target relative to the camera can be successfully described through computer vision techniques for target detection with the centroid of the target providing latitude and longitude
coordinates and the area of the target providing the altitude of the camera. The yaw orientation
can also be estimated based on the area of the target when using a disproportioned target. Endpoint open-loop visual servoing can then be executed, such that the Crazyflie is controlled to
remotely mirror the translational and rotational motion of the target and maintain the altitude of
the camera by generating setpoints on the Raspberry Pi with transmission over the Crazyradio.
• The real-time effectiveness of the end-point open-loop visual servoing exhibited an average lag
of 0.732s, average position deviations of 83.4mm, minimum yaw orientation deviation of 9.54o
,
and maximum yaw orientation deviation of 9.87o
. However, if the average lag is compensated in
each test, the average position deviation reduces to only 54.0mm, while the minimum and maximum yaw orientation deviations reduce to only 4.76o


def send_setpoint(self, roll, pitch, yawrate, thrust):


Sphere
Light Ray
z
Light Rays
Optical Centre
rd
ra
x or y
Principal Point
Image
Plane
Figure 11: Fish-eye distortion model representing projections of rays of light in a wide field of view
(left). Example of an image from a camera with very significant fish-eye distortion (right) [2].
Similarly, for the common type of camera, a rolling shutter mechanism is used and the image is not
actually captured at a single instant, but rather the imaging sensor sequentially scans through the
field of view either horizontally or vertically over a very short time period [2, 12]. As a result, there
is an opportunity for distortions to arise due to a target moving during the scanning process, which
would result in an inaccurate representation of the target in the captured image through shearing,
smearing, or deforming - although it should be emphasised that the speed of the target needs to be
exceptionally high for a significant effect [2, 12]. These effects are avoided with a global shutter
mechanism which captures the entire field of view in a single instant, but this is usually only available
on high-performance cameras. There are in-depth methods to compensate for rolling shutter, but for
11
the operating speeds in this research, it can confidently be assumed that there will be no distortions
from rolling shutter and the camera essentially performs as though it has a global shutter.
2.5 COMPUTER VISION
Computer vision focusses on gaining high-level understanding from images and basically uses a sequence of consecutive images, where each frame is processed and relevant information is extracted.
The camera used for target detection and tracking may be monocular with a single lens for monoscopic vision or binocular with two lenses for stereoscopic vision. The advantage of a binocular
camera is that three-dimensional information can be extracted more accurately since the environment
can be compared from two vantage points, but this comes at a much higher cost. However, only a
monocular camera is available and, so, only monoscopic vision from a single vantage point will be
considered. Notably, other sensors could actually be used to achieve a similar overall result of target
detection and tracking, with methods based on sound, lasers, optics, or tactile interaction [2].
The image from the camera will require processing in order to detect the target in the image, with
examples of this processing seen in Figure 12. A colour image is described by an array of red,
green, and blue (RGB) 8-bit integers for each pixel between 0 for each minimum and 255 for each
maximum (alternative colour modes are available which offer advantages in different situations, such
as hue, value, and saturation (HVS)). The image can be converted to grayscale where each pixel is
described by a single scalar as an 8-bit integer for each pixel between 0 for black and 255 for white,
which is usually performed to reduce the required computational effort when processing an image


[ ![github](/advisor/images/snyk-icon-github.svg) bitcraze / crazyflie-
clients-python / examples / basiclog.py  View on Github
![external](/advisor/images/snyk-icon-external.svg)
](https://github.com/bitcraze/crazyflie-clients-
python/blob/ceb5196a311161e2896381ed5f773677e9c9c047/examples/basiclog.py#L77)


; and ψq, quadrotor yaw angle, o
.
Table 4: Outline of the lag and angular deviation of the Crazyflie for the yaw orientation in the rotation
tests, with comparison for the deviation after adjustment accounting for the average lag.
Lag Time [s] Original Dψ [
o
] Adjusted Dψ [
o
]
Total Time [s]
Minimum
Average
Maximum
Minimum
Average
Maximum
Minimum
Average
Maximum
Yaw
Trail 1 69.36 0.34 0.45 0.88 -9.54 0.185 9.87 -4.51 0.136 4.46
Trail 2 67.06 0.12 0.38 0.64 -8.32 -1.08 8.95 -4.76 -1.04 3.96
Overall Averages - 0.23 0.415 0.76 -8.94 -0.45 9.42 -4.64 -0.45 4.21
In each of the tests, there may be partial effects on the minimum and maximum values from the target
suddenly doubling-back which artificially fluctuates the lag and position deviation. This should be
minimal during the rotation tests due to the low average lag, but it may be noticeable on the translation
tests although the stationary periods before doubling-back should reduce these effect.
52
Overshoot
Average
Max.
Min.
Average
Deviation
Figure 48: Relevant results against time throughout the first trial of the yaw orientation test.
Average Overshoot
Max.
Average
Min.
Deviation


###  Get the Reddit app


[ ](https://play.google.com/store/apps/details?id=com.reddit.frontpage) [
](https://apps.apple.com/US/app/id1064216828)


open-loop control is not successful, the error will accumulate over time and rapidly diverge. It is also
necessary to discuss closed-loop control for understanding of the flight controller on a quadrotor.
For closed-loop control, the current state of the system is accurately estimated through measurements
from the sensors and this state is compared against the desired setpoint to find an error which needs
to be reduced. The controller then uses this error as an input and outputs the necessary commands
to correct the state of the system and bring it closer to the reference state. The error should be
minimised to converge to zero as quickly as possible while the system remains stable, preferably
15
s(t) = Setpoint Input
r(t) = Reference Signal
f(t) = Control Signal
m(t) = Manipulated Signal
n(t) = Resultant Signal
d(t) = Disturbance Input
c(t) = Plant Output
Input
Element
Control
Element
Actuation
Element
Plant
Element
Disturbance
Element
+
s(t) r(t) f(t) m(t) n(t) c(t)
d(t)
Forward Direction
Open-Loop Control
Figure 14: General schematic of an open-loop control system without feedback.
without oscillations - although it is unlikely for the error to be completely eliminated due to unknown
external disturbances. This continuous closed-loop is represented in the control system shown in
Figure 15, where a signal opposite to the forward direction is referred to as feedback.
Input
Element
Control
Element
Actuation
Element