Navigation of a Freely Walking Fruit Fly in Infinite Space Using a Transparent Omnidirectional Locomotion Compensator (TOLC)
Animal behavior is complex and hard to understand because it is affected by various factors, such as the external environment. Recently, an automated system and robot have been integrated into biological experiments for a better understanding of animal behaviors . Automated robotic tools allow researchers to observe animal behavior in controlled conditions by manipulating the animal or environment. A fruit fly, Drosophila melanogaster , is one of the most popular animal models to study behavior in the laboratory. The most common method to observe behavior in a walking fruit fly is using a custom-designed chamber with a limited spatial area . To overcome the spatial limitation, simple mechanical devices have been implemented to alter the chamber’s position to induce behavior in a fruit fly, such as negative geotaxis . Brain imaging in a freely walking fruit fly has been acquired within a specialized arena in the limited space . However, because the animal behaves in the confined chamber, these types of experiments have been necessarily limited. Other studies demonstrated behavioral observation with a tethered fruit fly . A head-fixed fly walks on top of the air suspended sphere and its behavior is observed by measuring the rotation of the floating ball, which enables a fruit fly to walk in the virtual infinite space. The optical brain imaging with the tethered fly which walks on a ball allows observing behavior and brain activity simultaneously . However, physical restriction by tethering could hinder the natural behavior of a fruit fly due to the potential risk of injury  even though the tethering may be inevitable for brain imaging. Virtual reality has been integrated with these systems to create a realistic environment . Experiments with tethered insects have been carried out for other insects such as ants  and crickets . To eliminate the risk of injury, locomotion compensators with a sphere have been utilized for various insects such as a fruit fly , honeybee , a pill bug , and a moth . Even though the locomotion compensators track behavior in insects without tethering, it is still challenging to integrate additional apparatus such as a microscope, due to the spatial limitation around the device. Even though the behavior of a fruit fly has been successfully investigated for a long time, a behavioral study of a completely mobile fruit fly remains challenging due to the spatial limitation and the complexity of the external environment. Because animal behavior is affected by various factors in the external environment, it is hard to observe animal behavior realistically. In this paper, we developed the Transparent Omnidirectional Locomotion Compensator (TOLC), which compensates movements of a freely walking fruit fly and measures its behavior without tethering and spatial limitation. The TOLC compensates not only a position but also the heading of a freely walking fruit fly. The inverted imaging system of the TOLC secures a space for additional experimental apparatus. We also demonstrated the practical functionality of the TOLC via phototaxis behavioral experiments in Drosophila . Because observation without tethering prevents a fruit fly from damages or injuries, the TOLC will offer a unique opportunity to investigate longitudinal studies of a wide range of behavior in an unrestricted walking Drosophila . 

2.1. Overview of Transparent Omnidirectional Locomotion Compensator (TOLC) The overall schematic drawing of the TOLC is illustrated in a. The design of the TOLC is similar to that of a balancing robot, a mobile robot that navigates with a single spherical ball only . The design of the TOLC is an inverted version of the balancing robot. While a fruit fly is walking on top of the ball, the TOLC compensates a fly’s motion by rotating a transparent sphere (Precision Plastic ball Company, Franklin Park, IL, USA, 14179) using three omnidirectional wheels (Nexus Robot, Hongkong, China, 100 mm diameter). The size of the sphere is 4 inches in diameter and is located on top of three omnidirectional wheels illustrated in a. The omnidirectional wheel is designed to achieve continuous contact with a sphere using alternated passive rollers , which creates a nearly continuous rotation of the sphere while translational motions are minimized. Three omnidirectional wheels were placed equally at every 120° and oriented at 40° with respect to the xy -plane to support the sphere. The omnidirectional wheels are directly attached to the DC servomotors (Robotis, Lake Forest, CA, USA, MX-64AT) to minimize a mechanical backlash. The near-infrared (NIR) camera underneath the transparent sphere records a motion of a walking fruit fly while light-emitting diodes (LEDs) illuminate a fly motion from the bottom of the sphere. Two optical encoders (Logitech, Newark, CA, USA, G502 HERO) were placed at the mid-plane of the sphere to measure the rotations of the sphere in the x -, y - and z -axis. To prevent a fruit fly from flying, we created a fly chamber on top of the sphere with a limited height (2 mm) using the cover glass (Brain Research Laboratory, Waban, MA, USA, 4550-1D) to let a fly walk in the chamber. Because the fruit fly is kept at the top of the sphere most of the time while tracking, the height of the chamber can be maintained during the experiment. b,c shows the overall assembly drawing of the TOLC. d,e shows a picture of the real system and the walking fly in the chamber of the TOLC, respectively. 2.2. Behavior Imaging System Because a fruit fly is walking on top of the transparent sphere, the behavior camera underneath the sphere is able to capture a fly motion, which is similar to the inverted microscope. The advantage of this inverted imaging system is the secured space around and above the behavior chamber where an additional apparatus can be integrated for various future experiments with an external stimulus such as olfactory, vision, thermal stimuli. In addition, we utilized the 850 nm NIR LEDs (Osram, Wilmington, MA, USA, SFH 4655-Z) to minimize visual distraction for a walking fruit fly. The arrays of the NIR LEDs are located around the sphere as illustrated in . The NIR LEDs illuminate a walking fruit fly with an oblique angle from the bottom. b shows a side view of the simulation of ray tracing. c shows detailed simulation results at the behavior chamber where a fruit fly walks. Because the rays diverge laterally while the refracted light by a walking fly is captured by the camera sensor, this creates a high contrast dark-field image. The NIR camera (FLIR, Wilsonville, OR, USA, CM3-U3-31S4M-CS) with the lens (Tamron, NY, USA, 12VM1040ASIR) and the bandpass filter (Thorlab, Newton, NJ, USA, FB850-40) are located on the bottom of the transparent sphere to image behavior of a single fly. To minimize the motion blur, the exposure time was set to less than 1.0 ms with the synchronized LED illumination and the images were captured at 200 Hz. The sample NIR image is shown in a, whose field of view is 16.4 mm by 9.66 mm (640 by 376 pixels) with a pixel distance of 25.7 μm/pixel. To detect the position and orientation of a walking fruit fly, the captured image is processed in real-time with the following step. First, we applied the Gaussian filter on the image and binarized the filtered image with the threshold that keeps the same fly size. Then, we find the centroid and orientation of the fly object from the binary image. The configuration of a fly in the camera coordinate is defined by p cam = [ x cam , y cam , ϕ cam ], where ϕ is the orientation of the fly’s heading with respect to the x -axis in the image shown in b. The imaging system includes the transparent ball as a spherical ball lens, which may cause image distortion. To measure the image distortion, we captured the image of the concave pattern, which makes full contact with the surface of the sphere using the stereolithography (SLA) 3D printer (Creality, Shenzhen, China LD-002R). a shows the captured images of the patterns, in which the distance between patterns is 1 mm in both the x - and y -axis. The yellow dots are the centroids of the patterns in the captured image. We evaluated the distortion with those centroids , and b shows the undistorted image of the pattern and the red dots are the centroids of the pattern. The dotted circle in a,b shows this 4 mm radius circle from the image center. At the center of the image, the differences between the centroids of the patterns in the original (yellow dots) and the undistorted image (red dots) are negligible. However, those at the corner of the image are noticeable. The result distortion in b is the barrel (or negative) distortion, which is typical in a zoom lens. To quantify the distortion, we defined the distortion as follows: (1) Distortion  = ( H * − H ) / H × 100 where the H and H * are the distances of the point on the image from the center in the undistorted image and the original image, respectively. c shows the distortion rate of the imaging system. The distortion at the radial distance from the center of 4 mm is −0.32%, which equals 12.8 μm or 0.5 pixels in the image. At the corner of the image (the radial distance from the center of 9.58 mm), the distortion is 1.89%, which equals 7.0 pixel or 181 μm. We assumed that there is no image distortion because the image distortion is not significant (<1 pixel) inside of the 4 mm circle where a freely walking fly is mostly staying (less than 2 mm error range for 99.3% of experiment time, see the discussion section) during tracking. 2.3. System Modeling a,b show the kinematics of the TOLC . We assumed that the transparent sphere rotates only without translational motion. Let θ sp denotes the angles of sphere rotations in x -, y -, and z -axis, and θ w represents the angles of three omnidirectional wheel rotations. The angular velocity ω sp (= θ ˙ s p ) is related to the angular velocity of three omnidirectional wheels ω w (= θ ˙ w ) as (2) ω s p = M ω w where M is the matrix, which is identified based on the configuration of the TOLC in . The matrix M is expressed as (3) M = r w r s p [ sin α cos β 1 sin α cos β 2 sin α cos β 3 − sin α sin β 1 − sin α sin β 2 − sin α sin β 3 cos α cos α cos α ] where r w is a radius of a wheel, r sp is a radius of a sphere, α is the oriented angle of omnidirectional wheels with respect to the xy -plane, and β is the angle between omnidirectional wheels. We set α = 40°, and β = [0, π/3, 2π/3]. The radius of the omnidirectional wheels r w is 50 mm, and the radius of the transparent sphere r sp is 50.8 mm. The equation of motions of the TOLC, described in , is as follows: (4) I θ ¨ s p + D θ ˙ s p = U s p where I is the moment of inertia of the sphere, D is the damping coefficient, and U sp = [ u x , u y , u ϕ ] is the input. The control input U sp is converted to control input of three omnidirectional wheels U using the matrix M shown in Equation (3) as follows: (5) U s p = M U For the tracking control, the position error of a fly e fly = [ e x , e y , e ϕ ] is defined as a distance between a fly position p cam from the center of the image p c = [ x c , x c , ϕ c ] as follows. (6) e f l y = p c a m − p c which is also aligned with the top plane of the sphere shown in a. The error e fly in image coordinate then converted to the error e sp = [Δ θ x , Δ θ y , Δ θ z ] = [ e y / r sp , − e x / r sp , e ϕ ] in the sphere coordinate, which is the amount of the rotation of a sphere required to compensate for the current position error. Then we computed the control input U sp = [ u x , u y , u z ] from on the error e sp using the proportional-integral-derivative (PID) control. The overall process is repeated for every image, and the average computational time, including image processing and computing control inputs, takes less than 2 ms, which is sufficient to run the tracking system at 200 Hz. 2.4. Tracking Path Generation A travel path of a walking fly is reconstructed based on a fly position p cam in the camera coordinate and a position of the sphere p sp in the virtual xy -plane. A virtual sphere position p sp is defined as [ x sp , y sp , ϕ sp ], which shows a path of the top point of the sphere, where a fly is walking. The optical encoders measure the rotation changes of the sphere Δ θ sp = [Δ θ x , Δ θ y , Δ θ z ] in the x -, y - and z -axis and then converted them into the position and orientation changes in the virtual plane Δ p sp = [Δ x sp , Δ y sp , Δ ϕ sp ] = [ r sp Δ θ y , − r sp Δ θ x , Δ θ z ]. The virtual path of the sphere p sp is updated by adding the change of the virtual position into the current virtual sphere position as follows. (7) p s p , i + 1 = p s p , i + R Δ p s p = p s p , i + [ cos ( ϕ s p , i ) − sin ( ϕ s p , i ) 0 sin ( ϕ s p , i ) cos ( ϕ s p , i ) 0 0 0 1 ] Δ p s p where p sp,i and p sp,i+ 1 are the virtual sphere positions in the current and next step respectively. The matrix R is the rotational matrix with the current heading of the sphere ϕ sp,i in the virtual coordinate. The virtual fly position p fly = [ x fly , y fly , ϕ fly ] is computed by adding the virtual sphere position p sp and a fly position p cam using Equation (8). The virtual path of a walking fly is a sequence of p fly , which shows a travel trajectory of a walking fly in the infinite space. (8) p f l y = p s p + [ cos ( ϕ s p ) − sin ( ϕ s p ) 0 sin ( ϕ s p ) cos ( ϕ s p ) 0 0 0 1 ] p c a m 

The overall schematic drawing of the TOLC is illustrated in a. The design of the TOLC is similar to that of a balancing robot, a mobile robot that navigates with a single spherical ball only . The design of the TOLC is an inverted version of the balancing robot. While a fruit fly is walking on top of the ball, the TOLC compensates a fly’s motion by rotating a transparent sphere (Precision Plastic ball Company, Franklin Park, IL, USA, 14179) using three omnidirectional wheels (Nexus Robot, Hongkong, China, 100 mm diameter). The size of the sphere is 4 inches in diameter and is located on top of three omnidirectional wheels illustrated in a. The omnidirectional wheel is designed to achieve continuous contact with a sphere using alternated passive rollers , which creates a nearly continuous rotation of the sphere while translational motions are minimized. Three omnidirectional wheels were placed equally at every 120° and oriented at 40° with respect to the xy -plane to support the sphere. The omnidirectional wheels are directly attached to the DC servomotors (Robotis, Lake Forest, CA, USA, MX-64AT) to minimize a mechanical backlash. The near-infrared (NIR) camera underneath the transparent sphere records a motion of a walking fruit fly while light-emitting diodes (LEDs) illuminate a fly motion from the bottom of the sphere. Two optical encoders (Logitech, Newark, CA, USA, G502 HERO) were placed at the mid-plane of the sphere to measure the rotations of the sphere in the x -, y - and z -axis. To prevent a fruit fly from flying, we created a fly chamber on top of the sphere with a limited height (2 mm) using the cover glass (Brain Research Laboratory, Waban, MA, USA, 4550-1D) to let a fly walk in the chamber. Because the fruit fly is kept at the top of the sphere most of the time while tracking, the height of the chamber can be maintained during the experiment. b,c shows the overall assembly drawing of the TOLC. d,e shows a picture of the real system and the walking fly in the chamber of the TOLC, respectively. 

Because a fruit fly is walking on top of the transparent sphere, the behavior camera underneath the sphere is able to capture a fly motion, which is similar to the inverted microscope. The advantage of this inverted imaging system is the secured space around and above the behavior chamber where an additional apparatus can be integrated for various future experiments with an external stimulus such as olfactory, vision, thermal stimuli. In addition, we utilized the 850 nm NIR LEDs (Osram, Wilmington, MA, USA, SFH 4655-Z) to minimize visual distraction for a walking fruit fly. The arrays of the NIR LEDs are located around the sphere as illustrated in . The NIR LEDs illuminate a walking fruit fly with an oblique angle from the bottom. b shows a side view of the simulation of ray tracing. c shows detailed simulation results at the behavior chamber where a fruit fly walks. Because the rays diverge laterally while the refracted light by a walking fly is captured by the camera sensor, this creates a high contrast dark-field image. The NIR camera (FLIR, Wilsonville, OR, USA, CM3-U3-31S4M-CS) with the lens (Tamron, NY, USA, 12VM1040ASIR) and the bandpass filter (Thorlab, Newton, NJ, USA, FB850-40) are located on the bottom of the transparent sphere to image behavior of a single fly. To minimize the motion blur, the exposure time was set to less than 1.0 ms with the synchronized LED illumination and the images were captured at 200 Hz. The sample NIR image is shown in a, whose field of view is 16.4 mm by 9.66 mm (640 by 376 pixels) with a pixel distance of 25.7 μm/pixel. To detect the position and orientation of a walking fruit fly, the captured image is processed in real-time with the following step. First, we applied the Gaussian filter on the image and binarized the filtered image with the threshold that keeps the same fly size. Then, we find the centroid and orientation of the fly object from the binary image. The configuration of a fly in the camera coordinate is defined by p cam = [ x cam , y cam , ϕ cam ], where ϕ is the orientation of the fly’s heading with respect to the x -axis in the image shown in b. The imaging system includes the transparent ball as a spherical ball lens, which may cause image distortion. To measure the image distortion, we captured the image of the concave pattern, which makes full contact with the surface of the sphere using the stereolithography (SLA) 3D printer (Creality, Shenzhen, China LD-002R). a shows the captured images of the patterns, in which the distance between patterns is 1 mm in both the x - and y -axis. The yellow dots are the centroids of the patterns in the captured image. We evaluated the distortion with those centroids , and b shows the undistorted image of the pattern and the red dots are the centroids of the pattern. The dotted circle in a,b shows this 4 mm radius circle from the image center. At the center of the image, the differences between the centroids of the patterns in the original (yellow dots) and the undistorted image (red dots) are negligible. However, those at the corner of the image are noticeable. The result distortion in b is the barrel (or negative) distortion, which is typical in a zoom lens. To quantify the distortion, we defined the distortion as follows: (1) Distortion  = ( H * − H ) / H × 100 where the H and H * are the distances of the point on the image from the center in the undistorted image and the original image, respectively. c shows the distortion rate of the imaging system. The distortion at the radial distance from the center of 4 mm is −0.32%, which equals 12.8 μm or 0.5 pixels in the image. At the corner of the image (the radial distance from the center of 9.58 mm), the distortion is 1.89%, which equals 7.0 pixel or 181 μm. We assumed that there is no image distortion because the image distortion is not significant (<1 pixel) inside of the 4 mm circle where a freely walking fly is mostly staying (less than 2 mm error range for 99.3% of experiment time, see the discussion section) during tracking. 

a,b show the kinematics of the TOLC . We assumed that the transparent sphere rotates only without translational motion. Let θ sp denotes the angles of sphere rotations in x -, y -, and z -axis, and θ w represents the angles of three omnidirectional wheel rotations. The angular velocity ω sp (= θ ˙ s p ) is related to the angular velocity of three omnidirectional wheels ω w (= θ ˙ w ) as (2) ω s p = M ω w where M is the matrix, which is identified based on the configuration of the TOLC in . The matrix M is expressed as (3) M = r w r s p [ sin α cos β 1 sin α cos β 2 sin α cos β 3 − sin α sin β 1 − sin α sin β 2 − sin α sin β 3 cos α cos α cos α ] where r w is a radius of a wheel, r sp is a radius of a sphere, α is the oriented angle of omnidirectional wheels with respect to the xy -plane, and β is the angle between omnidirectional wheels. We set α = 40°, and β = [0, π/3, 2π/3]. The radius of the omnidirectional wheels r w is 50 mm, and the radius of the transparent sphere r sp is 50.8 mm. The equation of motions of the TOLC, described in , is as follows: (4) I θ ¨ s p + D θ ˙ s p = U s p where I is the moment of inertia of the sphere, D is the damping coefficient, and U sp = [ u x , u y , u ϕ ] is the input. The control input U sp is converted to control input of three omnidirectional wheels U using the matrix M shown in Equation (3) as follows: (5) U s p = M U For the tracking control, the position error of a fly e fly = [ e x , e y , e ϕ ] is defined as a distance between a fly position p cam from the center of the image p c = [ x c , x c , ϕ c ] as follows. (6) e f l y = p c a m − p c which is also aligned with the top plane of the sphere shown in a. The error e fly in image coordinate then converted to the error e sp = [Δ θ x , Δ θ y , Δ θ z ] = [ e y / r sp , − e x / r sp , e ϕ ] in the sphere coordinate, which is the amount of the rotation of a sphere required to compensate for the current position error. Then we computed the control input U sp = [ u x , u y , u z ] from on the error e sp using the proportional-integral-derivative (PID) control. The overall process is repeated for every image, and the average computational time, including image processing and computing control inputs, takes less than 2 ms, which is sufficient to run the tracking system at 200 Hz. 

A travel path of a walking fly is reconstructed based on a fly position p cam in the camera coordinate and a position of the sphere p sp in the virtual xy -plane. A virtual sphere position p sp is defined as [ x sp , y sp , ϕ sp ], which shows a path of the top point of the sphere, where a fly is walking. The optical encoders measure the rotation changes of the sphere Δ θ sp = [Δ θ x , Δ θ y , Δ θ z ] in the x -, y - and z -axis and then converted them into the position and orientation changes in the virtual plane Δ p sp = [Δ x sp , Δ y sp , Δ ϕ sp ] = [ r sp Δ θ y , − r sp Δ θ x , Δ θ z ]. The virtual path of the sphere p sp is updated by adding the change of the virtual position into the current virtual sphere position as follows. (7) p s p , i + 1 = p s p , i + R Δ p s p = p s p , i + [ cos ( ϕ s p , i ) − sin ( ϕ s p , i ) 0 sin ( ϕ s p , i ) cos ( ϕ s p , i ) 0 0 0 1 ] Δ p s p where p sp,i and p sp,i+ 1 are the virtual sphere positions in the current and next step respectively. The matrix R is the rotational matrix with the current heading of the sphere ϕ sp,i in the virtual coordinate. The virtual fly position p fly = [ x fly , y fly , ϕ fly ] is computed by adding the virtual sphere position p sp and a fly position p cam using Equation (8). The virtual path of a walking fly is a sequence of p fly , which shows a travel trajectory of a walking fly in the infinite space. (8) p f l y = p s p + [ cos ( ϕ s p ) − sin ( ϕ s p ) 0 sin ( ϕ s p ) cos ( ϕ s p ) 0 0 0 1 ] p c a m 

3.1. Drosophila Melanogaster We obtained wild-type Canton S from the Drosophila stock center (Bloomington, IN, USA) and used them for our experiments in this paper. All flies were raised in standard food (Genesee Scientific, San Diego, CA, USA, 66-117) bottles or vials at 25 °C in a 12-h dark–light cycle. The flies were collected from the vial when they are between 20 to 40 days old and transferred to the device under CO 2 anesthesia. A fruit fly normally begins to move within a few seconds after transfer and the tracking device begins to track a fruit fly and compensate its motion immediately. Flies were left for five minutes on the device in order to habituate them before collecting the experimental data. All the experiments were performed in a dark enclosed box. The light source for the phototaxis experiment was white LEDs with a light intensity of 0.09 mW/cm 2 measured using the optical power meter (Thorlabs, Newton, NJ, USA, PM100D). 3.2. Step Response to the Control Input shows the response of the angular velocity of the sphere in the x -, y - and z -axis. These step responses were measured when the input is (a) U sp = [1, 0, 0], (b) U sp = [0, 1, 0] and (c) U sp = [0, 0, 1], respectively. The output angular velocity ω sp of the sphere was measured using optical encoders. To remove the noise, we applied a low-pass Gaussian filter to the output and averaged multiple measurements. In , the final angular velocity of all three responses is reached at around 100 ms. The final angular velocity in the x - and y -axis in a,b are faster than that in the z -axis in c even though the magnitudes of the step inputs are the same. In a, there are still some transient responses of ω y and ω z even though those become zero in steady-state response. The same transient responses of angular velocities in other axes are found as well in b,c. Because two or three motors should be actuated simultaneously to apply the step input U sp as described in Equation (5), the different transient responses of each motor can cause transient responses in other axes. 3.3. Tracking of Unstimulated Behavior To demonstrate the performance of the developed TOLC, a walking fruit fly was tracked on the device without any external stimulus in the dark chamber. The locomotion of the freely walking fruit fly was measured for five minutes. The regenerated trajectory in the virtual coordinate shows the overall path, whose total travel distance of a fly was 957 mm ( a). The position and orientation during the experiment in b show that the fly walks and stops spontaneously on the TOLC. c shows the trajectory information of the single bout illustrated in the red inset box in a. During this bout, the fly moved forward and turned right (clockwise), and the tracking system compensated its error to maintain the fruit fly position in the field of view. The NIR images of the tracked fly at each time point is shown in d. Once the fly walked, the TOLC system began to compensate its position and orientation error and followed behind the fly ( e–g) because our feedback control system relies on the error only. There are some fluctuations of the heading of the fruit fly in g because the body of a fruit fly oscillates when it walks. shows the error e fly during a single bout movement in d. The distance error in a shows that the error increases once the fly moves, and the error is kept around 0.7 mm after the TOLC begin to cancel the error. When the fly accelerated at around 0.7 s, the error in the x -axis increased, but it decreased back after the system compensated the error. In b, the error e x is normally larger than e y because a fly mostly walks forward and the TOLC controls a heading angle toward the positive direction of the x -axis in the image coordinate. Thus, an error normally increases more along its longitudinal direction ( e x ) than its lateral direction ( e y ) during motion. In c, the heading error increased when a fly turned at around 1.1 s, but the error decreased after the system compensated the error. At the end of the movement, the position and heading error decreased to zero. 3.4. Positive Phototaxis of Drosophila Normally, a fruit fly moves toward the light, called positive phototaxis . We observed live behavior during positive phototaxis in a freely walking fruit fly to demonstrate the potential use of the TOLC. In a dark chamber, LEDs emit broadband white light either on the left or right side of the fly walking direction as illustrated in a. The location of LEDs is 183 mm apart from the fly behavior chamber and maintained because the TOLC maintains a heading of the walking fruit fly toward the same direction. Before the experiment, the fruit fly is placed on the TOLC in the dark chamber for five minutes. Then, the LEDs were turned on and toggled between the left and right sides every 30 s. b shows the heading angles ϕ fly for 180 s. The fly turned and moved to the right (clockwise) for the first 30 s because the LEDs on the right side were on while those on the left side were off. After 30 s, the left LEDs turned on while the right LEDs were off. The fruit fly switched its direction to the left (counterclockwise) where when LEDs light toggled. During the experiment in , the total travel distance is 1430 mm with an average speed of 12.38 mm/s, which is measured when it is walking only. This experimental data clearly shows the positive phototactic movement of an adult fruit fly. 

We obtained wild-type Canton S from the Drosophila stock center (Bloomington, IN, USA) and used them for our experiments in this paper. All flies were raised in standard food (Genesee Scientific, San Diego, CA, USA, 66-117) bottles or vials at 25 °C in a 12-h dark–light cycle. The flies were collected from the vial when they are between 20 to 40 days old and transferred to the device under CO 2 anesthesia. A fruit fly normally begins to move within a few seconds after transfer and the tracking device begins to track a fruit fly and compensate its motion immediately. Flies were left for five minutes on the device in order to habituate them before collecting the experimental data. All the experiments were performed in a dark enclosed box. The light source for the phototaxis experiment was white LEDs with a light intensity of 0.09 mW/cm 2 measured using the optical power meter (Thorlabs, Newton, NJ, USA, PM100D). 

shows the response of the angular velocity of the sphere in the x -, y - and z -axis. These step responses were measured when the input is (a) U sp = [1, 0, 0], (b) U sp = [0, 1, 0] and (c) U sp = [0, 0, 1], respectively. The output angular velocity ω sp of the sphere was measured using optical encoders. To remove the noise, we applied a low-pass Gaussian filter to the output and averaged multiple measurements. In , the final angular velocity of all three responses is reached at around 100 ms. The final angular velocity in the x - and y -axis in a,b are faster than that in the z -axis in c even though the magnitudes of the step inputs are the same. In a, there are still some transient responses of ω y and ω z even though those become zero in steady-state response. The same transient responses of angular velocities in other axes are found as well in b,c. Because two or three motors should be actuated simultaneously to apply the step input U sp as described in Equation (5), the different transient responses of each motor can cause transient responses in other axes. 

To demonstrate the performance of the developed TOLC, a walking fruit fly was tracked on the device without any external stimulus in the dark chamber. The locomotion of the freely walking fruit fly was measured for five minutes. The regenerated trajectory in the virtual coordinate shows the overall path, whose total travel distance of a fly was 957 mm ( a). The position and orientation during the experiment in b show that the fly walks and stops spontaneously on the TOLC. c shows the trajectory information of the single bout illustrated in the red inset box in a. During this bout, the fly moved forward and turned right (clockwise), and the tracking system compensated its error to maintain the fruit fly position in the field of view. The NIR images of the tracked fly at each time point is shown in d. Once the fly walked, the TOLC system began to compensate its position and orientation error and followed behind the fly ( e–g) because our feedback control system relies on the error only. There are some fluctuations of the heading of the fruit fly in g because the body of a fruit fly oscillates when it walks. shows the error e fly during a single bout movement in d. The distance error in a shows that the error increases once the fly moves, and the error is kept around 0.7 mm after the TOLC begin to cancel the error. When the fly accelerated at around 0.7 s, the error in the x -axis increased, but it decreased back after the system compensated the error. In b, the error e x is normally larger than e y because a fly mostly walks forward and the TOLC controls a heading angle toward the positive direction of the x -axis in the image coordinate. Thus, an error normally increases more along its longitudinal direction ( e x ) than its lateral direction ( e y ) during motion. In c, the heading error increased when a fly turned at around 1.1 s, but the error decreased after the system compensated the error. At the end of the movement, the position and heading error decreased to zero. 

Normally, a fruit fly moves toward the light, called positive phototaxis . We observed live behavior during positive phototaxis in a freely walking fruit fly to demonstrate the potential use of the TOLC. In a dark chamber, LEDs emit broadband white light either on the left or right side of the fly walking direction as illustrated in a. The location of LEDs is 183 mm apart from the fly behavior chamber and maintained because the TOLC maintains a heading of the walking fruit fly toward the same direction. Before the experiment, the fruit fly is placed on the TOLC in the dark chamber for five minutes. Then, the LEDs were turned on and toggled between the left and right sides every 30 s. b shows the heading angles ϕ fly for 180 s. The fly turned and moved to the right (clockwise) for the first 30 s because the LEDs on the right side were on while those on the left side were off. After 30 s, the left LEDs turned on while the right LEDs were off. The fruit fly switched its direction to the left (counterclockwise) where when LEDs light toggled. During the experiment in , the total travel distance is 1430 mm with an average speed of 12.38 mm/s, which is measured when it is walking only. This experimental data clearly shows the positive phototactic movement of an adult fruit fly. 

The range of the average velocity of a walking fruit fly on the TOLC is from 10 to 40 mm/s, where it is a similar range of the walking velocity in the previous studies . The overall average walking velocity is 14.2 ± 4.20 mm/s, which is measured from 270 min of data across 20 flies in total. We measured the mobility, the ratio between bout (walking) and rest (interbout), and it shows that a fruit fly moves actively 50.4 ± 17.13% of the time on the TOLC. We assumed that a fly is at rest when the position change is less than one pixel which is equal to 5.08 mm/s to eliminate the noise. a,b show the relationship between the distance error ( e x 2 + e y 2 ) 1/2 and the speed of a walking fruit fly on the TOLC. The distance error increases when a fly moves faster ( b). However, the distance error is mostly less than 1.0 mm regardless of the velocity of a walking fly. The distribution in a shows that a fly is normally walking less than 20 mm/s, and the TOLC maintains its position within a 1.0 mm distance error. c shows the cumulative probability of distance error of walking fruit flies. In c,d, the error data during entire experiments (blue) is higher than that while a fruit fly is in motion (orange) because error becomes almost zero when a fruit fly stops between walking bouts. We found that the distance error is less than 1 mm for 90.3% of the experiment time and less than 1.25 mm for 95.9% of the experiment time. While the fruit fly is in motion, the position error was less than 1 mm for 82.4% of the experiment time and less than 1.25 mm for 92.4% of the experiment time. Even though we exclude the data when a fruit fly is at rest, the system still keeps a position error less than 2 mm for more than 99.3% of the time. Similarly, d shows the cumulative probability of heading error e ϕ of walking fruit fly. The heading error was maintained less than 5° for 80.2% of the experiment time and less than 8° for 90.7% of the experiment time. While a fruit fly is in motion, the heading error was maintained less than 5° for 68.8% of the experiment time and less than 8° 85.0% of the experiment time. 

Here, we demonstrate the utility of the transparent omnidirectional locomotion compensator (TOLC) for tracking behavior in a freely walking fruit fly. The inverted imaging system with the transparent sphere secures the space around and above the chamber, where an additional apparatus such as virtual reality, argument reality, and the optical microscope can be integrated into the TOLC system. In the TOLC system, a fruit fly walks on the transparent sphere while the sphere rotates in the opposite direction of the movement to cancel out its motion. Because three omnidirectional wheels and servomotors control rotations of the sphere, the TOLC system can compensate not only the position but also the heading of a fly simultaneously. The results have demonstrated that a fly can walk freely at 14.2 mm/s speed on average on the TOLC system, and the ratio of active walking is greater than 50% of the time. The TOLC system maintains the position and heading error of a freely walking fly less than 1 mm for 90.3% of the time and 5° for 80.2% of the time, respectively. An advanced control method such as Model Predictive Control  can improve the tracking performance of the TOLC system in the future. In addition, the phototaxis behavior of a freely walking fruit fly has been demonstrated to validate the potential utilization of the TOLC system in various behavioral experiments. The TOLC system allows us to observe a freely walking fly without physical tethering. Thus, there is no potential injury during the experiment due to tethering, which is ideal for long-term research for multiple days and weeks. Additionally, we remain future work to improve the tracking performance of the TOLC system and integrate it with other apparatus such as an optical microscope for internal organ and brain imaging while behaving. 

