PIMOD: A Tool for Configuring Single-Board Computer Operating System Images

Computer systems used in the field of humanitarian technology are often based on general-purpose single-board computers, such as Raspberry Pis. While these systems offer great flexibility for developers and users, configuration and deployment either introduces overhead by executing scripts on multiple devices or requires deeper technical understanding when building operating system images for such small computers from scratch. In this paper, we present PIMOD, a software tool for configuring operating system images for single-board computer systems. We propose a simple yet comprehensive configuration language. In a configuration profile, called Pifile, a small set of commands is used to describe the configuration of an operating system image. Virtualization techniques are used during the execution of the profile in order to be distribution and platform independent. Commands can be issued in the guest operating system, providing access to the distribution specific tools, e.g., to configure hardware parameters. The implementation of PIMOD is made public under a free and open source license. PIMOD is evaluated in terms of user benefits, performance compared to on-system configuration, and applicability across different hardware platforms and operating systems.

I. INTRODUCTION
When applying technology in the humanitarian field, it is particularly important that the equipment used is available and that the installations are traceable and maintainable by the user groups. For this reason and for reasons of low cost, single-board computers (SBCs), such as the Raspberry Pi, are often used as the basis for research and especially for practical applications. Various use cases have been posted, where such devices are key enablers for the proposed solutions, be it technical or general education [24], [27], monitoring of technology [25] or monitoring in the health care sector [16], or various communication technologies [1], [2], [20].
When dealing with single-board computers, either for software development or when deploying hardware based on these boards, there is a lack of support for creating operating system (OS) images. There are several cases in which readily configured images and use case specific distributions need to be distributed to users or operators. Devices like a Raspberry Pi are used at home, in applications for multimedia centers or smart homes, but also in challenging applications such as emergency response, environmental monitoring, Internet-of-Things (IoT), and smart city infrastructures.
Single-board platforms that do rely on an operating system regularly use images provided by vendors or third parties. Typically, an image is flashed to an SD card and then booted in a system. Since there is no installation process, the OSes heavily depend on defaults, e.g., username, password, installed software, or on scripts executed on the first boot, e.g., cryptographic parameters or partition size adjustments. Software can then be installed and configurations can be adapted in the running system. While this seems to be convenient for single deployments and fast progress compared classical installation routines, it is not suitable for larger deployments.
When custom software and additional configurations need to be added to an OS image, this can either be achieved by a) creating an OS image from scratch, b) adding scripts to be run on the first boot, or c) create an image from a previously configured system. However, these methods each have their individual drawbacks. Bootstrapping images from scratch requires deep technical understanding. When using first boot scripts, a network connection is required on this first boot. Creating an image from a configured system requires additional steps to revert OS specific first boot configurations.
In this paper, we present PIMOD, a tool for modifying an existing operating system image by executing commands described in a configuration file. In the proposed line-based configuration, a Pifile, i.e., a small set of commands, can be used to describe how an image will be created. These commands are then interpreted by PIMOD and executed accordingly. In our approach, the target image is based on an existing image, it then can be resized, and files from the host system can be included in the image. The special RUN command allows running commands inside the image, so that guest OS specific packet managers and configuration tools can be used. Our approach can easily be used with continuous integration (CI) systems and enable reproducible builds of single-board computer operating system images. The software as described in this paper is released under the free and open source GPL-3.0 license 1 and is available online 2 .
To summarize, we make the following contributions: • We propose a novel method of configuring single-board computer operating system images. • We present a simple yet comprehensive operating system image configuration language. • We provide a free and open source implementation of PIMOD. • We conduct an evaluation of PIMOD in terms of user benefits, performance, and language flexibility. The paper is organized as follows. Section II discusses related work. In Section III, we present requirements and designs decisions. Section IV discusses implementation issues. Section V presents experimental results. Section VI concludes the paper and outlines areas of future work.

II. RELATED WORK
In general, single-board computers (SBCs) use operating system images provided by a) the hardware vendor, b) a third party operating system distributor, or c) a software provider, bundling up its software and dependencies to create a softwarespecific distribution. The images are flashed to a SD card and then booted on a SBC. Since there is no installation process, the OSes heavily depend on defaults and can only be adapted by running them and changing software or configurations.
As a first option, use case specific images can be created by using the tools provided by the SBC vendor, such as pi-gen provided by the Raspberry Pi Foundation [23] or alternative approaches [15]. The tool is designed to create images from scratch and highly adjusted to the specific use case. The open source wireless router distribution OpenWRT features its own build system [8]. This build system is created modularly, and own packages can easily be integrated into the build process. In addition to build images from scratch, the authors created an image builder, specifically targeted for OpenWRT, which installs precompiled packages to an image. However, the image builder is targeted specifically for their operating system and does not work for others. These tools often result in long execution times, since all components are installed or even cross-compiled from scratch.
The second available option consists of tools that add custom scripts to be executed on the devices itself. With PiBakery, a graphical configuration interface for Raspbian is provided, which then creates scripts that are executed on the first boot or on every boot accordingly [9]. Some distributions use the first boot for configuration, e.g., ssh keys in the case of Raspbian, which need to be taken care of manually [22]. These tools have the drawback that, e.g., requested software needs to be installed on every device independently, which results in multiple identical installation processes that may lead to high network overheads. The approach also lacks the possibility of being integrated into Continuous Integration (CI) build systems.
The third widely available option is to use an existing SD card with installed software and a finished configuration. While this is a straightforward approach, it can hardly be automated. To be storage efficient, the copied image, including the partition table and file system, would need to be shrinked, which requires additional tools, such as PiShrink [6].
When dealing with configuration of systems, Docker is a well known virtualization system, designed for dependency management and containerization of applications [18]. Docker features a simple imperative configuration language. A new image is built based on an old image and extended by copying files, altering a Docker specific configuration or running commands inside the container. When using Docker to provide and install software, it is necessary to install Docker and the corresponding software images on the live system. Therefore, this approach does not overcome the problem of multiple installations on individual devices and does not offer a full operating system image.
For configuration management tools like Ansible [12], Saltstack [13], and Puppet [17], the main concept is to have a central server that ships a configuration to every node. The node then adapts the installed system in the manner defined in the configuration. This method has the drawback that it uses more network resources because every single node has to download updates and the installer for itself. Also, the nodes have to be booted so that the first boot scripts are executed. The possibility of configuring and reconfiguring a running system is quite helpful, but we focus on the creation of full OS system images with preinstalled and configured software. Furthermore, the client part has to be installed on every single node, and the master node has to run when a new node should be configured.

III. PIMOD DESIGN
The goal of PIMOD is to facilitate the creation of singleboard computer operating system images with custom software in an easy and reproducible manner and simplify the deployment of such devices. To reach this goal, a simple yet comprehensive configuration language is provided, which is interpreted to modify a system image. The language should be manageable through versioning systems to support the overall goal of reproducibility. A generic configuration language cannot rely on distribution-specific configuration parameters and thus should provide an interface to the distribution's configuration mechanisms. With PIMOD we target Linuxbased operating systems, which are widespread in several communities using SBCs [3], [14].

A. The PIMOD Language
In this section, the PIMOD language used in a Pifile is presented. To reach the goal of easy learnability, the language was inspired by the Dockerfile language, which is already widely known. A Pifile is a line-based document where each line may either contain a) an empty line that may contain white space, b) a comment indicated by a hash symbol, c) a PIMOD command written in caps followed by parameters.
a) FROM <source> [partition]: The required source parameter declares a base image to be found in the local file system, a block device to create an image from, or an URL to be downloaded and extracted. Optionally, the partition number resized and mounted in the further process can be declared. It defaults to the second partition, since most operating systems use one boot as one system partition.
b) TO <destination image>: When a Pifile is executed, the resulting image is written next to the Pifile and named after the respective Pifile. The image destination can be changed by running the TO command. When a block device is specified, the defined source is written to the respective device and further commands are executed directly on the device. c) INPLACE <image>: Using the INPLACE command, an image can be specified on which the commands are executed.
d) PUMP <bytes>: Using the PUMP command, the image is increased by the given amount of bytes, SI prefixes such as k, M, G or, T are supported. e) PATH <location>: By default, the local PATH variable of the host system is used inside the guest system. With this command, it can be extended by another location.
f) RUN <cmd>: Commands specified using the RUN command are executed inside of the operating system image. Note that the operating system of the image is not started, but the run time environment of the target system is modeled. g) INSTALL [mode] <source> <destination>: Installing custom files from the host system is especially useful when custom software is used, or for configuration purposes. The source parameter relates to a file in the host file system, the destination describes a path in the file system of the target system. The optional mode parameter can be useful when installing executables, e.g., cross-compiled software.
h) HOST <cmd>: When a command is specified using the HOST command, it is executed on the local system rather than inside the image. Issuing a local command can especially be useful for preparing configuration files or cross-compiling software, which later is installed to the guest system.
In Listing 1, a Pifile is presented that features all commands of the PIMOD language. Line 1 defines a source image to be found in the local file system and the partition to be resized and mounted as the primary system partition. In Line 2, we declare that the file should be written to an alternative location. Line 5, PUMP 100M, causes the image and the second partition to be increased by 100 mebibytes. In Line 8, a distributionspecific configuration tool is used to enable the serial console available at the target hardware. Line 11 and 12 are used to upgrade the operating system by first updating the sources of the packet manager and then running a distribution update. Note that in Line 12 an environment variable is set by running the command inside a bash shell. Finally, in Line 15, a ssh public key is copied to allow remote login.

B. Linux Support
The Pifile language is designed to be a simple yet comprehensive operating system image configuration language. To reach this goal, some assumptions were made during the design phase. First, to enable fast execution of Pifiles, we do not want to use full system emulation, which would result 1 FROM 2020-05-27-raspios-buster.img 2 2 TO raspbian-buster-upgraded.img 3 4  in booting the guest system kernel. This would have the disadvantage that, e.g., the first boot scripts of the distribution would be executed and other parameters would be initialized, such as cryptographic keys, as discussed in the introduction. We decided to use a QEMU-based system emulation, which allows us to execute Linux ELF binaries across multiple different instruction set architectures [4]. Second, especially mounting the partitions of the image according to the distribution requires specific knowledge, which is hard to generalize. Therefore, we decided to use the file system table defined by the Filesystem Hierarchy Standard, /etc/fstab, which itself is used by many Linux operating systems. Third, the executed binaries are searched according to a path variable, which itself is distribution specific. In PIMOD, this variable is initialized from the host system and can be extended by using the PATH command in a Pifile.

C. Continuous Integration Support
Continuous Integration (CI) is a technique used to overcome integration problems in the development cycle during software engineering. It has been shown that continuous integration improves the productivity of project teams and boosts the integration of external contributions without a reduction in code quality [26]. When combining version management and modern CI systems, every commit of a software under development is automatically integrated into a larger context and tested. PIMOD is designed to be used in combination with CI to create software-specific operating system images in a reproducible and easy manner.

D. Host System Support
Another goal of our approach is extensibility, such that it can be integrated into workflows of the communities using PIMOD. Thus, the configuration language should provide options to interface the host system. One option to enable interfacing in this manner is the already described HOST command. In addition, users should be able to use environment variables defined in the host system and program a control flow.

IV. IMPLEMENTATION
The target of modifying system images and executing code inside of a system image can be achieved best by using system tools. GNU/Linux ships several helpful tools for the individual tasks implemented by PIMOD. To make use of and integrate existing tools in a simple manner, PIMOD was implemented using the Bash programming language [21].
The interpretation of a Pifile is implemented in several stages, since some commands can only be executed after others, as shown in Fig. 1. We define three stages to which the commands are assigned: First, setup, managing the FROM, TO and INPLACE configuration. Second, prepare, handling changes of the image itself, currently only depicting PUMP. Third, chroot, covering all commands interacting with the file system of an image, namely RUN, PATH and INSTALL. During the execution of a Pifile, it is actually executed one time for each stage, only executing the commands belonging to the individual stage. This mechanism insures that source and target are defined before resizing an image, which itself needs to happen before modifying content on the guest.

A. Stage 1: Setup
In the setup stage, the FROM, TO, and INPLACE commands are executed, checking and setting source and destination system images. FROM is not only able to handle local images, but can also download images from a remote location by specifying an HTTP(s) or FTP URL. A local cache minimizes network load and timely overhead. When no destination is defined via TO, it is derived from the name of the Pifile. The stage is concluded by copying the source image to the destination location. In the special case of an identical source and destination path and when using INPLACE, the modifications are executed in place.

B. Stage 2: Prepare
The stage implements the PUMP command. Enlarging an image requires to a) increase the image file size, b) enlarge a partition inside the image, and c) expand the file systems to the size defined by the partition. These subtasks are implemented using the GNU/Linux utilities dd 3 , fdisk 4 , and resize2fs 5 .

C. Stage 3: Chroot
Before executing the RUN and INSTALL commands implemented by this stage, some preparations need to be taken: first, the system image file is associated with a loop device of the host system. Then, the main partition's file system is mounted inside the host system. A working chroot environment requires system interaction, which can easily achieved by importing /dev, /sys, /proc and /dev/pts using a bind mount. The network interfaces are available through the host system kernel, the domain name system configuration is done by bind mounting /etc/resolv.conf. After this step, the statically linked QEMU binaries for the supported platforms are also bind mounted in the chroot environment. Ultimately, additional partitions defined in the file system table of the guest system are mounted. INSTALL is implemented by copying the requested files to the target file system and optionally adjusting the permissions. Running a command inside the target image is easily achievable using chroot: a command is executed in a specified root directory and thus using all binaries, libraries and resources of the mounted image.

D. Continuous Integration
PIMOD is designed to work in combination with continuous integration services. We provide an example integration for two different CI services. Travis CI is a free and open source CI service, which has been shown to be used by a wide variety of software projects [5]. GitHub Actions is a CI service integrated with GitHub, a software hosting platform, widely used for Open Source projects [7]. In both integrations, first the dependencies need to be installed, then the resources, such as a base image are downloaded, and finally the Pifile is executed. The output of PIMOD is presented inside the web interfaces of the individual service. Also, both implementations offer the possibility to release the created image in the form of a downloadable image. Hence, the developers of a use case specific distribution can test their progress locally and tag a specific git commit. This indeed triggers a cloud build using the discussed CI integration and uploads an image to the corresponding releases web page. Our example integration is also available free and open source in a separate repository 6 and can easily be forked and adapted.

V. EVALUATION
In this section, we evaluate PIMOD. First, the benefits of using PIMOD from the view of a sensor network operator are discussed. Then, the performance of the approach is evaluated by comparing execution times of exemplary commands. Third, the generalization of PIMOD is investigated by testing the software with Linux OS images of different distributions and made for different hardware.

A. PIMOD vs. Manual Integration
The first goal of PIMOD is to facilitate the modification of single-board computer operating system images and thereby simplify deployment. We evaluate this goal by discussing the use case of deploying nodes in a sensor network scenario. The deployment of such sensor nodes can be done in different ways. A straightforward approach would be to repeat the deployment on each node manually. First, a chosen operating system is installed, then the operator connects to each node, installs dependencies and software, and configures the system. A more complex yet more efficient way would be to install only one system by the steps presented above and clone this installation to the other systems. In some cases, the operator would need to manually alter some configurations, done during the first boot. This, however, requires the appliance to be connected to a fast Internet connection and checks on every device that the initial scripts did run correctly. The third alternative for the operator would be to build the system image by him-or herself and execute the required steps in the process. This would, however, require a deep understanding of the build process of an operating system, which can take time to understand and which itself takes a certain time to execute. In addition, some software build systems require vast amounts of resources, e.g., TensorFlow, a machine learning toolkit. Building the software requires certain tricks, e.g., swap partitions to allow for larger amounts of memory, which can be circumvented using more powerful hardware. With PIMOD, the operators can create a configuration file in which all the required steps can be described. The resulting Pifile can then be executed either locally resulting in an image to be flashed or integrated into a CI system, e.g., to build and upload an image to a certain online location. The approach can also be used to write the resulting image to a SD card, e.g., to write device-specific configuration files.

B. Performance Evaluation
To evaluate the performance of PIMOD, multiple example commands are executed on a single-board computer and using PIMOD. The experiments presented in Table I  Intel SSDPEKKW512G7, with 512 GB storage and a read and write speed of up to 1775 MB/s and 560 MB/s. All tests were repeated 5 times and averaged. Although the systems themselves are quite different in terms of performance, we try to mimic a build server that might be running in a continuous integration pipeline.
In Table I, the runtimes of various commands on different operating systems are presented, namely OpenWRT and Raspbian Buster. With opkg update and apt-get update, the individual packet managers update the list of available packages. The OpenWRT packet manager introduces a small overhead using PIMOD compared to native execution. Updating the package list consists of downloading the compressed lists and verifying the signature, especially the latter requires many ARM instructions to be simulated on the x86_64 hardware and therefore introduces some overhead. Installing software using opkg is faster on the native hardware compared to PIMOD, which is mostly related to the used gzip compression executed through qemu.
When looking at Raspbian Buster's packet manager apt, PIMOD achieves better runtimes compared to native execution, for both updating the package. One reason for this is the much faster write speed of the desktop system compared to the microSD card of the Raspberry Pi. Since OpenWRT is designed for low footprint systems, such as routers, the packages are relatively small compared to the respective Raspbian packages, based on Debian and designed for desktop class computers. For the wget examples, the downloaded data is not written to disk, and thus only shows the performance of the networking stack. Downloading the file using HTTP shows the superiority of the build server, only requiring a tenth of the time compared to native execution. Since most build servers do have larger than 1 GBit/s Internet connections, the runtimes can be up to 10 times as fast compared to a Raspberry Pi limited to 100 MBit/s. However, when using the same protocol with TLS encryption, the cost of encryption and decryption adds an overhead of 13.6%. In the examples using dd, 100 MB of random data is read using the command dd if=/dev/urandom bs=1M count=100. The computations needed to generate the data happen in the kernel and thus natively on both systems. When discarding the data using of=/dev/null, no heavy computation is required and PIMOD performs better, because the main load happens in the kernel of the host. From the runtimes of the second example presented in the table it becomes evident that for read/write intensive tasks, the disk is the bottleneck; the overhead gain of 91.24% is comparable to the download task. In the last example, data is read from /dev/urandom and encrypted using the OpenSSL AES encryption. The speed advantage of PIMOD fades, since the computationally intensive encryption is performed through the QEMU emulation layer. Nevertheless, our approach is still around 30% faster compared to native execution.

C. Testing Linux Distributions
To test the generalization properties of our approach, two directions are important: hardware and operating system variety. In our tests using the well known Raspberry Pi, we were able to use PIMOD for the widespread Debian-based operating systems, such as Raspbian and Ubuntu Server, as well as OpenWRT, CentOS, Fedora, Kali, and OpenSUSE. Android as well as NixOS did not implement the Filesystem Hierarchy Standard and therefore do not satisfy our assumptions. PIMOD can be adapted to work with both distributions, but a generalization is not easily feasible. In addition, we evaluated different hardware platforms using the operating system recommended and distributed by the vendor. In our tests, we found that PIMOD can be used with the Libre Computer boards ALL-H3-CC, AML-S805X-AC and ROC-RK3328-CC, BananaPi M4, Nvidia Jetson Nano (AI development board), ODROID C2 and N2, OrangePi 3, all models of RaspberryPi, and the RockPi 4. Most of the operating systems distributed by the vendors are Debian-based and include specific changes for the individual boards.

D. PIMOD Language Flexibility
One secondary design goal was to support flexibility for the users of PIMOD. The presented tool is written in the Bash scripting language, a Pifile itself is a script sourced in the individual stages. Thus, in a Pifile all features available in Bash scripts can be used. PIMOD's flexibility features will be discussed by the examples presented in Listing 2, the respective output log is shown in Listing 3. 1) Environment Variables: Environment variables can be helpful to adjust the build to the runtime. In the example presented in Line 5 of Listing 2, a block device to work on can be specified through the environment variable DEVICE. In Line 11, another example is shown, where the ssh public key from the local user is added to the guest system.
2) Redirections: Redirections of input and output streams are a key feature in shell programming and can be helpful in our use cases. The task of adding an ssh key from the host system, presented in Line 11, is implemented by appending to a file and redirecting the input stream to a file in the host system. The HOME variable provides the location of the host system user's home directory. A second example is presented in Lines 14-19, where a network configuration file is written by using a here document. When using documents, the configuration resides inside of the Pifile itself, and is easily understandable. Environment variables are also evaluated inside of here documents, such that device-specific configuration, e.g., a hostname, can be set using this mechanism.
3) Cross-Compilation: For some cases, it is beneficial to compile software on the host system, rather than running the virtualized compiler of the guest system, especially for larger software projects. In the above example of Line 22, the delaytolerant routing software DTN7 [19] is cross-compiled on the host system by using the HOST command and installed to the guest system afterwards.  be reused afterwards. In the example presented above, the Wi-Fi configuration is included from another Pifile, as presented in Line 8. The file is included by using the source command, which executes the commands from the file in the current script. Thus, PIMOD offers a wide variety of interactions with the host systems and can thereby be integrated into user's workflows well.

E. Applications of PIMOD
The goal of the Nature 4.0 project is to develop a modular environmental monitoring system for high-resolution observation of species, habitats, and processes relevant to nature conservation [10]. For this use case, a set of sensor boxes has been designed, each dedicated to sightly different use cases, following the same principal networking functionalities and the same software basis, each based on a Raspberry Pi. In Fig. 3, the hierarchy derived from the requirements of the project are presented. The first box is the rather generic Sensorbox, which comes in different configurations, distinguished by the wired sensor set. It includes a camera, a microphone via the I2S bus, sensors for temperature, humidity, atmospheric pressure, air quality, spectral brightness, and various others. The second box is the RadioTracking configuration. This box is dedicated to read and analyze data flows of four software-defined radio devices per box, which point to the four directions of the compass in order to record signals from bats tagged with special senders [11]. The last configuration is a LoRa gateway receiving and forwarding messages of other sensor applications via the well known long range LoRa protocol. All images are derived from the Raspbian OS, in its Buster version, which is the de facto standard for Raspberry Pi SBCs. The Base Sensorbox image contains all shared software and configuration, e.g., an access option via a VPN and a SSH remote shell, a web server for convenient data retrieval and management in the field, as well as a mesh network configuration for sensor box interconnection. PIMOD is used in the project Nature 4.0 for over a year at the date of writing, and integrates well into the workflow 7 . A software or system developer can get a copy of the project using the git versioning system, including all sub modules, and can build am image from scratch in the matter of minutes. Additional software as well as adjustments can be implemented and a new image can be built and tested locally. After a successful test, the changes are committed, and a new release is built and using the CI integration mentioned above.

VI. CONCLUSION
In this paper. we presented PIMOD, a tool for configuring single-board computer operating system images. A simple yet comprehensive language for configuring these images was proposed, which offers an interface to operating system specific configuration tools. The implementation of the proposed language was presented and the staged execution of a Pifile was explained. PIMOD was evaluated by first discussing the benefits for operators of a sensor network. The performance of the proposed approach was evaluated by comparing runtimes of exemplary commands providing an overview of the overhead. Furthermore, it was demonstrated that PIMOD supports a wide variety of hardware platforms and operating systems.
An interesting area of future work is to extend PIMOD to enable reproducible image builds. However, some of the unique characteristics of PIMOD's implementation prevent hash identical builds. Several commands executed in the guest systems have non-deterministic side effects, such as creating temporary and logging files. Another example are the time stamps contained in the image file system, which would also need to be adjusted accordingly.


Design and Development of K-Band FMCW Radar for Nano-Drone Detection

Nano-drones, are insect-like size drones with a capability of intrusion to provide intelligence and potentially violate secure establishments and public privacy rights. Nano-drones are already an existing technology which is becoming more and more available, portable, affordable and easily operated. As such, they may soon become a plausible defence and security threat. This paper presents the design and development of a K-Band FMCW radar prototype for nano-drone detection. The FMCW radar prototype consists of connectorised components operating at a carrier frequency of 24 GHz and offer high parameter selection flexibility. Experiments have been carried out in order to evaluate the system performance. Results show that a small Arcade PICO Drone Nano Quadcopter (smaller than 5 cm) could be detected, and that its micro-Doppler signature could be extracted from data.

I. INTRODUCTION
The greatest technological transformation towards unmanned aerial vehicle started with the transition from fullsize aircraft to large remotely controlled and unpiloted aerial platforms. Then slowly the size of drones has significantly reduced to provide unmanned solutions that have become more portable, economical and affordable. Drones are no longer restricted to the military, and they have become a technology readily available for civilians, either for industrial or private purposes. A recent study has claimed that advanced research in nano-drones, e.g. smaller versions of drones up to insect-size, might result in a much more vulnerable environment to the society as drones may soon become a perfect tool for stealthy surveillance [1]. Smaller drones are characterised by a low Radar Cross Section (RCS) and, as a result, detecting them with conventional radar becomes a challenging task. Increasing the radar operating frequency (i.e. shorter wavelength) is a possible way to increase the target RCS. However, at the same time this reduces the radar maximum detection range as attenuation due to atmosphere and dissipation losses grow with frequency. As the radar performance is usually evaluated by the ability to detect a specific target at a given range, it is important to compromise between the operating frequency and the detectable range.
One of the techniques that have been deployed in micro-UAV detection is the extraction of micro-Doppler signature. Micro-Doppler signatures are Doppler signatures induced by micro-motions of the target, like the rotation of the propeller blades. Since micro-UAVs can hover or move at a relatively low speed, they are often characterised by a small main body Doppler return. However, small rotating or vibrating parts of micro-UAVs can largely contribute to the micro-Doppler.
A large and growing body of literature has investigated UAV detection and classification by micro-Doppler techniques. In particular, previous studies have focused on mini-UAVs (MUAVs), with typical dimensions in the range between 30 cm to less than 2 m and micro-UAVs (MAVs) with less than 30 cm wingspans [2]. Although there are no standard global dimension of nano-UAVs (NAVs) yet, a program that has been launched to design the NAV have specified the size to be less than 10 cm wingspan [3]. Authors in [4] made one the first attempts in mini-UAVs classification, by using an X-band CW radar to investigate the signatures of a variety of target types including small fixed-wing planes, small helicopters (main rotor size between 71 cm -104 cm) and a quadcopter (blade size 20 cm). In [5], the characteristic features extracted from the signatures of the mini-UAVs measured in [4] were used to investigate how to distinguish drones from birds. A different radar setup was deployed for micro-UAV detection in [6] and [7], whereby the authors developed an S-band (2.4 GHz) multistatic pulsed radar to detect a drone (DJI Phantom Vision 2+) versus birds and to differentiate between loaded and unloaded drones. In [8] and [9], FMCW radars were used to detect the Hexa-copter BumbleBee F820, with a propeller diameter of 38 cm, at Kband and to detect a DJI Phantom Drone Quadcopter at Wband. It was demonstrated that the proposed FMCW radar could provide an extra advantage with respect to detection range due to a frequency modification that enabled measurements of time delays and in return provide range information. Similar research has been carried out in [10] - [13] to investigate the micro-Doppler signatures of different types of drones (from MUAVs to MAVs) and how these could be used to classify the targets accordingly. Together, all these studies provide an important insight into micro-Doppler techniques applied to detection and classification of drones. Despite the work described above, the existing literature lacks of detailed investigations focussing on detection of nanodrones (or the insect-size drone). To the best of our knowledge, the smallest blade sizes been studied in the current literature are 13 cm and 13.5 cm, for blades belonging to DJI Phantom 3 Standard and DJI Mavic Air drones, respectively. Also, the total body size of both these drones is much bigger than the size of their blades. Because there is a large growing interest in pocket-size drone mass production, there is a need to give an account of nano-target detection as well [14].
This paper investigates the possibility of nano-drone detection using a home-brew radar. The radar system is designed and developed, using connectorised off-the-shelf components at 24 GHz. Unlike analogous off-the-shelf integrated radars, the developed K-band FMCW radar prototype offers a high level of parameters selection flexibility and as such allowed us to change key radar waveform design characteristics to analyse and obtain the desired performance. Several experiments have been carried out on an Arcade PICO drone to extract its micro-Doppler signature.

II. RADAR DESIGN MODEL
A periodic signal is continuously transmitted at a carrier frequency that consists of a sequence of linear frequency modulated ramps

2
(1) of amplitude , bandwidth and ramp duration . The transmit signal propagates to the target which produces an echo signal where is the amplitude of the received signal and is the time delay that accounts for the propagation path. FMCW radars that employ a double balanced mixer on receive generate an output signal of amplitude which is obtained by mixing the received signal with a copy of the transmitted signal . The frequency of the signal provides the target range information via a calculation of the time delay . The primary parameter in this design is the slope of the ramp / which, for each range bin, determines the expected beat frequency If an object is moving at a relative radial velocity with respect to the radar, the time delay measured by the n-th ramp is 2 / and the target Doppler shift induced by the movement of the target can be extracted by comparing the phase of the Fourier transform of the beat signal from ramp to ramp.
Our underlying strategy to this design was to minimise the RF components and the cost of the radar prototype. To achieve this, we excluded the use of IQ demodulator components that would have represented an additional cost to the prototype. For this radar design, a target at a range produces a beat signal with a spectrum characterised by a positive and a negative contribution centred at the target range, as shown in Fig 1. The use of IQ demodulators, instead, provides a onesided spectrum and therefore no blind range limitations. Selecting a high frequency bandwidth results in a much wider separation of the two spectrum components as it shifts the resulting beat frequency away from DC and the overlapping zone. This limitation does not affect targets that are farther away from the radar as these generate non-overlapping spectrum components. The minimum requirement in this case is that the range of the target must be higher than the range resolution of the radar. One must also bear in mind that, the greater the distance of the target, the higher the transmitted power required to optimise target detection. Doppler signatures are extracted by a double FFT and a time-frequency analysis. A fast-time FFT is applied ramp to ramp to construct a 2D matrix known as Range-Time Intensity (RTI). Then, a Range-Doppler map (RDM) is structured by applying second FFT for each range bin of the RTI matrix which will then transform the previous slow-time axis into the Doppler information. Lastly, a Short-Time Fourier Transform (STFT) is applied at the selected target range bin from the RDM to extract the micro-Doppler signature from the RTI.

III. PROTOTYPE IMPLEMENTATION
The K-band FMCW radar transmits linear frequency modulated ramps of a user-selectable duration spanning the frequency bandwidth from 24.58 GHz to 25.06 GHz [15]. A linear ramp voltage signal is generated by a Keysight Waveform Generator 33600A (see Fig 2) and fed into a HMC739LP4 Voltage Controlled Oscillator (VCO) to provide the frequency modulated signal at K-band. The VCO is an evaluation board manufactured by Hittite with an integrated chip mounted on the board. The output of the VCO is then passed to a mini-circuits ZC2PD-18263-S+ power divider which splits it into two signals; one is directed to the transmit antenna and the other one is fed to the LO port of a Marki Microwave MM1-1140HS double-balanced mixer and is used as the beating copy of the transmitted signal. Target echoes are collected by the receive antenna and are passed to a HD30172 Low Noise Amplifier (LNA) for signal amplification. The amplified received signal is then flowed to the RF port of the mixer and mixed down with the copy of the transmitted signal available at the LO port of the mixer. The IF product is filtered with a mini-circuits VLF-45+ low pass filter before being digitised with a Handyscope HS5 USB oscilloscope. The RF components of the radar is illustrated in Fig 3. The transmit and receive antennas are identical with a gain of 18 dBi, length 135 mm and aperture 40 x 50 mm. During the experiments both antennas were mounted on a bracket 1.5 m above the floor.
The main advantage of this prototype is that it provides full control of the radar components and the flexibility to choose the radar parameters, and hence the radar performance, on a case to case as desired. The operational frequency range of the VCO, for example, is between 23.8 GHz to 26.8 GHz allowing a wide selection of the frequency span and bandwidth within the system linear response interval.  During the measurements, the drone was tied up to a thin fisher line and hanged to the ceilings from the top and from the bottom to the floor, to help the drone hover in one place and to minimise uncontrollable trajectories. The drone was placed about antenna boresight and approximately 1.5 m above the ground (at the same height as the antennas of the radar prototype). A Radar Absorbent Material (RAM) panel was placed behind the radar prototype and another RAM panel was placed behind the target to limit multipath effects. The target range from the radar was about 3 m. Fig 5 shows a photo of the hanging drone which was then switched on for data collection with the rotating blades. The data was sampled with a 500 kHz sampling frequency and results generated for integration times of 2 s. A first set of measurements were collected with a ramp duration of 1 ms (Case I). Fig 6 shows the range doppler map of the target obtained while the blades were continuously rotating. Because of the presence of the fishing line, and due to the spinning blades, during the measurements the body of the drone moved on a small circular trajectory within the fixed place. The drone is detected at approximately 2.42 m away from the radar. The highest peak signal return corresponds to the main body of the drone which because of the rotational trajectory, was slightly oscillating about zero-Doppler. The exact rotational rate of the blades is unknown, but the range-Doppler map shows maximum Doppler frequency returns around 400 Hz. From these measurements alone, determining the rotational rate of the propeller blade is not obvious but the observed Doppler returns suggest periodicities of about 25,000 RPM. The length of STFT window was 60 ms. Results demonstrate that even smaller size drones, built-in plastic material body can be detectable in an enclosed building. Data of the same target was also collected for a shorter ramp duration of 0.5 ms (Case II). The data was sampled at 500 kHz and results obtained with the same integration time of 2 s. The drone was detected 2.5 m away from the radar. Fig  8 shows the target Doppler returns over a wider Doppler bandwidth and, for this case, the maximum Doppler frequency return is between 500 Hz to 600 Hz. The length of STFT window used to extract micro-Doppler signature was 60 ms. Fig 9 shows the target micro-Doppler signature with the same Doppler resolution over a larger Doppler bandwidth. The blade behaviour is more visible compared to Fig 7. Again, the plot corroborates that, for this case, the maximum Doppler frequency returns were contributing approximately around 500 Hz. From the two cases discussed, it is evident that there is a trade-off in having a shorter ramp duration and a longer ramp duration. One of the key requirements is for the ramp duration to be at least twice the maximum Doppler return of the drone to avoid ambiguous Doppler measurement. This project was undertaken to design, develop and prototyping a low-cost K-band (24 GHz) FMCW radar using the off-the-shelf connectorised components. The radar design offers a high parameter selection flexibility at a frequency range of 24 GHz. Results have demonstrated the workability of the radar by showing that the nano-drone could be detected and its micro-Doppler signature extracted using a STFT.


Guessing State Tracking for Visual Dialogue

The Guesser plays an important role in GuessWhat?! like visual dialogues. It locates the target object in an image supposed by an oracle oneself over a question-answer based dialogue between a Questioner and the Oracle. Most existing guessers make one and only one guess after receiving all question-answer pairs in a dialogue with predefined number of rounds. This paper proposes the guessing state for the guesser, and regards guess as a process with change of guessing state through a dialogue. A guessing state tracking based guess model is therefore proposed. The guessing state is defined as a distribution on candidate objects in the image. A state update algorithm including three modules is given. UoVR updates the representation of the image according to current guessing state, QAEncoder encodes the question-answer pairs, and UoGS updates the guessing state by combining both information from the image and dialogue history. With the guessing state in hand, two loss functions are defined as supervisions for model training. Early supervision brings supervision to guesser at early rounds, and incremental supervision brings monotonicity to the guessing state. Experimental results on GuessWhat?! dataset show that our model significantly outperforms previous models, achieves new state-of-the-art, especially, the success rate of guessing 83.3% is approaching human-level performance 84.4%.

Introduction
Visual dialogue has received increasing attention in recent years. It involves both vision and language processing and interactions between them in a continuous conversation, and brings some new challenging problems. Some different tasks of Visual Dialogue have been proposed, such as Visual Dialog [7], GuessWhat?! [9], GuessWhich [5], Image-Grounded Conversation [17] and Multimodal dialogs [20]. Among them, GuessWhat?! is a typical object guessing game played between a Questioner and an Oracle. Given an image including several objects, the goal of the Ques- tioner is to locate a target object supposed by the oracle oneself at the beginning of a game by asking a series of yes/no questions. The Questioner therefore has two subtasks: one is Question Generator (QGen) that asks questions, the other is Guesser that identifies the target object in the image. The Oracle answers questions with yes or no. The left part of Fig.1 shows a game played by the QGen, Oracle and Guesser. The Guesser, which makes the final decision, is the focus of this paper.
Compared with QGen, relatively less work has been done on Guesser. De Vries et al. [9,24] proposed the first Guesser model that outputs probabilities over objects in the image, where the full dialogue history is encoded by a Long Short-Term Memory (LSTM) [12], an object is represented by its category and spatial position, a dot product and a softmax are used to match between the dialogue history and any object. The object with maximum probability is selected as the answer. If the right object is picked, the game is successful. Some other Guesser models [34,2,22,21] were proposed, which are similar to the model in [9,24] except for encoding of dialogue history and images. Zhao et al. [34,2] introduced memory [25] and attention [6] to the model [9,24], the question-answer (QA) pairs are encoded using two-hops attention. Shekhar et al. [22,21] used a shared encoder in conjunction with the QGen task to process the dialogue history.
Most of existing Guesser models make a guess after fixed rounds of QA, such as [9,24,19] or [34,2,21,22]. It is obvious that different game might need different rounds of question-answer. Some work has therefore been done on choosing when to guess, i.e. make a guess after different rounds of question-answer for different game.
No matter the number of question-answer rounds is fixed or changed in different dialogues, existing Guesser models make one and only one guess after final round of questionanswer, i.e. Guesser is not activated until it reaches the final round of dialogue.
This paper models the Guesser in a different way. We think the Guesser is active through a dialogue. It keeps on updating a guess distribution after each question-answer pair from the beginning, and does not make a final guess until the dialogue reaches a predefined round or it can make a confident guess. For example, as show in Figure 1, a guess distribution is initiated as uniform distribution, i.e. each object has same probability as target object at beginning of the game. After receiving the first pair of question-answer, the guesser updates the guess distribution and continues to update the distribution in following rounds of dialogue. It makes a final guess after predefined 5 rounds of dialogue.
We think that modeling the Guesser as a process instead of a decision in single point provides more chances to not only make much more detailed use of dialogue history but also combine more information for making better guesses. One such information is monotonicity, i.e. a good enough guesser will never reduce the guess probability on the target object by making proper use of each question-answer pair. A good guess either raises the probability of target object in guess distribution when the pair contains new information about target object, or does not change the probability when the pair contains no new information.
This paper proposes a guessing state tracking (GST) based Guesser model to implement the above idea. Guessing state (GS) is at first time introduced into the game. A GS is defined as a distribution on candidate objects. A GST mechanism, which includes three sub-modules, is proposed to updating GS after each question-answer pair from the beginning. The update of GS brings a series of GS.
Two loss functions are designed on making better use of the GS, or the introduce of GS into visual dialogue makes the two new loss functions possible. One is called early supervision loss that tries to lead GS to the target object as early as possible, where ground-truth is used to guide the guesses after each round of question-answer, even the guess after first round where a successful guess is impossible at that time. The other is called incremental supervision loss that try to bring monotonicity mentioned above to the probability of target object in the series of GS.
Experimental results show that the proposed model achieves new state-of-the-art performances in all different settings on GuessWhat?!. To summarize, our contributions are mainly three-fold: • We introduce guessing state into visual dialogue for the first time, and propose a Guessing State Tracking (GST) based Guesser model, a novel mechanism that models the process of guessing state updating over question-answer pairs.
• We introduce two guessing states based supervision losses, early supervision loss and incremental supervision loss, which are effective to model training.
• Our model performs significant better than all previous models, and achieves new state-of-the-art in all different settings on GuessWhat?!. The accuracy of 83.3% approaches the humans level of 84.4%.

Related Work
Guesser models play a very important role in many visual dialogue tasks such as GuessWhat?!. Some different models have been proposed on modeling Guess. The first guesser model was proposed in [9,24]. The model makes a guess after a dialogue between the QGen and the oracle. Each dialogue includes given number of question-answer pairs. All question-answer pairs in a dialogue are encoded sequently by an LSTM into a vector as the representation of the linguistic dialogue. Each candidate object is embedded into a vector by concatenating theirs 512-dimensional category embedding and 8-dimensional spatial embedding. A dot product is then used to calculate similarity between the representation of the dialogue and each candidate object. A softmax classifier over the similarities is used to output final guess.
The second different guesser architecture was employed in [34,2]. Memory [25] and Attention [6] modules are integrated into the first guesser model. It is a two-hops of attention [26] module. They encode each QA-pair into a vector as a fact, which form a memory base of facts. The summary of all object embeddings is used as a key, where the encode of object like [9,24]. They perform two-hops attention and obtain a final linguistic representation. Similarly, a dot product and a softmax are used to output probabilities between the dialogue history and each candidate object.
The recent different guesser model was introduced in [22]. Differs from the first guesser model, they use a shared dialogue history encoder with the QGen. The encode of dialogue history and image are similar to [9,24]. Finally, a dot product and a softmax are used to output object probabilities between dialogue history and each object in the image.
As we can see, all existing guesser models make one and only one guess after a given number of question-answer rounds. They treat guess as a single time decision at the end of a dialogue. Different from them, Guesser model proposed in the paper treats guess as a process, and keeps a guessing state through the process. The guessing state is defined as a distribution over objects. The guessing state will be updated after each round of the dialogue. Furthermore, two new loss functions are proposed for making full use of the guessing state.
Moreover, there are some attention-based methods proposed in similar tasks such as VisDial [7,11,18,14,35,30], which are also suitable for solving guesser task. Deng et al. [10] proposed an accumulated attention (ATT) mechanism for guesser and VisDial task. They view guesser task and VisDial are similar and cast guesser task as a visual grounding task [16,32]. Specially, they compute three kinds of attentions respectively, i.e. query attention, image attention and objects attention, combine them together, refine them circularly and obtain a joint representation. Finally, they output probability between the representation and any object by a softmax function. To better consider the impact of imperfect dialogue history, Yang et al. [31] proposed history-aware co-attention network (HACAN) to encode the QA-pairs, a dot product is used to calculate similarities between history and object, the one with highest similarity is selected as the guess.
There are some methods on model learning. De Vries et al. [9] trained the guesser network by supervised learning. Zhao et al. [34] first considered guesser task as a deep reinforcement learning task [27,4,8], a guess is viewed as an action. Shekhar et al. [22] formulated Guesser and QGen as a multi-task learning problem [3] and trained by cooperative learning (CL) [4].

Model: Guessing State Tracking
The framework of our guessing state tracking (GST) model is illustrated in Fig.2. Three modules are implemented in each round of guessing. There are Update of Visual Representation (UoVR), Question-Answer Encoder (QAEncoder) and Update of Guessing State (UoGS). Where, UoVR updates representation of an image for guesser according to the previous round of guessing state, new visual representation is then combined into QAEncoder for synthesizing information from both visual and linguistic sides up to current round of dialogue for guesser. Finally, UoGS is applied to update the guessing state of guesser. We give details of each module in following sub-sections.

Update of Visual Representation (UoVR)
Following previous work [9,24], candidate objects in an image are represented by their category and spatial features as in Eq.1: where i , it is concatenation of a 512-dimensional category embedding o cate and an 8-dimensional vector o spat of the location in the image. To map the dimension of object embedding to the same as the word embedding, the concatenation passed through an MLP to obtain a d-dimensional vector.
Let π (j) ∈ R m be an accumulative probability distribution over m objects after jth round of dialogue. It is defined as the guessing state and will be updated with the guessing process. At the beginning of a game, π (0) is a uniform distribution.
With the progress of guessing, the visual representation for guesser would update accordingly. Two steps are designed for the update.
The first step is update of representations of objects. Pang and Wang [19] proposed an effective way of representation update for QGen model. We borrow it for our guesser model as shown in Eq.2: where O (j) ∈ R m×d . Second, the element-wise summary of all object repre- where v (j) ∈ R d denotes the updated visual representation especially for guesser after jth round of dialogue.

Question-Answer Encoder (QAEncoder)
For encoding linguistic information in current question with visual representation in hand, each word w (j) i in jth question is concatenated with the new visual representation and fed to a LSTM encoder as shown in Eq.4, The last hidden state of the LSTM is used as question representation h (j) , h (j−1) is used as initial input of the LSTM as shown in Fig.2. h (j) is then concatenated to a (j) which the embedding of the answer for jth question, and passes through an MLP as shown in Eq.5, where h (j) qa ∈ R d synthesizes information from both question and answer up to jth round dialogue for the guesser. It will be used to update guessing state in next module.

Update of Guessing State (UoGS)
Three steps are designed for Guessing State update.
where O (j) qa ∈ R m×d is a set of fused representation which combines information from current question-answer pair and visual objects.
Second, a softmax is used to measure how many changes this round of question-answer brings to the guess as shown in Eq.7,π Where, three vectors, including the fused representation and visual object representation as well as representation of question-answer h (j) qa are concatenated together. We find that this type of symmetric concatenation is an effective operation. d is the dimension of word embedding that used for controlling variance.
Finally, the previous rounds of guessing state π (j−1) is updated according toπ (j) as in Eq.8, where π (j) ∈ R m is the new guessing state after jth round of question-answer, norm is normalization keeping π (j) be a probability distribution.

Stop Questioning
When to stop questioning is also a problem in Guess-What?! like visual dialogue. Most of previous work choose a simple policy, i.e, a QGen model stops questioning after a predefined number of dialogue rounds, and the guess model select an object as the guess.
Our model can implement this policy by making use of π (j) , the guessing state after jth round dialogue. If K is the predefined number, the guesser model will keep on updating π (j) till j = K. The object with the highest probability in π (K) will be then selected as the guess.
A same number of questions are asked for any game under this policy, no matter how different the different games are. The problem of the policy is obvious. On the one hand, the guesser model does not select any object even if it is confident enough about a guess, and make a QGen model keep on asking till K questions are asked. On the other hand, the QGen model cannot ask more questions when K questions are asked even if the guesser model is not confident about any guess at that time. The guesser model must give a guess.
Our model provides a chance to adopt some other policies for stopping questioning. A simple way is to predefine a threshold of confidence. Once the biggest probability in a guessing state is equal to or bigger than the threshold, question answering is stopped, and the guesser model output the object with the biggest probability as the guess. Another way involves the gain of guessing state. Once the information gain from jth state to j+1th state is less than a threshold, the guesser model outputs the object with the biggest probability as the guess.

Early and Incremental Supervision
Besides on stopping questioning, the introduce of guessing states provides another useful information for model training.
Because the guessing states are tracked from the beginning of a dialogue, supervision of correct guess can be employed from early stage, which is called early supervision. Because the guessing states are tracked at each round of a dialogue, the change of guessing state can be also supervised to ensure that the guessing is alone a right way. We call this kind of supervision incremental supervision. Two supervision functions are introduced as follows.
Early Supervision Early supervision tries to maximize the probability of the right object from the beginning of a dialogue, and keep on used up to the penultimate round of the dialogue. It is defined as the summary of a series of cross-entropy between the guessing state after each round of dialogue and the ground-truth object as shown in Eq.9, CrossEntropy(π (j) , y GT ), (9) where y GT is a one-hot vector with 1 in the position of the ground-truth object, J max is the maximum number of rounds.
The cross-entropy at the final round, i.e. CrossEntropy(π (Jmax) , y GT ), we refer to as plain supervision loss (L P S in briefly).
Incremental Supervision Incremental supervision tries to keep the probability of the target object in guessing state increasing or nondecreasing as shown in Eq.10, where π (j) target denotes probability value of the target object in guessing state at jth round of dialogue, c is a parameter controlling input value of log function is valid.

Training
Our model is trained in two stages including supervised and reinforcement learning.
For supervised learning, the guesser network is trained by minimizing the following objective as shown in Eq.11, where α is a balancing parameter. For reinforcement learning, the guesser network is refined by minimizing the negative expected reward in Eq.12, (12) where π θ denotes a policy parameterized by θ which associates a guessing state over actions, e.g., an action corresponds to select an object over m candidate objects. We have three notes on Eq.12.
First, it use the REINFORCE [28,29] without baseline. Second, a reward is 1 if the right object is found at the final guess, and 0 otherwise.
Finally, it is trained only on the self-play successful dialogues, the unsuccessful ones are filtered. The reward 1 is spread uniformly over the guessing action at each round.

Experimental Setup
Dataset GuessWhat?! dataset containing 66k images, about 800k question-answer pairs in 150K games. It is split at random by 70%, 15%, 15% of the games into the training, validation and test set [9,24]. The number of object candidates in each image is between 3 and 20, the average number is around 8.
Baseline models A GuessWhat?! game involves Oracle, QGen and Guesser. Almost all existing work use a same Oracle model [9,24] which will be used in all our experiments. Two different QGen models are used for validating our guesser model. One is the often used model in previous work [24], the other is a new QGen model which achieves new state-of-the-art [19]. Several different existing Guess models are compared with our model. They are guesser [9,24], guesser(MN) [34], GDSE [22], ATT [10] and HA-CAN [31]. The models are first trained in supervised way on the training set, and then, one Guesser and one QGen model are jointly refined by reinforcement learning from self-play with the Oracle model fixed. Specifically, selfgenerated successful games are used to tune the Guesser model, while all self-play games are used to optimize the QGen.
Implementation Details The maximum round J max is set to 5 or 8. The balancing parameter in Eq.11 is set to 0.7, the parameter c in Eq.10 is set to 1.1 (We will discuss the influence of these parameters in ablation study). The size of word embedding is 512, LSTM hidden unit number in all three models are 512. Early stopping is used on the validation set. More details, including the source codes and other materials, will be published in the near future.
We use success rate of guessing for evaluation. Following previous work [9,24], both success rates on NewObject and on NewGame are reported. Results by three inference methods described in [2], including Sampling (S), Greedy (G) and Beam-search (BS, beam size is set to 20) are used on both NewObject and NewGame.
Supervised Learning (SL) We separately train the Guesser and Oracle model for 20 epochs, the QGen for 50 epochs using Adam optimizer [13] with a learning rate of 3e-4 and a batch size of 64.
Cooperative Learning (CL) In [22], the QGen and Guesser are first trained with SL. Guesser is further trained using the self-generated dialogues, then QGen is re-trained on the human dialogues with SL. The two models iteratively trained in this cooperative way.
Reinforcement Learning (RL) We use momentum stochastic gradient descent with a batch size of 64 and learning rate annealing. The base learning rate is 1e-3 and decayed every 25 epochs with exponential rate 0.99. The momentum parameter is set to 0.9. Table 1 reports the success rates of guessing with different combination of QGen and Guesser models with the same Oracle model [9,24] for the game GuessWhat?!. In the first part of table 1, all models are trained in SL way. We can see that no matter which QGen models are used, qgen [24] or VDST [19], our guesser model GST significantly outperforms other guesser models in both 5 and 8 rounds dialogue at all different settings. Specifically, GST achieves a new state-of-the-art of 54.10% and 50.97% on NewObject and NewGame in Greedy way by SL.

Comparison with the state-of-the-art
In the second part of table 1, two combinations trained in CL way are given. Our model is not trained in this way. So we do not have comparison in CL case with the performance of these models are lower than those in RL part.
In the third part of table 1, all QGen and Guesser models are trained by RL. We can see that our GST Guess model combined with VSDT QGen model achieves best perfor-mance in both 5 and 8 rounds dialogue at all different settings. It significantly outperforms other models. For example, it outperforms the best previous model at Sampling (S) setting on NewObject (i.e. guesser(MN) [34] + ISM [1] with 72.1%) by nearly 9 percent, outperforms the best previous model at Greedy (G) setting on NewObject (i.e. guesser(MN) [34] + TPG [34] with 74.3%) by more than 9 percent, outperforms the best previous model in NewObject at Beam-search (BS) setting on NewObject (i.e. guesser [24] + VDST [19] with 71.03%) by more than 12 percent. The same thing happens on NewGame case. That is to say, our model consistently outperforms previous models in all different setting on both NewObject and NewGame. Especially, GST achieves 83.32% success rate on NewObject in Greedy way, which is approaching human performance 84.4%. Specifically, with same QGen, no matter which QGen models are used, qgen [24] or VDST [19], our guesser model GST significantly outperforms other guesser models in both 5 and 8 rounds dialogue at all different settings. Table 2. Classification errors (%) for the guesser models (lower is better).

Model
Train err Val err Test err Max Q's Random [9] 82.9 82.9 82.9 -LSTM [9] 27.9 37.9 38.7 -HRED [9] 32 A previous work [10,31] evaluate Guess model only on training data by measuring the error rate of model. For compare, we also follow the error rate of classification for evaluation. Table 2 reports the results. We can see that our method trained in SL is comparable to the best model. After reinforcement learning, our GST based guesser achieves a new state-of-the-arts of 16.7%, 16.9%, 18.4% and outperforms the compared methods by a large margin. Effect of Individual Supervision In this section, we conduct ablation studies to separate the contributions of su-pervisions: Plain Supervision (PS), Early Supervision (ES) and Incremental Supervision (IS). Table 3 reports the success rate of guessing after supervised learning. Removing ES&PS from the full model, the game success rate significantly drops 11.52 (from 54.10% to 42.58%) and 11.49 (from 50.97% to 39.48%) points on NewObject and NewGame on Greedy case. Removing IS, the success rate drops 0.61 (from 54.10% to 53.49%) and 0.64 (from 50.97% to 50.33%), respectively. It shows that early supervision pair with ES&PS contributes more than incremental supervision.

Ablation Study
We then analyze the impact of individual supervision loss to guessing state after reinforcement learning. We train three GST networks using three different loss, i.e. PS, PS&ES and PS&ES&IS, respectively, and then count the average probability of the ground-truth object in guessing state on all successful games at each round for these cases. Fig.3(a) show the curves of average probability changing with rounds of dialogue for three cases.
In Fig.3(a), as is observed, we have three notes. First, the guess probability is progressively increasing in all three different loss. It demonstrates the effectiveness of the proposed guessing state tracking mechanism, i.e. modeling guess as a process helps to improve guess gradually.
Second, the average probability in the blue line is higher than that in the gray line, it demonstrates the effectiveness of early supervision loss.
Third, the average probability in the red line is better than that in the blue line, it demonstrates that incremental supervision gives further improvement to guess model.
Overall, these results demonstrate the effectiveness of early supervision and incremental supervision. It is the combination of these supervisions that train GST based model efficiently.  Figure 3. a, the average guessing state on the ground-truth object change with an increase in the number of dialogue rounds. b, the error rate at different α settings.
Effect of balancing parameter α in Eq.11 Fig.3(b) shows the error rate of classification changes smoothly with increasing of α in Eq.11 from 0.0 to 1.0. After 0.1, it makes no big difference to the error rate of classification.
Effect of Symmetric Concatenation Table 4 shows the effectiveness of the symmetric concatenation used in Eq.7.  Effect of c in Eq.10 Table 5 shows the error rate on three different c settings. As is observed, c is insensitive to error rate. In our experiment, we set c to 1.1, since it obtains lower error rate in Val err and Test err.

Qualitative Evaluation
In Fig.4, we show four successful dialogues to visualize the guess process. We only plot 4 candidate objects for simplicity in each example. The colorful barcodes represent an initial guessing state π (0) on candidate objects, the detailed numbers represent the subsequent rounds of guessing state.
Taking Fig.4(a) as an example. For an image with several candidate objects (including a target object), Guesser has an initial uniform guess on all candidates, i.e. initial guessing state π (0) . QGen starts a dialogue by asking the first question is it a cow?, Oracle responses with yes. The guesser then updates its initial guessing state π (0) to π (1) , e.g., the probability on the ostrich and tree approaches to zero, while the cow on both sides increases to 0.45 and 0.51 respectively. The guessing state updates in this way as shown in Fig.4(s). As we can see, the probability of the cow on the right side keeps to raise with the process of the dialogue.

Conclusion
The paper proposes a novel guessing state tracking (GST) based model for guesser, which models guess as a process with change of guessing state, instead of making one and only one guess over the dialogue history in all the previous work. To make full use of the guessing state, two losses, i.e. early supervision loss and incremental supervision loss, are introduced. Experiments shows that our GST based guesser significantly outperforms all of the existing methods, and achieves a new strong state-of-the-art performance that is close to humans, the success rate of guessing 83.3% is approaching human-level performance 84.4%.


Chronic Kidney Disease using Machine Learning Techniques

Interminable Kidney Disease (CKD) proposes the realm of kidney chance which may even crumble by means of time and through implying the factors. If it continues finishing all the more dreadful Dialysis is and most desperate conclusive outcomes believable it'd flash off kidney misery (End-Stage Renal Disease). Area of CKD in a starting period should help in filtering by means of the complexities and harm. In the past work portrayal applied are SVM and Naïve Bayes, it happened that the execution time took by methods for Naïve Bayes is irrelevant appeared differently in relation to SVM, confused events are substantially less with SVM that results in less request execution of Naïve Bayes, inferable from gentle exactness distinction. It can be corrected by methods for taking less improvements. Unsuspecting Bayes is a probabilistic classifier a fundamental count by utilizing Bayes Theorem with a prohibitive independence supposition. The artistic creations for the most segment brings around growing symptomatic exactness and decrease commitment time, this is the guideline factor. An undertaking is made to develop a form evaluating CKD data collected from a particular course of action of people. From the model data, recognizing verification should be conceivable. This work has enchanted on developing up a system relying upon gathering procedures: SVM, Naïve Bayes, glomerular filtration rate (GFR) is the best pointer of how well the kidneys are working.CKD has got no cure but it can be treated based on symptoms to reduce complications and

INTRODUCTION
Constant Kidney Disease or (CKD) alludes to a condition where kidneys lose their usefulness after some time.
On the off chance that the harm is serious, it will prompt ENRD. This is called kidney disappointment. On the off chance that kidneys fizzle, the elective like Kidney substitution or Dialysis can be picked. The Changes in the hard and fast you pee. These are huge signs of CKD. The term GFR (Glomerular Filtration Rate) is the goliath and best test to check how well kidney work. The GFR rate decrease in like manner with the age, so if GFR is under 60, by then its is seen as the patient is encountering CKD.

II. METHODOLOGY OF NAÏVE BAYES:
Is data notwithstanding basic data of prior probabilities of the various theories in a. This Classifier goes under the social occasion of probabilistic classifiers.

III. DECISION TREE
A decision tree is a classifier as a tree structure with two sorts of focuses or centers: Decision focus point: Specifies a decision or starter of some trademark, with one branch for everyresult. Leaf focus point: Indicates mentioning of a model. Key Regression is a structure in Machine Learning, more then likely it's a quantifiable assessment technique used to imagine an information worth dependent on earlier impression of an instructive record. As far as possible is in addition called sigmoid cutoff, it accomplishes a S-formed bend, it can take any real respected numbers or something along those lines and is mapped into values 0 or 1.1/(1 + e^-esteem) SVM Support vector machine goes under managed learning and generally, it is described by a detaching hyperplane. SVM is a notable and supportive portrayal for good figurings and tremendous data. SVM has a quick technique to hinder overfitting because they use various features without requiring an exorbitant measure of estimation. In SVM, fortify vectors are depicted and from this, we discover an edge with least length furthermore keeping the requirement so consolidates have remarkable conviction from the plane. The edges stick be 1) practical edge 2) geometrical edge. Occasion based learning is One strategy for understanding assignments of approximating discrete or authentic respected objective cutoff points and KNN is one this sort of assortment. In KNN not the comparable different depictions, we store preparing Examples, when a test model is given we discover the closest neighbor.

IV. WORK DONE
The dataset is isolated into two sub datasets both containing 14 qualities. Preparing information: preparing dataset is gotten from principle dataset and it contains 300 out of 400 records in primary dataset of CKD. Testing information: testing dataset is of 100 out of 400 records from principle CKD dataset. iv) Classifiers: Decision Tree: Decision tree is a graphical portrayal of explicit choice circumstance that utilized for prescient model, principle segment of choice tree includes root, hubs, and fanning choice. Choice tree is utilized in those territory of the clinical science where various parameters engaged with characterization of informational index. Since choice tree is most compressive methodology among all AI calculation. These plainly reflect significant highlights in the informational index. They can likewise produce the most influencing highlight in the mass of populace. Choice tree depends on entropy and Information gain obviously implies the significance of dataset. Disadvantage of choice tree is that it experiences two significant issues overfitting and it depends on avaricious strategy. overfitting occurred because of choice tree split dataset adjusted to hub it implies it need a great deal of hubs to part information, this issue is settled by J48 clarified in dependent on ravenous technique lead to less ideal tree, if dynamic methodology is taken it lead to exponential number of tree which isn't feasible [6]. Bolster Vector Machine: "Bolster Vector Machine" (SVM) is an administered AI calculation which can be utilized for both grouping and relapse difficulties. Be that as it may, it is for the most part utilized in grouping issues. Right now, plot every datum thing as a point in n-dimensional space (where n is number of highlights you have) with the estimation of each component being the estimation of a specific organize. At that point, we perform arrangement by finding the hyperplane that separate the two classes well indeed .Support Vectors are essentially the directions of individual perception. Bolster Vector Machine is a boondocks which best isolates the two classes (hyper-plane/line) [6].

V. FUTURE SCOPE
There is a need of an illustrative choice really consistent framework to improve master's judgment in picking a steady state of kidney affliction's patients. We proposed a structure that will help stars with treating their kidney disorder's patients. This framework was made dependent on AI procedures. The strategy of this assessment is partitioned into two significant stages: gathering appearing and structure movement. From strategy delineating, we discovered guidelines and model in the social occasion of kidney infirmity. Also, we executed that rules into a structure utilizing programming language python. This assessment choice truly consistent framework has some essential highlights, for example, Read File, Classification, Error Rate, and Diagnosis. This framework is relied on to help the master in picking a consistent state of kidney ailment patients with needed precision.

VI. RESULTS
By using this model, can fit anticipate whether the individual is having CKD or not. This relies upon the features of the patient like RC, WC, BGR, etc. By envisioning this can say that the individual is influenced or not. In case he/she is influenced we can offer some direction for extra methodology. Arbitrary FOREST: Random boondocks is an idea of the general system of unpredictable decision forests that are a social event learning technique for request, backslide and various endeavors, that work by building countless decision trees at planning time and yielding the class that is the strategy for the classes (game plan) or mean estimate (backslide) of the individual trees, the precision of the computation. KNN: In plan affirmation, the k-Nearest Neighbors estimation (or k-NN for short) is a non-parametric procedure used for portrayal and backslide. In the two cases, the information contains the k closest getting ready models in the part space, the precision of the figuring is 95.8%.
FIG.5-normal to incraese protien level SVM: In AI, support-vector machines (SVMs, furthermore support-vector frameworks) are directed learning models with related learning estimations that separate data used for plan and backslide assessment, the precision of the figuring is 95.8%.

VII. CONCLUSION
Right now, have done figure models by utilizing a fragment of the AI estimations, for example, SVM, Logistic Regression, Naïve Bayes, Decision Tree and through the prepared model, we imagine whether the people. From the outcomes, it will when all is said in done be seen that Naïve Bayes Classification check is having more exactness than SVM,Regression and Clustering estimations.