Gesture/Pattern Recognition Without Camera : TOF Sensor & SPAD Array
Introduction: Gesture/Pattern Recognition Without Camera : TOF Sensor & SPAD Array
UPDATE : 12/11/2021 : New video Online :
Grumpy Hedgehog is a connected device that allows you to read and interact with movements. Its LCD screen displays the Hedgehog's expressions according to his understanding.
We can easily create our own movements and patterns to transmit actions through several communication channels (USB HID, Wifi, IR...) or simply use its relay and servomotor.
Supplies
* Arduino MKR 1010 (any Arduino work for most parts)
* VL53L1X TOF Board
* TFT LCD 1.44
* SG90 Servo Motor
* TSOP2138 Receiver
* DS18B20 Temperature
* Single Relay Board
Github REPO
Step 1: What Problem Are We Going to Solve?
Today we are used to have physical contact with the objects that surround us. This applies to switches, touchscreens, remote controls... The current crisis forces us to question these habits and to enter a new digital era where physical contact with certain objects must be reduced to a minimum.
While speech recognition allows a complete approach for transmitting information, it is still quite complex for machines and humans to grasp. A child is usually able to understand and imitate many signs long before his or her first words.
Why not use signs to create simple interactions that can convey more information than ON/OFF?
Once you send a movement or a pattern, GrumpyHedgehog can :
* Control a computer by sending keyboard command (movie pause, volume...)
* Track the number of person in a shop on your smartphone
* Safely unlock a computer
* Send encrypted informations to a server
* Control light or any power components
* Send IR command to your TV
* Move a servo motor
Step 2: Key Elements
In addition to offering a modular solution with its different interfaces Grumpy Hedgehog is particularly distinguished in 4 aspects:
* It uses a VL53L1X Time Of Flight Sensor allowing complex motion recognition with 16x16 SPAD sensing array, it can run on most micro-controllers such as SAMD21 (MKR Wifi 1010) and Atmega328p (NANO/UNO board).
* The second aspect is the management of the LCD screen which is optimised to store in FLASH and display a moving image. The first tests on my Arduino UNO are promising!
* The third differentiating aspect is its design. The challenge here is to create a decorative object that you will gladly leave on display in your living room or office.
* It uses the ATECC508 secure element to create a strong security layer based on Public/Private Key to authenticates node IDs, ensures the integrity of messages, and supports key agreement to create session keys for message encryption. (This part isn't fully finished)
So you will understand that the GrumpyHedgehog with his thirst for learning, his gestural communication, his cute look and his grumpy character will quickly become your new companion.
Step 3: How It Work
By default GH detects three movements: Left, Right and Up. 152
The left and right movements allow you to send commands depending on the actual mode:
To switch from one mode to another, simply leave your hand within 15cm of the sensors for 3 seconds. You don't have to use all the mode, just comment the modes you are not using on the top of Arduino Program.
In addition to the different modes, you can always use Pattern Recognition with the upward movement the upward. GH will detect on a 4x4 pixel grid the shape of the object in front of it. The Arduino program allows to easily add Pattern and match it to an action.
To add security, a Pattern and a Keyboard Password can be saved in the secure element ATECC508. With the right configuration, it will be impossible to recover the Password unless the Pattern match. See the ATECC508 section for more informations. (This part isn't finished yet)
Step 4: Time of Flight Sensor and Algorithm
The use of VL53L1X TOF sensor is really interesting. But what makes it different from many competing proximity sensing technologies? (Ultrasonic, Sharp IR angle, LIDAR...)
Beyond the fact that it allows a detection up to 4 meters, a 50Hz refresh rate and a good immunity to light and surfaces reflexion. What I was particularly interested in is called SPAD Array (Single Photon Avalanche Diode) and ROI (Region Of Interrest).
This is because VL53L1X not only detects proximity, it can also target this information to a detection field. 152