Answers.Body: <p>Firstly, before we commence I will recommend that you refer to similar questions on the network i.e. <a href="https://stackoverflow.com/questions/6499880/ios-gesture-recognition-utilizing-accelerometer-and-gyroscope">https://stackoverflow.com/questions/6499880/ios-gesture-recognition-utilizing-accelerometer-and-gyroscope</a> and <a href="https://stackoverflow.com/questions/6368618/store-orientation-to-an-array-and-compare">https://stackoverflow.com/questions/6368618/store-orientation-to-an-array-and-compare</a></p>
  <p>Your problem can be divided into three parts.</p>
  <p>How to gather sensor data.</p>
  <p>How to use the gathered data to train a model</p>
  <p>How to use the trained model to make a prediction.</p>
  <p>A modern smartphone contains around six sensors packed into one device. To implement your application I recommend that you use raw sensor data from either the gyroscope or the accelerometer.</p>
  <p>On the android platform, you can access these sensors and acquire the raw sensor data by using the Android sensor framework. The Android sensor framework is part of the android.hardware package.</p>
  <blockquote>
  <p>To capture the raw sensor data, take regular samples (&gt; 20 Hz) and save the maximum values of x, y and z in an array each (to recognize in all 3 planes). If you want the gesture to be 5 seconds long, keep 100 samples (at 20 Hz). Then analyze if any of the three arrays has values which change sinusoidally. If it does, you've got a gesture.</p>
  <p>You could store these values into an array, if the user is in 'record mode'. And when the user tries to replicate that movement, the model could predict the replicated movement array from the recorded one's. The thing is, how can you compare the arrays in a smart way? (Randy M 2011)</p>
  </blockquote>
  <p>This leads us to the next step which is applying ML.</p>
  <p>To train your model, you can choose to either train the model on the cloud or train it locally on the device. In most cases the problem of training a model on a mobile device is about computational limitations. Machine Learning algorithms running on a mobile device need to be carefully designed and implemented since most mobile devices have weak processors and small RAM's.</p>
  <p>Its quite a challenge to squeeze a large neural network (NN) into the small RAM's that smartphones's have since NN's require that the model is fully loaded into memory. In many cases it is advisable to slim the model down and set some weights near zero to zero. (Chomba B 2017)</p>
  <p>Incase you decide to utilize the cloud. Your mobile app is required to simply send an HTTPS request to a cloud web service along with the required raw sensor data, and within seconds the service replies with the prediction results. Though there are several cloud services i.e. Microsoft Azure Cognitive Services, Clarifai and Google Cloud Cognition that you can leverage to host the server side of your application, personally I recommend that you consider using reality.ai which is specifically an AI tool for engineers working with signals and sensors.</p>
  <p>The next step will be to select an appropriate algorithm to be used in classifying the gesture. Here you can employ either Logistic Regression, Support Vector Machines, Random Forest or Extremely Randomized Trees depending on your app's use case. In order to train your model we then provide the learning algorithm with 'labeled examples'. The ML algorithm then extracts the features and constructs a mathematical model that can accurately describe gestures i.e. roll, pan from raw sensor data.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-31T00:55:18.647Z
Answers.Text: Thanks for the good description. I know about the Android sensors myself, just not the ML part. I wish there were some libraries or SDK that I could use for recognition experiments.
Answers.UserDisplayName: Mary
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-12-30T11:55:14.710Z
Answers.LastActivityDate: 2017-12-30T11:55:14.710Z
Answers.LastEditDate: 2020-06-17T09:57:20.710Z
Answers.UserDisplayName: Seth Simba
Body: <p>For instance, one task would be to detect if with an Android phone in hand, I'm panning the camera toward a circle shape in a 2D space.</p>
  
  <p>What is the best technology set and embedded application approach can be used for these types of motion independent pattern recognition tasks?</p>
  
  <p><a href="https://i.sstatic.net/36kBFs.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/36kBFs.png" alt="enter image description here"></a>
  <a href="https://i.sstatic.net/XIAIms.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/XIAIms.jpg" alt="enter image description here"></a></p>
  
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-30T18:36:14.453Z
Comments.Text: i can't find the article, but i recall there was a result where researchers were able to guess the smartphone pin by doing ML on the accelerometer / gyro. sounds like something you'd be interested in
Comments.UserDisplayName: k.c. sayz 'k.c sayz'
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-31T00:50:52.833Z
Comments.Text: @k.c.sayz'k.csayz' yes that would be something very related if you have it.
Comments.UserDisplayName: Mary
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-31T18:27:53.113Z
Comments.Text: @TinaJ it took me 3 words in google to search it up. you should do more research before asking a question like that (including this very question)
Comments.UserDisplayName: k.c. sayz 'k.c sayz'
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2017-12-31T22:05:49.007Z
Comments.Text: I'm not interested in papers per se. More like SDK or APIs or something.
Comments.UserDisplayName: Mary
Comments.ContentLicense: CC BY-SA 3.0
Comments.CreationDate: 2018-01-01T18:40:34.007Z
Comments.Text: @TinaJ you should first analyse the architecture of the hardware(android device),very well.I see there are some questions to ask more in android community,as well.
Comments.UserDisplayName: quintumnia
Tags: <neural-networks><machine-learning><pattern-recognition><embedded-design>
Title: Use mobile device camera for moving pattern recognition
UserDisplayName: Mary
ThreadText: # Use mobile device camera for moving pattern recognition
  
  (Asked by: username_0 on 2017-12-29)
  
  For instance, one task would be to detect if with an Android phone in hand, I'm panning the camera toward a circle shape in a 2D space.
  
  What is the best technology set and embedded application approach can be used for these types of motion independent pattern recognition tasks?
  
  [![enter image description here](https://i.sstatic.net/36kBFs.png)](https://i.sstatic.net/36kBFs.png)
  [![enter image description here](https://i.sstatic.net/XIAIms.jpg)](https://i.sstatic.net/XIAIms.jpg)
  
  **Tags:** neural-networks, machine-learning, pattern-recognition, embedded-design
  > **Comments:**
  > **username_1**: i can't find the article, but i recall there was a result where researchers were able to guess the smartphone pin by doing ML on the accelerometer / gyro. sounds like something you'd be interested in  
  > **username_0**: @k.c.sayz'k.csayz' yes that would be something very related if you have it.  
  > **username_1**: @TinaJ it took me 3 words in google to search it up. you should do more research before asking a question like that (including this very question)  
  > **username_0**: I'm not interested in papers per se. More like SDK or APIs or something.  
  > **username_2**: @TinaJ you should first analyse the architecture of the hardware(android device),very well.I see there are some questions to ask more in android community,as well.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2017-12-30. Score: 3
  
  Firstly, before we commence I will recommend that you refer to similar questions on the network i.e. <https://stackoverflow.com/questions/6499880/ios-gesture-recognition-utilizing-accelerometer-and-gyroscope> and <https://stackoverflow.com/questions/6368618/store-orientation-to-an-array-and-compare>
  
  Your problem can be divided into three parts.
  
  How to gather sensor data.
  
  How to use the gathered data to train a model
  
  How to use the trained model to make a prediction.
  
  A modern smartphone contains around six sensors packed into one device. To implement your application I recommend that you use raw sensor data from either the gyroscope or the accelerometer.
  
  On the android platform, you can access these sensors and acquire the raw sensor data by using the Android sensor framework. The Android sensor framework is part of the android.hardware package.
  
  > To capture the raw sensor data, take regular samples (> 20 Hz) and save the maximum values of x, y and z in an array each (to recognize in all 3 planes). If you want the gesture to be 5 seconds long, keep 100 samples (at 20 Hz). Then analyze if any of the three arrays has values which change sinusoidally. If it does, you've got a gesture.
  >
  > You could store these values into an array, if the user is in 'record mode'. And when the user tries to replicate that movement, the model could predict the replicated movement array from the recorded one's. The thing is, how can you compare the arrays in a smart way? (Randy M 2011)
  
  This leads us to the next step which is applying ML.
  
  To train your model, you can choose to either train the model on the cloud or train it locally on the device. In most cases the problem of training a model on a mobile device is about computational limitations. Machine Learning algorithms running on a mobile device need to be carefully designed and implemented since most mobile devices have weak processors and small RAM's.
  
  Its quite a challenge to squeeze a large neural network (NN) into the small RAM's that smartphones's have since NN's require that the model is fully loaded into memory. In many cases it is advisable to slim the model down and set some weights near zero to zero. (Chomba B 2017)
  
  Incase you decide to utilize the cloud. Your mobile app is required to simply send an HTTPS request to a cloud web service along with the required raw sensor data, and within seconds the service replies with the prediction results. Though there are several cloud services i.e. Microsoft Azure Cognitive Services, Clarifai and Google Cloud Cognition that you can leverage to host the server side of your application, personally I recommend that you consider using reality.ai which is specifically an AI tool for engineers working with signals and sensors.
  
  The next step will be to select an appropriate algorithm to be used in classifying the gesture. Here you can employ either Logistic Regression, Support Vector Machines, Random Forest or Extremely Randomized Trees depending on your app's use case. In order to train your model we then provide the learning algorithm with 'labeled examples'. The ML algorithm then extracts the features and constructs a mathematical model that can accurately describe gestures i.e. roll, pan from raw sensor data.
  
  > **Comments:**
  > **username_0**: Thanks for the good description. I know about the Android sensors myself, just not the ML part. I wish there were some libraries or SDK that I could use for recognition experiments.  
  
  ---
  
  