Computer Vision in nutshell is defined as “making computers see and understand things the way we humans do”. This seems easy to understand but it is really complex.

If you really want to grasp the importance and use case of Computer Vision then do watch the video. See yourself how Amazon is using this technology to build next generation shopping malls.

Pretty amazing, isn’t it?

So let’s just learn and understand about Computer Vision and how to get started with it using NodeJS.

Learning Computer Vision

It’s just not possible to explain the complete field of Computer Vision in one article or probably even in series of articles. Let’s learn the basics enough to go ahead with development.

You can, of course, take courses to learn more about it from the experts at sites like EDX.

There is an open source organization working towards computer vision called OpenCV. This library official supports C, C++, Python, and Java. There is no official module for Node.js but some awesome people work together to create OpenCV module called “node-opencv”. We will use it in the upcoming section.

In this article we will cover the following:

OpenCV Installation.

OpenCV Node.js module.

Building live face detector.

Let’s begin.

OpenCV Installation

You need to install OpenCV library in your computer. If you are using Ubuntu then these commands will install the OpenCV. I have tested it on Ubuntu 16.04.

First, we need to install the dependencies required by OpenCV.

COMMANDS

sudo apt-get install build-essential

sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev

sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev

Now we can install OpenCV using the following command.

COMMANDS

sudo apt-get install libopencv-dev

However, you can also install it using the latest code shipped by OpenCV team. Refer the official installation document here.

You can also install it in Windows by following this article.

And for Mac, you can either build OpenCV from source or install it using brew. I’ll mention the brew version here.

COMMANDS

brew tap homebrew/science

brew install opencv

Node-OpenCV Module

OpenCV officially does not provide a Node.js driver. However, there is alternative available on NPM made by Peter Braden and team. This module is still in development and does not cover all of the OpenCV API’s.

As of now, the latest build on Github is not merged with NPM module of node-opencv hence if you directly install from NPM you may face building issues.

Install the module directly from Github using following command.

npm install peterbraden/node-opencv

Let’s start coding!

Building Live WebCam Face Detector

Before we start detecting the faces in live webcam feed, let’s detect faces from images first.

So how come we detect faces from images? OpenCV provides us various classifiers which you can use to detect faces, eyes, cars etc. These classifiers, however, is a simple one and not trained one using machine learning, so we may expect approximately 80 percent (rough estimate) of accuracy in detecting faces.

Here is the image of my friends from one of the traditional party at the office.

Let’s write a code to detect faces. Here is the piece of code which reads the file from the disk and generates the output image.

face-detector.js

var cv = require ( 'opencv' ) ;



cv. readImage ( "./friends.jpg" , function ( err , im ) {

if ( err ) throw err ;

if ( im. width ( ) < 1 || im. height ( ) < 1 ) throw new Error ( 'Image has no size' ) ;



im. detectObject ( cv. FACE_CASCADE , { } , function ( err , faces ) {

if ( err ) throw err ;



for ( var i = 0 ; i < faces. length ; i ++ ) {

var face = faces [ i ] ;

im. ellipse ( face. x + face. width / 2 , face. y + face. height / 2 , face. width / 2 , face. height / 2 ) ;

}



im. save ( './friends-faces.jpg' ) ;

console. log ( 'Image saved as friends-faces.png' ) ;

} ) ;

} ) ;

Here is the image after face detection.

I know it’s not accurate, but that’s a good start.

Let’s do same using Webcam.

First, we need to connect to the webcam and show the live feed. Here is the code which does the same.

camera.js

var cv = require ( 'opencv' ) ;

try {

var camera = new cv. VideoCapture ( 0 ) ;

var window = new cv. NamedWindow ( 'Video' , 0 )

setInterval ( function ( ) {

camera. read ( function ( err , im ) {

if ( err ) throw err ;

console. log ( im. size ( ) )

if ( im. size ( ) [ 0 ] > 0 && im. size ( ) [ 1 ] > 0 ) {

if ( err ) throw err ;

window. show ( im ) ;

}

window. blockingWaitKey ( 0 , 50 ) ;

} ) ;

} , 20 ) ;

} catch ( e ) {

console. log ( "Couldn't start camera:" , e )

}

Run the code using the following command.

node camera.js

and you shall see the desktop window popping up with a feed from the webcam. Similar to as shown below, except with your face.



I swear I look better in person, this webcam sucks 😀

Let’s go through the code.

var camera = new cv. VideoCapture ( 0 ) ;

var window = new cv. NamedWindow ( 'Video' , 0 )

Here we are trying to access the webcam video feed and creating a new window to show that video.

Now, to show the live feed instead of the still image we need to refresh the window with the latest image received from the webcam. We do it by using interval function and refreshing the window in 20ms.

setInterval ( function ( ) {

camera. read ( function ( err , im ) {

if ( err ) throw err ;

console. log ( im. size ( ) )

if ( im. size ( ) [ 0 ] > 0 && im. size ( ) [ 1 ] > 0 ) {

if ( err ) throw err ;

window. show ( im ) ;

}

window. blockingWaitKey ( 0 , 50 ) ;

} ) ;

} , 20 ) ;

The initial image comes as 0x0 pixel hence we do not need to show that. That if condition takes care of that.

Alright, so now we know how to detect faces from the image and how to show live feed from the webcam, let’s merge both and we will have our live face detector ready to go!

Here is the code to detect faces from the webcam.

camera.js

var cv = require ( 'opencv' ) ;



try {

var camera = new cv. VideoCapture ( 0 ) ;

var window = new cv. NamedWindow ( 'Video' , 0 )

// face detection properties

var rectColor = [ 0 , 255 , 0 ] ;

var rectThickness = 2 ;



setInterval ( function ( ) {

camera. read ( function ( err , im ) {

if ( err ) throw err ;

console. log ( im. size ( ) )

if ( im. size ( ) [ 0 ] > 0 && im. size ( ) [ 1 ] > 0 ) {

im. detectObject ( cv. FACE_CASCADE , { } , function ( err , faces ) {

if ( err ) throw err ;

for ( var i = 0 ; i < faces. length ; i ++ ) {

face = faces [ i ] ;

im. rectangle ( [ face. x , face. y ] , [ face. width , face. height ] , rectColor , rectThickness ) ;

}

window. show ( im ) ;

} ) ;

}

window. blockingWaitKey ( 0 , 50 ) ;

} ) ;

} , 20 ) ;

} catch ( e ) {

console. log ( "Couldn't start camera:" , e )

}

Let’s run the code.

node camera.js

Here is the output.

It’s not accurate but it provides at least 80% of accuracy (rough estimate though).

Conclusion

OpenCV is an amazing open source project. We can use it for free to develop some useful real world application like motion tracker, face detector and hook it with machine learning to train the existing classifiers and provide deep learning. I hope they official support and work on Node.js client module to avail more API’s of OpenCV in Node.js client module.

Further study