In this post I will show you how to track multiple objects. But first we must learn to track a single object
Here I have made small finger cap type thingy with a blue paper and placed it on my finger. My aim was to control the mouse through my hand. Hence such an arrangement.
I have used emguCV a wrapper of C# over openCV. But openCV users can also refer to the post because the high level functions and the theories are exactly similar.
Step 1: The first part is to detect the object through its colour. Here our object has blue colour.
We would need to threshold this colour .But thresholding is difficult in RGB namespace, mainly because each channel is stored as grayscale image. The intensity changes the colour itself.Hence we have the HIS or HSV space .
HSV-Hue, Saturation and Value.
Hue represents the colour. The colour information irrespective of the intensity .
Saturation-Represents the amount of light in the colour.
Value(V) or Intensity(I) -> Brightness.
HSV color space is very similar to the way humans understand color. No one says that a car has a color of 50% blue, 20% Red and 30% Green. Instead we see the hue .So first we need to convert our image to HSV space.
Image<Hsv, Byte> hsvimage = frame.Convert<Hsv,Byte>();
Now that we have our image, we would proceed to thresholding. The result of thresholding can be seen left bottom camera feed.
MCvScalar min = new
MCvScalar(104, 150, 150);
MCvScalar max = new
MCvScalar(120, 255, 255);
CvInvoke.cvInRangeS(hsvimage, min, max, temp);
First to threshold the image we need to scalars .The pixels present in between the two pixels will be changed to 255 and the rest to black.The first parameter in MCvScalar is the Hue, second is the Saturation and the third is the intensity .You need to adjust these based on the colour you are thresholding.
After the thresholding, we now have to find the position of the object. This is done using moments. I will touch upon moments in some other post.
MCvMoments moment = new
double moment10 = CvInvoke.cvGetSpatialMoment(ref moment, 1, 0);
double moment01 = CvInvoke.cvGetSpatialMoment(ref moment, 0, 1);
First we create the object of MCvMoments type which is used to store the moments which will be calculated in the nextstep.the cvMoments(<thresholdedimage>,<reference to the MCvMoment object>,<isBinaryFlag>);
If the is binary flag is non zero,any non zero pixels are treated as one.
The moments are then calculated and stored.
Now to calculate the area .
double area = CvInvoke.cvGetSpatialMoment(ref moment, 0, 0);
From the moments and the area, we get the centre of gravity(the position of the object).
posX = (int)(moment10 / area);
posY = (int)(moment01 / area);
Image<Bgr, Byte> tracking = new
Point(posX, posY), 4, new
MCvScalar(0, 255, 0), 5, LINE_TYPE.EIGHT_CONNECTED, 0);
Thus we get the position by dividing the moments by the area.
The end result can be seen in the bottom left of the diagram.
Multiple object Tracking. Here I made a small app to draw with my fingers.I used a blue cap and yellow cap,made out of paper.I will paste the code for your reference.
void ProcessFrame(object sender, EventArgs e)
frame = _videoStream.RetrieveBgrFrame();//Retrieve the frame from thecamera
Point new_yellow = TrackYellow(frame); //Track the yellow colour through adifferent value of the scalar
Point new_blue=TrackBlue(frame); //Tracking blue colouras I did previously
//Sometimes when the fingers go out of frame, it may result in noise or the tracker circle going haywire.Since I made a paint app,I did not want unnecessary lines drawn
if (blue.X != 0 && blue.Y != 0 && new_blue.X!=0 && new_blue.Y!=0 )
//Draw a line from the previous position to the current position
CvInvoke.cvLine(img, blue, new_blue, new
MCvScalar(255, 0, 0), 5, LINE_TYPE.EIGHT_CONNECTED,0);
//Both the tracking takes place in the same frame.
if ( yellow.X != 0 && yellow.Y != 0 && new_yellow.X!=0 && new_yellow.Y!=0 )
CvInvoke.cvLine(img, yellow, new_yellow, new
MCvScalar(0, 255, 255), 5, LINE_TYPE.EIGHT_CONNECTED, 0);
The new position gets stored in the old position for the next iteration
yellow = new_yellow;
blue = new_blue;
TrackImagebox1.Image = img;
This is the final result. This had been made by hand movements J.
Sometimes there are background noises ,you can filter having a condition based on area.
Those that did not understand the moments part.I will be back with a blog post on the same. J