Project: Play games with a piece of paper using your webcam!
Finally I decided to make a game plugin which would help a gamer play games touchless mode, without a mouse & keyboard.
After days of requirements gathering, got my hands on eclipse & started coding. The basic idea was to analyze the continuous stream of images incoming from the webcam & accordingly send virtual keystrokes to the game.
Cutting the long story short, got a rectangular piece of paper, drew two fat circles on two of its sides. That piece of paper was basically the steering wheel used to drive the car in games (in my case, ‘Need for speed most wanted 2012’).
The images captured were processed manually. RGB to grayscale, grayscale to monospace using a tuned-up threshold. Finally the two circles of the paper were two white circles on the monospace image with black background.
The next part included getting the coordinates of the two circles without fluctuation, in a way optimization of receiving coordinates of the two points. After using a lot of math here, I decide to go for the easy way by adding increments of 5-7 in the loop which goes over every pixel. In this way, the average is extracted & a smooth transition is the outcome.
Finally used Pythagoras theorem to calculate the hypotenuse to know how far the paper is from the webcam. And simple ‘Cos’, ‘Sin’ & Tan’ functions to get the angle of the paper.
Next stop was using two webcams to get the depth of the paper. This was important for three main reasons. To isolate the paper from the background scene, to get coordinates in 3D & to eradicate the lighting issue so that it may work under all lighting conditions. (Did this part during the summers)
Took around two weeks to complete the whole project.
Here’s the source code! (excluding the stereo-cam code)