Last week was spent trying different camera calibration methods. A few days were spent on the checkerboard method, a few more on sticky notes and other color detection. Many problems cropped up with both these methods. I couldn’t get the program to truly recognize the checkerboard, and the way my program is set up it was very difficult to get the blob detection up and running.

This week I tried the last resort clicking method which surely was the least problematic out of all the methods. Some annoying issues I ran into were:
1) finding reliable Python tutorials and documentation as most resources were only available for C++
2) once a python resource was found, it was version 2.7 rather than 3.6, so a lot of time was spent fixing up indentation and syntax issues

Once all that was figured out, I wrote a tiny simple algorithm to take in the first two mouse clicks and stop after the first two, and then find the distance between the two. I have also finished up with the pixel -> mm conversion math, all that needs to be done now is testing with a reliable piece of paper / sticky note etc. taped to the forehead. I will probably do that on Sunday and edit this post with results, along with adding the option for command line arguments for more generality.

The only final issue is consistency, as hough circles distance in pixels detection seems to work on only 50% of the images, and I have yet to find out a solution for the short-scalar overflow. It shouldn’t be occurring based on the method I’m using, not sure what’s wrong.

It took some time to get it to work, but with trial and error and cutting out multiple combinations of checkerboards, I might have got OpenCV to detect the board. The problem was the pattern has to be larger than 2×2 (OpenCV detects only the inner circle and likes having a wide border, so in essence a 3×3 cutout would actually be detected as 2×2, however that works)

3×3 was the largest that I could get to fit on my forehead, so I decided to get a printout of a chessboard with smaller square sizes, and cut out a 5×5 piece to test it as 4×4 and this is the outcome:

I wrote an if statement that draws a pattern around the detected corners of the chessboard, and the screenshot appears to faintly show this, I couldn’t get the pattern to be colored to be more visible. Here’s the original image:

Or I might just be seeing things; it seems as though the closer I focus on the gray screenshot, I can see a pattern, but not further away. May just be an optical illusion and I’ll take a look at ret’s boolean value to know for sure.

However another problem that popped up with this specific input photo is overflow, which the command prompt shows in the screenshot. This leads me to get an inaccurate result for the pixels distance, despite it working for every other image. This is another hurdle that I spent time trying to fix but wasn’t able to.

For next I’ll try to get done with the process of getting the detected points of the chessboard to get the distance in pixels (minus the white border perhaps) for a conversion value, and then perhaps take another photo with the chessboard to get an accurate pixels distance, and then complete the calibration from there.

This week, a chessboard pattern was printed out and tested with multiple pictures of my face and my roommate Julian’s face. OpenCV’s chessboard detector method wouldn’t quite work on us but the problem seems to be the size of the squares. For it to fit on the forehead, it had to be cut down to 3×2. However, OpenCV’s method detects the inner corners only which makes it 2×1, and requires at least 2×2. So I will spend more time trying to work with that and if it doesn’t work out I’ll try other calibration methods.

In this screenshot, the algorithm didn’t quite accurately detect my eyes but that was a rarity as it works with most other images:

Applying the distance method to these photos, the result in pixels actually seems pretty accurate. The distance between the eyes turns out to be around half of the image’s width in pixels, more or less.

After some trial and error for finding the distance in pixel, a (potentially tentative) solution has been found. Actually, two different solutions were found which yielded me different results with the picture I was testing:

In this screenshot I used a method widely recommended by stackoverflow and other sources for finding the distance between two points, which can be seen in the code. To find the center of the two circles in the first place, a small and probably inefficient algorithm was used.

The current result from the image that I’m going to go with for now is 266.27 pixels.

The next task is to figure out the width in pixels of the image itself to determine whether this is a plausible solution, as the other solution I was getting was 92485 pixels.

Once that is figured out, I should very soon be able to begin with the cardboard detection and conversion values to find distance in mm. I will try to accomplish this with a picture of my own face.

This week’s post won’t involve many useful screenshots as most my time was spent
1) trying to get things to work
2) looking into how to calculate the IPD in pixels

This post comes a bit late as most of last Monday-Thursday was spent a bit scattered with a little lack of direction, with uncertainty of how the checkerboard pattern was supposed to be used. I tried reading through many, many tabs of documents and tutorials involving distance between objects in OpenCV but so far didn’t make much progress:

My meeting with Kevin Ponto and Alex Peer on Friday was pretty helpful for me to get back on track and have a better idea of what to do.

Since the meeting, a lot of my time has been consumed trying to get the Hough Circles method to work on a picture of my own face (so far I have been failing a lot, and haven’t yet been able to figure out why. Inputting an image of my face doesn’t even output the image when I run the program). So instead I tried taking a picture of my friend’s eyes, who has much more defined irises and pupils and the method worked on her:

As work is ongoing, I will keep trying to get a picture of my face working, in prep for the final step of the camera calibration with the checkerboard. In the meantime, the immediate goal is to figure out how to get the IPD in pixels. I have a good idea of the formula and math to be used, although I am still digging for documentation and tutorials to get the location/coordinates of the centers of the circles in the image.

These two weeks, a lot of time was spent debugging and working on both photos and webcam (although photos will be focused on from now on).

A lot of time was spent on getting to detect the face and eyes via Haar Cascades. This was accomplished relatively easily for photos with the help of the openly available tutorial, though there was a little hiccup where the program wouldn’t run at all. There was some trouble with the Haar Cascade parameters and learning how to integrate the xml files as the tutorial was based on a different version of OpenCV than I am currently using, but with some digging I was able to find a solution:

After this I tried applying the same method for the webcam by changing some lines of code around with the help of the OpenCV webcam tutorial, and spent a few hours trying to get that to work but wasn’t able to. I followed the steps at https://realpython.com/blog/python/face-detection-in-python-using-a-webcam/ with some changes (I changed the part where the code uses system arguments because I wasn’t using any) but I still struggled to get it to work. Then I came across another helpful resource in https://pythonprogramming.net/haar-cascade-face-eye-detection-python-opencv-tutorial/ and was able to get the face+eye detection working with the webcam, after adding the full paths to the xml files for the cascade parameters (apparently an older version of OpenCV didn’t need the full path)

I still felt like I wasn’t making much progress and all my time and effort was scattered in multiple places so I decided to try and focus on pupil detection for pictures, which I was struggling with using blob detection, so I looked into some other methods and Hough Circles worked out pretty well for me through the tutorials at http://docs.opencv.org/3.1.0/da/d53/tutorial_py_houghcircles.html and of course, changing the parameters around to accommodate for the objects we care about. (is hough circles a viable alternative?)

Using Hough Circles on the Ronaldo photo wasn’t getting me anywhere but soon it was pretty clear to me that the photo wasn’t zoomed in enough and the pupils weren’t visible at all so now I’m trying the method on other eye-focused pictures and that helped a lot:

If I use Haar Cascades, I am already able to detect the eyes which is the area we care about but the challenge there is that I’m not familiar with the concept of how to work with only the detected space (there is a square drawn around both eyes, and I’m not sure how I would go about using only that space and working with it to get to the pupils)

The goal for next week is to integrate the checkerboard pattern and starting to get to the distance tracking, though right now I’m pretty clueless as to how I’m going to accomplish that.

This week, I installed PyCharm and attempted to install all the necessary packages to it (nimpy and cv2). I tried to learn how to run a file remotely through the interpreter and how to organize my files in one location.

Currently I’m having trouble setting up my project in PyCharm (it won’t detect the necessary modules and I’ve been trying to figure out how to fix that)

EDIT: After spending some hours trying to figure out the issue, I found out that I had Python 2.7 installed from a while ago and that was getting mixed up with the 3.6 that I had installed recently, so that may have been causing PyCharm to not recognize the packages. I had the wrong interpreter selected in the IDE settings, so selecting the correct one fixed the issue!

In addition to that, I tested out cv2’s live video/webcam functionality in PyCharm and started working on face/eye/blob detection. I used the code from http://docs.opencv.org/3.0-beta/doc/py_tutorials/py_gui/py_video_display/py_video_display.html
and running the program correctly opened the laptop’s webcam (how cool) but I couldn’t get the window to close properly (it would keep reopening).

Now that I actually have some foundation and idea of what’s going on, the plan for next week is what the initial plan was for this week – to actually start playing with blob and face detection from pictures and live webcam.