Final Post with code and image

Here’s how the project turned out:

final

And the code for the project:

import cv2
import numpy as np
import argparse

import math
from scipy.spatial import distance


#ap = argparse.ArgumentParser()
#ap.add_argument("-i", "--image", required = True, help = "Path to the image")
#args = vars(ap.parse_args())

#img = cv2.imread(args["image"])
img = cv2.imread("test.jpg", 0)
img = cv2.medianBlur(img,5)
cimg = cv2.cvtColor(img,cv2.COLOR_GRAY2BGR)

cv2.namedWindow("Display", flags= cv2.WINDOW_AUTOSIZE)

circles = cv2.HoughCircles(img,cv2.HOUGH_GRADIENT,1,100,
                           param1=50,param2=25,minRadius=10,maxRadius=30)

circles = np.uint16(np.around(circles))

#coordinates of pupils
point1x = 0.0
point1y = 0.0
point2x = 0.0
point2y = 0.0

#coordinates of object on forehead
leftpointx = 0
leftpointy = 0
rightpointx = 0
rightpointy = 0

#coordinates of pupils in case hough circles doesn't work
leftpointx2 = 0
leftpointy2 = 0
rightpointx2 = 0
rightpointy2 = 0

objectexist = 0

m = 0
k = 0

dist = 0 #distance of pupils in pixels
dist2 = 0 #distance of object in pixels

mm = 100 #distance of object in mm



n = 1


for i in circles[0,:]:
    # draw the outer circle

    cv2.circle(cimg,(i[0],i[1]),i[2],(0,255,0),2)
    # draw the center of the circle
    cv2.circle(cimg,(i[0],i[1]),2,(0,0,255),3)
    if n == 1:
        n = 2
        point1x = float(i[0])
        point1y = float(i[1])
    elif n==2 :
        point2x = float(i[0])
        point2y = float(i[1])

    #print(" " + point1x + ", " + point1y + ", " + point2x + ", " + point2y)

print(point1x)
print(point1y)
print(point2x)
print(point2y)





dist = math.hypot(point2x - point1x, point2y - point1y) #ipd in pixels

#if dist < 10000:
print(dist)



def my_mouse_callback(event, x, y, flags, param):
    global m
    global k
    global leftpointx
    global leftpointy
    global rightpointx
    global rightpointy

    global leftpointx2
    global leftpointy2
    global rightpointx2
    global rightpointy2

    global objectexist

    global dist
    global dist2
    if event==cv2.EVENT_LBUTTONDBLCLK:
        m = m+1

        #print(x)
        if m == 1:
            leftpointx = x
            leftpointy = y
        elif m == 2:
            rightpointx = x
            rightpointy = y
            dist2 = math.hypot(rightpointx - leftpointx, rightpointy - leftpointy)
            print( "object distance = " + str(dist2))

            if dist < 10000:
                distanceInMM = dist * (mm / dist2)  # ipd in mm
                print( "IPD in mm = " + str(distanceInMM))
    if dist > 10000:
        if event==cv2.EVENT_RBUTTONDBLCLK: #used if hough circles doesn't work
            k = k+1
            if dist2 == 0:
                dist2 == 500

            #print(x)
            if k == 1:
                leftpointx2 = x
                leftpointy2 = y
            elif k == 2:
                rightpointx2 = x
                rightpointy2 = y
                dist = math.hypot(rightpointx2 - leftpointx2, rightpointy2 - leftpointy2)
                print("IPD in pixels = " + str(dist))

                distanceInMM = dist * (mm / dist2)  # ipd in mm
                print( "IPD in mm = " + str(distanceInMM))

cv2.setMouseCallback("Display",my_mouse_callback,cimg)










while(1):
    cv2.imshow("Display",cimg)

    if dist2 == 0 and dist < 10000:
        c = cv2.waitKey(0)
        if c == 32: #spacebar
            dist2 = 500
            distanceInMM = dist * (mm / dist2)  # ipd in mm
            print("IPD in mm = " + str(distanceInMM))

    if cv2.waitKey(15)%0x100==27:break    # waiting for clicking escape key
cv2.destroyWindow("Display")



#cv2.imshow('detected circles',cimg)




#cv2.waitKey(0)
#cv2.destroyAllWindows()

In conclusion

1) Describe your final project. What have you accomplished over the course of the semester?

The goal of my final project was to create an IPD (inter-pupillary distance) calculator, i.e a program that detects and calculates the distance in millimeters between the two pupils of the subject / research participant. This is done through circle detection via Hough Circles, and some camera calibration (pixels to mm) through mouse input. I have accomplished this goal for the most part, with some non-major consistency issues.

Other accomplishments were learning how to use the OpenCV library (and coding libraries in general), and the language Python. I came into this project with negligible knowledge of python and its syntax/rules, and ending the semester I feel to have a much better grasp on the language. Finally, this was the first project that forced me to use and learn the command line, due to the nature of python.


2) Describe your overall feelings about the project. Are you happy, content, frustrated, etc with the results?

I am content with the results, though frustrated with some inconsistency. Some methods I used and small algorithms I wrote work for some images and don’t work / work somewhat inaccurately for other images. Overall I am glad to have taken this opportunity to participate in research and feel that I have learned a lot, not only about Python/OpenCV but about the Virtual Reality field in general.
One particularly satisfying thing is overcoming my fears of not finishing the project. When I started it, everything seemed so intimidating as I wasn’t familiar with a lot of the tools I was using, so it was an uphill struggle.

3) What were some of the largest hurdles you encountered over the semester and how did you approach these challenges.

As mentioned in answer 2 and in many of my blog posts, the biggest hurdle I encountered was inconsistent results. A temporary – although admittedly not great -solution I have used is to alter parameters in my methods used by image basis as needed.

Other big hurdles were finding and understanding documentation and applying them to my project. Most OpenCV resources available were for C++, and the ones I found for Python were often for Python 2 rather than 3. Similarly I often encountered issues adjusting my code as a lot of online assistance I found was in Python 2.

One scary hurdle involved having to urgently factory-reset my computer a few weeks ago after an onslaught of viruses and this forced me to reinstall OpenCV, numpy, PyCharm, and some other stuff. Fortunately, I had all my code backed up so none of that was lost.

4) If you had more time, what would you do next on your project?
I would keep working on trying to make it more consistent and trying out other methods to detect circles. Since I had issues with blob detection and chessboard pattern detection, I would spend more time trying to figure out why these wouldn’t work for me and how to get them to work. I would clean up the code to make it more efficient, make it more user-friendly, generalize the program more, potentially add a webcam feature, and if I wanted to go above and beyond if I had more time I would try to create a GUI for the project.

Final Week

Last week was spent trying different camera calibration methods. A few days were spent on the checkerboard method, a few more on sticky notes and other color detection. Many problems cropped up with both these methods. I couldn’t get the program to truly recognize the checkerboard, and the way my program is set up it was very difficult to get the blob detection up and running.

This week I tried the last resort clicking method which surely was the least problematic out of all the methods. Some annoying issues I ran into were:
1) finding reliable Python tutorials and documentation as most resources were only available for C++
2) once a python resource was found, it was version 2.7 rather than 3.6, so a lot of time was spent fixing up indentation and syntax issues

Once all that was figured out, I wrote a tiny simple algorithm to take in the first two mouse clicks and stop after the first two, and then find the distance between the two. I have also finished up with the pixel -> mm conversion math, all that needs to be done now is testing with a reliable piece of paper / sticky note etc. taped to the forehead. I will probably do that on Sunday and edit this post with results, along with adding the option for command line arguments for more generality.

The only final issue is consistency, as hough circles distance in pixels detection seems to work on only 50% of the images, and I have yet to find out a solution for the short-scalar overflow. It shouldn’t be occurring based on the method I’m using, not sure what’s wrong.