1. Context

This summer I have been asked to write a program that will take the following picture of transistors…

…and measure lengths and angles of the gray shapes inside of each box. The scope of this writing is image processing so we’re not going to talk about what a transistor is or where the gray shapes come from. Just accept the fact that they exist and I’m trying to measure it. On to the image processing where the program I am using is MATLAB.

2. Binary Image

The entire process starts with edge detection: letting the computer decide where lines are. Googling edge detection tells you to start with converting the image into a binary image. Binary Images are the foundation of edge detection because there is a white pixel were a line is and a black pixel where there is not a line. To make a binary image simply type edge(image) into MATLAB and-

So it’s the first step and I have already come across an issue. The images I am working with are too noisy and produce real nasty looking pictures. So step 1 a) is to filter an image to reduce noise. There are multiple filters that can be applied and multiple methods for creating binary images and photos of these multiple iterations are below:

The best and most consistent thing that worked was the use if the ‘canny’ method for binary image creation and applying one filter.

So after all that, step one is completed on to step two.

3. Line detection

MATLAB has a built-in method for detecting lines called the Hough transform. It’s a popular method for line detection used in most computer vision programs today.

The graph is in terms of the length of the line (rho) and the angle it is at (theta). The brightest points are where the computer believes there is a line. Applying this to the current binary images I have yields…

An incomplete method for finding lines. Turns out the images I have are too pixelated to give clean lines to the program so MATLAB only finds the lines it is only 100% exist and anything else is left unnoticed. This led to a rabbit hole for finding other methods of line detection such as the Radon transform, and nothing worked. So eventually, I decided to tell the computer of multiple pixels are in a vertical or horizontal line then say it is a line.

Now I can find lots more lines. My next step was to make an outline of the transistor using the furthest outward line. As you may have guessed, there is a lot of room for error using this method.

This is an error I plan to fix later in the project because this does work on most samples I have. From this point I moved on to my next Step.

4. Meaningful Measurements

I currently have measurements in pixels, but pixels mean nothing to the Lab, so I decided to implement a method for recording the measurements in nm. There is a scale bar on every picture I receive so the easy approach is to use the scale bar to create a pixel to nm conversion.

Ahh yes, clearly defined lines in this scale will make this step trivial compared to everything I have done up to this poi-

Yeah this is an issue, being unable to consistently measure the scale bar leads to many issues as you may have guessed. So on to plan B. I noticed by this point that all photos gathered are in exactly two resolutions, and the magnification is consistent across each photo. Below are data points that show the best fit curve to describe the relationship between magnification and pixel to nm conversions.

MATLAB has a built-in word detector called the OCR which is a simple tool to use. I can find the phrase ‘mag’ and read the number underneath to find the magnification of the photo and use the trendlines from above to find the approximate conversion for pixels to nm. So now I have a consistent form of real -world measurements and my next step from here takes an exciting turn.

5. Object Detection

My next step is to use object detection to take a full photo of transistors and locate each one individually and perform all the steps described above without the need of the user to locate each one at a time. This is the meat and potatoes of computer vison you brag to everyone about and feel real cool about it. So how hard is it? Step 1: type imagelabeler into MATLAB. Step 2: draw boxes around what you want to find like so.

Step 3: repeat step 2 until mouse breaks. Step 4: run program and the result is…

Easy. All it takes is a premade algorithm and your sanity to do object detection. This is the progress I have made so far in my task but there is still more to do. So how do you learn image processing? The answer is to google everything and see what works for you. Hint: if your subject is really small then most things are not going to work on the first try.