opencv import
import cv2
opencv template matching
Useful opencv functions
Note:Many functions of opencv rely on the ndarray type: in input as
parameter and in output as returned result.
imread(filename: Any, flags: Any = None)->ndarray:
It loads an image from the specified file and returns a ndarray.
The dimension of the returned ndarray depends on the flags used:
– With cv2.IMREAD_COLOR: ndarray.shape is a tuple such as
(image_height,image_width,image_channel_number).
Generally the image channel number is 3.
– With IMREAD_GRAYSCALE : ndarray.shape is a tuple such as
(image_height,image_width).
imwrite(filename, img: ndarray[, params]) -> retval
It saves the specified image provided as a ndarray object to a specified file.
The image format is chosen based on the filename extension.
matchTemplate(image:ndarray, templ:ndarray, method:int [, result[, mask]]) -> result:ndarray
Search a template (That is the image we are looking for) in another image.
The function slides through image , compares the regions against
templ using the specified method and returns the result of the
comparison.
Concretely, it returns a ndarray object with 2-dimensions.
its shape follows this formula:
Width: image.shape[1] - template.shape[1] + 1
Height: image.shape[0] - template.shape[0] + 1
The elements of the returned array represents the probability of the match for each pixel:
High value (closer to 1) means high probability of matching and lower value means the contrary.
Note:
– The result parameter is ambiguous because actually the method return the result,
so
generally it is not required to specify it.
– The method parameter is a int. The different values we can specify as method are
defined in cv2.TM_* variables:
TM_CCOEFF = 4 TM_CCOEFF_NORMED = 5 TM_CCORR = 2 TM_CCORR_NORMED = 3 TM_SQDIFF = 0 TM_SQDIFF_NORMED = 1 |
It is important to not that the probability returned depends on the used method, so we cannot use the same threshold with different methods to interpret the results.
Useful numpy functions
Comparison operators
Numpy overloads most comparison operators such as: __eq__, __ge__,
__gt__, __le__, __lt__,…
Applying operators on a ndarray object such as array>=0.8 or array<0.6
creates a ndarray object with the same dimension and shape than the array object
but where
elements are boolean : true if the condition is true and false if the
condition is false for elements of array.
Its usage here: applying a threshold on the result returned by
matchTemplate().
ndarray_object.nonzero()->tuple[ndarray,ndarray]:
It is an instance method that returns indexes of elements which values are different of
zero.
Note that false value is interpreted as a zero value and that true value is interpreted as a
non-zero value.
The returned result is a tuple of 2 elements: 2 ndarray object of 1-dimension, one for each
dimension
of the array object we call the method.
The first array represents the row indices found and the second array represents the column
indices found.
Its usage here: filtering out the elements under the threshold(That we got thanks to the
comparison operators).
Examples
Template matching of a small image on a larger image with detailed explanations
We will use a template(a wall brick) with 44×43 dimension that we are looking for in a larger
image(a
screenshot of a game with many wall bricks)with 2100×1200 dimension.
Here some remarks:
– We load images with their color(IMREAD_COLOR).
– We use the method TM_CCOEFF_NORMED to perform matching.
We use a normed method because image sizes are different.
– We apply a threshold of 0.95. As explained above the threshold value depends on the method used.
– We draw read rectangles on the larger image in zones where the threshold is reached and we
store
the new image in a new file.
– We can notice on the final image that some rectangles are larger than others, the reason is
many
matches happen around this location.
import cv2 import numpy as np print('####LOAD TEMPLATE AND IMAGE####') # imread() read an image file and store its content (That is each pixel) in a two-dimensional # ndarray # The first dimension is the height of the image and the second dimension is its width img_rgb: np.ndarray = cv2.imread('sonic_screen.png', cv2.IMREAD_COLOR) # This image has the dimension 2100x1200 and has colors(3 channels), so the shape tuple is (1200, # 2100,3) print(f'img_rgb.shape={img_rgb.shape}') # img_rgb.shape=(1200, 2100, 3) # Same comment as above template: np.ndarray = cv2.imread('sonic_brick.png', cv2.IMREAD_COLOR) print(f'template.shape={template.shape}') # template.shape=(43, 44, 3) # We extract the height and width of the shape to ease their usage later h_template, w_template, _ = template.shape print(f'template w, h={w_template, h_template}') # template w, h=(44, 43) print('####MATCH TEMPLATE PROCESSING####') # matchTemplate() return a ndarray with 2-dimensions. # its shape follows this formula: # Width: image.shape[1] - template.shape[1] + 1 # Height: image.shape[0] - template.shape[0] + 1 # The elements of the returned array represents the probability of the match for each pixel: # high value (closer to 1) means high probability of matching and low value means the contrary: res: np.ndarray = cv2.matchTemplate(img_rgb, template, cv2.TM_CCORR_NORMED) print(f'res.shape={res.shape}') # res.shape=(1158, 2057) print(f'res={res}, type={type(res)}') # Here is a preview of the result: # res=[[ 0.35639524 0.31923023 0.2841785 ... -0.3894405 -0.36949798 # -0.35127434] # [ 0.35662106 0.31944606 0.28431392 ... -0.406504 -0.38580927 # -0.36710274] # [ 0.35695735 0.3198492 0.28469688 ... -0.42621684 -0.40453294 # -0.3850909 ] # ... print('#### KEEPING ONLY VALUES WITH A MINIMAL PROBABILITY####') threshold = 0.95 # We create an array with the same dimension and shape than the res array but where # elements are boolean : true if the condition is true and false if the condition is false for # elements of res res_threshold: np.ndarray = res >= threshold print(f'res_threshold.shape={res_threshold.shape}') # res_threshold.shape=(1158, 2057) print(f'res_threshold={res_threshold}') # Here is a preview of the result: # res_threshold=[[False False False ... False False False] # [False False False ... False False False] # [False False False ... False False False] # ... print('#### MAPPING KEPT VALUES TO INDEXES MATCHING ') # nonzero() returns indexes of elements which values are different of zero(Here it means # different of false) # The result is a tuple of 2 elements: 2 ndarray objects of 1-dimension : one for each dimension # of the array object we call the method. # The first array represents the row indices found and the second array represents the column # indices found. loc: tuple[np.ndarray,np.ndarray]= res_threshold.nonzero() print(f'loc with size={len(loc)} type={type(loc)},text={loc}') # loc with size=2 type=<class 'tuple'>, # text=(array([ 40, 41, 84, ..., 1134, 1134, 1134], dtype=int64), # array([1748, 1748, 1928, ..., 2007, 2008, 2009], dtype=int64)) print('#### PRINTING THE RESULT') # We iterate on the 2 arrays to draw on the full original image a rectangle with the dimension # of the template for each # couple of position for pt in zip(loc[1], loc[0]): # print(f'pt={pt}') cv2.rectangle(img_rgb, pt, (pt[0] + w_template, pt[1] + h_template), (0, 0, 255), 2) cv2.imwrite('res_sonic.png', img_rgb) |
opencv issues
problem:
Pycharm doesn’t recognize completion.
See Code completion doesn’t work for cv2
module
for more details.
solution
Install the 4.5.4.60 version of the opencv library because last may cause some issues:
python -m pip install opencv-python===4.5.4.60 --force-reinstall
problem:
Intellij doesn’t manage to display documentation.
It shows a warning message:
You need configured Python 2 SDK to render Epydoc docstrings
solution
Install Python 2 on your machine.You don’t need to configure your project to use python 2,
you just need to make it available as python interpreter for intellij.
problem:
Intellij outputs the documentation in an unreadable way.
solution
It is a dirty way but it works. The idea is to fix the documentation by replacing the character
‘.’ by the character blank.