I am developing a rover using Raspberry Pi , that will sweep a room and pick up objects fallen on the ground . To detect the object , I am using a reference Image , that is taken right at the start of the rover's operation , and an Image (new Image) that is clicked every 10 seconds . To determine if there is a change in the image frame , I do a image subtraction between the reference image and the new image . If any difference is found , it'll draw a contour around it , and if the contour area is greater than a certain threshold (cautionary step) , it concludes that there exists an object .
I am using the following code -
import numpy as npimport cv2,timeimg=cv2.imread("object1.jpg")img1=cv2.imread("object2.jpg")sub=cv2.subtract(img,img1)gray=cv2.cvtColor(sub,cv2.COLOR_BGR2GRAY)blur = cv2.GaussianBlur(gray,(3,3),0)_, contours, _= cv2.findContours(blur,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)c=max(contours,key=cv2.contourArea)print(cv2.contourArea(c))if cv2.contourArea>20000:print("Object detected !")The above code just uses 2 images to calculate their difference and determine if there is an object present .Please note that I have not posted the original code here that I will be using in my project .
Now , the above code works fine for very controlled situations, say ,when the image background is very constant or there is no presence of shadow in it . But considering the fact that the rover will be moving around in the room , and there are chances that the lighting variations might trigger a false object detection , even if there is no real object in the frame . The difference might be triggered due to false contouring from shadow effects .
I want to know , if there is any other way of achieving this object detection without doing foreground/background image subtraction . I have also considered using a ultrasonic sensor to detect the object's presence , however that is not a very reliable option . I would prefer a Image processing based solution .
Thank you .
==========================================================================
EDIT 1 -
So , I decided to change the algorithm a bit . I have done thresholding on both the foreground and background Images and then performed absdiff between the binary images , to obtain any frame change(object) . The code is as follows -
import numpy as npimport cv2,timeimg1=cv2.imread("back.jpeg")blur1 = cv2.GaussianBlur(img1,(5,5),0)gray1=cv2.cvtColor(blur1,cv2.COLOR_BGR2GRAY)ret,thresh1 = cv2.threshold(gray1,65,255,cv2.THRESH_BINARY_INV)img2=cv2.imread("front.jpeg")blur2 = cv2.GaussianBlur(img2,(5,5),0)gray2=cv2.cvtColor(blur2,cv2.COLOR_BGR2GRAY)ret,thresh2 = cv2.threshold(gray2,65,255,cv2.THRESH_BINARY_INV)diff=cv2.absdiff(thresh2,thresh1)diff=cv2.bitwise_xor(diff,thresh1)kernel = np.ones((2,2),np.uint8)diff=cv2.erode(diff,kernel,iterations = 1)diff=cv2.dilate(diff,kernel,iterations = 8)_, contours, _= cv2.findContours(diff,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)c=max(contours,key=cv2.contourArea)x,y,w,h = cv2.boundingRect(c)cv2.rectangle(diff,(x,y),(x+w,y+h),(125,125,125),2)cv2.imshow("thresh",diff)cv2.waitKey(0)"absdiff" is followed by Erosion and Dilation . After that , I find the largest contour and determine if there is an object . The images used in the algorithm are as follows -
Background Image -Background Image
Foreground Image - Foreground image
Foreground Threshold - Foreground threshold Image
Background Threshold - Background threshold Image
Difference Image - Final Image with contour and its boundary .
As you can see , the detection works fine . I have few other foreground images which I have used to test the algorithm . They have given satisfactory results .I wanted to know , if there is any other way of achieving the same result with better efficiency .
PS- All of the foreground Images have been taken with Flash ON . I have tried with Flash OFF but there seems to exist a lot of noise in the image .
=============================================================
EDIT 2-
Performance of the algo using other pictures -
Note :- The Background Image has remained same .
- Object 1 - Foreground Image 1
- Object 1 Detection - Foreground Image 1 Result
Best Answer
I doubt if this problem is that much simple as you have described in the question, it will be get very complex when we move in real world scenario.
But anyway assuming you have small objects only present in the room then you identify them by identifying connected components in the binary image captured and choose them based on their relative pixels sizes.
Here is Python implementation for the same:
img = cv2.imread('D:/Image/objects.jpg')gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)# binarize the imageret, bw = cv2.threshold(gray, 128, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)# find connected componentsconnectivity = 4nb_components, output, stats, centroids = cv2.connectedComponentsWithStats(bw, connectivity, cv2.CV_32S)sizes = stats[1:, -1]; nb_components = nb_components - 1min_size = 250 #threshhold value for objects in sceneimg2 = np.zeros((img.shape), np.uint8)for i in range(0, nb_components+1):# use if sizes[i] >= min_size: to identify your objectscolor = np.random.randint(255,size=3)# draw the bounding rectangele around each objectcv2.rectangle(img2, (stats[i][0],stats[i][1]),(stats[i][0]+stats[i][2],stats[i][1]+stats[i][3]), (0,255,0), 2)img2[output == i + 1] = colorImage containing objects:
Detected objects using connected components labels:
Another approach you can consider is to use structure from motion, reconstruct the environment / point cloud, and the point cluster above the floor surface belongs to your object. Also could combine this with your background subtraction method to remove false detection like shadows that are causing issue for that method.