2. SSIM: 85%
2.7.1 Conclusion
Today, there are various methodologies and algorithms being proposed and applied to detect OOS (Out-of-Stock) and OSA (On-Shelf Availability).
Almost every study mentioned the downside of using RFID (Radio- Frequency Identification) and weight sensors in detecting OOS on-shelf. RFID is considered expensive and time-consuming to be tagging since 2006 until current day. Smart shelf with weight sensors, on the other hand, is costly for installation even though it has been experimented successfully by Metzger, et al. (2007).
As proposed by Kejriwal, et al. (2015), product may be tallied by abstracting the image. They employed a mobile robot with a monocular camera attached to it. Rosado, et al. (2016), on the other hand, suggested a panorama stitching method for detecting the label on the shelf. This method employs supervised learning and necessitates the use of high-quality panoramic image.
Hu, et al. (2020) introduced DiffNet for the first time to extract and compare the difference between two images. This method can be applied to image before and after a product had been taken. These methods identify the products on-shelf by extracting the speeded up robust features (SURF). Before comparing the source and target images, image pre-processing is performed. The shortcoming, on the other hand, is clear. First, huge multimedia files necessitate high processing performance. In addition, training the network takes more calculation time compared to the one-stage algorithm.
The idea of Higa, et al. (2018) to detect the product changes (e.g., taken or returned) on-shelf is innovative and valuable. In their method, a surveillance camera captures low-quality videos that are analysed, compared, and classified.
However, low-quality videos hindered the tracking of moving objects in the video. Higa, et al. (2019) extended their study by introducing the Hungarian method to analyse the successive images when there are customers blocking the shelf. The result still has a classification error problem, prompting the author to recommend that the accumulated error be reset on regular basis via a scheme.
In addition, the author suggested using image segmentation to further divide the changing region. Also, the background subtraction method does not give us a satisfying succession rate (lower than 90%) in OOS identification.
Some other papers proposed method that can determine the distance or depth to identify if a product has been taken from the shelf. Qiao, et al. (2017) proposed to use ScaleNet to predict the scale of object on-shelf. As result, the solution failed to do product counting for overlapped products even though the scale prediction is a success. Milella, et al. (2020) proposed to use a depth sensor to monitor the shelf. The depth sensor camera was placed on top of the shelf. As result, the average error is only 5 out of 100. However, this method has a limitation where the object can only be placing on a table facing upwards. Also, this method is limited to OOS condition detection, but not for product identification.
Liu, et al. (2018) and Zhao, et al. (2019) utilised the fish-eye camera on top of the shelf to detect OOS condition and OSA. Mask-RCNN produced as high as 97.7% accuracy for product recognition. YOLOv3 with optic nerve micro-saccade (ONMS) as proposed by Zhao, et al. (2019) produced a great result of 97.38% accuracy to identify the object on-shelf. The fish-eye camera showed a good result in determining objects because it provides a wide angle.
However, this equipment is expensive when each row of the shelf must attach to at least one camera, and the angle of the camera must be specific.
From the study of Liu, et al. (2018) mentioned above, we can also see the shortcoming of Mask-RCNN when it comes to product mutual occlusion on- shelf. Hence, Xu, et al. (2020) proposed to use SSD to solve the problem.
However, Tensowflow framework applied by this study failed to classify well.
As the data imbalanced happened, samples had been recognised wrongly as Negatives. Later, Yilmazer and Birant (2021) proposed a new method called SOSA stands for the combination of “semi-supervised learning” and “on-shelf availability” for the first time. YOLOv4 as the architecture of this method outperforms YOLOv3 and RetinaNet in accuracy of product detection. In this method, no product labelling is required as the OSA is determined by the shelf situation. However, the accuracy of this method will be affected when the product on-shelf is placed too deep into the shelf.
As technology advances and improves over time, we can expect a huge development in product management using deep learning algorithms. There is no one-size-fits-all solution for the optimum method to use for OOS detection and OSA management system. However, the state-of-art method using SSD or
YOLO are promising as it uses one-stage algorithm. This algorithm is fast and accurate with less annotated dataset needed. In real-world scenario, RFID or a weight sensor should be used in conjunction with CCTV to efficiently monitor the shelf.
CHAPTER 3
3 METHODOLOGY AND WORK PLAN