opencv_zoo Object_Detection: YoloX (GSOC)

Updated files for object detection using YOLOX, files uploaded for review:

Model files (.onnx format FP16)
Models: YOLOX_s, YOLOX_tiny, YOLOX_nano
README
LICENSE (Apache 2.0)
demo.py (with image and webcam inference options)
YoloX.py (class YoloX)
examples (test_data: test images, results: inferred results)
COCO class names (coco.names)

COMPLETED:

[x] YOLOX ONNX models (FP16): YOLOX_S, YOLOX_tiny, YOLOX_nano
[x] ReadMe
[x] License
[x] Sample Images and Results
[x] Video Inference

TO DO:

[x] Model Quantization

Aug 19 '22 10:08 Sidd1609

This PR is for YoloX, so please remove the object_detection_nanodet directory.

Aug 23 '22 02:08 fengyuentau

removed nanodet directory

Aug 23 '22 05:08 Sidd1609

Yes, will check with cv2.dnn.NMSBoxes() to see if the results can be parsed when drawing boxes
Will resolve the other changes as well and update the PR

On Fri, Sep 2, 2022 at 1:11 PM Yuantao Feng @.***> wrote:

@.**** commented on this pull request.

In models/object_detection_yolox/YoloX.py https://github.com/opencv/opencv_zoo/pull/86#discussion_r961338576:
   with open('coco.names', 'rt') as f:
       self.classes = f.read().rstrip('\n').split('\n')
I think we should either make this optional (return id if self.classes is none), or place all classes as strings here in the class like this https://github.com/open-mmlab/mmdetection/blob/df28da98926bc410e16bed1e9fc7d425d9a89495/mmdet/datasets/coco.py#L25-L38 .

In models/object_detection_yolox/YoloX.py https://github.com/opencv/opencv_zoo/pull/86#discussion_r961341022:
   if len(image.shape) == 3:
       padded_img = np.ones((self.input_size[0], self.input_size[1], 3)) * 114.0
   else:
       padded_img = np.ones(self.input_size) * 114.0
Is there any case when image.shape is not equal to 3?

In models/object_detection_yolox/YoloX.py https://github.com/opencv/opencv_zoo/pull/86#discussion_r961342635:
   self.mean = np.array([0.485, 0.456, 0.406], dtype=np.float32).reshape(1, 1, 3)
   self.std = np.array([0.229, 0.224, 0.225], dtype=np.float32).reshape(1, 1, 3)
   if not p6:
       self.strides = [8, 16, 32]
   else:
       self.strides = [8, 16, 32, 64]
   self.confThreshold = confThreshold
   self.nmsThreshold = nmsThreshold
   self.objThreshold = objThreshold
def preprocess(self, image):
   if len(image.shape) == 3:
       padded_img = np.ones((self.input_size[0], self.input_size[1], 3)) * 114.0
   else:
       padded_img = np.ones(self.input_size) * 114.0
   img = np.array(image)
not necessary, it is already a numpy array.

In models/object_detection_yolox/YoloX.py https://github.com/opencv/opencv_zoo/pull/86#discussion_r961366916:
           while order.size > 0:
               i = order[0]
               keep.append(i)
               xx1 = np.maximum(x1[i], x1[order[1:]])
               yy1 = np.maximum(y1[i], y1[order[1:]])
               xx2 = np.minimum(x2[i], x2[order[1:]])
               yy2 = np.minimum(y2[i], y2[order[1:]])
               w = np.maximum(0.0, xx2 - xx1 + 1)
               h = np.maximum(0.0, yy2 - yy1 + 1)
               inter = w * h
               ovr = inter / (areas[i] + areas[order[1:]] - inter)
               inds = np.where(ovr <= self.nmsThreshold)[0]
               order = order[inds + 1]
               if len(keep) > 0:
                   cls_inds = np.ones((len(keep), 1)) * cls_ind
                   dets = np.concatenate([valid_boxes[keep], valid_scores[keep, None], cls_inds], 1)
                   final_dets.append(dets)
call cv2.dnn.NMSBoxes() to perform NMS. Documentation for NMSBoxes(): https://docs.opencv.org/4.x/d6/d0f/group__dnn.html#ga9d118d70a1659af729d01b10233213ee

In models/object_detection_yolox/YoloX.py https://github.com/opencv/opencv_zoo/pull/86#discussion_r961369189:
   if len(final_dets) == 0:
       return None
Return an emtpy numpy array if no boxes.

In models/object_detection_yolox/YoloX.py https://github.com/opencv/opencv_zoo/pull/86#discussion_r961369775:
               w = np.maximum(0.0, xx2 - xx1 + 1)
               h = np.maximum(0.0, yy2 - yy1 + 1)
               inter = w * h
               ovr = inter / (areas[i] + areas[order[1:]] - inter)
               inds = np.where(ovr <= self.nmsThreshold)[0]
               order = order[inds + 1]
               if len(keep) > 0:
                   cls_inds = np.ones((len(keep), 1)) * cls_ind
                   dets = np.concatenate([valid_boxes[keep], valid_scores[keep, None], cls_inds], 1)
                   final_dets.append(dets)
   if len(final_dets) == 0:
       return None
   res_dets = np.concatenate(final_dets, 0)
we dont need this if we call cv2.dnn.NMSBoxes()

In models/object_detection_yolox/coco.names https://github.com/opencv/opencv_zoo/pull/86#discussion_r961370650:

@@ -0,0 +1,80 @@ +person

no need to keep this file if we have all these classes in YoloX.py.

In models/object_detection_yolox/demo.py https://github.com/opencv/opencv_zoo/pull/86#discussion_r961374634:

+with open('coco.names', 'rt') as f:

classes = f.read().rstrip('\n').split('\n')

How about adding a member function in YoloX which returns the classes for the detetection results?

In models/object_detection_yolox/demo.py https://github.com/opencv/opencv_zoo/pull/86#discussion_r961378413:
       if cv2.waitKey(1) > -1:
           print("Stream terminated")
           break
i think we dont have to set limitations on the stop key. With while cv2.waitKey(1) < 0:, user can stop the stream with basically any key.

In models/object_detection_yolox/demo.py https://github.com/opencv/opencv_zoo/pull/86#discussion_r961379746:
   if(Video_save):
       clip = moviepy.video.io.ImageSequenceClip.ImageSequenceClip(frame_list, fps=fps)
       clip.write_videofile('Webcam_result.mp4')
OpenCV provides VideoWriter to save videos. Here is an example: https://stackoverflow.com/a/29317298/6769366

— Reply to this email directly, view it on GitHub https://github.com/opencv/opencv_zoo/pull/86#pullrequestreview-1094380104, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALTC7VY7WJXNIVWPGF4SXPDV4GVUDANCNFSM57AGGXVQ . You are receiving this because you authored the thread.Message ID: @.***>

Sep 02 '22 07:09 Sidd1609

@Sidd1609 Could you squash commits into one? I am going to work on this pr and merge it this weekend if possible.

Oct 21 '22 07:10 fengyuentau

Not sure what you mean by squash commits Please could you elaborate?

On Fri, Oct 21, 2022 at 12:39 PM Yuantao Feng @.***> wrote:

@Sidd1609 https://github.com/Sidd1609 Could you squash commits into one? I am going to work on this pr and merge it this weekend if possible.

— Reply to this email directly, view it on GitHub https://github.com/opencv/opencv_zoo/pull/86#issuecomment-1286549962, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALTC7VYEKYWQKTUVLLPLADLWEI6RJANCNFSM57AGGXVQ . You are receiving this because you were mentioned.Message ID: @.***>

Oct 21 '22 10:10 Sidd1609

You can see here in the picture you have 128 commits in total for this PR. When merging a PR, we don't need such a long commit history and we prefer only one commit. Learn more details here. You could squash commits with the following commands:

$ git rebase -i HEAD~128 # number 128 is the number of your commits in this PR
# Then an editor will pop up. Leave the first commit as pick,
# and change the other commits from pick to squash. Exit the
# editor and a new page on the commit message will be opened.
# Compose a short and clear commit message and exit the editor.
# Squashing commits is done.

There is a page talking details on squashing commits: https://www.git-tower.com/learn/git/faq/git-squash

Oct 21 '22 12:10 fengyuentau

Oh got it yuantao I think both the commits for nanodet and yolox have been taken into consideration I will work on it and send the updates

On Fri, Oct 21, 2022 at 5:51 PM Yuantao Feng @.***> wrote:

[image: image] https://user-images.githubusercontent.com/17219438/197192566-efaa8f4b-4c45-4325-9479-c49d1e3ffa61.png

You can see here in the picture you have 128 commits in total for this PR. When merging a PR, we don't need such a long commit history and we prefer only one commit. Learn more details here https://docs.gitlab.com/ee/user/project/merge_requests/squash_and_merge.html. You could squash commits with the following commands:

$ git rebase -i HEAD~128 # number 128 is the number of your commits in this PR# Then an editor will pop up. Leave the first commit as pick,# and change the other commits from pick to squash. Exit the# editor and a new page on the commit message will be opened.# Compose a short and clear commit message and exit the editor.# Squashing commits is done.

There is a page talking details on squashing commits: https://www.git-tower.com/learn/git/faq/git-squash

— Reply to this email directly, view it on GitHub https://github.com/opencv/opencv_zoo/pull/86#issuecomment-1286889272, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALTC7VY44KSEPTEVS66ACPTWEKDGJANCNFSM57AGGXVQ . You are receiving this because you were mentioned.Message ID: @.***>

Oct 21 '22 12:10 Sidd1609

I think both the commits for nanodet and yolox have been taken into consideration

That's why these two PRs should be based on different branches. I will start working on your PRs when commits are squashed.

Oct 21 '22 12:10 fengyuentau

Noted.

On Fri, Oct 21, 2022 at 6:07 PM Yuantao Feng @.***> wrote:

I think both the commits for nanodet and yolox have been taken into consideration

That's why these two PRs should be based on different branches. I will start working on your PRs when commits are squashed.

— Reply to this email directly, view it on GitHub https://github.com/opencv/opencv_zoo/pull/86#issuecomment-1286905836, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALTC7V6HZ4YH6HGI4YVWXSLWEKFA5ANCNFSM57AGGXVQ . You are receiving this because you were mentioned.Message ID: @.***>

Oct 21 '22 12:10 Sidd1609

close in favor of https://github.com/opencv/opencv_zoo/pull/104

Nov 13 '22 02:11 fengyuentau