Introduction

NeuralMarker offers a variety of state-of-the-art Pretrained Deep Learning Models for Image Classification from different available frameworks like PyTorch, Dlib, and Mediapipe.

These available Pretrained deep learning models cover wide categories of Keypoint tasks such as :


  • Facial landmark detection
  • Human Pose Estimation
  • Head Pose Estimation


This list of models at hand is not static, as more models in the Neural Network Library of NeuralMarker will be added, then accordingly the below list will be updated too:


Pretrained Models 


1.  Dlib-68-points


      Model ArchitectureDlib-68

      No of landmarks: 68 


     

      Convenient to Use: For Facial landmark Detection

      Dataset Used300 Faces In-the-Wild 



2.  Hrnet_w32_coco


      Model Architecture: pose_hrnet_w32

      Convenient to Use: For Human Pose Estimation

      Num_Joints: 17 Person Pose Keypoints

      Dataset Used: MS COCO



3.  Hrnet_w48_coco


      Model Architecture: pose_hrnet_w48

      Convenient to Use: For Human Pose Estimation

      Num_Joints17 Person Pose Keypoints

      Dataset Used: MS COCO



4. Detectron2_R50FPN


        Convenient to Use: For Human Pose Estimation

        Num_Joints: 17 Person Pose Keypoints

        Dataset Used: MS COCO



5. Detectron2_R50FPN_3x


      Convenient to Use: For Human Pose Estimation

      Num_Joints: 17 Person Pose Keypoints

      Dataset Used: MS COCO



6. Detectron2_R101FPN


      Convenient to Use: For Human Pose Estimation

      Num_Joints17 Person Pose Keypoints

      Dataset Used: MS COCO



7. Pose_Mediapipe


Convenient to UseBody Pose Tracking

No of landmarks: 25 upper body landmarks

      


    Dataset Used: MS COCO




8.  Hrnet_mpii_w32


      Model Architecture: pose_hrnet_w32

      Convenient to Use: For Human Pose Estimation

      Num_Joints7 Person Pose Keypoints [Head, Shoulder, Elbow, Wrist, Hip, Knee, Ankle]

      Dataset Used: MPII Human Pose dataset



9.  Hrnet_mpii_w48


      Model Architecture: pose_hrnet_w48

      Convenient to Use: For Human Pose Estimation

      Num_Joints7 Person Pose Keypoints [Head, Shoulder, Elbow, Wrist, Hip, Knee, Ankle]

      Dataset Used: MPII Human Pose dataset



10.  Hrnet_resnet152_mpii


      Model Architecture: pose_resnet_152

      Convenient to Use: For Human Pose Estimation

      Num_Joints7 Person Pose Keypoints [Head, Shoulder, Elbow, Wrist, Hip, Knee, Ankle]

      Dataset Used: MPII Human Pose dataset


11.  Hrnet_resnet101_mpii


      Model Architecture: pose_resnet_101

      Convenient to Use: For Human Pose Estimation

      Num_Joints7 Person Pose Keypoints [Head, Shoulder, Elbow, Wrist, Hip, Knee, Ankle]

      Dataset Used: MPII Human Pose dataset


12.  Hrnet_resnet50_mpii


      Model Architecture: pose_resnet_50

      Convenient to Use: For Human Pose Estimation

      Num_Joints7 Person Pose Keypoints [Head, Shoulder, Elbow, Wrist, Hip, Knee, Ankle]

      Dataset Used: MPII Human Pose dataset



13.  Hrnet_resnet152_COCO


      Model Architecture: pose_resnet_152

      Input Size256

      Convenient to Use: For Human Pose Estimation

      Num_Joints17 Person Pose Keypoints

      Dataset Used: MS COCO


14.  Hrnet_resnet101_COCO


      Model Architecturepose_resnet_101

      Input Size256

      Convenient to Use: For Human Pose Estimation

      Num_Joints17 Person Pose Keypoints

      Dataset Used: MS COCO


15.  Hrnet_resnet50_COCO


      Model Architecturepose_resnet_50

      Input Size256

      Convenient to Use: For Human Pose Estimation

      Num_Joints: 17 Person Pose Keypoints

      Dataset Used: MS COCO


16.  Hrnet_w48_coco_384


      Model Architecture: pose_hrnet_w48

      Input Size: 384

      Convenient to Use: For Human Pose Estimation

      Num_Joints17 Person Pose Keypoints

      Dataset Used: MS COCO


17.  Hrnet_w32_coco_384


      Model Architecture: pose_hrnet_w48

      Input Size: 384

      Convenient to Use: For Human Pose Estimation

      Num_Joints17 Person Pose Keypoints

      Dataset Used: MS COCO


18.  Alphapose_Resnet50


          Backbone: Resnet50

        Input Size: 800

      Convenient to Use: For Human Pose Estimation

      Num_Joints17 Person Pose Keypoints

      Dataset Used: MS COCO


19.  Superpoint_pytorch

    

        Input Size: 600

      Convenient to Use: For Human Pose Estimation

      Num_Joints17 Person Pose Keypoints

      Dataset Used: MS COCO


20.  Headpose_FSANet

    

        Input Size: 600

      Convenient to UseFor Head Pose Estimation

      Num_Joints4

      Dataset Used: MS COCO


21.  mediapipe_face_mesh

    

        Input Size: 1280

      Convenient to Use: For 3D face landmarks

      Num_Joints: 468 3D face landmarks

     


How to create facial/pose landmarks on a Keypoint Dataset using S-O-T-A Pre-Trained Models


Steps to Follow


1. log in to the tool

2. Click on the Add button.

3. The Add Dataset Form will appear.

4. Fill in all the fields such as dataset name, dataset description, category-type, and categories.


4 a. Select category-type as Keypoints.

    4 b. Choose Pretraining Models from the available list.


        4 c. Categories according to the chosen model will appear.

               such as  pose17 or face68 [keypoint task followed by the no of landmarks the Pretrained model covers]


All Pose Estimation Pre Trained Models are trained on MS COCO Dataset.



5. Add data to NeuralMarker using the available options listed below:


To Learn More, Click Here.




6. Click on Submit Button

7. A New Dataset will be created.

8. After New Dataset is created: 

  • The Dataset card will display the status "Pretrain Model running". 
  • The brain symbol on hover will display "AI labeling in Progress".



9. After Pretrain Model stops running over the entire dataset:

  •  The Dataset card will display the status "Pretrain Model Done & Ready For Annotation". 
  •  The brain symbol on the card will display "AI Labeling Report".



10. After Pretrain Model stops running over the entire dataset, an AI labeling report will be generated:

  • With a confidence score of pose/face landmarks annotations over the images in the dataset.