Introduction
NeuralMarker offers a variety of state-of-the-art Pretrained Deep Learning Models for Image Classification from different available frameworks like PyTorch, Dlib, and Mediapipe.
These available Pretrained deep learning models cover wide categories of Keypoint tasks such as :
- Facial landmark detection
- Human Pose Estimation
- Head Pose Estimation
This list of models at hand is not static, as more models in the Neural Network Library of NeuralMarker will be added, then accordingly the below list will be updated too:
Pretrained Models
1. Dlib-68-points
Model Architecture: Dlib-68
No of landmarks: 68
Convenient to Use: For Facial landmark Detection
Dataset Used: 300 Faces In-the-Wild
2. Hrnet_w32_coco
Model Architecture: pose_hrnet_w32
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
3. Hrnet_w48_coco
Model Architecture: pose_hrnet_w48
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
4. Detectron2_R50FPN
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
5. Detectron2_R50FPN_3x
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
6. Detectron2_R101FPN
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
7. Pose_Mediapipe
Convenient to Use: Body Pose Tracking
No of landmarks: 25 upper body landmarks
Dataset Used: MS COCO
8. Hrnet_mpii_w32
Model Architecture: pose_hrnet_w32
Convenient to Use: For Human Pose Estimation
Num_Joints: 7 Person Pose Keypoints [Head, Shoulder, Elbow, Wrist, Hip, Knee, Ankle]
Dataset Used: MPII Human Pose dataset
9. Hrnet_mpii_w48
Model Architecture: pose_hrnet_w48
Convenient to Use: For Human Pose Estimation
Num_Joints: 7 Person Pose Keypoints [Head, Shoulder, Elbow, Wrist, Hip, Knee, Ankle]
Dataset Used: MPII Human Pose dataset
10. Hrnet_resnet152_mpii
Model Architecture: pose_resnet_152
Convenient to Use: For Human Pose Estimation
Num_Joints: 7 Person Pose Keypoints [Head, Shoulder, Elbow, Wrist, Hip, Knee, Ankle]
Dataset Used: MPII Human Pose dataset
11. Hrnet_resnet101_mpii
Model Architecture: pose_resnet_101
Convenient to Use: For Human Pose Estimation
Num_Joints: 7 Person Pose Keypoints [Head, Shoulder, Elbow, Wrist, Hip, Knee, Ankle]
Dataset Used: MPII Human Pose dataset
12. Hrnet_resnet50_mpii
Model Architecture: pose_resnet_50
Convenient to Use: For Human Pose Estimation
Num_Joints: 7 Person Pose Keypoints [Head, Shoulder, Elbow, Wrist, Hip, Knee, Ankle]
Dataset Used: MPII Human Pose dataset
13. Hrnet_resnet152_COCO
Model Architecture: pose_resnet_152
Input Size: 256
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
14. Hrnet_resnet101_COCO
Model Architecture: pose_resnet_101
Input Size: 256
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
15. Hrnet_resnet50_COCO
Model Architecture: pose_resnet_50
Input Size: 256
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
16. Hrnet_w48_coco_384
Model Architecture: pose_hrnet_w48
Input Size: 384
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
17. Hrnet_w32_coco_384
Model Architecture: pose_hrnet_w48
Input Size: 384
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
18. Alphapose_Resnet50
Backbone: Resnet50
Input Size: 800
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
19. Superpoint_pytorch
Input Size: 600
Convenient to Use: For Human Pose Estimation
Num_Joints: 17 Person Pose Keypoints
Dataset Used: MS COCO
20. Headpose_FSANet
Input Size: 600
Convenient to Use: For Head Pose Estimation
Num_Joints: 4
Dataset Used: MS COCO
21. mediapipe_face_mesh
Input Size: 1280
Convenient to Use: For 3D face landmarks
Num_Joints: 468 3D face landmarks
How to create facial/pose landmarks on a Keypoint Dataset using S-O-T-A Pre-Trained Models
Steps to Follow
1. log in to the tool
2. Click on the Add button.
3. The Add Dataset Form will appear.
4. Fill in all the fields such as dataset name, dataset description, category-type, and categories.
4 a. Select category-type as Keypoints.
4 b. Choose Pretraining Models from the available list.
4 c. Categories according to the chosen model will appear.
such as pose17 or face68 [keypoint task followed by the no of landmarks the Pretrained model covers]
All Pose Estimation Pre Trained Models are trained on MS COCO Dataset.5. Add data to NeuralMarker using the available options listed below:
- Google Drive Link or S3 link
- CSV File with Image URL's
- Drag and Drop
6. Click on Submit Button
7. A New Dataset will be created.
8. After New Dataset is created:
- The Dataset card will display the status "Pretrain Model running".
- The brain symbol on hover will display "AI labeling in Progress".
9. After Pretrain Model stops running over the entire dataset:
- The Dataset card will display the status "Pretrain Model Done & Ready For Annotation".
- The brain symbol on the card will display "AI Labeling Report".
10. After Pretrain Model stops running over the entire dataset, an AI labeling report will be generated:
- With a confidence score of pose/face landmarks annotations over the images in the dataset.
