Introduction


NeuralMarker through its queue management system makes sure that no annotator among a data annotation team gets the same or already annotated image while working parallelly on the same dataset of an Image annotation task.




How NeuralMarker Queue Management System Works


NeuralMarker Queue Management system built with a need to address a couple of measures such as :

  • How to distribute unlabeled training data among team members of a big Annotation team.


  • How to make a failproof workflow, so that no two labelers of a team:

a. At the same time get the same unlabeled images in their Annotation workflow.


b. At the same or at different intervals of time get the already annotated images in each other Annotation workflow.





Queue Mangement System for Unlabeled Data is applied to all category types present in NeuralMarker. 



How its Works


1. Suppose if 2 or more than 2 Annotators click on Annotate button available on the dataset card in the "Add dataset" page of NeuralMarker at the same time.



2. So once Annotators land on the Annotation page through Annotate button, NeuralMarker in order to provide unique and unlabeled images to these annotators in their annotation workflow.




  • Creates a queue and puts 5 unique images in each queue which includes:


  • The current image on which an individual annotator landed by clicking on annotate button 


    + 


  • 4 images following the current image.


3. After creating the queue, NeuralMarker locks these 5 unique images for each individual annotator in their respective queue.


4. Then till all individual annotators annotate or skip (if the object of interest is not present) the respective images in their queue, these images remain locked in.




5. So that no other annotator except to whom it is allotted can change or manipulate these images by going through the preview page section. 



6. Once all the images inside the queue are annotated or skipped (if the object of interest is not present) by annotators in their respective annotation workflow.



7. Only then, these 5 images are released, and only then anybody i.e. another annotator or QA Manager can review or make changes in those images.


8. This entire process of queue creation + image locking process which popularly known as the Queue Management system in NeuralMarker. Goes on and on until the entire dataset of relevant images gets annotated by every annotator in their respective annotation workflow for a training dataset.