Introduction

In a broader sense, NeuralMarker allows Admin account holders to monitor annotator's performance over multiple datasets on an everyday real-time basis till they annotate those datasets entirely.




How to reach Stats Page


1. Click on the View Stats" symbol present on the top right border of a dataset , then after that stats page will appear.  




2. "View Stats" is also available on the right-most corner of the preview page. Click on the View Stats button then after that stats page will appear.


Attributes of Stats Page


NeuralMarker for every dataset and for every Category type monitors the Annotator's performance on both images as well as annotations level.


Information about various attributes of a Stats Page is given below:


Attribute 1. Name of the Dataset: 


On the Stats page, always at the top left most corner of the page, DataSet Name will be present.





Attribute 2. DataSet Summary: 


This section on the stats page displays the overall information about the dataset.



A. It gives information about the total no of images present in the dataset.


B. It gives information about the category/categories present in the dataset.


C. It gives information about the Dataset Creation Date.


D. It gives Information about  "how many images in the dataset" annotated till now.


E. It gives Information about the "Expected time of completion" of the dataset.



Attribute 3. Calendar Filter: 


This filter gives more control to users to analyze annotator's stats not only on a real-time basis but also on a daily and even on a per/hour basis for an image dataset.


How It Works:


1.  Click on the date range visible on the left-hand side of the stats page, just next to Annotation Summary.


Initially, the date range on the stats page will represent the start date and the current date which a user can change easily by using the calendar.




2. A Calendar with multiple filters in the drop-down will appear 



3. Once a user selects any of the available options/filters from the drop-down then accordingly the date range in the calendar changes.


A user can select a Date Range in the calendar in two ways:


Method 1: Through Available Filters on the drop-down





Method 2: Manual Selection


If a user instead of filters want to select manually a date range, then they can do it by :


a. First selecting the start date range by just single clicking on a date, then scroll through all the dates for which they want to analyze the annotator's performance.


b. Then they can scroll through and click on the end date range (i.e. the date till which they want the stats of the annotators).




IMP: Calendar Filter also allows users to change time in the calendar in order to access the stats of the annotators with respect to a particular time.



E. After selecting the date range using any of the available methods and a suitable time, click on Apply present on the right-side bottom of the filter, and then stats based upon the selected date range will appear.



Attribute 4. Refresh Button:


In NeuralMarker Stats for the current in-progress datasets are refreshed on an hourly basis. Therefore Admin has to use or click this below-included button to get the latest stats details.




Attribute 5. Annotation Summary on Image Level:


This section on the stats page displays the overall information about the total no of images skipped by the annotators and also about how many annotations were drawn over the images by them.



A. It gives information about the average time taken by annotators in order to draw a single annotation on the object of interest present in the image.


B. It gives information about the "Number of annotations done" till now on the image dataset.


C. It gives information about the total number of images skipped by the entire team of labelers.



Attribute 6. Individual's Annotator Stats:


This section on the stats page displays the information about the overall work done by all the annotators individually on the dataset.



A. It gives information about the annotator's name.


B. It gives information about the total time spent by the annotator while annotating images present in the dataset.


C. It gives information about the total images annotated by the annotator till the selected date range.


D. It gives Information about the "Average time per annotation" taken by the annotator while annotating the object of interest in an image of a dataset.


E. It gives Information about the total annotations done by the annotators on the available images of the dataset.


F. It gives Information about the Average time spent by the annotator to annotate an image in the dataset.



  • NeuralMarker records time on the annotation page of an annotator from the moment the annotator lands on the annotation page till the moment he/she clicks on the "Save and Next button".


  • If in-between while creating annotations over the object of interest, the annotator opens a new tab or spent some time on another website, so during that time, the timer on the annotation page will get paused until the annotator comebacks to the annotation page.


On Dataset card, view stats button is also available for annotators or non-admin account holders so that they can view their own stats.



G. It gives Information about the no of images skipped by the annotator in the dataset.