Administrator Guide

This guide covers managing annotation campaigns: setting up authentication, monitoring progress, ensuring quality, and exporting results.

Setting Up Authentication

Choose an authentication method based on your deployment:

Method Best For Guide
In-memory Local development Default, no config needed
Password + file Team annotation Password Management
Database Production deployments Password Management
OAuth / SSO Institutional login SSO & OAuth
Passwordless Low-stakes, easy access Passwordless Login

For user registration and access control, see Users & Collaboration.

Quality Control

Ensure annotation quality with multiple strategies:

  • Quality Control - Attention checks, gold standards, and inter-annotator agreement
  • Training Phase - Require annotators to pass practice rounds before the real task
  • Adjudication - Expert review of annotator disagreements
  • MACE - Statistical estimation of annotator competence

Monitoring Progress

  • Admin Dashboard - Real-time monitoring of annotation progress, annotator performance, and suspicious activity detection
  • Behavioral Tracking - Timing, click patterns, and annotation change history
  • Annotation History - Complete audit trail of all annotation actions
  • Productivity - Keyboard shortcuts, tooltips, and efficiency features

Managing Annotators

Data Management

Troubleshooting