Multi-Phase Workflows¶

Potato supports multi-phase annotation workflows that guide annotators through a structured sequence of stages. This is useful for setting up consent forms, pre-screening questions, training/qualification, the main annotation task, and post-study surveys.

Phase System Overview¶

Potato's workflow consists of the following phases (in order):

Phase	Purpose	Required
`consent`	Collect informed consent	Optional
`prestudy`	Pre-screening questions, demographics	Optional
`instructions`	Task instructions and guidelines	Optional
`training`	Qualification/training with feedback	Optional
`annotation`	Main annotation task	Required
`poststudy`	Post-study surveys, feedback	Optional

Users automatically progress through enabled phases in this order.

By default, the Back button is hidden when there is no previous page to navigate to. This means:

On the very first page of the workflow (e.g., the consent page), the Back button is not shown.
On the first annotation instance, the Back button is hidden unless phase back-navigation is enabled.

To allow users to navigate backward across phases (e.g., from the annotation phase back to the instructions phase), set:

allow_phase_back_navigation: true

When this is enabled, the Back button will appear on all pages, allowing users to return to previous phases. When disabled (the default), users can only navigate backward within the current phase — for example, moving between annotation instances or between pages within a multi-page instructions phase.

Configuration¶

Modern Phases Configuration (Recommended)¶

The recommended way to configure multi-phase workflows is using the phases section in your YAML config:

phases:
  order: [consent, prestudy, instructions, annotation, poststudy]

  consent:
    type: consent
    file: surveyflow/consent.json

  prestudy:
    type: prestudy
    file: surveyflow/prescreening.json

  instructions:
    type: instructions
    file: surveyflow/instructions.html

  poststudy:
    type: poststudy
    file: surveyflow/demographics.json

Phase Types¶

Each file-based phase has a type that tells Potato which stage of the workflow it belongs to. Valid values are:

`type`	Purpose
`consent`	Informed consent
`prestudy`	Pre-screening / demographics shown before annotation
`instructions`	Task instructions (supports `.html` or survey JSON)
`training`	Qualification/training with feedback
`annotation`	The main annotation task
`poststudy`	Post-study survey / feedback

Always set type explicitly. As a convenience, if you omit type and the phase's name is itself one of the values above (e.g. a phase literally named consent), Potato infers the type from the name. Custom-named phases (e.g. intro, feedback) must declare a type:

phases:
  order: [intro, consent, instructions, annotation, feedback]

  intro:
    type: prestudy          # custom name -> type is required
    file: surveyflow/intro.json

  consent:
    type: consent
    file: surveyflow/consent.json

  instructions:
    type: instructions
    file: surveyflow/instructions.json

  feedback:
    type: poststudy         # custom name -> type is required
    file: surveyflow/feedback.json

Note: the annotation phase needs no file — it uses the project's top-level annotation_schemes.

Phase Data Files¶

Each phase references a data file containing the questions or content for that phase.

JSON Format for Survey Questions¶

[
  {
    "id": "consent_1",
    "text": "I certify that I am at least 18 years of age.",
    "schema": "radio",
    "choices": ["I agree", "I disagree"],
    "label_requirement": {"right_label": ["I agree"]}
  },
  {
    "id": "consent_2",
    "text": "I have read and understood the information above.",
    "schema": "radio",
    "choices": ["Yes", "No"],
    "label_requirement": {"right_label": ["Yes"]}
  }
]

Supported Schema Types¶

radio - Single choice from options
checkbox / multiselect - Multiple selections
text - Free text response
number - Numeric input
likert - Likert scale rating
select - Dropdown selection

Label Requirements¶

Control which answers are acceptable to proceed:

{
  "id": "age_check",
  "text": "Are you at least 18 years old?",
  "schema": "radio",
  "choices": ["Yes", "No"],
  "label_requirement": {
    "right_label": ["Yes"],
    "required": true
  }
}

right_label: List of acceptable answers (user must select one to proceed)
required: Whether the question must be answered

Pre-Screening Questions¶

Pre-screening questions example

Pre-screening questions appear before the main annotation task. Use them to: - Verify eligibility (age, consent) - Collect demographic information - Assess prior knowledge

Example Prestudy Configuration¶

phases:
  order: [prestudy, annotation]
  prestudy:
    type: prestudy
    file: surveyflow/prescreening.json

prescreening.json:

[
  {
    "id": "1",
    "text": "What is your native language?",
    "schema": "select",
    "use_predefined_labels": "language",
    "label_requirement": {"required": true}
  },
  {
    "id": "2",
    "text": "How familiar are you with this topic?",
    "schema": "likert",
    "choices": ["Not at all", "Slightly", "Moderately", "Very", "Extremely"],
    "label_requirement": {"required": true}
  }
]

Post-Study Surveys¶

Post-study questions example

Post-study surveys appear after annotation is complete:

phases:
  order: [annotation, poststudy]
  poststudy:
    type: poststudy
    file: surveyflow/demographics.json

demographics.json:

[
  {
    "id": "gender",
    "text": "What gender do you most closely identify with?",
    "schema": "radio",
    "choices": ["Male", "Female", "Non-binary", "Prefer not to say"],
    "label_requirement": {"required": true}
  },
  {
    "id": "feedback",
    "text": "Please share any feedback about this study (optional)",
    "schema": "text"
  }
]

Built-in Question Templates¶

Potato provides predefined label sets for common survey questions:

Countries¶

{"schema": "select", "use_predefined_labels": "country"}

Languages¶

{"schema": "select", "use_predefined_labels": "language"}

Ethnicity¶

{"schema": "select", "use_predefined_labels": "ethnicity"}

Religion¶

{"schema": "select", "use_predefined_labels": "religion"}

Complete Example¶

Here's a complete multi-phase workflow configuration:

annotation_task_name: "Sentiment Annotation Study"

# Enable multi-phase workflow
phases:
  order: [consent, prestudy, instructions, training, annotation, poststudy]

  consent:
    type: consent
    file: phases/consent.json

  prestudy:
    type: prestudy
    file: phases/demographics.json

  instructions:
    type: instructions
    file: phases/instructions.html

  poststudy:
    type: poststudy
    file: phases/exit_survey.json

# Training phase configuration (optional)
training:
  enabled: true
  data_file: phases/training_questions.json
  passing_criteria:
    min_correct: 3
    max_attempts: 2

# Main annotation configuration
data_files:
  - data/instances.json

annotation_schemes:
  - name: sentiment
    annotation_type: radio
    labels: [Positive, Negative, Neutral]
    description: "Select the sentiment of this text"

Free Response Fields¶

Add optional text input to any question:

{
  "id": "gender",
  "text": "What is your gender?",
  "schema": "radio",
  "choices": ["Woman", "Man", "Non-binary", "Prefer not to disclose", "Prefer to self-describe"],
  "has_free_response": {"instruction": "Please specify:"},
  "label_requirement": {"required": true}
}

Conditional Questions (Display Logic)¶

Survey questions can be shown or hidden based on earlier answers, so you can build adaptive surveys without splitting the workflow into extra phases — reveal a follow-up only when it is relevant, show a free-text box when "Other" is picked, or ask for an explanation only on a low rating.

Add a display_logic block to any survey question. Because survey questions go through the same schema pipeline as annotation schemes, they reference other questions by their name (not id):

[
  {
    "name": "prior_experience",
    "annotation_type": "radio",
    "description": "Have you previously worked on a text-annotation task?",
    "labels": ["Yes", "No"],
    "label_requirement": {"required": true}
  },
  {
    "name": "experience_details",
    "annotation_type": "text",
    "description": "Please briefly describe your previous annotation experience.",
    "label_requirement": {"required": true},
    "display_logic": {
      "show_when": [
        {"schema": "prior_experience", "operator": "equals", "value": "Yes"}
      ]
    }
  }
]

experience_details stays hidden — and is excluded from required-field validation, so it never blocks the Continue button — until the participant answers "Yes".

Key behaviors specific to SurveyFlow:

Cross-page conditions. A poststudy question can condition on a prestudy answer; references resolve against questions on any phase. Earlier-phase answers are injected into each page so the condition is evaluated on load.
Startup validation. Invalid survey display_logic (unknown operator, a reference to a question that exists on no phase, or a circular dependency) fails fast when the server starts.
Server-side enforcement. With export_include_phase_data: true, Potato re-evaluates each participant's answers server-side and excludes answers to questions they never actually saw. Opt out with exclude_hidden_survey_answers: false (answers are kept, tagged hidden).
JSON/instrument questions only. An instructions phase pointing at a raw .html file bypasses the schema pipeline, so display_logic does not apply there.

The full operator reference (equals, contains, matches, numeric/range, emptiness, text length), all/any logic, and troubleshooting are documented in Conditional Logic. A runnable example lives at examples/advanced/surveyflow-conditional-logic/:

python potato/flask_server.py start examples/advanced/surveyflow-conditional-logic/config.yaml -p 8000

Page Headers¶

Customize the header text displayed on each survey page:

phases:
  prestudy:
    type: prestudy
    file: surveyflow/consent.json
    header: "Please answer all consent questions"

Legacy Surveyflow Configuration¶

Note: The configuration format below is deprecated but still supported for backward compatibility. New projects should use the phases configuration format shown above.

The legacy surveyflow configuration uses this format:

surveyflow:
  on: true
  order:
    - pre_annotation
    - post_annotation
  pre_annotation:
    - surveyflow/consent.jsonl
    - surveyflow/demographics.jsonl
  post_annotation:
    - surveyflow/exit_survey.jsonl

Migration Guide¶

To migrate from legacy surveyflow to the modern phases system:

Legacy	Modern
`surveyflow.on: true`	Use `phases` section
`surveyflow.pre_annotation`	`phases.prestudy` or `phases.consent`
`surveyflow.post_annotation`	`phases.poststudy`
`surveyflow.testing`	`training.data_file`
`.jsonl` files	`.json` files (array format)

Legacy surveyflow_html_layout¶

If you were using surveyflow_html_layout to customize survey page appearance:

# Legacy
surveyflow_html_layout: "templates/survey-layout.html"

# Modern equivalent - customize via UI configuration
ui_configuration:
  phase_layout: "templates/survey-layout.html"

Conditional Logic - Show/hide survey questions based on prior answers
Training Phase - Configure qualification training with feedback
Category-Based Assignment - Assign tasks based on training performance
Configuration - Full configuration reference