Skip to content

Release Notes: v2.4.5 — Validated Refinement, Config Validator & Stability

A stability and tooling release: a pluggable validated-refinement framework for solo-mode guideline improvement, a new config-validator CLI, a security fix for path validation, and a broad set of navigation, Prolific, and solo-mode fixes. Documentation was also reorganized into topic-based subdirectories.

Solo Mode & Validated Refinement

  • Validated refinement framework — pluggable strategies for iteratively improving annotation guidelines, with guardrails against guideline contradictions and metadata leakage, a consistent-disagreement filter to reduce refinement noise, and evaluation at temperature=0.
  • Solo-mode pipeline fixes — corrected overlap, refinement, and output-parsing in the end-to-end pipeline; non-string labels in _fuzzy_match_label no longer crash; sampling-diversity confidence fixes plus ICL examples.
  • Performance — batched n-parameter sampling for ~4× faster diversity sampling.
  • Tooling — CLI to view the refinement log from the state directory; hybrid-strategy configs and a comparison script.

Tooling & Validation

  • potato.validate_cli — a config-validator CLI; user_config and solo_mode keys added to the known-config schema; dead config keys removed from all 115 example configs.

Security

  • Path-traversal fix — fixed a sibling-prefix bypass in validate_path_security (GHSA-q9m2-fhv9-3jcf).

Bug Fixes

  • Fixed save/navigation state-sync bugs that could cause silent data loss.
  • Fixed Prolific integration, form submission, logout, and block-loop bugs.
  • Fixed annotation-page back-navigation crash and centralized required-annotation logic (#147).
  • Fixed crash on consent/instructions pages due to undefined instance_index (#146).
  • Show instance number in the navbar; hide nav controls when jumping is disabled (#145).
  • Hide the Back button on the first page of multi-phase workflows (#144).
  • Fixed QC feedback display; prune stale attention-check items from user state.
  • Fixed coding_trace empty-data path to include the span wrapper when span_target is set.

Documentation

  • Reorganized documentation into topic-based subdirectories.
  • Removed potato list/get commands and the hardcoded project hub.
  • Added Playwright browser tests for concurrency, network resilience, and scale.

Upgrade

pip install --upgrade potato-annotation==2.4.5