Annotating Studies

This guide is for reviewers who annotate studies in SyRF projects. If you are a project administrator looking to design annotation questions, see Annotation Questions.

Getting Started with Annotation

Accessing Your Annotation Stage

To begin annotating studies:

1. Navigate to your project
2. Select the annotation stage from the left navigation panel
3. Click “Start Reviewing” to begin

When you enter an annotation stage, you will be presented with studies from your project’s systematic search. The study’s title and abstract are displayed, and if PDFs have been uploaded, you can view them as well.

Understanding the Study Queue

Studies are presented to you based on:

• Studies that need annotation (not yet completed by enough reviewers)
• Studies you have started but not finished (incomplete studies)
• Random selection from available studies

You will not be shown studies that have been sufficiently annotated by other reviewers.

When Proportional Allocation Is Enabled

Proportional allocation is disabled by default and is not generally available yet. If it is enabled for your service and stage, new annotation work is restricted to your allocated share. Opening a direct link does not bypass that restriction: a study outside your share returns a not-found response, and a new annotation submission for it is not saved. Return to the stage’s review queue for eligible work.

Your own saved annotation sessions in that stage remain accessible for resuming or correcting work, subject to the usual access checks. Reconciliation is unchanged. To see your allocation, open the stage’s Overview and find Study allocation. My assigned studies shows your assigned, completed, in-progress and remaining studies, with links to annotate new work or resume/review saved work. Use Previous and Next to browse the list and Refresh allocation for current counts.

Remaining includes work in progress. New work available can be smaller because other reviewers have filled the required slots. Counts include your preserved saved work and are snapshots; opening or saving a study checks access again.

If allocation says paused, the settings or configured project membership no longer match the allocation. This does not mean you have completed all assigned work. Saved work remains available, but no new work is allocated. Ask your project administrator to check the configuration. Studies cannot be opened for review while the stage is disabled.

Administrators with stage-design access also see Reviewer allocation progress on the overview: each configured reviewer’s slot share, assigned/completed/in-progress/ remaining counts and completion percentage. A 50% slot share does not necessarily mean 50% of studies; in a two-review stage it can mean one slot on every study. Verify that each reviewer can access the stage, since allocation does not grant access. When allocation is paused, the counts show preserved saved work rather than the original split; any configured reviewers no longer active are listed for administrators. Shares are not automatically redistributed. Restore membership where supported, or use a new annotation stage if the team or split needs to change after reviewing has begun.

Working with the Annotation Form

Form Layout

The annotation form is organised into tabs (also called categories). Each tab contains questions relevant to a specific aspect of the study:

• Study Level - Questions about the overall study
• Disease Model Induction - Questions about disease models used
• Treatment - Questions about treatments and interventions
• Outcome Assessment - Questions about outcome measures

If data extraction is enabled on your stage, you may also see additional tabs for Cohort and Experiment.

Accelerated Annotation Form

If the accelerated annotation form is enabled on the SyRF service you use, a supported ordinary annotation stage shows the same questions in a workspace designed to stay responsive on large forms:

• category tabs separate Study, Cohort, Disease Model Induction, Treatment, and Outcome Assessment questions;
• unit cards let you open only the items you are working on, with ten units per page;
• repeatable category items can be added, renamed, duplicated, hidden from the workspace, or deleted. Add cohort, Add treatment, and the equivalent category action put the complete new card at the top and focus its required label. Edit the label in the same card; press Enter or choose Save label to keep it, or press Escape or choose Cancel to leave the previous form unchanged; and
• a focused full-screen view is available when one unit needs more space.

Your answers and notes use Save progress and Complete. The accelerated form is not used for data-extraction, Experiment, reconciliation, or preview workflows yet, so those workflows continue to show the established annotation form.

Comments in the Accelerated Form

Choose Add comment beside an answer to write a note, or Edit comment to change an existing note. Cancel discards your typed changes in that comment editor; quotes inserted independently through Quote into comment… are retained. Save comment applies the note to your current form and displays it beside the answer; use Save progress to persist your review in the database. Saving progress also includes any comment editor you have left open.

Where quoting is available, select study text, choose Quote into comment…, and choose the destination answer. Use Save progress to persist the quote. Switching answer tabs or closing a full-screen entity view retains unsaved comments. Revert restores the last saved review.

Question Types

Your project administrator will have designed questions using different input types:

Required vs Optional Questions

• Required questions are marked with an asterisk (*)
• You cannot submit your annotation until all required questions are answered
• Optional questions can be left blank if not applicable

Nested and Conditional Questions

Some questions only appear based on your previous answers:

• A question might appear only if you select “Yes” to a parent question
• Nested questions are indented under their parent question
• If you change your answer to the parent question, nested questions may appear or disappear

Repeatable Answers and Dependent Questions

In the accelerated annotation form, repeated answers appear as tabs. Choose a tab to work on that answer and its dependent questions; switching tabs keeps your drafts and validation. Use Left/Right arrows, Home and End while focused on the tab strip. Each tab has a distinct status symbol and an accessible status description, and its answer supplies its name.

Use the branch set’s Add action for another answer. Deleting a populated branch asks for confirmation and also removes its dependent answers and comments. After deleting the active branch, the previous answer is selected; if there is no previous answer, the next one is selected. This may return you to an earlier question page. Tab selection is remembered while you work in the current review; save progress to persist your answers.

Some questions let you record several separate answers, each with its own dependent questions. Use the Add another … action shown directly below that question’s answers. When there is more than one answer, SyRF labels and visually groups each answer so you can tell which dependent questions belong to it.

In the established form, select a branch heading to collapse or expand it. A collapsed heading keeps the branch’s identifying answer visible so you can scan the list without losing your place. With three or more branches, use Collapse all to scan them or Expand all to return to the full form. Adding a branch opens the new branch without changing the others.

The heading also says whether a branch is Complete, Incomplete, or Needs attention. These states use both an icon and text. If saving finds an invalid answer, SyRF opens every affected branch and takes you to the first answer to correct. Collapsing never clears an answer or its validation state.

Deleting one of these answers also deletes its dependent answers and comments. SyRF shows a confirmation warning before removing them.

Managing Your Work

Saving Progress

You don’t have to complete all questions in one session:

• Click “Save” to save your progress and return later
• Your incomplete annotations are saved and can be resumed
• Each saved study counts towards any session limit your administrator has set

Completing an Annotation

When you have answered all required questions:

1. Review your answers across all tabs
2. Click “Submit” to complete your annotation
3. SyRF will record your annotation and move you to the next study

Returning to Incomplete Studies

To continue working on a study you previously started:

1. Look for the “Incomplete Studies” panel on your annotation stage
2. Click on a study to resume where you left off
3. Complete and submit when ready

Session Limits

Your project administrator may have set a limit on incomplete studies:

• This limits how many studies you can have “in progress” at once
• If you reach the limit, you must complete or submit existing studies before starting new ones
• This encourages reviewers to finish studies rather than accumulate many incomplete ones

Understanding Your Progress

Progress Indicators

The progress bar shows:

• Completed - Studies you have fully annotated
• In Progress - Studies you have started but not finished
• Available - Studies waiting to be annotated
• Unavailable - Studies already sufficiently annotated by others

Multi-Reviewer Workflows

Your project may require multiple reviewers to annotate each study:

• By default, SyRF expects two independent reviewers per study
• If reviewers disagree, a third reviewer may reconcile
• You will not see studies already sufficiently annotated by others

Working with Study Information

Reading the Study

While annotating, you have access to:

• Title and Abstract - Displayed prominently in the form
• PDF Viewer - Click the PDF button to view uploaded documents
• Study Metadata - Authors, publication year, and other details

Navigate between the study information and the annotation form as needed to answer questions accurately.

Tips for Efficient Annotation

1. Read the study first - Skim the abstract before starting questions
2. Work through tabs in order - This often follows the logical flow of information
3. Use keyboard navigation - Tab between fields, Enter to select
4. Save frequently - Don’t lose work if you need to step away
5. Check conditional questions - Changing an answer may reveal new questions

When Data Extraction is Enabled

If your administrator has enabled data extraction for your stage, you will see additional question categories and a more complex form structure. This allows you to:

• Create units (disease models, treatments, outcomes)
• Link units together to form cohorts
• Group cohorts into experiments
• Enter quantitative outcome data (means, medians, error values)

For detailed instructions on data extraction, see Data Extraction Guide.