Post-judging QA (PJQA)

After preliminary judging is complete, there is a 48-hour period during which the sponsor team and certified wardens who submitted at least one finding can raise questions or concerns for the judge's review. This phase of the judging process is called Post-Judging QA, or PJQA for short; you may be familiar with terms like "escalations" or "appeals," but at Code4rena the focus is on quality assurance for the judging process.

PJQA's objectives are to:

• ensure that Code4rena's judging criteria are implemented consistently, and

• surface any errors regarding duplication (e.g. overlooked duplicates or mis-groupings).

PJQA rules

PJQA comment limits

PJQA comment limits vary according to signal score.

One submission per comment

Each PJQA comment should focus on a single submission (or set of duplicate submissions).

No new evidence

All evidence should be presented in your original submission.

• Elaborations of the already introduced information can be considered (e.g. tweaking a POC), but they will only count towards the validity of the issue, not its quality score.

• Screenshots of private conversations with members of the project team, judge, etc. are not acceptable as evidence, and will be removed.

Comments only via app

All feedback for the judge should be provided in a comment on the relevant submission, via the C4 app interface.

Ad hoc discussions about judging in the C4 Discord, DM-ing judges, etc. violate the Good citizenship policy and may result in disciplinary action.

48 hour comment window

Warden comments are only allowed during the 48-hour PJQA comment window. C4 staff will post a notice in the ⁠C4 Discord when the comment window opens.

You can opt in to receive Discord notifications when post-judging QA opens.

🔔 To enable notifications:

1. Visit the #⁠manage-notifs channel

2. React to the post with a 🔔

3. You should now have the pjqa-notifs role, which will be tagged in future PJQA announcements.

🔕 To disable notifications:

1. Visit the #⁠manage-notifs channel

2. De-select the 🔔 - this will remove the pjqa-notifs role.

No solicitation of others to comment for you

Wardens must not solicit other wardens, judges, or C4 staff to perform PJQA actions on their behalf; doing so is a violation of C4's Good citizenship policy.

If someone asks you to comment on their behalf, decline and report it to C4 staff if it persists.

Fact-based and respectful dialogue

Focus on technical facts, evidence, and analysis.

Comments that are insulting, accusatory, or that question others' motives will be removed and may result in loss of PJQA privileges.

Violations of these guidelines may result in comment removal, suspension of PJQA privileges, or other disciplinary action at the discretion of C4 staff.

Logistics

Public vs private comments

The comment form includes a checkbox option for Private discussion. If this option is checked, only you, the judge, sponsor, and C4 staff will be able to see the comment and its child replies.

If the Private discussion option is not checked, your comments will be:

• visible to all wardens, the sponsor team, judge, and staff until the report is published, and

• public once the audit report is published.

Judge replies

• Judges may respond during the comment window, but often they review all comments together afterward.

• Judges cannot respond to every PJQA comment in detail.

• If the judge replies to you during the comment window, you may reply to either a) answer their question, or b) correct a factual error in their response.

• Further back-and-forth will result in suspended commenting privileges.