'Calibration is usually only talked about when there are dedicated reviewers, while I think it's especially helpful for peers. It is basically an empathy challenge/learning opportunity/team cohesion exercises rolled all in one.'
Valentina - Empress of Product, Klaus
What is Calibration?
Calibration is the practice of all of your reviewers rating the same batch of conversations and comparing what scores and comments they left. This is something which should be done regularly to make sure that your reviewers are on the same page when giving reviews. This means that your agents will receive similar reviews each week irrelevant of who they are being reviewed by.
How can you set up Klaus for calibration?
Calibration can be done within the existing Workspace and the Calibration scores won't affect the existing IQS.
Calibration settings can be accessed under Workspace Settings. There you can select also the visibility of the calibration reviews. You can either allow your Workspace Reviewers to only see their own reviews or you can allow them to see their peers' reviews after they submit the calibration review. Workspace Managers can have a bigger overview of the calibration process if allowed to always see all reviews.
Agents don't see anything conversation-related in Calibration.
No notifications are sent to reviewers so it's up to the Workspace Manager to let reviewers know about the Calibration Session and its due date.
How to do the Calibration?
You can do the calibration within the same Workspace where you conduct your regular reviews. The calibration section is under the Conversation view and you can add a new Calibration session by clicking on the + button. Here you can also find the list of previous Calibration sessions.
All Calibration sessions have a deadline that can be extended.
When you find a conversation you want to add to the Calibration session, you just need to use the button to add/remove it to the session.
Once the reviews have been submitted, you can select a Baseline (final review) - mark one review as a baseline for others, so it would be easier to compare their score against it. The Baseline review is always on top and is highlighted with the label “Baseline review”. Workspace managers can add and remove baseline reviews.
Where do I find the results?
The IQS of the chosen Workspace will not be affected by the Calibration reviews. Instead, you can find the list of conversations and the reviews attached to them in the Calibration Dashboard. No notifications are triggered, calibration reviews, comments and scores won't be visible on the main dashboard, only in the Calibration Dashboard.
Baseline review is always the first one and is highlighted with the label “Baseline”.
Actions not possible in Calibration review
Leave a comment without score
@ mention a user
Add a # hashtag
Highlight conversation for later
Notification sent to reviewee
Start a dispute
See "review seen" status
Reply to a review
Leave a comment for reviewee
Review several reviewees
See review count