AutoScoring removes subjectivity and saves time for reviewers who can focus on categories that require more attention.
There are six categories that will get an automatic scoring in Klaus:
Greeting - fixed binary scale 👍 👎
Grammar - fixed 5-point scale
Empathy - fixed binary scale 👍 👎
Closing - binary scale 👍 👎
Tone - fixed 5-point scale
Solution - binary scale 👍 👎
How do the AutoQA categories work?
Greeting - Answers the question “Did the agent greet the customer?”
Closing - Assesses the conversation answering question: "Did the agent close the conversation, take into consideration that offering further help and thanking the customer is a good closing message?"
Empathy - Assesses the conversation answering question “Was the Agent empathetic towards the customer and their problems?"
Tone - Recognizes the following 27 tones and calculates a score in 5 point scale based on that. Each tone has different weight, e.g. joyful 2.9 and frustrated -3. A conversation can have multiple different tones, each found tone has different positive or negative weights that are aggregated together. Tones: accusatory, worried, curious, surprised, formal, confident, appreciative, concerned, sad, informal, regretful, encouraging, egocentric, joyful, optimistic, excited, polite, professional, frustrated, angry, disgusted, fun, friendly, supportive, polite, professional, apologetic.
Solution - Identifies whether or not a solution was proposed in the conversation. 👍 if the agent offer a solution to the customer. If not then 👎 And if not sure "N/A".
Spelling & Grammar - Detects all kinds of grammar mistakes which are grouped into 3 groups: grammar mistakes, misspelling mistakes and style mistakes.
To set up AutoScoring, one needs to go to Settings > Users & Workspaces > choose the Workspace > Scorecard > add new or choose an existing category > enable Auto QA for that category to start using automatic scores.
You can then choose the automatic category data that will be attached from a pre-populated list and add or change the name and description of the category.
In the Conversation view, Auto Scores that have been found in the conversation will be marked as graded with a hologram icon. Auto Scores can be different for different agents, and if a reviewer does not agree with the Auto Score and prefers to change it, they can click on a different rating and submit the review from the bottom right Submit button as usual.
The reviewer can also dig deeper by enabling Highlight and check which part of the conversation exactly has been graded. The Auto Score will only be calculated once the reviewer submits the review and the result will be visible only on the Auto QA Dashboard.
If the reviewer needs to find conversations that were Klaus automatically found errors in grammar for example, they can use the Conversation view filter for that.
The Auto QA Dashboard provides information about auto-reviewed conversations and you can learn more about it here.
Disclaimer:
The AutoQA scoring will show results for any new (eligible for scoring - for example, grammar doesn't work on calls), closed conversations that have been imported after the feature has been enabled. It might take a bit for the scores to load due to the necessary calculations.
MS Azure OpenAI processing and option to opt out is under account settings.
AutoQA uses Microsoft Azure OpenAI Service for automatically scoring some categories. Your data is processed under a strict DPA only to provide AutoQA to you and is never used to train any OpenAI models.
Microsoft Azure OpenAI Service enables a wider range of AutoQA categories; out-of-the-box multilingual support; fast paced addition of new categories; Improved evaluation accuracy. GPT-3.5 shows human-level contextual awareness; benefit from the latest advancements in AI technology.
For most common questions around AutoQA Auto Scoring, please check this article.
*If you opt out of Azure OpenAI processing, then AutoQA Grammar is only available for English (US and UK), French, German and Polish.