These are bot metrics I found Googling. It will be updated as I find more.
While bot metrics can include KPIs like daily active users, being in the early days of bots, it would be better to focus on how well bots work in fulfilling the customers’ experience. Most other metrics related to funnels and sales should still be applicable.
|Metric||Type||Data Collection Method|
|Total elapsed time||Efficiency||Quantitative Analysis|
|Total number of user/system turns||Efficiency||Quantitative Analysis|
|Total number of system turns||Efficiency||Quantitative Analysis|
|Total number of turns per task||Efficiency||Quantitative Analysis|
|Total elapsed time per turn||Efficiency||Quantitative Analysis|
|Number of re-prompts||Qualitative||Quantitative Analysis|
|Number of user barge-ins||Qualitative||Quantitative Analysis|
|Number of inappropriate system responses||Qualitative||Quantitative Analysis|
|Concept Accuracy||Qualitative||Quantitative Analysis|
|Turn correction ratio||Qualitative||Quantitative Analysis|
|Ease of usage||Qualitative||Questionnaire|
|Robustness regarding misunderstandings||Qualitative||Questionnaire|
|Willingness to use system again||Qualitative||Questionnaire|
- Dialogue efficiency in terms of matching type.
- Dialogue quality metrics based on response type.
- Users’ satisfaction assessment based on an open-ended request for feedback.