next up previous
Next: Semantic accuracy. Up: Criteria Previous: Criteria

String accuracy.

String accuracy measures the distance of the test sentence and the best sentence. String accuracy is expressed in terms of sentence accuracy (SA, the proportion of cases in which the test sentence coincides with the best sentence), and word accuracy (WA). The string comparison on which word accuracy is based is defined by the minimal number of substitutions, deletions and insertions of words that is required to turn the best sentence into the test sentence (Levenshtein distance). Word accuracy is defined as

\mbox{\it WA} = 1 - \frac{d}{n}
\end{displaymath} (4)

where n is the length of the actual utterance and d is the Levenshtein distance. For example, if the analysis gives 'a b a c d' for the utterance 'a a c e', then the Levenshtein distance is 2, hence the WA is 1-2/4 is 50%.