Contents

• 13 Evaluation
  • 13.1 Overview of Evaluation in Speech and Natural Language Processing
    • 13.1.1 Introduction to Evaluation Terminology and Use
      • Adequacy Evaluation
      • Diagnostic Evaluation
      • Performance Evaluation
    • 13.1.2 The Successes and Limitations of Evaluation
      • Successes of Evaluation
      • Limitations of Current Evaluation Methods
    • 13.1.3 Future Directions
  • 13.2 Task-Oriented Text Analysis Evaluation
  • 13.3 Evaluation of Machine Translation and Translation Tools
    • 13.3.1 Future Directions
  • 13.4 Evaluation of Broad-Coverage Natural-Language Parsers
    • 13.4.1 State of the Art
    • 13.4.2 Improving the State of the Art
    • 13.4.3 Future Directions
  • 13.5 Human Factors and User Acceptability
    • 13.5.1 State of the Art
    • 13.5.2 Current Work
  • 13.6 Speech Input: Assessment and Evaluation
    • 13.6.1 Speech Recognition (Input) Assessment
    • 13.6.2 Speech (Spoken Language) Understanding Assessment
    • 13.6.3 Speaker Recognition Assessment
    • 13.6.4 Future Directions
    • 13.6.5 Appendix: Assessment and Evaluation Defined
  • 13.7 Speech Synthesis Evaluation
    • 13.7.1 Modular Diagnostic Evaluation
    • 13.7.2 Global Overall Performance
    • 13.7.3 Towards International Standards
    • 13.7.4 Future Directions
  • 13.8 Usability and Interface Design
    • 13.8.1 Future Directions
  • 13.9 Speech Communication Quality
    • 13.9.1 Subjective Intelligibility Tests
    • 13.9.2 Quality Rating or Mean Opinion Scoring (MOS)
    • 13.9.3 Objective Measures
  • Character Recognition
    • Future Directions
  • 13.11 References