Jan P. H. van Santen

Center for Spoken Language Understanding
Computer Science & Electrical Engineering Department
OGI School of Science & Engineering
Oregon Health & Science University
20000 NW Walker Road,Beaverton, Oregon 97006
vansanten-at-cslu.ogi.edu
phone: (503) 748-1138

Current Responsibilities

Prof. van Santen's current responsibilities consist of teaching, conducting research, and managing the Center for Spoken Language Understanding (CSLU) as well as the Computer Science & Electrical Engineering (CSEE) Department. His background includes mathematical modeling of prosody, signal processing, and computational linguistics. A key growing focus of his work and of the Center and the Department is on biomedical applications of computer science and electrical engineering. Current research projects include several neurologically focused projects (quantitative characterization of prosody in autism, instrumental approaches to neurogenic speech disorders, automated neuropsychological assessment), as well as basic speech technology research on speech synthesis. He is currently the Director of CSLU and the Chair of the CSEE Department.

Current and Previous Positions

2004-Present	Chair, Computer Science & Electrical Engineering Department
2004-Present	President, BioSpeech Inc.
2001-Present	Member, Board of Directors, SVOX AG, Zurich, Switzerland
2000-Present	Professor of Biomedical Engineering and Computer Science & Engineering
2000-Present	Director, Center for Spoken Language Understanding
1984-2000	Member of Technical Staff, Bell Laboratories
1981-1984	Associate Research Scientist, New York University
1979-1981	Postdoctoral Member of Technical Staff, Bell Laboratories

Education

1979	PhD, Mathematical Psychology, University of Michigan, Ann Arbor, Michigan
1974	MA, Statistical Psychology, University of Amsterdam, Amsterdam, The Netherlands
1971	BA, Psychology, University of Amsterdam, Amsterdam, The Netherlands

Organizational Activities

Associate Editor, Journal of Mathematical Psychology

Member, Speech Technical Committee, IEEE.

Co-organization of the Third European Speech Communication Association Workshop on Speech Synthesis.

Co-organization of the Second European Speech Communication Association Workshop on Speech Synthesis.

Participation in and co-organization of the 1998 NSF Workshop for Discussing Research Priorities and Evaluation Strategies in Speech Synthesis.

Reviews:

Journals:

J. Acoustical Soc. Amer., J. Optical Soc. Amer., Vision Research, J. Exp. Psych., Speech Communication, J. Math. Psych., Comp. Language & Speech, Phonetica, J. Phonetics

Conferences:

Eurospeech 1997-2003, ICSLP 1996-2002, ICPhS, ESCA/IEEE Synthesis Workshops 1994-2001, HLT, ICASSP 2000-2003

Granting agencies:

National Institutes of Health, National Science Foundation, Engineering and Physical Sciences Research Council (UK), Netherlands Organisation for Scientific Research, Australian Research Council, Swiss Science and Technology Council

Scientific and Technological Contributions

These contributions consist of a number of quantitative models of various aspects of human behavior, and of associated computational or quantitative methods. Over the years, these contributions have ranged from human motion perception, and clinical judgment, to language and speech. Language and speech has been his primary focus since 1986.

Measurement of degree of coarticulation in Dysarthria

Small footprint text-to-speech synthesis system

Formant Tracking Based On Phoneme Information

Methods for measuring polysemy based on unannotated text corpora.

High-accuracy phonetic feature based speech segmentation.

Methods for minimizing spectral discrepancies in acoustic inventories for concatenative synthesis.

Statistical modeling of phonetic segment durations via sums-of-products models.

Quantitative description of fundamental frequency alignment.

Development of "greedy" algorithms for optimal text selection.

Automatic phonetic accuracy scoring of orthographic transcriptions of speech.

Concept of "command forms" as a user interface for command line based operating systems.

Development of a model of clinical judgment in subtyping of developmental language disorders.

Development of a human motion perception model.

Recent Funded Research Projects

Expressive and Receptive Prosody in Autism (NIH)
Objective Methods for Predicting and Optimizing Synthetic Speech Quality (NSF)

Prosody Generation for Child Oriented Speech Synthesis (NSF)

Making Dysarthric Speech Intelligible (NSF)

Modeling Degree of Articulation for Speech Synthesis (NSF)

Creating the Next Generation of Intelligent Animated Conversational Agent (NSF)

Use of Computer-Enhanced Prosody to Improve Comprehension and Retention of Verbal Information in Children with Autistic Spectrum Disorder (Medical Research Foundation, Oregon)

Novel Computerized Behavioral Assessment Methods for Attention Deficit Hyperactivity Disorder (OHSU Foundation)
Small Footprint Speech Synthesis (NSF-STTR)
Automated Test of Word Recognition (NIH-STTR)
Voice Transformation for Dysarthria (NIH-STTR)
Speech Supplemented Word Prediction Program (NIH-SBIR)

Publications

Invited Papers

Invited paper at University of Aix-en-Provence, France, 2004
Invited paper at Johns Hopkins University Summer School, 2004
Invited paper at IBM Watson Research Laboratories, 2004
Invited paper at Eurospeech 2003, Geneva, Switzerland, 2003

Invited paper at the International Congress of Phonetic Sciences, Barcelona, Spain, 2003

Invited paper at Speech Prosody 2002, Aix-en-Provence, France, 2002

Keynote address at Prosody and Speech Processing conference, Japanese Ministry of Education, Tokyo, 2002.

Keynote address at IEEE Speech Synthesis conference, Santa Monica, 2001.

Invited paper at Eurospeech 2001, Aalborg, Denmark, 2001
Invited paper at University of Trondheim, Norway, 2000

Invited paper at the Joint European and American Acoustical Society Meetings, Berlin, Germany, 1999.

Keynote address at Eurospeech, Rhodes, Greece, 1997.

Invited paper at the Joint Japanese and American Acoustical Society Meetings, Honolulu, Hawaii, 1996.

Invited paper at the American Acoustical Society Meetings, Denver, Colorado, 1995.

Invited paper at the American Acoustical Society Meetings, Cambridge, Massachusetts, 1995.

Keynote address at Eurospeech, Berlin, Germany, 1993.

Books

van Santen, J., Sproat, R., Olive, J., and Hirschberg, J. (Eds.) "Progress in Speech Synthesis", New York: Springer-Verlag, 1997.

Journal articles, book chapters, and conference proceedings

1. Prosodic Modeling:

1.1 General:

van Santen J. "Prosodic modeling in text-to-speech synthesis" . In Proceedings Eurospeech 1997 , Rhodos, Greece, 1997.

van Santen J., "Segmental Duration and Speech Timing". In Y. Sagisaka, W.N. Campbell, N. Higuchi (Eds.), Computing Prosody. New York: Springer, 1996.

van Santen J. "Using statistics in text-to-speech system construction" . In Proc. ESCA/IEEE Workshop on speech synthesis , pages 240-243, 1994.

1.2 Timing:

Son R.J.J.H. and Van Santen, J.P.H. (2005). "Duration and spectral balance of intervocalic consonants: A case for efficient communication", Speech Communication 47, 100-123.
van Santen, J., and Shih, C. "Suprasegmental and Segmental Timing Models in Mandarin Chinese and American English". Journal of the Acoustical Society of America. V. 107 (2), 1012-1026, 2000.
Gu W., Shih C., and van Santen J. "An Efficient Speaker Adaptation Method for TTS Duration Model". Proceedings of EUROSPEECH 99, Budapest, Hungary, 1999.
van Santen L. and D'Imperio M., "Positional effects on stressed vowel duration in Standard Italian". International Conference of Phonetic Sciences 99, San Francisco, U.S.
van Santen J. "Timing". In Multilingual Text-to-Speech Synthesis: The Bell Labs Approach , pages 115-140. Kluwer, Dordrecht, 1998.
Shih C., Gu W., van Santen J.. "Efficient Adaptation of TTS Duration Model to New Speakers", ICSLP 98 and Third ESCA Workshop on Speech Synthesis 98, 1998.
Venditti J., van Santen J. "Modeling segmental durations for Japanese text-to-speech synthesis". Third ESCA Workshop on Speech Synthesis 98, 1998.
van Santen J., Shih C., Möbius B., Tzoukermann E., Tanenblatt M.. "Multilingual duration modeling". Proceedings of EUROSPEECH 97, Rhodes, Greece, 2651-2654, 1997.
R. van Son and van Santen J. "Modeling the interaction between factors affecting consonant duration" . In Proceedings Eurospeech 1997 , Rhodos, Greece, 1997.
Möbius B., van Santen J.. "Modeling Segmental Duration in German Text-to-Speech Synthesis". ICSLP 96, 2395-2398, 1996.
van Santen J. "Computation of timing in text-to-speech synthesis". In W. B. Kleijn and K. K. Paliwal, editors, Speech Coding and Synthesis, Elsevier, 1995.
van Santen J. "Assignment of segmental duration in text-to-speech synthesis" . Computer Speech and Language , 8:95-128, 1994.
van Santen J. "Timing in text-to-speech systems". In Proc. Eurospeech , Berlin (Germany), Sept. 1993.
van Santen J. "Quantitative modeling of segmental duration". In Proc. ARPA Human Language Workshop , Princeton, March 1993.
van Santen J. "Exploring N-way tables with Sums-of-Product models". J. Mathematical Psychology , 37(3):327-371, 1993.
van Santen J. "Deriving text-to-speech durations form natural speech". In G. Bailly and C. Benoit, editors, Talking Machines: Theories, Models & Application . Elsevier, 1992.
van Santen J. "Contextual effects on vowel duration". Speech Communication , 11(6):513-546, 1992.
van Santen J. "Deriving text-to-speech durations from natural speech". In G. Bailly and C. Benoit, editors, Proc. ESCA Workshop Speech Synthesis , pages 157-160, 1991.
van Santen J. and Olive J. "The analysis of contextual effects on segmental duration". Computer Speech & Language , pages 359-390, 1990.

1.3 Intonation:

J. van Santen, E. Klabbers, and T. Mishra (2005). Towards measurement of pitch alignment. Italian Journal of Linguistics. Special issue on "Autosegmental-metrical approaches to intonation in Europe: tonal targets and anchors" (in press, 2006).
Miao Q., Niu X., Klabbers E., and van Santen J. "Effects of Prosodic Factors on Spectral Balance: Analysis and Synthesis", Prosody 2006, Dresden, Germany (2006)
T. Mishra, J. van Santen and E. Klabbers, Decomposition of pitch curves in the general superpositional model, Prosody 2006, Dresden, Germany (2006)
J. van Santen, A. Kain, E. Klabbers, and T. Mishra. "Synthesis of Prosody using Multi-level Unit Sequences". Speech Communication, Volume 46, Issues 3-4, Pages 365-375, July 2005.
Esther Klabbers, Jan van Santen and Johan Wouters (2005). Minimizing the amount of pitch modification in speech synthesis, in S. Narayanan and A. Alwan (eds). Text to Speech synthesis: New Paradigms and Advances, Prentice Hall Professional Technical Reference, Upper Saddle River, NJ.
Jan van Santen, Alexander Kain and Esther Klabbers (2004). Synthesis by recombination of segmental and prosodic information, International Conference on Speech Prosody 2004, Nara, Japan.
Esther Klabbers and Jan van Santen (2004). Clustering of foot-based pitch contours in expressive speech, ISCA Speech Synthesis Workshop 5, Pittsburgh, PA, 2004
J. van Santen, T. Mishra, and E. Klabbers, "Estimating phrase curves in the general superpositional intonation model", ISCA Speech Synthesis Workshop 5, Pittsburgh, PA, 2003.
Klabbers, E., and van Santen, J.. "Control and Prediction of Impact of Pitch Modification on Synthetic Speech Quality", Proceedings Eurospeech 2003, 2003.

van Santen, J. "Quantitative Models of Pitch Alignment", Speech Prosody 2002, Aix-en-Provence, France, 2002

van Santen, J., and Möbius, B. "A quantitative model of F0 generation and alignment". In Antonis Botinis (ed.) Intonation - Analysis, Modeling and Technology (Kluwer, Dordrecht), 269-288, 2000.

Möbius, B., and van Santen, J.. "Phonetically motivated modeling of prosody". Prosody 2000: Speech Recognition and Synthesis (Kraków, Poland), 161-166.

Venditti, J. and van Santen, J.. "Japanese intonation synthesis using superposition and linear alignment models". In Proceedings of the International Conference on Spoken Language Processing (ICSLP). Beijing, China, 2000.

van Santen J., Shih C., and Möbius B. "Intonation" . In E. Sproat, editor, Multilingual Text-to-Speech Synthesis: The Bell Labs Approach , pages 141-189. Kluwer, Dordrecht, 1998.

van Santen J., Möbius B., Venditti J., Shih C. "Description of the Bell Labs intonation system". Third ESCA Workshop on Speech Synthesis 98, 1998.

Maeda K., Venditti J., van Santen J., "Modeling Japanese boundary pitch movements for speech synthesis". Third ESCA Workshop on Speech Synthesis 98, 1998

van Son R., van Santen J., "Modeling the interaction between factors affecting consonant duration", Proceedings of EUROSPEECH 97, Rhodes, Greece, 1997.

van Santen J., Shih C., Möbius B., E. Tzoukermann, and M. Tanenblatt. "Multilingual duration modeling" . In Proc. European Conf. Speech Communication & Technology (EUROSPEECH '97) , volume 5, pages 2651-2654, Rhodes, Greece, 1997.

van Santen J. and Möbius B. "Modeling pitch accent curves" . In Intonation: Theory, Models and Applications - Proceedings of an ESCA Workshop , pages 321-324, Athens, Greece, 1997.

van Santen J., Möbius B.: "Modeling pitch accent curves". Intonation: Theory, Models and Applications. Proceedings of an ESCA Workshop, Athens, Greece. 321-324, 1997.

Prieto P., van Santen J., and Hirschberg J. "Tonal alignment patterns in Spanish". J. of Phonetics , 23, 1995.

van Santen J. and Hirschberg J. "Segmental effects on timing and height of pitch contours". In Proc. ICSLP-94 , pages 719-722, 1994.

Prieto P., van Santen J., and Hirschberg J. "Patterns of FO peak placement in Mexican Spanish". In Proc. ESCA/IEEE Workshop on Speech Synthesis , pages 33-36, New Paltz, 1994.

Prieto P. and van Santen J. "Secondary stress in Spanish: Some experimental evidence". In Proc. Twenty-fourth Annual Linguistic Symposium on Romance , Los Angeles, CA, 1994.

2. Statistical text analysis

Mishra, T., Klabbers, E., and and van Santen, J.. "Detection of list-type sentences", Proceedings Eurospeech 2003, 2003.

Sproat R., van Santen J., "Automatic Ambiguity Detection". Proceedings of ICSLP 98. Sydney, Australia. 1998.

van Santen J. and Buchsbaum A.L. "Methods for optimal text selection" . In Proceedings Eurospeech 1997 , Rhodos, Greece, 1997.

Buchsbaum A.L. and van Santen J. "Selecting training inputs via greedy rank covering". In Proc. 7th ACM-SIAM Symp. on Discrete Algorithms, pages 288-295, 1996.

van Santen J., "Combinatorial issues in text-to-speech synthesis", Proceedings of EUROSPEECH 97, Rhodes, Greece, 1997.

3. TTS Evaluation

van Santen J. "Evaluation". In Multilingual Text-to-Speech Synthesis: The Bell Labs Approach . Kluwer, Dordrecht, 1998.

van Santen J., Pols L.C.W., Abe M., Kahn D., Keller E., Vonwiller J., "Report on the Third ESCA TTS Workshop Evaluation Procedure". Third ESCA Workshop on Speech Synthesis 98, 1998.

Pols L.C.W., van Santen J., Abe M., Kahn D., Keller E., "The use of large text corpora for evaluating text-to-speech systems". Proceedings First International Conference on Language Resources and Evaluation. Granada, Spain, 1998.

Pols L.C.W., van Santen J., Abe M., Black A., House D., Liberman M.Y., Wu Z., "Easy access via a TTS web site to mono- and multilingual text-to-speech systems". Proceedings Elsnet in Wonderland. Soesterberg, The Netherlands, 1998

van Santen J. "Perceptual experiments for diagnostic testing of text-to-speech, systems". Computer Speech & Language , 7:49-100, 1993.

van Santen J. "Diagnostic perceptual experiments for text-to-speech system evaluation". Proc. ICSLP , pages 555-558, 1992.

4. Signal Processing

X. Niu, A. Kain, J. van Santen. "Estimation of the Acoustic Properties of the Nasal Tract during the Production of Nasalized Vowels". Proceedings of EUROSPEECH, September 2005.
H. Duxans, A. Bonafonte, A. Kain, J. van Santen. "Including Dynamic and Phonetic Information in Voice Conversion Systems". Proceedings of ICSLP, October 2004.
A. Kain, X. Niu, J. Hosom, Q. Miao, J. van Santen. "Formant Re-synthesis of Dysarthric Speech". Presentation at IEEE Workshop on Speech Synthesis, June 2004.
A. Kain and J. van Santen. "A speech model of acoustic inventories based on asynchronous interpolation". EUROSPEECH 2003, 2003

A. Kain and J. van Santen. "Compression of Acoustic Inventories using Asynchronous Interpolation". Proceedings of IEEE Workshop on Speech Synthesis, September 2002.

van Santen J., and Sproat R., "High-accuracy automatic segmentation". Proceedings of EUROSPEECH 99, Budapest, Hungary, 1999.

Lee M., van Santen J., Möbius B. , and Olive J. "Formant tracking using segmental phonemic information". Proceedings of EUROSPEECH 99, Budapest, Hungary, 1999.

Olive J., van Santen J., and Möbius B. "Synthesis" . In Sproat R., editor, Multilingual Text-to-Speech Synthesis: The Bell Labs Approach , pages 191-228. Kluwer, Dordrecht, 1998.

Ljolje A., van Santen J., Hirschberg J. "Automatic speech segmentation for concatenative inventory selection". In van Santen J., Sproat R., Olive J. and Hirschberg J. (eds.), Progress in Speech Synthesis, New York, Springer-Verlag, 305-312, 1996.

Ljolje A., Hirschberg J., and van Santen J. "Automatic speech segmentation for concatenative inventory, selection". In Proc. ESCA/IEEE Workshop on speech synthesis , pages 93-96, New Paltz, 1994.

5. Speech Technology Overviews

van Santen, J. "Phonetic Knowledge in text-to-speech Synthesis", Barry, W. and van Dommelen, W. (Ed.), Integration of Phonetic Knowledge in Speech Technology, Kluwer, Dordrecht, in press.

van Santen, J. "Speech Synthesis", in W. S. Bainbridge (Ed.), Encyclopedia of Human-Computer Interaction, Great Barrington, MA: Berkshire Publishing Group, in press.

van Santen, J. "Review of the Handbook of Standards and Resources for Spoken Language Systems, edited by Dafydd Gibbon, Roger Moore, and Richard Winski", Computational Linguistics, 24(3), 1998.

Van Santen, J. P. H. "The Role of Phonetics in Speech Synthesis", Proceedings of the 15-th International Congress of Phonetic Sciences, Barcelona, Spain, 2003.

Pavlova E., Pavlov Y., Sproat R., Shih C., and van Santen J. "Bell Laboratories Russian Text-to-Speech System" . In Proceedings Eurospeech 1997 , Rhodos, Greece, 1997.

Möbius B., Sproat R., van Santen J., Olive J.. "The Bell Labs German text-to-speech system: an overview". Proceedings of EUROSPEECH 97, Rhodes, Greece. 2443-2446, 1997.

van Santen J., Sproat R.W., Olive J., and Hirschberg J. (Eds.). Progress in speech synthesis . Springer Verlag, New York, 1996.

Möbius B., Schröter J., van Santen J., Sproat R., and Olive J. "Recent advances in multilingual text-to-speech synthesis". In Fortschritte der Akustik DAGA '96 , pages 82-85, DPG, Bad Honnef, 1996.

6. Mathematical Psychology

Bamber D. and van Santen J. "How to Tell Whether a Model is Testable: The Example of Discrete-State Selective-Influence Models". Journ. Math. Psych., 44, 2000.

Bamber D. and van Santen J. "How many parameters can a model have and still be testable?" Jour. Math. Psych. , 29:443-473, 1985.

van Santen J. and Bamber D. "Finite and infinite state confusion models" . Jour. Math. Psych. 24:101, 1981.

van Santen J. "A new axiomatization of portfolio theory". Jour. Math. Psych. , 17:14, 1978.

7. Neuroscience and Technology Applications to Neuroscience

Black L., and van Santen J. "Social-emotional problems and affect sensitivity in subtypes of developmental language disorder", Meeting of the International Neuropsychological Society, Zurich, 2006
Niu, X. and van Santen, J. "A formant trajectory model and its usage in comparing coarticulatory effects in dysarthric and normal speech", Proceedings of the 3rd International Workshop on Models and Analysis of Vocal Emissions for Biomedical Applications, Florence, Italy, 2003.
J. Hosom, A. Kain, T. Mishra, J. van Santen, M. Fried-Oken, J. Staehely. "Intelligibility of modifications to dysarthric speech". Proceedings of ICASSP, May 2003.
J. van Santen, L. Black, G. Cohen, A. Kain, E. Klabbers, T. Mishra, J. de Villiers, X. Niu. "Applications of computer generated expressive speech for communication disorders". EUROSPEECH, 2003
van Santen J., Black L., Wilson B., and Risucci D. "Modeling clinical judgment: A reanalysis of data from Wilson and, Risucci's (1986) Paper, A model for clinical-quantitative, classification. Generation I: Application to language disordered, preschool children". Brain and Language , 46:469-481, 1994.
van Santen J. and Sperling G. "Elaborated Reichardt detectors". JOSA , 2:554, 1985.
Sperling G., van Santen J., and Burt P.W. "Mathematical models of motion perception". Spatial Vision , 1:47-56, 1985.
Johnston J.C., van Santen J., and B. L. Hale. "Repetition effects in word and pseudoword identification:, Comment on Feustel, Shiffrin, and Sassaloo" . Jour. Exp. Psych., 114:498-508, 1985.
van Santen J. and Sperling G. "Applications of a Reichardt-type model of two-frame motion". Investigative Ophthal. and Visual Sci. , 25:14, 1984.
van Santen J. and Sperling G. "A temporal covariance model of human motion perception". JOSA , 1:451, 1984.
van Santen J. and Sperling G. "A temporal covariance model of human motion perception". Investigative Ophth. and Visual Sci. , 124:79, 1983.
Samuel A.G., van Santen J., and Johnston J.C. "A reply to Matthei". Jour. Exp. Psych, 9:321, 1983.
Samuel A.G., van Santen J., and Johnston J.C. "Length effects in word perception" . Jour. Exp. Psych , 1982.
van Santen J. and Jonides J. "A replication of the face superiority effect" . Bull. Psychonomic Soc. , 12:378, 1978.

8. Applied Psychology

Poller M.F. and van Santen J. "ASSIST: An interface for direct interaction with the UNIX system". In Proc. UniForum UNIX Conf., 1987.

Groeneboom P.J., Hoogstraten J., Mellenbergh G., and van Santen J. "Relevante variabelen bij het doorverwijzen na, de lagere school". Tijdschrift voor Onderwijs Research , 3:262, 1978.

Patents:

"Speech Synthesizer Having An Acoustic Element Database "

"System and Method For Determining Pitch Contours"

"System and Method for Selecting Training Text"

"Formant Tracking Based On Phoneme Information"

"A Method And Apparatus For Measuring The Degree Of Polysemy In Polysemous Words"

"System and Method for Compressing Concatenative Acoustic Inventories for Speech Synthesis"
"Methods and apparatus for speaker specific durational adaptation"

"System and Method for Concatenating Acoustic Contours for Speech Synthesis" (filed)