1	Speaking in Minor and Major Keys Maartje Schreuder Laura van Eerten Dicky Gilbers
2	Outline Speech and music: melody Modality: major/minor Research question Experiment Methods Analyses Results Conclusion
3	Speech and music: melody Speech and music have a lot in common: structure, rhythm, phrasing, melody/intonation (cf. Lerdahl & Jackendoff 1983, Gilbers & Schreuder 2002) Speech prosody: intonation Intonation: ‘musical melody’
4	Major and minor modality in music Modalities: major / minor Cheerful and sad music: Major third: 4 semitones: C – E Minor third: 3 semitones: C – Eb
5	Major and minor modality in music Modalities: major / minor
6	Research question: emotional intonation? Are differences in emotional speech also characterized by different modalities? In other words, can we find major third intervals in the intonation contours of cheerful speech, and minor thirds in sad speech? Already found (independently) in Japanese speech (Norman Cook et al. 2004).
7	The experiment Method: 5 professional readers (primary school teachers) each read out 4 stories of Tigger and Eeyore (Winnie the Pooh, A.A. Milne) Tigger: cheerful character Eeyore: sad character Recorded on hard disk as wav files Cluster analysis intonation contour (F0) Music scores
8	The experiment Sampling the pitch contour: The passages in which Tigger and Eeyore were speaking were extracted Concatenated to 10 files, varying from 8 to 53 seconds Pitch contour (F0) sampled every 10 milliseconds (with PRAAT): obtain pitch values of the entire contour
9	The experiment Cluster analysis: Macro in Excel (Cook (2002) and Cook et al. (2004)) Raw clusters: number of occurrences of a certain pitch value Clustered in semitones, depicted as musical note values: Pitch values rounded off to nearest semitone Abstractions (musical phonemes) of the real frequencies (musical allophones)
10	The experiment Cluster analysis: Modality can only be determined if graphs contain more than 1 peak thirds This leaves 7 cases for analysis
11	Cluster analyses Clusteranalysis: Tigger
12	Cluster analyses Semitones: Tigger 2 x major third
13	Cluster analyses Clusteranalysis: Eeyore (smaller range)
14	Cluster analyses Semitones: Eeyore minor third
15	Results Cluster analyses Many Tigger stories have larger intervals than thirds Many Eeyore stories only contain a single peak From the 7 stories with thirds: All 3 Tigger stories: only major thirds All 4 Eeyore stories: only minor thirds No counterexamples!
16	Musical scores Cluster analysis ignores absolute intervals in time Incorporate time as a factor: musical scores From speech intonation contours to scores: Tweaking Wav to MIDI
17	Musical scores Tweaking: round off to nearest semitone
18	Musical scores
19	Musical scores
20	Results Tigger: Most intervals between notes in sequences are larger intervals than thirds Most phrases appear to be spoken on a single tone However, we find some thirds on stressed syllables: major thirds Major part of score built upon notes which form major thirds with each other. Ultimate feeling of a major key: happy, cheerful, and energetic story
21	Musical scores
22	Results Eeyore: Longer story: more notes Sequences of thirds between stressed syllables Minor thirds!
23	Conclusion The cluster analyses seem to give a good account of the internal relations in the melodies. The mood of emotional prosody in speech is very similar to musical modality: A sad mood can be expressed by using intervals of three semitones, i.e. minor thirds Eeyore speaks in a minor key! Cheerful speech mostly has bigger intervals than thirds, but when thirds are used, these thirds are major thirds Tigger speaks in a major key!
24	References AmazingMIDI (1998-2003). Araki Software, Japan. http://www.pluto.dti.ne.jp/~araki/amazingmidi/. Boersma, P. and D. Weenink (1992-2004). Praat: a system for doing phonetics. www.praat.org. Cook, N.D. (2002). Tone of voice and Mind. The connections between intonation, emotion, cognition and consciousness. Amsterdam: John Benjamins. Cook, N.D., T.Fujisawa and K. Takami (2004). Application of a Psycho-acoustical Model of Harmony to Speech Prosody. Proceedings of Speech Prosody. Nara, Japan: 147-150. Eerten, L. van (2004). Mineur en Majeur in emotionele spraak; een intonatieonderzoek. [Minor and major in emotional speech: an intonation investigation]. BA thesis, University of Groningen. Gilbers, D.G. and M.J. Schreuder (2002). Language and music in Optimality Theory. [ROA 571-0103]. Lerdahl, F. and R. Jackendoff (1983). A Generative Theory of Tonal Music. The MIT Press, Cambridge, Massachusetts, London, England. Milne, A.A. (1994). Winnie de Poeh [Winnie the Pooh]. Dutch translation by Mies Bouhuys. Van Goor, Amsterdam. Milne, A.A. (1996). Het huis in het Poeh-hoekje [House at Pooh Corner]. Dutch translation by Mies Bouhuys. Van Goor, Amsterdam. Schreuder, M. (2005). Prosodic Processes in Speech and Music. PhD dissertation. University of Groningen. Schreuder, M., Eerten, L. van and D. Gilbers (to appear). Speaking in Minor and Major Keys.