Some Comments on the Relation Between Music and Motion

Preface

[1] This text is a comment on a family of formal models of the “final ritard,” the typical slowing down atthe end of a music performance, and how the shape of thetiming patterns might relate to, or can be explained by, models of human motion.⁽¹⁾ The discussion is presented in the form of a tale, with three fictitious characters (P, M, and their musical friend MF) who represent the different disciplines involved in this research (psychology, mathematics/computer science, and musicology). It reflects some of my experiences in research that is concerned with the computational modeling of music cognition, using an interdisciplinary approach combining expertise from musicology, psychology, and computer science. See ⁽²⁾ for a discussion on this approach.

Part 1: What Happened Before⁽³⁾

[2] Quite some time ago P, who is interested in psychology, and M, an amateur mathematician, got together during the Christmas holidays with their musical friend MF. Those were the days before cellular telephones, a time of herbal tea and the just-arrived technology of MIDI. MF, while duly impressed by Messrs. P and M’s well-equipped music studio and expertise in computer modeling, remained unimpressed by their musical results and, sadly, left rather irritated to spend his Christmas elsewhere.

Part 2: In Which MF Had an Important Insight and P Found the Appropriate Literature

[3] Not so long ago MF remembered those Christmas holidays while he was reading a book on the history of tempo rubato. He was still convinced his friends were on the wrong track with their silly computer models. But the more he read about tempo rubato, the more he was convinced that they might have overlooked an obvious link between music and biological motion. Blatantly obvious—once he realized it—was the explicit reference of music terminology, words like andante or accelerando, to qualities of human movement. And therefore, he reasoned, a successful model of expressive timing—unlike the unsuccessful models made by his friends—should be based on the rules of movement and the human body.

[4] MF couldn’t help making a phone call to P, the amateur psychologist, to tell him about his new insight. “My dearfriend P,” he said, “for expressive timing to sound natural in a performance, it must conform to the principles of human movement. Isn’t knowledge about the body—the way it feels, moves, reacts—what musicians share with their listeners?” P almost immediately became enthusiastic. He saw a new opportunity to continue the investigations that had ended so brusquely before. P decided to go to the library and there he found a lot of interesting psychological literature on the relation between motion and music. Much of it, however, involved some formidable mathematics. MF then proposed to have a new gathering with the “old team,” including their mathematical friend, and this time at MF’s home, safe from modern technology!

Part 3: In Which the Friends Met Again, Explored Elementary Mechanics, and Built a “True” Physical Model

[5] A few days later P and M found themselves at MF’s kitchen table, which was well stocked with a pot of tea and a tin full of cookies. They returned to a lively discussion on expressive timing in music. After browsing through the books that P brought, M (the amateur mathematician) stated with some authority that “these models borrow from elementary mechanics and kinematics. They talk about mass, force, and speed of an object in terms of velocity, time and place. And, interestingly, tempo variations in music performance are compared with the behavior of physical objects in the real world.” P was all ears; MF just took another sip of his tea.

[6] M wrote most of the formulas, one below the other, on a piece of paper, patiently explaining their formal differences (a tidier version of M’s jottings is shown in Example 1. MF protested “But M, please! We are investigating music here, not mechanics!” “Look,” P swiftly interrupted, “I found the studies of these music researchers. They explain ritardandi in music performanceas alluding to human motion, like the way runners come to a standstill. Let me read a passage for you: ‘Performers aim at this allusion, and listeners, with some education, find it aesthetically pleasing.’⁽⁴⁾ Isn’t this exactly what you described to me on the phone!” (See Example 1: Formal models of the ‘final ritard’ in music performance.)

[7] P and M seemed confident that they had now found what they had been searching for all the time. MF too was quite pleased with the fact that these respected researchers had found evidence for his intuitive ideas about bodily motion. But he still had reservations. “How does the math of elementary mechanics compare to a final ritard in music? Can’t we listen to these formulas?” M replied with a frown on his face, “Well, if we would have met in our studio we could have programmed them for you. Now we have to think of something else.” But after a small pause he began to smile. “Let’s see how far we can get with the material in your garage.”

[8] That morning MF’s kitchen turned into a real workshop. “Can we use one of your music boxes?” P asked sheepishly. With some hesitation MF collected one of his beloved machines from the living room. And after some hours of trifling and hammering they had built it—a “true” physical model of constant braking force! (See Example 2: A mechanical implementation of a constant braking force model, and Example 3 for a brief RealPlayer movie (56K) showing the machine at work.)

[9] MF inserted his favorite piano roll, a Bach fugue, into their newly made contraption. He turned the flywheel and the music started playing. A few bars before the end he released the handle, and the music came slowly to a standstill over the last few notes. “Wonderful, wonderful!” They all jumped with joy. MF thought his antique music box had finally become truly musical.

Part 4: In Which Some Disappointment was Unavoidable and They Decided to Look at Real Performances

[10] When they had calmed down a bit, M had a second look at his paper full of formulas, and said with a tone not atypical of a young mathematician: “But I have to say that these models are actually under-specified. They make no claims about how to derive the ‘metaphorical’ mass or speed from the music. In our contraption we just arbitrarily decided on the mass of flywheel, and we can freely decide the speed at which the handle is released.” M also realized that their contraption had some shortcomings. “Our flywheel has a fixed braking force, caused by the friction of the contraption. But it should actually be dependent on when, and at what speed you release the handle, and stop when the right final tempo is reached, like the equations show. That’s difficult to make mechanically.” But P responded, “Oh come on M, don’t be so strict. Let’s just try another one, a slightly more modern piece. What do you think?” After some searching, MF returned with a piano roll of Beethoven’s Paisiello Variations. “Remember this?” he teased. MF inserted the piano roll and they listened again for the last measures of each variation. But whatever they tried, releasing the handle early or late, at higher or lower speeds, it never sounded quite right. “It doesn’t do the rhythmical figures right,” MF complained. “Apparently it only works with the repeated eighth notes of the fugue.”

[11] “We could be here forever trying to change this or that factor,” P warned. He was convinced they had to return tothe empirical approach. “Why don’t we look at how MF performs final ritards?”

Part 5: In Which They Looked at Graphs from Famous Pianists, But Couldn’t Please Their Musical Friend

[12] P opened his case and pulled-out a folder with the performance data they had collected during that first Christmas gathering. “These are the graphs of MF performing the final measures of Träumerei by Schumann.” And enthusiastically holding up an article, P added, “And here are some interesting measurements made from recordings by some of your colleagues. Look, you played it just like Alfred Brendel!” (See Example 4: Final ritards in performances of the last three measures of R. Schumann’s Träumerei.)

[13] There was quite some diversity among these famous pianists; they all seemed to play the final measures differently. MF said, questioningly, “I do not see how one single curve could describe all these performances?” Presponded “But the point here is to model the average, normative performance.” To which M added, while pointing at Equation 3, “This research showed that the last six notes of these averaged performances can be fitted closely by a quadratic function. That is an important finding, isn’t it?” “Indeed, M” P confirmed, “but we must be aware that an average curve is a statistical abstraction, not a musically reality.” Their musical friend smiled and took another close look at the diagrams. “So if I understood your explanations,” he asked M, “this function should have a hollow, concave shape. But doesn’t our contraption generate a convex shaped deceleration?” M confirmed this. “A convex shape indeed is what the other research found. Apparently there is evidence for a variety of shapes. However, what worries me is the complete freedom in deciding on the mass and amount of force applied; fitting these curves to the data is too flexible.” “Maybe all these pianists have their own specific force and mass?” MF interjected optimistically. They looked at each other with some disappointment. It seemed that once again they had failed to find a model of expressive timing that could please their musical friend. MF, who this time wanted to end their endeavors in a moreoptimistic manner, proposed “Lets go to the living room. I will play my favorite fugue for you.” To be continued. . .

Epilogue

[14] This tale talks about kinematical models of expressive timing, and it questions in how far expressive timing can be explained by models of physical motion. The formalizations discussed above are based on the notion of a tempo curve (a continuous function of time or score position) regressing, for example, a linear tempo function to the performance data. One point of critique is that the predictions made by these models are insensitive to the actual rhythmic structure of the musical material: they make the same predictions for different rhythms.⁽⁵⁾ However, more central is the objection that these descriptions do not, in principle, teach us anything about the nature (whether “motional” or not) of the underlying perceptual or cognitive mechanisms. Even if we assume that these curves do give a good approximation of the empirical data (despite the contrasting results in the research discussed above), the mere fact that the overall shape (e.g. of a square root function) can be predicted by the rules that come with human motion is not enough evidence for an underlying physical model of expressive timing in music performance, however attractive such a model might be. The relation between music and motion turns out to be not that simplistic.⁽⁶⁾ The challenge here is to formulate appropriate alternatives.⁽⁷⁾

Henkjan Honing
Music, Mind, Machine Group
Music Department, University of Amsterdam
Spuistraat 134, NL-1012 VB
Amsterdam, The Netherlands
http://www.hum.uva.nl/mmm
honing@hum.uva.nl

Works Cited

Baily, John. 1985. “Music structure and human movement.” In Musical structure and cognition, edited by Peter Howell, Ian Cross, and Robert West, 237–258. London: Academic Press.

Clarke, Eric F. 1993. “Generativity, Mimesis and the Human Body in Music Performance.” In “Music and the Cognitive Sciences,” edited by Ian Cross and Irene Deliege. Contemporary Music Review 9: 207–220.

—————. 1999. “Rhythm and Timing in Music.” In Psychology of Music, 2nd ed., edited by Diana Deutsch, 473–500. New York: Academic Press.

Desain, Peter and Henkjan Honing. 1993. “Tempo Curves Considered Harmful”. In “Time in Contemporary Musical Thought,” edited by Jonathan D. Kramer. Contemporary Music Review 7/2: 123–138.

—————. 1994. “Does Expressive Timing in Music Performance Scale Proportionally with Tempo?” Psychological Research 56: 285–292.

—————. 1996. “Physical motion as a metaphor for timing in music: the final ritard.” Proceedings of the International Computer Music Conference: 458–460.

Desain, Peter, Henkjan Honing, Huub Van Thienen, and Luke Windsor. 1998. “Computational modeling of music cognition: problem or solution?” Music Perception 16: 151–166.

Epstein, David. 1994. Shaping time. New York: Schirmer.

Feldman, Jacob, David Epstein, and Whitman Richards. 1992. “Force Dynamics of Tempo Change in Music.” Music Perception 10/2: 185–204.

Friberg, Anders and Johan Sundberg. 1993. “Does music performance allude to locomotion? A model of final ritardandi derived from measurements of stopping runners.” Journal of the Acoustical Society of America 105/3: 1469–1484.

Gabrielsson, Alf. 1999. “Music Performance.” In Psychology of Music, 2nd ed. edited by Diana Deutsch, 501–602. New York: Academic Press.

Honing, Henkjan. 2001. “From time to time: The representation of timing and tempo.” Computer Music Journal 35/3: 50–61.

Kronman, Ulf, and Johan Sundberg. 1987. “Is the musical ritard an allusion to physical motion?” In Action and Perception in Rhythm and Music, edited by Alf Gabrielsson. Royal Swedisch Academy of Music no. 55: 57–68.

Repp, Bruno H. 1992. “Diversity and commonality in music performance: An analysis of timing microstructure in Schumann’s Träumerei.” Journal of the Acoustical Society of America 92: 2546–2568.

Shove, Patrick and Bruno H. Repp. 1995. “Musical motion and performance: theoretical and empirical perspectives.” In The Practice of Performance: Studies in Musical Interpretation, edited by John Rink, 55–83. Cambridge: Cambridge University Press.

Sundberg, Johan and Ronald T. Verrillo. 1980. “On the anatomy of the ritard: A study of timing in music.” Journal of the Acoustical Society of America 68: 772–779.

Todd, Neil P. M. 1985. “A model of expressive timing in tonal music.” Music Perception 9: 33–58.

—————. 1992. “The Dynamics of Dynamics: a Model of Musical Expression.” Journal of the Acoustical Society of America 91/6: 3540–3550.

—————. 1999. “Motion in Music: A Neurobiological Perspective.” Music Perception 17/1: 115–126.

Footnotes

1. David Epstein, Shaping time (New York: Schirmer, 1994); Anders Friberg and Johan Sundberg, “Does music performance allude to locomotion? A model of final ritardandi derived from measurements of stopping runners,” Journal of the Acoustical Society of America 105/3 (1999): 1469–1484; Neil P. M. Todd, “Motion in Music: A Neurobiological Perspective,” Music Perception 17/1 (1999): 115–126.
Return to text

2. Peter Desain, Henkjan Honing, Huub van Thienen and Luke Windsor, “Computational modeling of music cognition: problem or solution?,” Music Perception 16 (1998): 151–166.
Return to text

3. This tale is a continuation of Peter Desain and Henkjan Honing, “Tempo Curves Considered Harmful”, in Jonathan D. Kramer, ed., Contemporary Music Review 7/2 (1993): 123–138 (“Time in Contemporary Musical Thought”). See http://www.nici.kun.nl/mmm/tc for additional sound examples.
Return to text

4. Bruno H. Repp, “Diversity and commonality inmusic performance: An analysis of timing microstructure in Schumann’s Träumerei,” Journal of the Acoustical Society of America 92 (1992): 2546–2568.
Return to text

5. Peter Desain and Henkjan Honing, “Does Expressive Timing in Music Performance Scale Proportionally with Tempo?,” Psychological Research 56 (1994): 285–292; Henkjan Honing, “From time to time: The representation of timing and tempo,” Computer Music Journal 35/3 (1999): 50–61.
Return to text

6. For a further discussion see Henkjan Honing, “The Final Ritard: On Music, Motion, and Kinematic Models,” Computer Music Journal, submitted (see http://www.nici.kun.nl/mmm/tc).
Return to text

7. Special thanks to Robert Gjerdingen for valuable suggestions on an earlier version of this paper, and to Bruno Repp for his constructive criticisms. Thanks also to the Department of Mechanics, University of Amsterdam, for actually making the contraption described in this paper.
Return to text

Copyright Statement

Copyright © 2003 by the Society for Music Theory. All rights reserved.

[1] Copyrights for individual items published in Music Theory Online (MTO) are held by their authors. Items appearing in MTO may be saved and stored in electronic or paper form, and may be shared among individuals for purposes of scholarly research or discussion, but may not be republished in any form, electronic or print, without prior, written permission from the author(s), and advance notification of the editors of MTO.

[2] Any redistributed form of items published in MTO must include the following information in a form appropriate to the medium in which the items are to appear:

This item appeared in Music Theory Online in [VOLUME #, ISSUE #] on [DAY/MONTH/YEAR]. It was authored by [FULL NAME, EMAIL ADDRESS], with whose written permission it is reprinted here.

[3] Libraries may archive issues of MTO in electronic or paper form for public access so long as each issue is stored in its entirety, and no access fee is charged. Exceptions to these requirements must be approved in writing by the editors of MTO, who will act in accordance with the decisions of the Society for Music Theory.

This document and all portions thereof are protected by U.S. and international copyright laws. Material contained herein may be copied and/or distributed for research purposes only.