 |  |  |  |  |  |  |
| | ||||||||
| ||||||||
Volume 9, Number 1, March 2003
Copyright © 2003 Society for Music Theory
Henkjan Honing*Some Comments on the Relation Between Music and Motion | Get the free RealPlayer to view the video examples. Get the free DjVu browser plug-in to best view the examples. |
KEYWORDS: music performance, music cognition, computational modeling, music and motion, expressive timing
ABSTRACT: This text is a comment on a family of formal models of the "final ritard," the typical slowing down at the end of a music performance, and how the shape of the timing patterns might relate to, or can be explained by, models of human motion. The discussion is presented in the form of a tale, with three fictitious characters (P, M, and their musical friend MF) who represent the different disciplines involved in this research (psychology, mathematics/computer science, and musicology).
Submission received 3 October 2002
[1] This text is a comment on a family of formalmodels of the "final ritard," the typical slowing down atthe end of a music performance, and how the shape of thetiming patterns might relate to, or can be explained by,models of human motion.(1) The discussion is presented in the form of a tale, with three fictitious characters (P, M, and their musical friend MF) who represent the different disciplines involved in this research (psychology, mathematics/computer science, and musicology). It reflects some of my experiences in research that is concerned with the computational modeling of music cognition, using an interdisciplinary approach combining expertise from musicology, psychology, and computer science. See (2) for a discussion on this approach.
[2] Quite some time ago P, who is interested in psychology, and M, an amateur mathematician, got together during the Christmas holidays with their musical friend MF. Those were the days before cellular telephones, a time of herbal tea and the just-arrived technology of MIDI. MF, while duly impressed by Messrs. P and M's well-equipped music studio and expertise in computer modeling, remained unimpressed by their musical results and, sadly, left rather irritated to spend his Christmas elsewhere.
[3] Not so long ago MF remembered those Christmas holidayswhile he was reading a book on the history of tempo rubato. He was still convinced his friends were on the wrong track with their silly computer models. But the more he read about tempo rubato, the more he was convinced thatthey might have overlooked an obvious link between music andbiological motion. Blatantly obvious--once he realized it--wasthe explicit reference of music terminology, words like andante or accelerando, to qualities of human movement.And therefore, he reasoned, a successful model of expressivetiming--unlike the unsuccessful models made by hisfriends--should be based on the rules of movement and thehuman body.
[4] MF couldn't help making a phone call to P, the amateurpsychologist, to tell him about his new insight. "My dearfriend P," he said, "for expressive timing to sound naturalin a performance, it must conform to the principles of humanmovement. Isn't knowledge about the body--the way it feels,moves, reacts--what musicians share with their listeners?" Palmost immediately became enthusiastic. He saw a newopportunity to continue the investigations that had ended sobrusquely before. P decided to go to the library and therehe found a lot of interesting psychological literature onthe relation between motion and music. Much of it, however,involved some formidable mathematics. MF then proposed tohave a new gathering with the "old team," including theirmathematical friend, and this time at MF's home, safe frommodern technology!
[5] A few days later P and M found themselves at MF'skitchen table, which was well stocked with a pot of tea anda tin full of cookies. They returned to a lively discussionon expressive timing in music. After browsing through thebooks that P brought, M (the amateur mathematician) statedwith some authority that "these models borrow fromelementary mechanics and kinematics. They talk about mass,force, and speed of an object in terms of velocity, time andplace. And, interestingly, tempo variations in musicperformance are compared with the behavior of physicalobjects in the real world." P was all ears; MF just tookanother sip of his tea.
[6] M wrote most of the formulas, one below the other, on apiece of paper, patiently explaining their formaldifferences (a tidier version of M's jottings is shown in Example 1 [DjVu] [GIF]). MF protested "But M, please! We are investigating music here, not mechanics!" "Look," P swiftly interrupted, "I found the studies of these music researchers. They explain ritardandi in music performanceas alluding to human motion, like the way runners come to astandstill. Let me read a passage for you: 'Performers aimat this allusion, and listeners, with some education, findit aesthetically pleasing.'(4) Isn't this exactly what youdescribed to me on the phone!" (See Example 1: Formal models of the 'final ritard' in music performance [DjVu] [GIF].)
[7] P and M seemed confident that they had now found what they had been searching for all the time. MF too was quite pleased with the fact that these respected researchers had found evidence for his intuitive ideas about bodily motion. But he still had reservations. "How does the math of elementary mechanics compare to a final ritard in music? Can't we listen to these formulas?" M replied with a frown on his face, "Well, if we would have met in our studio we could have programmed them for you. Now we have to think of something else." But after a small pause he began to smile. "Let's see how far we can get with the material in your garage."
[8] That morning MF's kitchen turned into a real workshop. "Can we use one of your music boxes?" P asked sheepishly. With some hesitation MF collected one of his beloved machines from the living room. And after some hours of trifling and hammering they had built it--a "true" physical model of constant braking force! (See Example 2: A mechanical implementation of a constant braking force model [DjVu] [GIF], and Example 3 ([56K / LAN]) for a brief RealPlayer movie (56K) showing the machine at work.)
[9] MF inserted his favorite piano roll, a Bach fugue, intotheir newly made contraption. He turned the flywheel and themusic started playing. A few bars before the end he releasedthe handle, and the music came slowly to a standstill overthe last few notes. "Wonderful, wonderful!" They all jumpedwith joy. MF thought his antique music box had finallybecome truly musical.
[10] When they had calmed down a bit, M had a second look athis paper full of formulas, and said with a tone not atypical of a young mathematician: "But I have to say thatthese models are actually under-specified. They make noclaims about how to derive the 'metaphorical' mass or speedfrom the music. In our contraption we just arbitrarilydecided on the mass of flywheel, and we can freely decidethe speed at which the handle is released." M also realizedthat their contraption had some shortcomings. "Our flywheelhas a fixed braking force, caused by the friction of thecontraption. But it should actually be dependent on when,and at what speed you release the handle, and stop when theright final tempo is reached, like the equations show.That's difficult to make mechanically." But P responded, "Ohcome on M, don't be so strict. Let's just try another one, aslightly more modern piece. What do you think?" After somesearching, MF returned with a piano roll of Beethoven's Paisiello Variations. "Remember this?" he teased. MFinserted the piano roll and they listened again for the lastmeasures of each variation. But whatever they tried,releasing the handle early or late, at higher or lowerspeeds, it never sounded quite right. "It doesn't do therhythmical figures right," MF complained. "Apparently itonly works with the repeated eighth notes of the fugue."
[11] "We could be here forever trying to change this or thatfactor," P warned. He was convinced they had to return tothe empirical approach. "Why don't we look at how MFperforms final ritards?"
[12] P opened his case and pulled-out a folder with theperformance data they had collected during that firstChristmas gathering. "These are the graphs of MF performingthe final measures of Träumerei by Schumann." Andenthusiastically holding up an article, P added, "And hereare some interesting measurements made from recordings bysome of your colleagues. Look, you played it just likeAlfred Brendel!" (See Example 4: Final ritards in performances of the last three measures of R. Schumann's Träumerei [DjVu] [GIF].)
[13] There was quite some diversity among these famouspianists; they all seemed to play the final measuresdifferently. MF said, questioningly, "I do not see how onesingle curve could describe all these performances?" Presponded "But the point here is to model the average,normative performance." To which M added, while pointing atEquation 3, "This research showed that the last six notes ofthese averaged performances can be fitted closely by aquadratic function. That is an important finding, isn't it?""Indeed, M" P confirmed, "but we must be aware that anaverage curve is a statistical abstraction, not a musicallyreality." Their musical friend smiled and took another closelook at the diagrams. "So if I understood yourexplanations," he asked M, "this function should have ahollow, concave shape. But doesn't our contraption generatea convex shaped deceleration?" M confirmed this. "A convexshape indeed is what the other research found. Apparentlythere is evidence for a variety of shapes. However, whatworries me is the complete freedom in deciding on the massand amount of force applied; fitting these curves to thedata is too flexible." "Maybe all these pianists have theirown specific force and mass?" MF interjected optimistically.They looked at each other with some disappointment. Itseemed that once again they had failed to find a model ofexpressive timing that could please their musical friend.MF, who this time wanted to end their endeavors in a moreoptimistic manner, proposed "Lets go to the living room. Iwill play my favorite fugue for you."To be continued ...
[14] This tale talks about kinematical models of expressivetiming, and it questions in how far expressive timing can beexplained by models of physical motion. The formalizationsdiscussed above are based on the notion of a tempo curve (a continuous function of time or score position) regressing, for example, a linear tempo function to the performance data. One point of critique is that the predictions made by these models are insensitive to the actual rhythmic structure of the musical material: they make the same predictions for different rhythms.(5) However, more central is the objection that these descriptions do not, in principle, teach us anything about the nature (whether "motional" or not) of the underlying perceptual or cognitive mechanisms. Even if we assume that these curves do give a good approximation of the empirical data (despite the contrasting results in the research discussed above), the mere fact that the overall shape (e.g. of a square root function) can be predicted by the rules that come with human motion is not enough evidence for an underlying physical model of expressive timing in music performance, however attractive such a model might be. The relation between music and motion turns out to be not that simplistic.(6) The challenge here is to formulate appropriate alternatives.(7)
Baily, John. "Music structure and human movement." In Musical structure and cognition, edited by Peter Howell,Ian Cross, and Robert West, 237-258.London: Academic Press, 1985.
Clarke, Eric F. "Generativity, Mimesis and the Human Body in Music Performance." In "Music and the Cognitive Sciences," edited by Ian Cross and Irene Deliege.Contemporary Music Review 9 (1993): 207-220.
_______________. "Rhythm and Timing in Music." In Psychology of Music, 2nd ed., edited by Diana Deutsch, 473-500. New York: Academic Press, 1999.
Desain, Peter and Henkjan Honing. "Tempo Curves Considered Harmful". In "Time in Contemporary Musical Thought," edited by Jonathan D. Kramer. Contemporary Music Review 7/2 (1993): 123-138.
_______________. "Does Expressive Timing in Music PerformanceScale Proportionally with Tempo?" Psychological Research 56 (1994): 285-292.
_______________. "Physical motion as a metaphor for timing in music: the final ritard." Proceedings of the International Computer Music Conference (1996):458-460.
Desain, Peter, Henkjan Honing, Huub Van Thienen, and Luke Windsor. "Computational modeling of music cognition: problem or solution?" Music Perception 16 (1998): 151-166.
Epstein, David. Shaping time. New York: Schirmer, 1994.
Feldman, Jacob, David Epstein, and Whitman Richards. "Force Dynamics of TempoChange in Music." Music Perception 10/2 (1992): 185-204.
Friberg, Anders and Johan Sundberg. "Does music performance allude to locomotion?A model of final ritardandi derived from measurements of stopping runners." Journal of the Acoustical Society of America 105/3 (1993): 1469-1484.
Gabrielsson, Alf. "Music Performance." In Psychology of Music, 2nd ed. edited by Diana Deutsch, 501-602. New York: Academic Press, 1999.
Honing, Henkjan. "From time to time: The representation of timing and tempo."Computer Music Journal 35/3 (2001): 50-61.
Kronman, Ulf, and Johan Sundberg. "Is the musical ritard an allusion to physical motion?" In Action and Perception in Rhythm and Music, edited by Alf Gabrielsson. Royal Swedisch Academy of Music no. 55 (1987): 57-68.
Repp, Bruno H. "Diversity and commonality in music performance:An analysis of timing microstructure in Schumann's Träumerei." Journal of the Acoustical Society of America 92 (1992): 2546-2568.
Shove, Patrick and Bruno H. Repp. "Musical motion and performance: theoretical and empirical perspectives." In The Practice of Performance: Studies in Musical Interpretation,edited by John Rink, 55-83. Cambridge: Cambridge University Press, 1995.
Sundberg, Johan and Verrillo, Ronald T. "On the anatomy of the ritard: A study of timing in music." Journal of the Acoustical Society of America 68 (1980): 772-779.
Todd, Neil P. M. "A model of expressive timing in tonal music." Music Perception 9 (1985): 33-58.
_______________. "The Dynamics of Dynamics: a Model of Musical Expression." Journal of the Acoustical Society of America 91/6 (1992): 3540-3550.
_______________. "Motion in Music: A Neurobiological Perspective." Music Perception 17/1 (1999): 115-126.
1. David Epstein, Shaping time (New York: Schirmer, 1994);Anders Friberg and Johan Sundberg, "Does music performanceallude to locomotion? A model of final ritardandi derived from measurements of stopping runners," Journal of the Acoustical Society of America 105/3 (1999): 1469-1484; Neil P.M. Todd, "Motion in Music: A Neurobiological Perspective," Music Perception 17/1 (1999): 115-126.
Return to text
2. Peter Desain, Henkjan Honing, Huub van Thienen and Luke Windsor, "Computational modeling of music cognition: problem or solution?," Music Perception 16 (1998): 151-166.
Return to text
3. This tale is a continuation of Peter Desain and Henkjan Honing, "Tempo Curves Considered Harmful", in Jonathan D. Kramer, ed., Contemporary Music Review 7/2 (1993): 123-138 ("Time in ContemporaryMusical Thought"). See http://www.nici.kun.nl/mmm/tc for additional sound examples.
Return to text
4. Bruno H. Repp, "Diversity and commonality inmusic performance: An analysis of timing microstructure inSchumann's Träumerei," Journal of the Acoustical Society of America 92 (1992): 2546-2568.
Return to text
5. Peter Desain and Henkjan Honing, "Does Expressive Timing in Music Performance Scale Proportionally with Tempo?," Psychological Research 56 (1994): 285-292; Henkjan Honing, "From time to time: The representation of timing and tempo," Computer Music Journal 35/3 (1999): 50-61.
Return to text
6. For a further discussion see Henkjan Honing, "The FinalRitard: On Music, Motion, and Kinematic Models," ComputerMusic Journal, submitted (see http://www.nici.kun.nl/mmm/tc).
Return to text
7. Special thanks to Robert Gjerdingen for valuablesuggestions on an earlier version of this paper, and toBruno Repp for his constructive criticisms. Thanks also to the Department of Mechanics, University of Amsterdam, for actually making the contraption described in this paper.
Return to text
Copyright © 2003 by the Society for Music Theory. All rights reserved.
[1] Copyrights for individual items published in Music Theory Online (MTO) are held by their authors. Items appearing in MTO may be saved and stored in electronic or paper form, and may be shared among individuals for purposes of scholarly research or discussion, but may not be republished in any form, electronic or print, without prior, written permission from the author(s), and advance notification of the editors of MTO.
[2] Any redistributed form of items published in MTO must include the following information in a form appropriate to the medium in which the items are to appear:
This item appeared in Music Theory Online in [VOLUME #, ISSUE #] on [DAY/MONTH/YEAR]. It was authored by [FULL NAME, EMAIL ADDRESS], with whose written permission it is reprinted here.
[3] Libraries may archive issues of MTO in electronic or paper form for public access so long as each issue is stored in its entirety, and no access fee is charged. Exceptions to these requirements must be approved in writing by the editors of MTO, who will act in accordance with the decisions of the Society for Music Theory.
This document and all portions thereof are protected by U.S. and international copyright laws. Material contained herein may be copied and/or distributed for research purposes only.
prepared by
Brent Yorgason, Managing Editor
Updated 07 March 2003
