Computers Compose Music, But Do We Listen?

[0] NOTE: This paper has been reproduced with permission from World Scientific from their volume Multimedia Modeling.

EMI: The State of the Art in Computer Composition

[1] Members of the IEEE Computer Society may recall that the July 1991 issue of their monthly magazine, Computer, was a special issue devoted to the topic of “Computer-Generated Music.” There has long been a fascination with mechanical means for generating original musical compositions. Indeed, that fascination preceded the invention of the computer itself by several centuries.⁽¹⁾

[2] Of the articles which appeared in this special issue, the one which probably best characterizes the current state of the art is “Recombinant Music: Using the Computer to Explore Musical Style,” by David Cope.⁽²⁾ The topic of the article is a component of a more general project called Experiments in Musical Intelligence (EMI). Cope provides an excellent summary of the fascinating behavior of this component in his introduction:

This EMI subprogram . . . separates and analyzes musical pitches and durations and then mixes and recombines the patterns of those pitches and durations so that, while each new composition is different, it substantially conforms to the style of the original. The new works generally inherit aspects of the style of the period and, to a lesser degree, the style of the composer of the recombined works. Called recombinant music, this is not just a parlor game but a serious attempt to understand how listeners recognize the style of a composer or period, one of the more elusive and difficult to describe musical phenomena.

[3] Cope’s claim is as fascinating as it is bold. His article leads the reader through an exercise in the synthesis of a piano sonata in the style of Wolfgang Amadeus Mozart. In an earlier paper⁽³⁾ he demonstrated that his technique works for not only Mozart but also Giovanni Pierluigi da Palestrina, Johann Sebastian Bach, Tomaso Albinoni, Frederick Chopin, Bela Bartok, Igor Stravinsky, and Scott Joplin. These are extensive efforts. If we are to consider whether or not we are on the threshold of the age of computer composers, we need to examine both the assets and the liabilities of Cope’s EMI system. Having done so, we may then be able to ask whether artificial intelligence has now solved the problem of turning a computer into a successful composer.

EMI’s Assets

[4] Perhaps the most important thing about Cope’s results is that, at first hearing, they do a good job of sounding convincing. One can play the “EMI-Mozart sonata movement” illustrated in his Computer article and be reasonably fooled into taking it for “real” Mozart. Serious Mozart scholars are less likely to be fooled, but even they may have to listen for a bit before identifying what they hear as an imitation. If nothing else, Cope may have latched on to getting a computer to perform an important exercise in the study of “free composition:” the task of using an existing composition as a model which guides the structuring of a new one.⁽⁴⁾

[5] Such a model-based approach is important because it takes the computer away from the ultimately misguided approach to music composition as a process of putting notes together.⁽⁵⁾ Composition operates at a larger granularity, and Cope’s recombinant music may be regarded as a hypothesis of a more viable grain size. Just how those grains are determined—whether they are products of pattern matching algorithms which lie at the heart of EMI’s operation or of Cope’s own skill at musical analysis—is not made entirely clear; but the results are impressive enough to justify taking his thesis seriously.

[6] However, the possibility that Cope’s own skill may figure in EMI’s results may, itself, be one of the system’s assets. Cope’s work may be distinguished from many other results in computer-generated music by the fact that Cope, himself, is a serious composer. This makes EMI a system which is based on practice as much as theory, if not more so. In other words the judgments exercised by Cope in developing EMI may very well have more to do with how practical experience guides his decisions than with any attempts at theoretical models which dictate how decisions should be made.

EMI’s Liabilities

[7] Nevertheless, it is important to observe that the extent to which EMI is convincing as a composer in its own right is superficial, in the literal sense of the word. In other words the decisions which EMI makes in composing a piece of music are all concentrated on the surface structure. However, much of music theory has been concerned with the question of whether or not music has a “deep structure,” such as that which Chomsky claims is in natural language.⁽⁶⁾ Cope is well aware of the importance of deep structure in both natural language and music. Indeed, much of his success is based on the power of the augmented transition network (ATN) to handle both the analysis and synthesis of natural language syntactic structures.⁽⁷⁾ The heart of Cope’s work involves adapting the ATN so that it may accommodate the synthesis of musical structures.⁽⁸⁾

[8] At this point, however, one can raise the question as to just who is doing the composing. Ultimately, the success of the computer at “synthesizing” Mozart or Joplin depends on Cope’s success at designing an ATN which controls that synthesis. This technique is not that different from the one Mozart engaged for his own “Dice Composer,” a toy which enabled his contemporaries to compose simple dances without any prior knowledge of music just by rolling a pair of dice.⁽⁹⁾ I recently demonstrated that this toy was basically a very well designed random sentence generator.⁽¹⁰⁾ Mozart’s Dice Composer, for example, generated (to use the terminology of computational linguistics) random sentences by selecting productions from a simple context-free grammar. Each of the terminal symbols in this case was an entire measure of keyboard music. Each measure of the score, in turn, corresponded to 11 productions, each of which filled in that measure with one of those terminals. The bulk of Mozart’s work with the Dice Composer was to make sure that the terminals chosen for any given measure were interchangeable. It would thus be fair to say that the real composer of anything produced by the Dice Composer is Mozart, himself, rather than whoever happens to be tossing the dice. By similar reasoning, then, in designing specifications with ATN representations, Cope is the one doing the composing, even if EMI is actually “generating the output.”

[9] Of course Mozart never drew upon use of the Dice Composer for his own compositions. One may assume that he felt that music it produced was “not good enough” for his personal standards. Indeed, he used his standards for what it meant to be “not good enough” as a basis for another composition, his K. 522 sextet, subtitled Ein Musikalischer Spass (a musical joke).⁽¹¹⁾ This composition serves as a sort of catalogue raisonne of the sorts of faulty devices one could find in the work of mediocre (or poorer) composers of Mozart’s day. Such composers would often have a few good ideas but simply never got the details correct, and Mozart was capable of picking out examples of such inattention as a subject for lampoon.

[10] This is not to say that Cope’s work is a joke. Nevertheless, it in unclear just how seriously an experienced listening ear may be expected to react. Certainly, it will not react to the Mozart examples as if it has just encountered a previously undiscovered Mozart composition. More likely, it will respond with an uneasy sense of familiarity, as if it had been reminded of Mozart but then recognized at least one spanner in the works which caused the result to be “not quite right.”

[11] Of course it is important to remember that the first letter of EMI stands for “Experiments.” In the world of science, not all experiments succeed; but we assume we can learn something even if the outcome is not successful. Can we learn anything from the results of Cope’s experiments?

[12] One thing we may learn is that, whether or not music has a deep structure, much of our response, as individuals, is to surface features. Thus, we can pick up on surface cues and start to follow through with them, even if the music does not follow through in the same way. This is particularly the case if the music is performed with the appropriate air of seriousness. Cope has been fortunate enough to have his work performed by skilled instrumentalists and singers participating in the Santa Cruz Baroque Festival. Hearing these results performed with the same seriousness of purpose which one would expect for a performance of Palestrina or Bach quickly dispels any notion that this is just another variation on a Mozart-type musical joke. Because the performers are not the village amateurs which Mozart chose to ridicule, we assume that the composer is no less mediocre. Ultimately, whether or not the results are laughable depends upon whether or not they receive very loving performances in a very convincing manner. Whether or not we come away any the wiser about “Musical Intelligence” or the stylistic characteristics of Mozart or Joplin may ultimately be beside the point.

Music Is in the Ear of the Listener

[13] Erik Satie was a key forerunner of the avant garde approach to musical composition which developed at the beginning of this century. There is a story that Satie was once (if not many times) asked if he felt that the sorts of things he did were really music. It is said that Satie’s favorite answer to this question was: “Music is what happens at concerts.” This seems to capture a key element of what makes output from EMI “work.” Whether or not it was music when it came out of the computer, it was certainly music when it happened at the Santa Cruz Bach Festival!

[14] Thus, the real lesson to be learned is that, in matters of music, practice is more important than theory. Music is what musicians do at concerts, and practice has more to do with performing behavior than with writing scores. Performing behavior remains a great mystery to us, but it has tremendous power. Skilled actors are capable of picking up a telephone directory and reading aloud from it with the same command of emotional manipulation they can apply to Shakespeare. This does not mean that a telephone directory is “great literature;” but it does mean that a good enough actor can fool a willing audience into taking it as such! This is because we, as audiences, do not listen to compositions; we listen to the performances. Composition is simply the armature which is then fleshed out by performance.

[15] For many centuries this approach to music behavior was pretty much taken for granted. It has only been in the last century or two that the composer has begun to assume the role of “significant individual” in his own right. In today’s world of competitions and grants, the situation has become even more exacerbated. The consequence is that we seem to be losing sight of the behavior side of the story, particularly the significance of the behavior of the performer. Yet it is that performer behavior which ultimately determines whether or not we, as audience, decide to devote enough attention to listen, regardless of whether what is performed has come from a struggling genius, a commercial hack, chance decisions, or even a computer program.

Stephen W. Smoliar
National University of Singapore
Institute of Systems Science
Heng Mui Keng Terrace
Kent Ridge, SINGAPORE 0511
smoliar@iss.nus.sg

Footnotes

1. Hiller, L. A. and Isaacson, L. M. Experimental Music: Composition with an Electronic Computer, New York: McGraw-Hill (1959).
Return to text

2. Cope, D. “Recombinant Music: Using the Computer to Explore Musical Style,” Computer 24 (7) (1991): 22–28.
Return to text

3. Cope, op. cit.
Return to text

4. Schoenberg, A. Models for Beginners in Composition, New York: Schirmer (1943).
Return to text

5. Smoliar, S. W. “Algorithms for Musical Composition: A Question of Granularity,” Computer 24 (7) (1991): 54–56.
Return to text

6. Chomsky, A. N. “Language: Chomsky’s Theory,” in The Oxford Companion to the Mind, Gregory, R. L., editor, New York: Oxford University Press (1987): 419–421.
Return to text

7. Woods, W. A. “Transition Network Grammars for Natural Language Analysis,” Communications of the ACM 13 (10) (1970): 591–606.
Return to text

8. Cope, op. cit.
Return to text

9. Mozart, W. A. The Dice Composer, New York: Guild Publications of Art and Music (1941).
Return to text

10. Smoliar, op. cit.
Return to text

11. Apel, W. “Musikalischer Spass, Ein,” in Harvard Dictionary of Music, Apel, W., editor, Cambridge: Harvard University Press (1969): 560.
Return to text

Copyright Statement

Copyright © 1994 by the Society for Music Theory. All rights reserved.

[1] Copyrights for individual items published in Music Theory Online (MTO) are held by their authors. Items appearing in MTO may be saved and stored in electronic or paper form, and may be shared among individuals for purposes of scholarly research or discussion, but may not be republished in any form, electronic or print, without prior, written permission from the author(s), and advance notification of the editors of MTO.

[2] Any redistributed form of items published in MTO must include the following information in a form appropriate to the medium in which the items are to appear:

This item appeared in Music Theory Online in [VOLUME #, ISSUE #] on [DAY/MONTH/YEAR]. It was authored by [FULL NAME, EMAIL ADDRESS], with whose written permission it is reprinted here.

[3] Libraries may archive issues of MTO in electronic or paper form for public access so long as each issue is stored in its entirety, and no access fee is charged. Exceptions to these requirements must be approved in writing by the editors of MTO, who will act in accordance with the decisions of the Society for Music Theory.

This document and all portions thereof are protected by U.S. and international copyright laws. Material contained herein may be copied and/or distributed for research purposes only.