Slashdot Mirror

← Back to Stories (view on slashdot.org)

Simulating Human Musical Performance

Posted by CmdrTaco on from the will-for-food dept.

GFD writes "The EETimes has a story about a software program that can mimic the subtlties that humans give to a musical performance to give it a particular emotion or style. Apparently it is already so convincing that some TV producers are using it rather than live musicians. Long range plans are to clone the musical styles of famous musicians. Interesting question - there is no question that a Jimmy Hendrix could copyright his music but could he copyright his style? " Bet they can't mimic all 3 of my power chords.

28 of 218 comments (clear)

  1. Re:Jazz improvisation by jumpinin · · Score: 2

    I've got quite a bit of jazz theory - enough to know that truly great jazz musicians don't just rely on picking the "right" notes. They often use the wrong notes, they use the space between the notes. One of the differences between guitar and sax in jazz is guitarists don't have to breathe. As my jazz improv class prof used to say - "You're phrasing like a sax player would if he could run a hose up his ass for air!". But a computer doesn't have to breath. You can program it to only play so many notes before it has to leave some space but it will never know what it is to breathe. Plus great improvisors know when you can ignore the chords or change the chord qualities to fit a melodic idea. When does a quote of another melody sound cool and when does it sound corny? How closely does a melodic idea need to fit the changes to work. If you can get a computer to play "Goodbye" like Art Pepper on "Thursday Night at the Village Vanguard", I'm selling my bass!

    --
    Verbing wierds language --Calvin
  2. Re:Downloadable samples? by vectro · · Score: 2

    The product's website has a page where you can download and play some sibelius scores. But you have to install a plugin to do so, which is only availible for Windows. :\

  3. iMuse by Spire · · Score: 2

    What you describe sounds just like iMuse, which was developed around a decade ago by LucasArts for its own line of adventure games. I first heard it used in Monkey Island 2, and it really blew me away. The background musical score truly changed dynamically, seamlessly, and continuously to suit the action.

    iMuse has since been used in (AFAIK) every LucasArts adventure game, including Day of the Tentacle, Indiana Jones and the Fate of Atlantis, and Sam and Max Hit the Road.

    Newer LucasArts adventures, such as Full Throttle, Monkey Island 3, and Grim Fandango, use a digital-sample-based version of iMuse (as opposed to the original MIDI-based iMuse). Using digital samples increases the audio quality and fidelity of the music, while decreasing flexibility somewhat, since the musical repertiore is limited to a library of prerecorded phrases and segments. Still, it's a terrific, innovative system that's been put to great use over the years.

    --
    begin 644 .sig22&%I;"P@9F5L;&]W(&=E96 LA`end
  4. Re:great! by Stephen+VanDahm · · Score: 3

    "now all pop music can be created by formulaic computer programs!"

    Better than being created by formulaic record producers. At least now when they interview sucky artists on MTV, they'll have to be honest about where their inspiration is coming from. Instead of "Tom -- the producer -- is such an artist. He, like, totally understood where I coming from on this track" we'll finally hear the truth. "Well, like, you know, Carson, the businessmen at the record company used, like, Microsoft Hit-Maker Pro or something to generate the backing tracks and vocal melody for, like, the entire album, you know. Then, you know, some other businessman brought me some lyrics that, like, you know, test well with 14-year-old girls . . ."

    Of course, the MTV dopes will probably market this as the Next Big Thing -- "Cyber Rock -- the all-digital multimedia experience" or something dumb like that.

    take care,

    Steve

  5. Forget about instrumentals by webslacker · · Score: 4

    What I wanna know is, can they come up with a way to make Britney Spears' lip-synching more realistic?

  6. Copyright and celebrities by empty · · Score: 2

    Clearly a "style" can't be copyrighted. Even Elvis complains to me that he doesn't get royalties on all the imitators out there.

  7. What about *new* teachers? by Imperator · · Score: 2
    It will be a sad day when a teacher can't get hired because someone thinks a machine (television) can do his job better.

    In a worst-case scenario the proliferation of such devices will discourage young people from exploring the opportunities of teaching as a career, and may decrease our chances of.....

    Machines can replace people who perform manual labor. Machines can replace people who perform glorified forms of manual labor. Machines cannot replace creativity. (At least, not in the forseeable future. :)

    You can either lament that machines have replaced most of the need for humans in coal mines, or you can recognize that the continuing division of labor will obsolete certain professions and create others. If a machine can play music better than humans, then so it will be. But until a machine can duplicate human creativity, there'll be plenty of interesting things for us to do.

    --

    Gates' Law: Every 18 months, the speed of software halves.
  8. Two Issues by Local+Loop · · Score: 3

    There are really two issues here. One is the simulation of time (rythm), notes, and dynamics (volume). The other is simulation of the expressive sounds of the instruments themselves.

    These computer programs all work along the same lines, manipulating rythm, dynamics and notes. I have yet to see any program duplicate the variations in timbre (tone) that a real musician can achieve on his instrument.

    I can tell a computer generated score everytime by the fact that the attacks don't vary, and the timbre doesn't change with the volume.

    Besides, there's something magical about a group of musicians, all together, when then lock into a groove. It's something that a transcends the abilities of any single human, and certainly any single computer. And that space is where most of the best ideas are born.

    On the other hand, working composers banging out MIDI scores on demand will probably find this a very useful time-saving tool. Just lay down kick, snare and hi-hat right on the beats, and turn on the 'humanizer' feature to make it all sound loose. No more twiddling with each note to make it sound right.

    Rambling Again,
    Loopy

  9. Yes, but... by walnut · · Score: 2

    ...can it improvise?

    This sounds like a great tool for recording strict classical songs with no varriation, or garage-band formula songs. It might even play like Jimi, but it doesn't sound like it handles the once in a lifetime sound one might find from a show of his, or the instant inspiration and back-tracked tangent he takes. - not that I've ever seen Jimi play... But it's the same as a B.B. King, or a DMB (as much as I don't care for them), - all of these people are musicians...

    Play the notes, add technical dynamics and a realistic sound base - but if you can't make music - it'll still just be crap.

    Use of this only looks to stiffle human creativity.

    --
    You say you want a revolution?
    1. Re:Yes, but... by Jon-o · · Score: 3

      You betray some ignorance of classical music when you say that - not that it's different from the vast majority of people!

      Classical music, though usually lacking in the wholehearted improvisation of jazz (though even that almost always relies on pre-existing chords and styles), still involves a lot of variation from performance to performance. The notes might be the same, but that just means that the musicians are concentrating more on the subtle things than on the notes themselves.

      As well, just for the record, it's only quite recently that "classical music" (when will we get a better term?!?!) has not been improvised. Mozart, Beethoven, Rachmaninoff, Bach, Paganini, etc.. all improvised - most estimates say they improvised (in concert!) much more music than they ever wrote down. Some classically-trained people still improvise a bit - most decent church organists do, for one. Early music specialists (of which I am one) are always trained in some extemporization.

      Perhaps more subtly, but also more importantly IMHO, is that ALL music is improvised when performed half decently - not necessarily everything, but aspects of it.

      One example is a saxophone player (mainly classical) who told me in a master class that he takes into account the acoustics of the hall when he's playing. In a boomy, echoey hall, notes that he wants to be short much be played EXTRA short, in order to not be stretched by the reverb. Likewise, in a dry acoustic, notes need to have resonance added to them by the player, as a kind of performing trick, because the room won't create the sound on its own.

      Even more importantly than little technical things like this is the fact that any performance is unique, no matter what style. A musician will be playing for the moment - judging his interpretation based on the audience, his feelings, the sound produced, the physical sensations, ambient sound in the hall, etc etc etc... For a truly exceptional performance, all of these things need to be taken into account.

      Any music played "without variation" is far from a classical song, in my book! Rather, it's a technical excercise, devoid of music. I'm skeptical of the abilities of this program to do much more than that, though I'll reserve judgement until I can hear it for myself.

  10. MIDI is a control standard, not "mimicking" by Anonymous Coward · · Score: 2
    MIDI is a standard for mimicking instruments

    No, MIDI is not a standard for mimicking instruments. It's a communication standard for controlling Musical Instruments using a Digital Interface. If you look at MIDI messages, they are almost all of the "turn this note on", "turn this note off", "turn the volume up" and "use this patch" variety. (BTW: here, the term patch refers to a set of parameters that make up a sythesised sound, kinda like a sample, but not really.) There are a few uses of MIDI to dump samples and sysex data, but MIDI is kinda slow for that so most people use disks or SCSI interfaces.

    For more information, check out http://www.harmony-central.com/MIDI/Doc/doc.html .

  11. Lip-sync to what? by Morgaine · · Score: 5

    Oh, you mean you have the sound turned on?

    --
    "The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
  12. Forget Jimi Hendrix... by dfay · · Score: 2

    I won't be impressed until it can simulate my junior high school band with a substitute teacher leading.

  13. Here's a way to do it by Chris+Johnson · · Score: 2

    The assumption seems to be that you have to switch to entirely separate scores for different parts. That's not so. This is something I've put some thought into, not that I can code it right now, but USE this, anybody who can, gpl it or something.
    Take an extended musical piece, let's say something as MIDI for the purposes of the example (the idea was designed around MIDI). Set up some chord changes, an entire little score. Then do an extended improvisation or composition. Have it be major-key, and long enough to avoid too much repetitiveness.
    Now copy it to another track, just duplicate it straight across- then change every major third to a minor third. Boom! Suddenly you have a programmatic switch between two almost-identical scores, where at any point you can subtly shift the mood according to what's happening on the screen at that second. It's not a grand score change, it's imperceptible, but serves as a subtle cue that something ominous is going on. The solo turns foreboding, and you turn to look and WHAM! ;)
    Then add drums and bass, in a relaxed groove. Fine- now record them again, busy and violent. Then for kicks record a drum track that's all fills. Now you have another level- an activity monitor. If the game picks up, the groove picks up as well, and if things happen suddenly you can hit a drum fill at any point (if desired, doubled with bass).
    Now add some synth pads- let's say the game is like Joust or something, where you might be going up or going down or staying at a level. If the player is going up, up_pad is on: if the player's going down, down_pad is on: otherwise they are muted.
    Your result is a completely fluid piece of composition in which the dynamic nature of it was also human-composed. Every game will be different, and the score tracks the activity so closely that it might be a sound effects track, but it does it in a completely fluid manner. You could easily have other lead voices to accompany different monsters or prizes- the whole score is so completely dynamic that there is no such thing as 'the score', it's completely a product of how you play the game and is never quite the same.
    Computer involvement is a tool not a replacement for human input... in fact the example I've given could be implemented with current technology while using entirely human-performed musical tracks.

  14. Teaching tool by DreamerDude · · Score: 2

    A decent use of a performing tool like this
    might be for teaching.

    Back in high school (many years ago) I used a
    computer to program difficult etudes (musical
    exercises) in to see how a technically perfect
    rendition would sound.

    The computer helped with difficult rhythms or
    runs, but it had no concept of style. everything
    sounded "square".

    Such a tool could now be used to play prewritten
    music in such a way that beginning and intermediate
    musicians have help "visualizing/aurealizing"
    pieces that are beyond the scope of their present
    capabilities.

  15. You'll still be able to create your own works by Morgaine · · Score: 2

    I can't see any reason to feel bothered about computers being able to do what you do, eg. write music in this case. After all, you'll still be able to compose and perform your own original material.

    It's no different to the existence of other human musicians. They can do everything that you do as well, yet this doesn't in any way stop you from being original.

    --
    "The question of whether machines can think is no more interesting than [] whether submarines can swim" - Dijkstra
  16. Jimi Hendrix performance of Star Spangled Banner by Chris+Johnson · · Score: 2

    *grin* *chuckle* *ROFL*
    Of all the examples they could have picked...
    For those puzzled, Jimi did a freeform version of the Star Spangled Banner at woodstock, at deafening volume with feedback and complete digressions into other musical quotes like "Purple Haze" if I remember the story, and mutated the bit about 'the rockets' red glare' into a raging shriek of atonal feedback followed by a bomb explosion by way of political commentary.
    If anybody thinks a computer can take the notion of a national anthem and twist it into such a powerful, creative, and politically significant statement on its own volition, they are quite mad. You could program the thing to do it- but then YOU become the musician, indirectly. At any rate, these silly people (was it the reviewer or the developers who didn't notice this?) who suggest seeing what would happen if you fed the Hendrixbot 'The Star Spangled Banner' have inadvertently illustrated in the most graphic and unforgettable manner... just how much can be _lost_ >;)

  17. us poor musicians by jbf · · Score: 2

    First off, I'm primarily a computer scientist... music is a [serious] hobby.

    When we talk about computers _replacing_ musicians, I don't think that will happen. Even if a computer can write better music than I ever can, I'll still compose, because it helps me respond to emotion. In fact, many people will likely do the same. With reasonable probability, some of these types of music will be emotionally more satisfying than computer generated score and computer generated lyrics sung and played by computers. After all, if this technology was invented when only classical music was played, pop music would probably still have evolved (maybe differently, but that's the subject for another post) because of the change in culture and expression.

    Saying that computers will ever kill music is like saying they'll kill painting or what-not. Technology can simulate some instance of a human's creative spark, but IMO it will never be able to simulate the spark itself. Perhaps this will make it difficult to make a living as a professional recording/performing artist (not that it isn't difficult even now), but such is the nature of technology.

    I'd argue that this won't make us less cultured, because people will still pursue music for personal fulfilment; it will just change how people are compensated for music. The change would be similar to how if all software was open-source by law, some people would still program, and in fact the people who stick around will be among the best.

  18. Have any of you read the article? by DzugZug · · Score: 2
    This program does not write music. It performs music. For TV they still hire a composer to write the music but they don't need to hire musicians to play it and record it. So no, whoever asked, they can't just play a track off a CD. Also this is different from the program that could compose in a composer's style. This can perform a piece with emmotion. It adds accents and vibratto the way a human player would.

    I find it annoying that people post responces like "I don't believe it. A computer can't do x." When the article said that this new computer program can do y. Slashdot has been invaded by the posters on Amazon that say "Well I havn't read the book but this is what I think..."

    Just a request... before you flame me or moderate me down read the article and read some of the potings on this story.

  19. No. Sorry. Music = Interpretation. by Monty+Worm · · Score: 3
    (disclaimer: I am not a professional musician. I have, however, been taking making music for 17 years (with lessons for the first 12 or so...))

    I'm sorry, no. I really don't think the music scene is equipped for this sort of thing.

    Music is interpretation, it isn't an equation. You should not simply use some sort of hokey Scoring + Emotion = Music equation and expect to have something worthwhile come out. Maybe for genuine on-the-fly stuff like context-sensitive game music stuff it could be welcome.

    The key here, is art. Music, like other more physical art forms, is all about human imput. If Mike Oldfield's classic 'Tubular Bells' album was released today, and he'd played synthesiser, instead of actually learning and playing that myriad my instruments, it just wouldn't get it.

    Even for a group, flexibility is key. I've played with (concert) band conductors who refused to say 'in the concert we will do it this way'. He preferred to deal with the specifics as the situation developed. Things like the crowd are getting bored, let's play this bit faster; the acoustics here really suck, let's make that instrument a bit louder..)

    Maybe it would be good for bland elevator/Mc Donalds background music, but not for anything else.

    Music is my recreation (listening, playing, creation). Anything that takes that away is bad.

    --
    ... and today's pet project has ... been discarded for lack of time.
  20. Re:fake can never be real enough by jareds · · Score: 2
    I am a musician, and I want to hear my computer making music just as much as all you programmers want computers to write your code for you, and be better than you at it. :-)

    When you think about it, there's quite a difference between the two for our respective economic futures. I think there would be many, many people who would continue to listen to human-written music, even if people in double-blinded tests were unable to tell the difference. People might even claim to tell the difference, and look on an ability to tell the difference as indicative of cultural refinement or something. However, I doubt anyone would care whether the program they use is coded by humans or a not.

  21. Replace audio CDs with "style" CDs by tap · · Score: 2

    Instead of the latest pop group comming out with a album of mundane audio tracks, they could come out with a style album. It would have samples, wavetables, voice samples, lyric generator tables, and a musical style description for use with a computer music generator. You stick it in your computer and push go, and it spits out an "original" track in the style of the group. If you hear one you really like, you can dump it to an MP3. An infinite number of tracks on a single CD. And given the originality of most popular music, who would know the difference? Probably already in use now.



  22. Already happenening... sort of... by Overt+Coward · · Score: 3
    The (now cancelled, please bitch to Sierra -- it was 3-6 months from completeion, at most) Babylon 5 space combat simulator was supposed to have a dynamically created soundtrack. Not quite up to the level we're talking about here, but it would seamlessly blend in fragments written by series composer Chris Franke in such a way that it would fit the on-screen action. This of course would be based on a rule set as to which fragment(s) would be appropriate and how to make a smooth transition into it. Still, the obvious benefit to games should be readily apparent.

    Moving this same type of system to a completely independent composition is obviously an order or two of magnitude more difficult, but essentially, it would require using style fragments instead of music fragments, a way to provide it's own "mood" (instead of being fed by an external source [the game]), and some more complex rules on music theory in general to make things cohesive.

    All in all, pretty cool. It'll be unlikely that such a system produces anything particularly innovative or moving, but it will certainly help lower budget TV shows or movies build up better scoring -- it's a sad shame that music is one of the first things cut when budget gets tight. Looking back at B5, Franke's score adds so much to the on-screen scenes, often more than 20 minutes of original music for a 44 minute show...


    --

  23. The awful truth by konstant · · Score: 2

    It's just a MIDI file with random delays between notes. It is so realistic that when they played it at a session of my local Oompah Band we could hardly believe our ears!
    -konstant

    --
    -konstant
    Yes! We are all individuals! I'm not!
  24. Used in TV instead of real musicians? by turbohavoc · · Score: 2

    Excuse me but this just sound plain stupid, if you ever would want to replace a musician because you dont have the money to have real ones, you probably would replace them with prerecorded pieces of music in appropriate lengths that you have on a CD or DAT.

    I dont really know how their system would work, but i thought of some alternatives.:

    If the thing is about dynamically created music content, I doubt that you could get any interesting music from it, you will get musically correct results that follows correct scales and such, but i doubt that you get anything that affect the listener or anything like that. It would only sound like a cheap home-multimedia hobby midifile.. Computers can not feel.

    If its about ready made midi-style music thats "groove-quantitized" in real-time, then whats the deal? You can just have a digital recording of the music, and that would mean much better sound quality.. (by sound quality i dont mean the quality the music is made in, like cd-quality or such, but how professional the results sounds after mixing and mastering and all the other stuff)

    All these ways of doing things would still only give you midi-style quality, since its realtime rendered sound, and just wont sound real enough for most instruments. Even if todays midi-intruments can sound real enough, you need very clever sequencing from the musicians to get it realistic, and its totally different on different instruments and is higly dependant of someone clever that can make it feel real, and a computer just cant do that until our computers have evolved a lot.. (think androids)

    just a thought :)

  25. Re:fake can never be real enough by SurfsUp · · Score: 2

    I think there would be many, many people who would continue to listen to human-written music, even if people in double-blinded tests were unable to tell the difference.

    Please let me know when the computer passes the double-blind test where it goes up against Vanessa-mae singing "Johnny". Oh, and don't forget, she wrote it too. If it passes that test, I'll happily listen to the computer, and maybe even try to get a date :)

    --
    Life's a bitch but somebody's gotta do it.
  26. This bothered me at first.... by Jack+William+Bell · · Score: 2

    ...After all I am a musician myself and am not sure I want to think that a computer can do everything that I can do.

    But I am also a computer programmer, a Science Fiction writer and a (to a small extent) an extroprian believer in a future where computers do do all the things that people do. Even including emotions and creativity. Given time I do think that true AI will happen, it may just be so strange that we don't recognize it as such at first.

    So I stepped back and looked at this a little more carefully -- and dang, it is just a simple rule based system. Easy, I could do it myself if I wanted! If the software produces a note that sounds expressive, it isn't because the computer is experiencing the feeling I know when I bend a note up and sustain it for a second before I do a full tremelo. It is just following rules that say 'Bend up at this speed, sustain this long, do tremelo with these parameters'.

    Hell, perhaps that sounds good enough for a TV Producer, but it also means that any difference between one performance and another is based on random numbers. Not due to the mood of the performer and how he or she reacts to their audience. It will be a while before a computer can really play the blues...

    Jack

    --
    - -
    Are you an SF Fan? Are you a Tru-Fan?
  27. fake can never be real enough by chocolatetrumpet · · Score: 3

    It will always take real musicians to do something truely great.

    Just as well as your computer can generate a believable landscape, with natural subtleties, a computer can generate a sound;

    Perhaps some day it may progress to the point of resolution where it is very nearly indistinguishable from real musicians.

    But, there is always something to be said for those who take art to a new and exciting level.

    I am a musician, and I want to hear my computer making music just as much as all you programmers want computers to write your code for you, and be better than you at it. :-)

    -stuff to think about

    --
    Spoon not. Fork, or fork not. There is no spoon.