Home

 

Music-Driven Motion Editing

Video and Slides Download Page

 

 

1. PowerPoint Slides

 

Siggraph2002MusicDrivenSketchTalk.ppt (1.8MB)

 

2. Slide Videos

 

Link Description Audio Codec Video Codec File Size Details

Quality

MIDIAudio_features_inv.avi  

400 x 300 Divx5 AVI video with sound MPEG-3 160kb/s Divx

Free Codec for Windows

Free Codec for Linux

Free Codec for Mac

7.6MB An animated representation of the extracted musical features.

You can see the original MIDI data on the top row, and all subsequent rows show various extracted MIDI (and later AUDIO) features. 
MEDIUM

readBalls_complete_orig2.avi  

460 x 476 Divx5 AVI video with sound MPEG-3 160kb/s Divx

Free Codec for Windows

Free Codec for Linux

Free Codec for Mac

21MB In this first example, we start out with 7 user-defined keyframes (depicted here as the red cones). 
The red ball repeatedly passes through these using linear interpolation.
As you might expect, the motion bares no relation to the music.
MEDIUM

readBalls_complete.avi  

460 x 476 Divx5 AVI video with sound MPEG-3 160kb/s Divx

Free Codec for Windows

Free Codec for Linux

Free Codec for Mac

21MB We modify the trajectory of the ball between the keyframes by changing the Tension and Continuity on a Kochanek-Bartels spline.
These params are controlled by the segmentation based on vertical note density
For good measure, we also add Perlin noise to the scale of the ball to mark at each change in horizontal density. The frequency of the noise is proportional current register.
MEDIUM

beforeAfter_final.avi  

400 x 300 Divx5 AVI video with sound MPEG-3 160kb/s Divx

Free Codec for Windows

Free Codec for Linux

Free Codec for Mac

9.3MB In this second example, we alter a motion capture sequence. First, the end-effector positions of all the limbs are used to calculate the sum of the absolute velocities of these end-effectors.
We then match the motion at these minimums to peaks in the -spread of the notes-, using localised timewarping.

The soundtrack is at the bottom. The chosen musical feature is designated by black dots. 

The original motion is on the left and the modified on the right.

Note we also add a some head-nodding to better mark these density peaks.

MEDIUM

drunkwalk_withaudio_colorblended.avi

400 x 300 Divx5 AVI video with sound MPEG-3 160kb/s Divx

Free Codec for Windows

Free Codec for Linux

Free Codec for Mac

6.3MB
This time, the audio-based mean zero-crossing rate is extracted [in blue] and used to change -what we refer to as- the Scale of the motion.

This is a modified version of the Chiís EMOTE laban Scale parameter.

The intensity of the inclination of the avatarís back is associated with the mean RMS volume.

What you should look for here is: 

the motion will be closer to the original one proportionally to the amount of Ďvariationsí in the music, the motion will be scaled down otherwise 

Basically, The busier the music, the busier the motion
MEDIUM

 

3. Other Example Video

 

Link Description Audio Codec Video Codec File Size

Quality

Sodemo_mp3.avi  

320 x 240 Cinepak AVI video with sound MPEG-3 160kb/s Included 12MB MEDIUM
SodemoDetails.ppt   PowerPoint Slides describing video     662KB  

 

 

This page was last updated in April 2003.
Please send any comments to mpc33 at cam.ac.uk.