Description

Source Motion with Sound

Motion: A cowboy walk followed by shooting.

Soundtrack: Harbour sounds and pistol shots.

Target Motion

Another cowboy walk followed by shooting.

Synthesis Constraints

Automatically defined by motion matching algorithm.

Comments

The RMS volume is used here to control the strength of the audio constraints.

The gunshots motion is accompanied by shooting sounds, which are louder than the surrounding harbour sounds. The average volume of shooting sound fragments will consequently be larger than the surrounding harbour sounds. The constraint weight is controlled by the volume. This ensures that a tighter control of the gunshots sounds, ensuring that they are given high priority during gunshot motions, and lower priority elsewhere (since we don't want a gunshot sound outside gunshot motions).

See for this example algorithm details.

 

 

 

Self-Similarity Synthesis

 

Source Motion with Sound

 

 

 

Shows motion of the 3D barycentre of the character for the Source motion.

Shows the norm of the 3D barycentre of the character for the Source motion. The minima are selected as fragment boundaries. The minima correspond to stoppages in the motion, which are typically good markers of changes in the motion.

Shows motion of the 3D barycentre of the character for the Source motion, along with selected minima.

Target Motion

 

 

 

Shows motion of the 3D barycentre of the character for the Target motion.

Shows the norm of the 3D barycentre of the character for the Target motion. The minima are selected as fragment boundaries. The minima correspond to stoppages in the motion, which are typically good markers of changes in the motion.

Shows motion of the 3D barycentre of the character for the Target motion, along with selected minima.

 

 

New Sound

 

(top) Shows the segmented Source soundtrack along the dynamically generated constraints source segments. (middle) Shows the synthesized sound for the Target motion. (bottom) Shows Constraint targets along with the origin of the grains in these targets regions.

Shows the average volume of the segments from the original soundtrack, which were used as audio sources for the constraints of the target sound. Each segment (i.e. constraint source segment) is represented by a line of a random color which is Y-position representing the average RMS volume accross that entire segment. Notice how the gunshot sounds have much higher average volume. These will consequently have a higher constraint enforcement weight in the synthesis.

Shows the Euclidean distances between the each fragment of the Source motion with that of the Target motion's.