Replicated Contextualised Motion Warping

 

=============================== BEGIN Pasted From Thesis ===============================

In the following example, we demonstrate the use of contextualisation in the case of displacement mapping. First, a query defining a walking sequence with a rising left arm motion is submitted to the matching engine. A series of matches are returned. The original query sequence is edited by adding a displacement to the right arm to synchronously match the motion of the left arm. The first row of Figure 131 illustrates the alteration. Both arms are thus made to move upwards in unison. The goal then is to replicate the edit onto all matches. If we apply the original edit onto each match, the synchronicity between the left and right arm will be lost. This is because the matches will not necessarily be perfectly synchronized with the query, especially if we use DTW as in this case. Therefore, the optimal timewarp is calculated between the query’s left arm movement and that of each match. The original edit’s displacement map is then resampled by a match’s timewarp before being applied. The new edit accounts for the time difference between the query’s and match’s left arm motions. The effect is that the replicated edit conserves the synchronicity  between both arms motion across all matches, as in the edited query. The bottom row of Figure 131 show the differences between the original edit and its contextualised counterpart. The plots for the corresponding arm motions are given in Figure 132.

 

A. Query

 

B. Edited Query

C. Match

D. Edited Match

E. Edited Match in Context

        

Figure 131: Contextualised Replicated Edit. (A) A frame of the query motion. (B) The same frame of the edited query motion where the right arm motion is made to match that of the left arm. (C) The best match to the query for the same frame. (D) If we apply the original displacement map, the synchronicity between the arms is lost. (D) By resampling the original displacement map by the optimal timewarp between the query and the best match, we preserve the unison upwards motion of both arms.

 

 

Figure 132: Plot showing the left and right arm movements for Figure 131. (Top row) Original query motion with edited right arm motion in blue. The applied displacement is in green. Notice how the right arm movements match perfectly. (Second row) Best match onto which the unaltered edit is applied. Notice the lack of synchronicity between the motions. (Third row) Same match onto which the resampled displacement is applied: now the arm motions are better synchronized. (Last row) The top reddish curves show the difference between the query and the best match’s left arm motions. The bottom curves in greenish show difference between the original displacement map and the resampled one, which corresponds to the difference between the upper red curves.

=============================== END Pasted From Thesis ===============================

 

Matching parameters: LCSS, rotational parameterization, full-body area, no global position, no global rotation.

 

Replicated Filtering Example 1

 

Video Description

(in order of appearance)


Full-size video

  1. The query.
  2. The query (again).
  3. The query displaced by animator.
  4. Pause
    The best match.
  5. The best match displaced without contextualisation.
  6. The best match displaced with contextualisation.
  7. Pause
  8. The 2nd best match.
  9. The 2nd best match without contextualisation.
  10. The 2nd best match with contextualisation.
  11. Pause
  12. etc...