"Mimicking" Gestures

Mimicking

Currently the way our original prototype works, as well as the general theory for mirror therapy is to have a vertical mirror in between the hands so that the virtual hand is simply a mirror of the "real" hand. Holger has suggested a different type of "mirroring" which potentially may work better and could be of use to Chris for his project as a comparative technique. It would instead be a combination of mirroring & translating which may be better explained by this video:

Potential Variation:

One thing about this technique is the rotation of the hands means that if you have your control hand facing downwards the virtual hand will be facing upwards & vice versa.

I suspect that this may not be a very natural movement because if we decide to implement physics into the hands it would be difficult to interact with in our case a box for example.





Another potential way to implement the movements would be to change the hand rotations from the ones shown in the previous video, to the same as in the previous mirrored system like in the video above.

The final product would look something like the video above. It is more of a combination of the methods, where for moving the hands horizontally we would translate the second hand according to the original hand, but for rotations they would mirror like in a normal system.

I imagine that finger movements would be mirrored as well as it would be more natural to move an index finger and the virtual index finger mimicking the movement. However with this being said to keep consistency with the hands, moving a finger to the left should also result in a "mirrored" finger move to the left as well. 

To do this we will need to decouple the data from the visualization so that we can take the frames read by the controller and map the gestures onto the virtual hand ourselves (basically the MVC). This will give more flexibility to do different gestures than the ones provided to us by leap. In my previous blog post I looked more into how I could go about doing this, by drawing a rough outline around what classes have to do with reading in data and what are more to do with the representations.

When reading through the code it's not immediately obvious where and how this would be done, it seems that it's relatively coupled. I think in general it will just take some playing around with till I get anywhere with it.

Another thing to keep in mind is the possibility that Tobi raised of doing a translation based system where we have four hands but the left hand controls the hand on the left and the right hand, the hand on the right. Could be another good thing to add for comparision measures.