Psuedo Mirroring


Current State:
So currently I have the new system functioning the same way the old was with the ability to have four hands using mirroring (by mirroring the frame) as well as a left hand mirror, right hand mirror mode. Also added in a four handed mode where the left controls the left and the right hand, the right etc. These are at the moment controlled by key board pushes (so that eventually the system can be run from by an "administrator" or something, rather than in VR)

The rigged hands still are looking the same as before with a few of the polygons flipped or something (the problem we discussed at the last meeting) however because the system is now desk mounted the "drifting" issue is eliminated.









Psuedo Mirroring:
As far as psuedo mirroring goes I haven't gotten much further than last week. I started by seeing how easy it would be to gain information about the hands (I know that it would be in the code somewhere buried deep) and it turned out it was a relatively easy task.


From a frame I could work my way deep enough to find out:
  • Left or Right Hand
  • Palm Position
  • Wrist Position
  • Normal and direction of palm
  • Finger:
    • Finger type
    • Finger tip position
    • List of bones within the finger (and further positions of these)
  • These are all provided in the Leap coordinate system which means that it would be trivial to get the Unity position of all of these (which we could in turn use to power a new hand)



Animation/Hand Model:
  • The bigger challenge has been implementing the psuedo mirroring because as we discussed I can't work out how to create a hand model from scratch that doesn't use the Unity data to in turn put our own coordinates into the hand to move it. 
  • From last week we discussed the potential of using a different hand model to do this where it would be an entirely separate model from unity that takes x amount of parameters to move/rotate etc. However this is proving more difficult then expected so far as I don't know too much about animation within Unity.  I'm not sure if I'm going down the right track with looking at models but the ones that I've found are 3d models created on some kind of modelling software which are rigged and can be animated in Unity (I assume). It's difficult to compare the types as I don't really know how animation works (something I will look into in the morning) but for now this is what I've found:
    • https://3dhaupt.com/3d-model-anatomy-rigged-hands-low-poly-vr-ar-game-ready-blender/ 
      • A free 3d model of rigged hands, I tried to add this to my system but didn't know how to move the parts yet
      • Also has a youtube video which makes it look like it's quite flexible and potentially a viable option:
        • https://www.youtube.com/watch?v=PW5dJVQe83U
    • https://www.turbosquid.com/3d-models/hands-rigged-3ds-free/932376
      • This one specifies that each finger can only be controlled by one bone which may be what we need if we only have a few points to work with
  • Am currently a little stuck with how I would be able to translate coordinates into animations and I think it's a bit of a road block when looking at models. 

Comments

Post a Comment

Popular posts from this blog

Texturing Hands

Image
Texturing the 3D Hands The purpose of texturing the box was so that eventually I could move on to texturing realistic hands.  The first issue I had was that Leap Motion export their prefabs in the ASCII format of .fbx (i'm guessing because it's human readable?) and Blender doesn't allow importing of ASCII file so I had to convert to a Binary file using an external application.  After importing it into Blender I realized there was something called Armature which I hadn't heard of before and as I started reading I figured it would help to look a little bit into how animation works so I could understand the Leap Motion documentation a little bit better. Leap Motion uses something called Skeletal Animation for their hand prefabs. When using this method we split the character object (in this case hands) into two parts, the skin/mesh and the interconnected bones/rig. The bones are connected in a hierarchy so that when you move for example the tip of the fin...
Read more

Day 2

Image
Sam's Demo To start today, as per Holger's request I captured an admittedly very amateur screen cast of Sam's Demo. I wasn't quite sure how to best show the different functions of the program. After a few failures this was the best I could come up with: Will have to add video capturing skills to the list of things that will (hopefully) improve throughout the duration of this student ship. I'm not sure how to get four hands up and going currently. Creating a trial leap hands project Today I decided to start by attempting to create a basic program that could render two hands as an attempt to get my head around the software a little bit.  I first downloaded a bunch of stuff: Leap Orion (The SDK for leap) Leap Core Assets (Package with basic functionality) Leap Graphic Renderer (Aims to create a system that allows you to to treat a collection of objects as a group) I followed a set up tutorial found here: https://developer.leapmotion.com/do...
Read more

5th of Dec Update

Image
Haven't updated in a while because I haven't really made any major achievements, everything has been very gradual in growing my understanding with the Leap API as well as how Sam's demo actually works. I've been focusing the last few days on how Leap works, from how to access any leap data within a script to detecting hands using the HandModels[] from the current Frame. I had a slight play around with the concept of four hands simply by duplicating the LeapController and HandModels twice over and position them in a way as to replicate four hands. Movement is that where both the left hand controls both hands on the left side and vice versa with the right. This was a good way to visualise the task and see the gaps in why this technique wouldn't work. My current challenge is to replicate the mirroring from Sam's demo. Now that I understand the code better I can see that the way that Sam has implemented it is no longer possible with the new Orion API. He mir...
Read more