Breakdown of Leap Motion classes


This blog post isn't really for reading, it's more about me trying to understand what everything in the classes do so that it will be easier to write my own classes when it comes to decoupling them. It's mainly a break down, line by line of how the bottom two classes, which deal with the actual frames/data work. 

Leap Service Provider

Image result for unity update vs fixedupdate vs lateupdate
Order of method calls in Unity.

Start()

  • checkShouldEnableHeadMounted()
    • This checks to see if VR is enabled and whether or not head mounted is enabled and logs an error if both are true. This is because by default they enforce that you use head mount when in virtual reality. This is an important thing to note for when I go to implement a table mounted version of the system. Would have to disable this check. 
    • Returns true if we should enable head mount, false if not necessary
  • createController()
    • "Create an instance of a Controller, initialize its policy flags and subscribe to connection event" - Leap Motion
    • Create the instance of the controller and then if its connected we call the initializeFlags
    • initializeFlags()
      • "Initializes the Leap Motion policy flags. The POLICY_OPTIMIZE_HMD flag improves tracking for head-mounted devices."
      • If head mounted then set policy flag for optimizing HMD otherwise clear it. 
      • I wasn't really sure what policies were so after some googling I think they are to do with C++, someone on SO defined it as: Policies are classes (or class templates) to inject behavior into a parent class, typically through inheritance. It sounds like it's kind of like dependency injection in Java i think?
      • The above screenshot is from the Leap motion website and explains a bit about the ones that they use in Leap. The initialiseFlags is only referring to the optimize HMD flag in this case though.
  • Start() ends by creating 4 new frame objects called respectively:
    • _transformedUpdateFrame
    • _transformedFixedFrame
    • _untransformedUpdateFrame
    • _untransformedFizedFrame

Update()

  • Update the fixed offset which is a "smoothed float" class created by Unity which is made to make the movements between states more fluid (i think)
  • DispatchUpdateFrameEvent(transforedfixedframe)
    • If the updateFrame event is not null fire the current frame to unity
  • If using interpolation:
    • GetInterpolatedFrameFromTime
      • Takes a frame to fill, a time and a source time. (untransformedUpdateFrame). Is this where we get the data from the controller and put in frame?
  • Else:
    • Similar to previous method but doesn't take any time values. Just a frame to fill. 
  • We get the timeStamp
    • Not really sure what this is/what it does. 
  • if (_untransformedUpdateFrame != null)
    • transformFrame(_untransform, _transform)
      • Takes a source frame and a destination frame  and if we are using temporalWarping apply a bunch of transforms to the LeapTransform.
      • This is the method where we are currently mirroring the frame by mirroring the LeapTransform
      • At the end we copy the data from the source frame into the destination frame with the transform applied. I assume this is because frames are mutable so we should save a copy of it. 
  • DispatchUpdateFrameEvent
    • Again fire the frame into Unity

FixedUpdate()

Occurs before Update and the method calls are almost exactly the same as Update. 

IHandModel/Rigged Hands

"Methods that drive transform based on data"

UpdateHand() 

  • First iterate through fingers and update them
  • if(palm!= null)
    • get the center and rotation using the skeletahand implementation of the abstract ihandmodel and if there is a rigid body use the movePosition/moveRotation methods otherwise directly set the transform of the palm. 
    • Potential for mirroring/translating could occur in this method and would be interesting to try out. 
  • if(forearm!=null)
    • Get the collider
    • Initialise the capsule
    • Update radius and height using methods from skeletal hand
    • Move like palm depending on whether or not there is a rigid body attached.

This method plus Skeletal Hands etc seems to be what I will need for doing this myself. 

\

Comments

Post a Comment

Popular posts from this blog

Texturing Hands

Image
Texturing the 3D Hands The purpose of texturing the box was so that eventually I could move on to texturing realistic hands.  The first issue I had was that Leap Motion export their prefabs in the ASCII format of .fbx (i'm guessing because it's human readable?) and Blender doesn't allow importing of ASCII file so I had to convert to a Binary file using an external application.  After importing it into Blender I realized there was something called Armature which I hadn't heard of before and as I started reading I figured it would help to look a little bit into how animation works so I could understand the Leap Motion documentation a little bit better. Leap Motion uses something called Skeletal Animation for their hand prefabs. When using this method we split the character object (in this case hands) into two parts, the skin/mesh and the interconnected bones/rig. The bones are connected in a hierarchy so that when you move for example the tip of the fin...
Read more

Day 2

Image
Sam's Demo To start today, as per Holger's request I captured an admittedly very amateur screen cast of Sam's Demo. I wasn't quite sure how to best show the different functions of the program. After a few failures this was the best I could come up with: Will have to add video capturing skills to the list of things that will (hopefully) improve throughout the duration of this student ship. I'm not sure how to get four hands up and going currently. Creating a trial leap hands project Today I decided to start by attempting to create a basic program that could render two hands as an attempt to get my head around the software a little bit.  I first downloaded a bunch of stuff: Leap Orion (The SDK for leap) Leap Core Assets (Package with basic functionality) Leap Graphic Renderer (Aims to create a system that allows you to to treat a collection of objects as a group) I followed a set up tutorial found here: https://developer.leapmotion.com/do...
Read more

5th of Dec Update

Image
Haven't updated in a while because I haven't really made any major achievements, everything has been very gradual in growing my understanding with the Leap API as well as how Sam's demo actually works. I've been focusing the last few days on how Leap works, from how to access any leap data within a script to detecting hands using the HandModels[] from the current Frame. I had a slight play around with the concept of four hands simply by duplicating the LeapController and HandModels twice over and position them in a way as to replicate four hands. Movement is that where both the left hand controls both hands on the left side and vice versa with the right. This was a good way to visualise the task and see the gaps in why this technique wouldn't work. My current challenge is to replicate the mirroring from Sam's demo. Now that I understand the code better I can see that the way that Sam has implemented it is no longer possible with the new Orion API. He mir...
Read more