Leap points to Unity space


Post Holger & Tobi meeting:

  • Work on leap to unity conversions
  • Visualize the interaction box/sensor frustum/ even potentially rift viewing space??
  • Move on to the decoupling hands from data
    • Step 1 have hands just in front of you
    • Step 2 mimic movement for chris

Leap Point to Unity 

So what I now know is that when leap returns a position vector, the point is relative to the origin of the leap controller rather than an absolute point. To convert this I followed the following steps:
  1. Divide the vector by 1000 to convert from mm to m
  2. Negate the Z value to change from rh to lf coordinate system
  3. Find the unity position of the leap controller's origin
  4. Add the origin to the vector  
Drawing Bounds

This proved to be more of a challenge for me then I expected. I was struggling to work out how to draw these straight lines at run time using the GL library. GL is a basic graphics tool which can be used to draw lines and triangles (potentially more I'm not sure). 

I started with the interaction box as it had the most information readily available. The IB class had fields for the center vector aswell as the height, width and depth of the box. The way GL works is it draws lines between pairs of vectors so I needed to work out the position vectors of each of the 8 corners so that I could draw each edge of the box. This was easy to work out because I had all the information necessary given to me. For example the Top back left point would equal (origin.x-.5 width, origin.y + .5height, origin.z-.5depth). After this I was still having issues with any of the graphics actually showing up so I changed two things.
  1. The first was that I nicked the bounds shader from Sam's demo. I did a little tiny bit of reading into what a shader is but I still don't really understand why it was necessary to use a Shader in his demo and why I need a shader at all to draw a line but for now it seems like a good bandaid fix. 
  2. The second and main reason it wasn't rendering was because I'd put my code in the OnPostRender() method because I'd read that if the graphics were done before rendering the camera might wipe it out so it's best to do it after the rendering has done. What I hadn't realised was that this method would only be called if the script was attached to an enabled camera object. Once I realized this I moved the script and managed to have my Interaction box working. (Although right now it's not fully centered over the leap controller model, I think this must be a problem with the model placing rather than the interaction box coordinates)
After I had this working I moved on to the rift sensor frustum. The frustum is a pyramid shape which means that even though I'd already worked out how to draw the lines this ended up being more difficult then the Interaction box as I had less information and triangles are by default more difficult then squares. The issue I had was that I had the following pieces of data:
  1. Vertical angle in degrees 
  2. Horizontal angle in degrees
  3. Top/Origin point
  4. Depth
And I needed to find the bottom four points of the pyramid. I knew this was going to require some maths and unfortunately my maths was/is very rusty. What I decided to do was take a right angled horizontal slice of the pyramid (might be easier to refer to below picture) because that would mean I would have two angles, the top angle (given by 1/2 of horizontal angle) and a 90 degree angle aswell as the height of the triangle. This meant I could calculate the width of the base of the triangle giving me half of the width of the frustum. Using this I could also find out the height of the frustum and then apply the same maths as what I used for the IB to find the four points of the base of the pyramid.
Diagram of frustum shape

Frustum with information we have added
Slice with real values added

I still have to add the near Z plane to the frustum bounds and the leap motion frustum but I think these both will be relatively trivial now that I have the basics down.











Comments

Post a Comment

Popular posts from this blog

Texturing Hands

Image
Texturing the 3D Hands The purpose of texturing the box was so that eventually I could move on to texturing realistic hands.  The first issue I had was that Leap Motion export their prefabs in the ASCII format of .fbx (i'm guessing because it's human readable?) and Blender doesn't allow importing of ASCII file so I had to convert to a Binary file using an external application.  After importing it into Blender I realized there was something called Armature which I hadn't heard of before and as I started reading I figured it would help to look a little bit into how animation works so I could understand the Leap Motion documentation a little bit better. Leap Motion uses something called Skeletal Animation for their hand prefabs. When using this method we split the character object (in this case hands) into two parts, the skin/mesh and the interconnected bones/rig. The bones are connected in a hierarchy so that when you move for example the tip of the fin...
Read more

Day 2

Image
Sam's Demo To start today, as per Holger's request I captured an admittedly very amateur screen cast of Sam's Demo. I wasn't quite sure how to best show the different functions of the program. After a few failures this was the best I could come up with: Will have to add video capturing skills to the list of things that will (hopefully) improve throughout the duration of this student ship. I'm not sure how to get four hands up and going currently. Creating a trial leap hands project Today I decided to start by attempting to create a basic program that could render two hands as an attempt to get my head around the software a little bit.  I first downloaded a bunch of stuff: Leap Orion (The SDK for leap) Leap Core Assets (Package with basic functionality) Leap Graphic Renderer (Aims to create a system that allows you to to treat a collection of objects as a group) I followed a set up tutorial found here: https://developer.leapmotion.com/do...
Read more

5th of Dec Update

Image
Haven't updated in a while because I haven't really made any major achievements, everything has been very gradual in growing my understanding with the Leap API as well as how Sam's demo actually works. I've been focusing the last few days on how Leap works, from how to access any leap data within a script to detecting hands using the HandModels[] from the current Frame. I had a slight play around with the concept of four hands simply by duplicating the LeapController and HandModels twice over and position them in a way as to replicate four hands. Movement is that where both the left hand controls both hands on the left side and vice versa with the right. This was a good way to visualise the task and see the gaps in why this technique wouldn't work. My current challenge is to replicate the mirroring from Sam's demo. Now that I understand the code better I can see that the way that Sam has implemented it is no longer possible with the new Orion API. He mir...
Read more