Sonifying UDK Skeletal Mesh Components with UDKOSC and Supercollider

4. Bone Tracking in UDKOSC

Here’s where depending on who you are, things either start to get reeeaaaalllly interesting, or a bit strange. When I started down this path, all I really wanted to do was make some great sounding musical sonifications using idiomatic gaming controls and a custom SkeletalMesh. Now that we’ve hooked up a plausible control schema to our Pawn’s SkeletalMesh, we’re now in position to simply track some properties for its bones (in this case the Local X, Y, and Z coordinates) and then output those values over Open Sound Control for sonification in SuperCollider. Before we get into the SuperCollider nitty-gritty, lets quickly look at how to grab the bones we want.

A little bit of this knowledge was already dropped earlier in this discussion back at the end of section 1, namely:

From some earlier poking around, I had isolated the specific bones at the end of each wing-tip which I wanted to use for sonification, namely 'valkordia_01Lwing_front_4' (left) and 'valkordia_01Lwing_front_4' (right). For that usage, since Mesh is a component of the Pawn itself, it's pretty easy to grab the location of a given bone with it's name, something like:

BoneLocationVector = Mesh.GetBoneLocation('valkordia_01Rwing_front_4');

In this way, given the name of any bone in our current SkeletalMesh, we can track that bone’s World Space location (X,Y,Z in UU) as well as any other properties we’ve (or the UDK has) exposed. To get the full picture on what you can access here, take a look into the UDK SkeletalMeshController.uc class, found in your UDK installation under Development/Src/Engine/Classes/SkeletalMeshController.uc with a quick search for “GetBone”. All kinds of good stuff, including:

native final function quat GetBoneQuaternion( name BoneName, optional int Space ); // 0 == World, 1 == Local (Component)
native final function vector GetBoneLocation( name BoneName, optional int Space ); // 0 == World, 1 == Local (Component)
native final function name GetBoneName(int BoneIndex);
native final function matrix GetBoneMatrix( int BoneIndex );
native final function GetBoneNames(out array BoneNames);
native final function vector GetBoneAxis(name BoneName, EAxis Axis);
native final function bool GetBonesWithinRadius( Vector Origin, FLOAT Radius, INT TraceFlags, out array out_Bones );

Lots more to play with down the road, but for now all we’re going to use is GetBoneLocation().

For now, I’m taking a shortcut and manually tracking 2 bones, rather than setting up a structure where I can dynamically pick and choose which bone locations to send over OSC at any given time. The deeper we go into this the faster implementing something like that will become necessary. But for now, it’s up and running.

Then, like all of our data-sharing between UDK and our Open Sound Control DLL, our bone data is written to a struct, and shared between UDK and the DLL. I won’t go into the details on DLL Bind here, there are a number of great tutorials and resources on UDN and the UDK Forums that discuss implementing the basics of DLLBinds.

After the Break: Sonification with Supercollider…

Tagged with: ,
Posted in UDKOSC

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: