Kinect Player Radar Using C# and WPF

In this post i'm going to explain how to make a simple top down player radar for the Kinect using wpf and c#

The radar provides a top down view of user locations relative to the Kinect sensor.

I needed something like this for part of a larger r&d project i'm working on, but wasn't able to find any examples of how to do it. Turns out it's actually pretty simple.

SkeletonStream

The Kinect's SkeletonStream allows us to track the position of a user infront of the Kinect, and also the positions of up to 20 joints on the user's body.

SkeletonStream provides position tracking for up to six users. Full skeletal tracking is only possible for  two users, however for the purposes of this project i only needed the global position of each user.

The Kinect SDK provides a Skeleton Class (Microsoft.Kinect.Skeleton) which is used to contain the data for a tracked user skeleton.

There are three members of the Skeleton Class that we are interested in.

1. TrackingState : this property tells us the tracking state of the skeleton, either Tracked, PositionOnly, or NotTracked
2. TrackingId : a unique id for each tracked skeleton
3. Position : provides the 3d position of a tracked user

Position data is represented by the SkeletonPoint structure which has 3 distance properties, X: which represents the horizontal distance from the sensor, Y: which represents the vertical distance from the sensor, and Z: which represents depth. The values returned are in meters and the Kinect sensor is at position 0,0,0. For this project i was only concerned with the X and Z values as this is a top down view.

For this simple example i'm going to draw a circle on a wcf canvas panel to represent each tracked player. Each player will be represented by a different colored circle.

I'm using a canvas panel with a width of 600px and a height of 1050px
300 pixels on the canvas represents 1 meter in the real world, so the canvas size maps to a 2m x 3.5m area infront of the kinect.
The values i'm using are oriented so that the kinect sensor is effectively at the top center of the canvas

The Code

I'm going to detail some of the code that i used for this project below.
You can find full example code for this article on github

The xaml markup for the radar is very simple. It's pretty much just a canvas panel.

<Window x:Class="KinectPlayerRadar.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="KinectPlayerRadar" Width="600" Height="1050" Loaded="WindowLoaded" Closing="WindowClosing">

    <Canvas Width="600" Height="1050" Name="canvas" Background="Gray">
    </Canvas>
</Window>

Jumping straight into the codebehind. First up we define some variables to contain a reference to the Kinect sensor, and a list of the skeleton ids that we are currently tracking.
We then initialize the Kinect sensor, enable the SkeletonStream, and subscribe to the sensor's SkeletonFrameReady event. This event fires when a new skeleton frame for the SkeletonStream is available.

private KinectSensor _sensor;
private List<int> _skeletonTrackingIds;

const int PixelsPerMeter = 300;
const string BaseMarkerName = "player";

private void WindowLoaded(object sender, RoutedEventArgs e)
{
    if (!KinectSensor.KinectSensors.Any())
    {
        throw new ApplicationException("no kinect sensor detected");
    }

    _sensor = KinectSensor.KinectSensors[0];
            
    _skeletonTrackingIds = new List<int>();
            
    _sensor.SkeletonStream.Enable();
    _sensor.SkeletonFrameReady += new EventHandler<SkeletonFrameReadyEventArgs>(sensor_SkeletonFrameReady);            
    _sensor.Start();
}

SkeletonFrameReady event handler

In the event handler we copy the Skeleton Frame data into a new array. The array will always contain 6  Skeletons, even if nothing is being tracked, so we need to loop through each of the Skeletons and check the TrackingState property. If the Skeleton is being tracked we call the SetMarkerPosition method, passing in the Skeleton and its index in the array as parameters.

Once we have processed each skeleton, the RemoveUnusedMarkers method is called to remove any markers for players that are no longer being tracked.

void SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
    Skeleton[] skeletons = new Skeleton[0];

    using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
    {
        if (skeletonFrame != null)
        {
            skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength];
            skeletonFrame.CopySkeletonDataTo(skeletons);
        }
    }

    var index = 0;

    foreach (Skeleton skeleton in skeletons)
    {
        if (skeleton.TrackingState != SkeletonTrackingState.NotTracked)
        {
            SetMarkerPosition(skeleton, index);
        }

        index++;
    }

    RemoveUnusedMarkers(skeletons);
}

SetMarkerPosition 

In this method we get the X and Z positions of the tracked skeleton and calculate the top and left values  for our player marker on the canvas.

The X position is negative if the player is to the right of the sensor, and positive if the user is to the left of the sensor. The Y value will always be positive. So for example an X value of -0.8 and a Y value of 1.5 would mean that the user is 0.8 meters to the left of the sensor, and 1.5 meters back from the sensor.

Given these position values we can easily calculate the position of the marker on the canvas.

Since our sensor is assumed to be at the top center of the canvas, the "top" value for our marker will be skeleton.Position.Z * PixelsPerMeter and the "left" value for our marker will be (canvas.Width / 2) + (skeleton.Position.X * PixelsPerMeter)

If the skeleton is already in our list of tracked skeletons then we simply update the position of the player's marker on the canvas.

If the skeleton is not currently being tracked, we add a new marker to the canvas and then set its position.

private void SetMarkerPosition(Skeleton skeleton, int index)
{         
    System.Windows.Shapes.Ellipse marker;
    var canvasCenter = canvas.Width / 2;
    var top = skeleton.Position.Z * PixelsPerMeter;
    var left = (canvas.Width / 2) + (skeleton.Position.X * PixelsPerMeter);

    if (!_skeletonTrackingIds.Contains(skeleton.TrackingId))
    {
        marker = AddToCanvas(skeleton.TrackingId, index);
        _skeletonTrackingIds.Add(skeleton.TrackingId);
    }
    else
    {
        marker = canvas.FindName(BaseMarkerName + skeleton.TrackingId) as System.Windows.Shapes.Ellipse;
    }

    if (marker != null)
    {
        Canvas.SetTop(marker, (skeleton.Position.Z * PixelsPerMeter));
        Canvas.SetLeft(marker, canvasCenter + (skeleton.Position.X * PixelsPerMeter));
    }
}

AddToCanvas

The AddToCanvas method dynamically adds a new ellipse shape to the canvas to represent our player.

private Ellipse AddToCanvas(int skeletonTrackingId, int skeletonIndex)
{
    var ellipse = new System.Windows.Shapes.Ellipse();
    ellipse.Name = BaseMarkerName + skeletonTrackingId;
    ellipse.Fill = GetMarkerColor(skeletonIndex);
    ellipse.Width = 30;
    ellipse.Height = 30;
            
    canvas.Children.Add(ellipse);
    canvas.RegisterName(ellipse.Name, ellipse);
          
    return ellipse;
}

GetMarkerColor

The GetMarkerColor method returns a different color brush for each possible player index.

private Brush GetMarkerColor(int playerIndex)
{
    if (playerIndex == 1) return Brushes.Blue;
    if (playerIndex == 2) return Brushes.Red;
    if (playerIndex == 3) return Brushes.Green;
    if (playerIndex == 4) return Brushes.Orange;
    if (playerIndex == 5) return Brushes.Purple;

    return Brushes.Turquoise;
}

RemoveUnusedMarkers

The RemoveUnusedMarkers method is called after our player positions have been set in order to remove markers for players that are no longer being tracked.

private void RemoveUnusedMarkers(Skeleton[] skeletons)
{
    _skeletonTrackingIds.ForEach(id =>
        {
            if (!skeletons.Select(s => s.TrackingId).Contains(id))
            {
                var marker = canvas.FindName(BaseMarkerName + id) as Ellipse;
                if (marker != null)
                {
                    canvas.Children.Remove(marker);
                }
            }
        });
}

That's pretty much all there is to it. Obviously this is a very basic example, but hopefully someone else out there might find it useful.

If you liked this article you can also find me on twitter.

Get the code from GitHub