Azure Kinect–Loading the RGB Camera with the new SDK 1.2–WPF – Part 2

Hello!!!

Welcome to the part two of this series of posts i will be writing about Azure Kinect. This post is special because yesterday Microsoft launched Azure Kinect Sensor SDK 1.2 update. It includes support for C#, color exposure get & set API fixes, public availability of C++ playback API, and firmware updates for improved USB compatibility.

Before we start programming this part of our tutorial. I will first update the code I did in my previews post in which i used the NuGet package: https://www.nuget.org/packages/K4AdotNet developed by Andrey Bibichev. We started by following some of the basic steps described in this image.

Code using the package K4AdotNet 1 . Open VStudio 2 . File > New WPF App .net Application 3. Select the Project > Properties > Build and make sure option “Prefer 32-bit” is disabled 4 . Manage Nuget Package 5 . Browse > Type and select: “K4AdotNet” and Install it: 6 . In your MainWindow.xaml.cs  file include:  using K4AdotNet.Sensor; 7. Lets declare a device and open it. Device device = null; device = Device.Open(); 8 . Now we will add a textblock in our MainWindow.xaml file. <TextBlock x:Name=”txtResult” HorizontalAlignment=”Left” TextWrapping=”Wrap” Text=”TextBlock” VerticalAlignment=”Top”/> 9 . Assign the property of the opened device to that textblock txtResult.Text = device.SerialNumber; 10 . Close the device. device.Dispose();

Code using the new SDK 1.2 Sensor 1 . Open VStudio 2 . File > New WPF App .net Application 3. Select the Project > Properties > Build and make sure option “Prefer 32-bit” is disabled 4 . Manage Nuget Package 5 . Browse > Type and select: “Microsoft.Azure.Kinect.Sensor” and Install it: 6 . In your MainWindow.xaml.cs  file include: using Microsoft.Azure.Kinect.Sensor; 7. Lets declare a device and open it. Device device = null; device = Device.Open(); 8 . Now we will add a textblock in our MainWindow.xaml file. <TextBlock x:Name=”txtResult” HorizontalAlignment=”Left” TextWrapping=”Wrap” Text=”TextBlock” VerticalAlignment=”Top”/> 9 . Assign the property of the opened device to that textblock txtResult.Text = device.SerialNum; 10 . Close the device. device.Dispose();

Now we need to understand, the Sensor is just a device which sends only the raw data to our system, now it is upon a developer to use that raw data in whatever way he feels like. In this tutorial we will try to get the RGB Camera in color and display it in our UI.

So we will start from zero. We alredy know how to open our device, now we need to configurate the camera in order to use it.

DeviceConfiguration config = new DeviceConfiguration();
             config.ColorFormat = ImageFormat.ColorBGRA32;
             config.ColorResolution = ColorResolution.R1080p;
             config.DepthMode = DepthMode.NFOV_2x2Binned;
             config.SynchronizedImagesOnly = true; / /when frame data comes from the data sometimes not all of the frame can come in… this will help you get only the ones that are correctly
             config.CameraFPS = FPS.FPS30;
             device.StartCameras(config);

So now… lets go to our MainWindow.xaml and we will assign an event when the Window Loads. Our xaml code will be something like this.

<Window x:Class=”WpfDemo.MainWindow”
         xmlns=”http://schemas.microsoft.com/winfx/2006/xaml/presentation”
         xmlns:x=”http://schemas.microsoft.com/winfx/2006/xaml”
         xmlns:d=”http://schemas.microsoft.com/expression/blend/2008″
         xmlns:mc=”http://schemas.openxmlformats.org/markup-compatibility/2006″
         xmlns:local=”clr-namespace:WpfDemo”
         mc:Ignorable=”d”
         Title=”MainWindow” Height=”450″ Width=”800″ Loaded=”Window_Loaded” >

In our Window_Loaded method we will use a global variable we will declare as true, we will name it “running”. This way we will be using the information of the camera until we decide to close it. We then will set a capture to get the image from the event GetCapture() which is a synchronous call and we are using Task.Run to make it asynchronous. We are looping with a “using” because the capture has all the memory we are using with all the images that come from the camera and they are disposable so we wanna make sure we dispose of those once we are done with them. 

private async void Window_Loaded_1(object sender, RoutedEventArgs e)
         {
             while (running)
             {         
                 using (Capture capture = await Task.Run(() => { return this.device.GetCapture(); }) )
                 {
                 }
             }
         }

In order to test what we just created, lets add a textblock to our xaml code and name it “_resultTxt”.

<TextBlock x:Name=”_resultTxt” HorizontalAlignment=”Left”  TextWrapping=”Wrap” Text=”TextBlock” VerticalAlignment=”Top”/>

And now inside our “using” declaration we will assign value to this textblock.

_resultTxt.Text = “capture data ” + capture.Depth.DeviceTimestamp;

If we test our app now we are still not displaying anything, but we can see DeviceTimestamp.

To understand better what each of the items we will be working with involves.. we need to understand this.

Capture
  – Container for multiple related images
  – Reference counted type in c
  – Disposable type in C#

Image
  – Represents a buffer of pixels
  – width, heigh, stride, format, timestamp
  – Reference couted type in C
  – Disposable type in C#

Image Types
  – Depth (16-bit X distance in mm)
  – Infrared (16-bit image of illumination)
  – Color Image

Now in order to draw something in our UI, we will add an Image control type and name it “displayImg”.

<Image Name=”displayImg” HorizontalAlignment=”Left”  VerticalAlignment=”Top” />

Now under the txt assignment we will add the following code. If we get a color capture different than null, we will assign that value to a var which will help us get to the information of the capture more easy.

We then assign to the source of our Image, our BitmapSource.Create value.

if (capture.Color != null)
                     {
                         var color = capture.Color;

                             Display.Source = BitmapSource.Create(color.WidthPixels, color.HeightPixels, 96, 96, PixelFormats.Bgra32, null, color.Memory.ToArray(), color.StrideBytes);

                     }

Lets run our WPF application.