Kinect Calibration

This program adjusts Kinect to better see the user, this way the user doesn’t have to move back or forward (unless Kinect’s angles aren’t enough to see the user). I think this is a useful thing to have when you start your Kinect application.

The code is very simple to follow. Use Kinect’s elevation angle to change the Kinect’s “view” and track the user in sight. For each angle count how many joints Kinect is tracking and save that angle as the best if we reach a new maximum number of tracked joints.

void KinectSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
	// Get skeleton information
	using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
	{
		if (skeletonFrame != null && m_skeletonData != null)
		{
			skeletonFrame.CopySkeletonDataTo(m_skeletonData);
		}
	}

	m_curTime = m_watch.ElapsedMilliseconds;

	if (m_calibrationState == CalibrateState.GoingDown)
	{
		// If the sensor reach its lowest angle, let's do a full scan from min angle to max angle.
		if (m_curTime >= WaitTime)
		{
			m_watch.Reset();
			m_watch.Start();

			m_bestAngle = m_kinect.MinElevationAngle;
			m_maxNumTracked = 0;

			m_calibrationState = CalibrateState.GoingUp;
			m_kinect.ElevationAngle = m_angles[m_curAngleIndex++];
		}
	}
	else if (m_calibrationState == CalibrateState.GoingUp)
	{
		if (m_curTime >= WaitTimeGoingUp)
		{
			m_watch.Reset();
			m_watch.Start();

			// If we scanned all the angles, lets adjust kinect to the best angle.
			if (m_curAngleIndex > m_angles.Length - 1)
			{
				m_calibrationState = CalibrateState.GoingBest;
				m_kinect.ElevationAngle = m_bestAngle;
				return;
			}

			m_kinect.ElevationAngle = m_angles[m_curAngleIndex++];
		}

		// For each skeleton, count the number of tracked joints and save the best
		// angle when the number of tracked joints is greater than previous values.
		foreach (Skeleton skeleton in m_skeletonData)
		{
			if (skeleton == null)
				continue;

			if (skeleton.TrackingState == SkeletonTrackingState.Tracked)
			{
				//TODO: Improve algorithm by using the number of inferred joints.
				int numTracked = 0;
				int numInferred = 0;
				foreach (Joint joint in skeleton.Joints)
				{
					if (joint.TrackingState == JointTrackingState.Tracked)
						numTracked++;
					else if (joint.TrackingState == JointTrackingState.Inferred)
						numInferred++;
				}

				if (numTracked >= m_maxNumTracked)
				{
					m_maxNumTracked = numTracked;
					m_bestAngle = m_kinect.ElevationAngle;
				}
			}
		}
	}
	else if (m_calibrationState == CalibrateState.GoingBest)
	{
		// Just wait until kinect adjusts itself to match the best angle.
		if (m_curTime >= WaitTime)
		{
			m_watch.Reset();

			m_calibrationState = CalibrateState.Idle;
			m_kinect.SkeletonFrameReady -= KinectSkeletonFrameReady;

			// Reset Kinect state.
			DisableStreams();
			EnableStreams();

			// Signal that we finished the calibration.
			if (OnCalibrationComplete != null)
				OnCalibrationComplete();
		}
	}
}

You may notice two things about the code:

  • First is that I’m using time to control between Kinect movements. The reason for this is that if we try to compare Kinect with the angle we set, we might not get the same value for different reasons (Kinect sensor is not 100% accurate or it couldn’t physically rotate the sensor to that exact angle).
  • And second is the fact that I’m only scanning the skeleton for some angles. I don’t know if this is a Kinect’s limitation or if I did something wrong in the code but I couldn’t track any joints while Kinect was moving. So what I did was move the sensor between some angles and wait there a bit to count the bones for each one of those angles.

Known limitations:

  • it only supports 1 user at this moment;
  • the scanning process is not ideal.

Download:

Kinect Calibration (13Kb)

Interactive portfolio using Kinect

I like to explore different technologies that relate with computers and that’s why I own a Lego Mindstorms NXT . More recently I bought a Kinect to transform some ideas into real applications.

I’m currently looking for job in software development in the Toronto (Canada) area and it occurred to me that I could present my portfolio in a fun and entertaining way. So I decided to create my first Kinect application to be this interactive portfolio. Although I have some years of experience in business applications, portals, intranets, etc., I decided to compile only my experience in game development and interactive 3d projects.

If you don’t own a Kinect you can watch the video below to see how it works. I encourage you to skip the video in case you want to try the application or you’ll ruin the experience.

Interactive portfolio

The code uses the following technologies: C#, .NET, XNA and WPF. I also used two open source libraries to help with the gestures and the XNA integration with WPF.

DOWNLOAD

Software requirements:

Kinect application:

Note: Please make sure you install the software requirements or you won’t be able to install the application.

THANKS TO

  • My parents for helping me out recording the video.
  • Anabela Faria for the video editing.
  • Paulo Silva for the animated silhoutte.

Windows Phone 7 – Tips for programming games

Puzzle Jumble started as an experiment to test and learn a little bit about XNA and Windows Phone 7. I decided to write some tips to help those who like me started experimenting with XNA and WP7.

Stencil

The stencil was used to cut the hole for doors so that pieces would appear to be sliding over the doors but under the board. The stencil was active in every frame which lowered the frame rate quite a lot (18-20 FPS). The solution was to activate the stencil only when a piece is going to leave through a door, so be careful when using the stencil buffer on the WP7.

DXT

GPUs like the DXT format and the WP7 is no exception, but unlike desktop’s GPUs the WP7 processor only loads DXTs that are power of two. My regret is not using DXT from the beginning when I started working on Puzzle Jumble. When I changed all my textures to DXT I had to change a lot of code, all the draw calls where I was using a position to draw a sprite had to be changed to a rectangle because the textures had a different resolution (power of two).
But the change was very useful:
– I gain some FPS;
– the memory footprint was reduced from 95MB (total peak) to about 45MB;
– the XAP file was reduced from 25MB to 9MB;
– thus, as a consequence reduced the loading times.

Floats (culture)

When parsing floats it’s easy to add bugs in your application if you ignore the culture format. Make sure to use the correct format provider or at least use the culture independent System.Globalization.NumberFormatInfo.InvariantInfo.

Float precision

Another problem that I found when using floats was that this comparison: if(a < float.Epsilon) (when a = 0) would work everywhere except on the WP7 device itself where it would always return false.

HTML page requests

This is not a WP7 specific problem but I thought it was worth mentioning it. Since Microsoft didn’t provide us with an API for leader boards I implemented an online score system using our servers to keep track of the users’ scores and best times.
The system was working fine to post data online but there was a problem when retrieving it. The first time the data was retrieved everything was fine but the subsequent retrievals would always be the same as if the data was cached somewhere.
After searching for a while I found out that the problem was that the HTTP request was being cached since the URL didn’t change, it would cache the result for that URL. The solution was to add a random attribute to the URL so that the the request couldn’t be cached. Not pretty but it was the only way I found to make it work.

And those were the problems that I found during the Puzzle Jumble’s development phase. I hope this useful for you!

Folder Creator

A few days ago a friend asked me if I had or knew a program that would create folders given a two numbers. Example: if we put 01 – 04 it would create folders from 01 to 04 and so on. Well, I didn’t have nor knew any program that did what was asked so I just thought: this is pretty simple, why not do it myself? And after a hour or so I had the first version. The program asks where you want to create your folders, the base folder name and the number range.
Folder Creator

Instructions:

  • Insert/Choose the base folder. You may enter a folder manually or you can double click the field to open a folder browser dialog an choose a folder. The base folder indicates where the folders will be created. Example: c:\test\
  • Insert a base name (not required). The base name will serve as the name which will be appended by the number of each folder. Example: inserting folder in the base directory textbox will create folders in the path: c:\test\folderXXX where XXX is a number.
  • Insert a start number. Start number is the starting number of the range (start-end) that will be appended to the base name (if there is one). Continuing with our example: 003 will create folder in the path: c:\test\folder003 and so one until we reach the end number.
  • Insert an end number. End number is the number that ends the range (start-end) which will be end appended to the base name (if there is one). Again, in our example: 010 will create folders in the path: c:\test\folder003 (which was our start number), c:\test\folder004 … an so on … ending with c:\test\folder010.
  • Escape key closes the application.

Note: The start number must be less than or equal to the end number.

The program hasn’t been fully tested so it’s most likely that you’ll find bugs in the program. If you find some please report them to me. With time I may add more things if needed, so feel free to suggest new features.

You’re free to use this code as long as you don’t blame me for any changes/bugs, as well as, don’t demand anything from me. Download folder creator here and/or download source code here.

Let me know if you have any bugs/questions/comments/suggestions.

Before I go, I want to thank Victor Romão for the idea for this program.

Diagramming for dummies

Hi, in the past months I’ve been working with the .Net framework either in C# and in C++/CLI. I’ve been more focused in the GUI part of the framework playing with controls and developing some simple ones. Now I needed to make a more challenging control that would draw diagrams, so I went looking for references and I found this great article http://www.codeproject.com/cs/miscctrl/NetronLight.asp by NetronProject. I liked the design of the code and decided to use it and make some modifications to my needs. Below you can find the modifications I did to the code.

Changelog (2006-04-02):

  • Added scroll bars (maybe a bit more of tuning is needed.)
  • Added type for connectors (none, input, output. An input connector can’t connect to another input connector and the same goes for the output connector.)
  • Added a new type of shape (the actionbox that has some inputs, in green, and outputs, in blue.)
  • You can indicate if a shape allows internal connections, i.e., a connection from two connectors within the same shape.
  • Delete key deletes the selected component.
  • When deleting a component, it will delete all connections associated with it.
  • The controls gains focus automatically as the mouse passes over it. This was added so we can use the mouse whell (for scrolling the scrollbars) and keyboard keys, like Delete, more easily.
  • Mouse cursor changes to a resizer when hovering a shape and to a hand when hovering a connector.
  • Fixed some problems with the drawing/refreshing of shapes, connections and connectors.

These modifications haven’t been fully tested yet so it’s most likely that you’ll find bugs in the control. If you find some please report them to me. With time I may add more things as I need, so if you interested in this control, just check this page from time to time.
You’re free to use this code as long as you don’t blame me for any changes/bugs that were introduced and don’t demand anything from me. Just download it here if you accept these conditions.

If you have any bugs/questions/comments/suggestions just send me an e-mail.
Before I go, I want to thank NetronProject team for this great control and tutorial.

References:

http://www.codeproject.com/cs/miscctrl/NetronLight.asp – Diagramming for dummies – The Code Project – C# Controls

Four-Way Splitter Control

Hi, for some time now I wanted a control that would allow me to have four panels separated with splitters which could be resized horizontally, vertically or all at the same time and that didn’t redraw the panels when resizing. I searched for a solution but didn’t find one that fitted my needs either because it would redraw the panels or it wouldn’t draw a shadow splitter like the default behaviour of the windows’ splitters, thus, I decided to make one.

THE IMPLEMENTATION

Since the normal splitter control from the .Net framework always draws the shadow splitter when you try to move it, I had to make a splitter that wouldn’t do this since I didn’t want it to happen in certain occasions. So this SimpleSplitter derives from the System.Windows.Forms.Splitter and overrides some methods that would draw the shadow splitter: OnMouseDown, OnMouseUp, OnSplitterMoved and OnSplitterMoving but I wanted to still be able to catch these events so what I did was to call the base event but changing the number of clicks because in the base class (the Splitter) will only draw the shadow splitter if the number of clicks in the splitter is one (Thanks to Hans Passant):

Simple Splitter code:

protected override void OnMouseDown(MouseEventArgs e)
{
    MouseEventArgs ebase = new MouseEventArgs(e.Button, 2, e.X, e.Y, e.Delta);
    base.OnMouseDown(ebase);
}
protected override void OnMouseUp(MouseEventArgs e)
{
    MouseEventArgs ebase = new MouseEventArgs(e.Button, 2, e.X, e.Y, e.Delta);
    base.OnMouseUp(ebase);
}
protected override void OnSplitterMoved(SplitterEventArgs e)
{
}
protected override void OnSplitterMoving(SplitterEventArgs e)
{
}

Other possible solution was was to create new events to catch these: MouseDown2 and MouseUp2:

public delegate void MouseEventHandler(object sender, MouseEventArgs e);
public event MouseEventHandler MouseDown2;
public event MouseEventHandler MouseUp2;
protected override void OnMouseDown(MouseEventArgs e)
{
    if (MouseDown2 != null)
        MouseDown2(this, e);
}
protected override void OnMouseUp(MouseEventArgs e)
{
    if (MouseUp2 != null)
        MouseUp2(this, e);
}

Next, I’ll describe the algorithm that defines this control works.

  • Event handlers for MouseDown/MouseUp/MouseMove/MouseEnter/MouseLeave events for every splitter.
  • The MouseDown handlers start by checking the mouse button (only the left button is accepted) and check if the cursor is in the middle (function MouseInCenter).
private bool MouseInCenter(object sender)
{
  Point p = this.PointToClient(MousePosition);
  // are we in the intersection?
  if ((Math.Abs(splitterHorLeft.SplitPosition - p.Y) < m_centerDelta &&
    Math.Abs(splitterVertical.SplitPosition - p.X) < m_centerDelta))
  {
    // yup, let's change the cursor to SizeAll.
    Cursor = Cursors.SizeAll;
    return true;
  }
  else
  {
    // Let's see to which cursor we have to change depending on the splitter we're on.
    Splitter splitter = sender as Splitter;
    if (splitter != null)
    {
      if (splitter.Dock == DockStyle.Left || splitter.Dock == DockStyle.Right)
      {
        Cursor = Cursors.VSplit;
      }
      else if (splitter.Dock == DockStyle.Top || splitter.Dock == DockStyle.Bottom)
      {
        Cursor = Cursors.HSplit;
      }
    }
    return false;
  }
}

This function is easy to understand (as any other function in this code), converts the mouse coordinates to the control’s local coordinates and checks if we’re not apart from the center more than a delta distance. If this is not the case, checks if we’re on top of any splitter and if so, change the cursor to the right shape.

  • Save the contents of the panels so we can restore them when we move the splitter(s) (function SaveGraphicContents).
private void SaveGraphicContents()
{
  // save the graphics contents before we move the splitter
  for (int i = 0; i < m_numPanels; i++)
  {
    // save top panels
    using (Graphics graphics = topPanels[i].CreateGraphics())
    {
      topImgs[i] = new Bitmap(topPanels[i].ClientSize.Width, topPanels[i].ClientSize.Height, graphics);
      Graphics g = Graphics.FromImage(topImgs[i]);
      IntPtr hdc = graphics.GetHdc();
      IntPtr memdc = g.GetHdc();
      BitBlt(memdc, 0, 0, topPanels[i].ClientSize.Width, topPanels[i].ClientSize.Height, hdc, 0, 0,
        RasterOperations.SRCCOPY);
      graphics.ReleaseHdc(hdc);
      g.ReleaseHdc(memdc);
    }
    // save bottom panels
    using (Graphics graphics = bottomPanels[i].CreateGraphics())
    {
      bottomImgs[i] = new Bitmap(bottomPanels[i].ClientSize.Width, bottomPanels[i].ClientSize.Height,
        graphics);
      Graphics g = Graphics.FromImage(bottomImgs[i]);
      IntPtr hdc = graphics.GetHdc();
      IntPtr memdc = g.GetHdc();
      BitBlt(memdc, 0, 0, bottomPanels[i].ClientSize.Width, bottomPanels[i].ClientSize.Height, hdc, 0, 0,
        RasterOperations.SRCCOPY);
      graphics.ReleaseHdc(hdc);
      g.ReleaseHdc(memdc);
    }
  }
}

Here a bitmap is created for each panel and the panel’s surface is copied to the bitmap through the function BitBlt. Notice that this function is not available in .Net so we have to access it through the unmanaged dll gdi32.dll using DllImport.

  • Still in the MouseDown handlers, we hide the child controls of the panels. This is done because if we didn’t hide the controls, the shadow splitters would get drawn under the controls and we wouldn’t see the shadow splitters (Thanks to David Júlio):
private void ShowHidePanelContents(bool show)
{
  for (int i = 0; i < MAX_NUM_PANELS; i++)
  {
    foreach(Control c in m_topPanels[i].Controls)
    {
      c.Visible = show;
    }
    foreach (Control c in m_bottomPanels[i].Controls)
    {
      c.Visible = show;
    }
  }
}

This function goes through every panel in the splitter control and hide all the controls inside each panel. It’s also used to show the controls once we’re done with the drawing.

  • The MouseMove handler is the one that draws the splitters in the right place when we’re moving them. First the old panel’s images are drawn and then auxiliary functions draw a shadow splitter (it’s just a rectangle with a hatchbrush at 50%) in the right place.
  • Finally we have the MouseUp handler that finishes the movement by setting the splitters and their new position, shows the child controls again and refreshes/redraws all panels and, consequently, their child controls.
  • The MouseEnter and MouseLeave handlers only change the shape of the cursor if necessary.

THE DESIGNER

Since we want to be able to add controls to each panel, we have to make those panels available for the designer. First we create a new control designer and then we open the access to the panels by calling the EnableDesignMode function on each panel.

public class FourWaySplitterDesigner : ControlDesigner
{
  public override void Initialize(IComponent comp)
  {
    base.Initialize(comp);
    FourWaySplitter usercontrol = (FourWaySplitter)comp;
    EnableDesignMode(usercontrol.TopLeftPanel, "TopLeftPanel");
    EnableDesignMode(usercontrol.TopRightPanel, "TopRightPanel");
    EnableDesignMode(usercontrol.BottomLeftPanel, "BottomLeftPanel");
    EnableDesignMode(usercontrol.BottomRightPanel, "BottomRightPanel");
  }
}

And that’s it! Pretty simple uh?! :)
Keep in mind that with time, I’ll add more functionalities to this control, most of them will be improve its behaviour in the VS designer so, keep checking this page for updates.

You’re free to use this code as long as you don’t blame me for any changes/bugs that were introduced and don’t demand anything from me. Just download it here if you accept these conditions.

Any bugs/questions/comments/suggestions are always welcome. Enjoy! ;)

Changelog:

v0.1.2

  • Changed the way how we handle the OnMouseDown and OnMouseUp events. (Thanks to Hans Passant.)
  • Now shadow splitters are drawn even if the panels in the control contain any controls inside them. (Thanks to David Júlio.)
  • Fixed bug in the designer that didn’t add other controls to the splitter panels.

References:

http://msdn.microsoft.com – MSDN Home Page

Managed DirectX Tutorial 3 – Cameras (Part 1)

VIEWPORT’S CAMERAS

In this tutorial I’ll explain how you can setup a scene with different cameras for each viewport. This will be a very short tutorial but in the following ones we’ll improve our camera system.
So, how does these cameras work in editors like 3D Studio Max? Just like in the movies, you have a scene (your 3D scene, in this case I used a cube) surrounded by cameras and each one of these cameras is filming your scene and presenting the result in different places (viewports).

A SIMPLE MESH

Before we setup the cameras, we need something to show so I decided to create a simple mesh, a cube (I used the sample that comes in the DirectX documentation). A mesh is just a bunch of lines, triangles, polygons that define the basic structure of a 3D model.

m_mesh = new Mesh(numberVerts * 3, numberVerts, MeshFlags.Managed,
  CustomVertex.PositionColored.Format, m_device);

In the creation of the mesh we indicate:

  • the number of faces, since it’s a cube, it has 24 faces/tris.
  • the number of vertices, a cube has 8 vertices.
  • some mesh flags, we indicate that vertex and index buffers are managed by DirectX.
  • the format of the vertices, our contains position and color information.
  • finally we indicate the device.

Now two important things here are the vertex and index buffers. A vertex buffer is nothing more than a collection (an array) of vertices and an index buffer stores how these vertices are used to draw a triangle. This can be very useful to save space when we render something and in this case you can see clearly how we can save space, just imagine that you have to send the vertices for every triangle in the cube, you’d have to repeat each vertex several times since the vertices are shared by different triangles. This way we send all the vertices we need then we just have to indicate how we build a triangle from these vertices.

short[] indices =
{
  0,1,2, // Front Face
  1,3,2, // Front Face
  4,5,6, // Back Face
  6,5,7, // Back Face
  0,5,4, // Top Face
  0,2,5, // Top Face
  1,6,7, // Bottom Face
  1,7,3, // Bottom Face
  0,6,1, // Left Face
  4,6,0, // Left Face
  2,3,7, // Right Face
  5,2,7 // Right Face
};

using (VertexBuffer vb = m_mesh.VertexBuffer)
{
  GraphicsStream data = vb.Lock(0, 0, LockFlags.None);

  data.Write(new CustomVertex.PositionColored(-5.0f, 5.0f, 5.0f, Color.Red.ToArgb()));
  data.Write(new CustomVertex.PositionColored(-5.0f, -5.0f, 5.0f, Color.Green.ToArgb()));
  data.Write(new CustomVertex.PositionColored(5.0f, 5.0f, 5.0f, Color.Red.ToArgb()));
  data.Write(new CustomVertex.PositionColored(5.0f, -5.0f, 5.0f, Color.Green.ToArgb()));
  data.Write(new CustomVertex.PositionColored(-5.0f, 5.0f, -5.0f, Color.Blue.ToArgb()));
  data.Write(new CustomVertex.PositionColored(5.0f, 5.0f, -5.0f, Color.Blue.ToArgb()));
  data.Write(new CustomVertex.PositionColored(-5.0f, -5.0f, -5.0f, Color.Yellow.ToArgb()));
  data.Write(new CustomVertex.PositionColored(5.0f, -5.0f, -5.0f, Color.Yellow.ToArgb()));

  vb.Unlock();
}

using (IndexBuffer ib = m_mesh.IndexBuffer)
{
  ib.SetData(indices, 0, LockFlags.None);
}

Here we set the mesh vertex and index buffers. Check the indices variable, it contains the indices of the vertices that build up a triangle of the cube. When we add a vertex, we indicate a color so if we’re indicating different colors for each vertex, what will be the color of a triangle? DirectX does a smooth transition between the vertex colors. I added different colors so we can easily see that the cameras are pointing to different sides of the cube.

VIEWPORT SETUP

To support different cameras in each viewport, some minor things were modified in our viewport. If you remember, in the last tutorial we saved the swap chain in the tag field of the viewport, but now, we have also the camera to save in each viewport:

class ViewportInfo
{
  public SwapChain swapChain;
  public CameraInfo camInfo;

  public ViewportInfo()
  {
  }

  public ViewportInfo(SwapChain nswapChain, CameraInfo ncamInfo)
  {
    swapChain = nswapChain;
    camInfo = ncamInfo;
  }
}

In the previous tutorial we already had the swap chain, but now we add a new attribute, the camera info. This CameraInfo, for now, will only save simple data to identify our viewport’s cameras.

class CameraInfo
{
  public Vector3 position;
  public Vector3 lookAt;
  public Vector3 up;

  public bool ortho;
  public float scale;
  public float zNearPlane;
  public float zFarPlane;

  public CameraInfo(Vector3 nposition, Vector3 nlookAt, Vector3 nup, bool northo)
  {
    position = nposition;
    lookAt = nlookAt;
    up = nup;
    ortho = northo;

    zNearPlane = 1f;
    zFarPlane = 100f;
    scale = 0.2f;
  }
}

These structures may (will, almost for sure) suffer changes in future, to accommodate new features.

CAMERA SETUP

How do we setup these cameras? Easy, just setup the look at and projection matrices.
What are these matrices? The look at matrix is also known as view matrix and this matrix transforms our world points into camera space, that is, all objects are relocated around our camera point. The projection matrix is another transformation matrix used to project three dimensional points (from the camera space) onto a two dimensional plane (our computer screen). I’m not going to enter in detail with these matrices, if you want to see how they work or how they’re built, check the DirectX documentation and search for Transformations.

Let’s see how we set these matrices in DirectX:

if (camInfo.ortho)
{
  m_device.Transform.Projection = Matrix.OrthoLH(viewport.Width * camInfo.scale,
    viewport.Height * camInfo.scale, camInfo.zNearPlane, camInfo.zFarPlane);

  m_device.Transform.View = Matrix.LookAtLH(camInfo.position, camInfo.lookAt, camInfo.up);
}
else
{
  m_device.Transform.Projection = Matrix.PerspectiveFovLH((float)Math.PI / 4,
    (float)viewport.Width / (float)viewport.Height, camInfo.zNearPlane, camInfo.zFarPlane);

  m_device.Transform.View = Matrix.LookAtLH(camInfo.position, camInfo.lookAt, camInfo.up);
}

We use two types of cameras, the orthogonal and the perspective.

Orthogonal:

  • Orthogonal cameras map the objects into a 2D view.
  • We create an orthogonal matrix using Matrix.OrthoLH (LH stands for left handed, DirectX uses left handed system) and set the projection matrix. Why is that scale multiplication by width and height? Because the width and height of the viewport are very large when compared to the size of the object, so we zoom a bit.
  • Create a look at matrix with Matrix.LookAtLH and set the view matrix. We indicate our camera’s position, the poitnt where we’re looking at and the camera’s up vector.

Perspective/Projection:

  • These cameras map the objects into a 3D view.
  • Create a perspective matrix from Matrix.PerspectiveFovLH and set the projection matrix. We define a FOV of 90º (degrees), the viewport aspect ratio width/height and the near and far planes that define the boundaries of what we can see (we can see everything that’s between these planes.)
  • Set the view matrix like in orthogonal cameras.

OTHER CHANGES

Viewport has been improved and I decided to add a title to know which camera we have in each viewport. Another thing that was missing in the viewport was the ability to see it rendered in the designer, when used in our Four way splitter control and this has also been fixed. Some other minor improvements here and there were done.

And we reached the end of another tutorial. I’ll add the C++/CLI code shortly (only C# for now), and other updates I might do to the code. Next time we’ll continue with cameras to help us move through the scenario. Feel free to contact me about anything wrong or good with the tutorial, you have my e-mail at the end of the page. See ya till next time! ;)

managed_directx3

Download Files:

C# version
You’re free to use this code as long as you don’t blame me for any changes/bugs that were introduced and don’t demand anything from me. Only download it if you accept these conditions.

References:

http://msdn.microsoft.com/directx/ – Microsoft DirectX Development Center
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_m/directx/directx9m.asp – DirectX 9.0 for Managed Code
http://forums.microsoft.com/ – MSDN Forums

Previous Tutorial

Managed DirectX Tutorial 2 – Introduction to swap chains: Rendering to two panels at the same time

VIEWPORTS AND SWAP CHAINS

In this tutorial I’ll explain how you can use swap chains to create different views of the same scene. This tutorial should be fairly simple for you to understand and follow.

A swap chain is basically the process that DirectX uses when presenting a scene to the user. It draws the scene to an off-screen buffer (back buffer) and when it’s ready, it presents the image to the user swapping the back buffer (that contains the current frame) with the front buffer (that contains the last frame). Many swap chains can be created depending on the memory you have available. These are useful when you want to render a scene in more than one place. Think of an editor or a 3D program like 3D Studio Max, it has different views (using different cameras and projections) of the same scene. By default, DirectX creates one swap chain whenever you create the device.
The creation of a new swap chain is very easy. Let’s see what’s needed:

SwapChain sc = new SwapChain(device, presentParams);

All we need is the device a present parameters structure. So, if we only need these two things to create a new swap chain, how do we indicate to where we will render it? When creating a new device we indicate the handle to the control where we’re going to render our scene but in the case of swap chains we already have the device created, so, we use a field in the present parameters structure to indicate the handle:

presentParams.Window = panel.Handle;

4 WAY SPLIT VIEWPORTS

Now let’s draw something in 4 different viewports. I built a new control for this purpose, the 4 way splitter. To continue we need to take some special measures to assure that our viewports are rendered/updated correctly. What happens if we resize one or more viewports? we need to create new swap chains since viewport’s area has changed. I created a new control to help us on this task, the viewport control. This is a very simple control (for now, we might add some stuff to it later), it has two function callbacks: one for the render and one for the creation of the swap chain. This two callbacks are called whenever the control is painted or resized, respectively.

Viewport code

public delegate void RenderFuncDelegate(Viewport viewport);
private RenderFuncDelegate m_renderFunc;

public delegate void SwapChainFuncDelegate(Viewport viewport);
private SwapChainFuncDelegate m_swapChainFunc;

public RenderFuncDelegate RenderFunc
{
	[Browsable(false)]
	set { m_renderFunc = value; }
}

public SwapChainFuncDelegate SwapChainFunc
{
	[Browsable(false)]
	set { m_swapChainFunc = value; }
}

protected override void OnPaint(PaintEventArgs e)
{
	// render our scene
	if (m_renderFunc != null)
		m_renderFunc(this);
}

protected override void OnResize(EventArgs e)
{
	base.OnResize(e);

	// create new swap chain
	if (m_swapChainFunc != null)
		m_swapChainFunc(this);
}

Our main window is composed by a four way splitter with four viewports in each corner. We do some initializations with the device (see managed directx tutorial (part 1)) and create our swap chain and render functions so they can process requests for the viewports:

public void InitializeViewports()
{
  // save our viewports in an array
  m_viewports[0] = viewportTopLeft;
  m_viewports[1] = viewportTopRight;
  m_viewports[2] = viewportBottomLeft;
  m_viewports[3] = viewportBottomRight;

  // for each viewport register the delegate functions
  foreach(Viewport viewport in m_viewports)
  {
    viewport.SwapChainFunc = new WindowsControls.Viewport.SwapChainFuncDelegate(CreateSwapChain);
    viewport.RenderFunc = new WindowsControls.Viewport.RenderFuncDelegate(Render);
    CreateSwapChain(viewport);
  }
}

First we initialize an array with our viewports and for each viewport we associated the delegate functions that will handle the creation of the swap chain and render of the screen. Next our two delegate functions:

private void CreateSwapChain(Viewport viewport)
{
  // get the swap chain of the viewport
  SwapChain swapChain = ((SwapChain)viewport.Tag);

  // if we already have a swap chain created, get rid of it
  if(swapChain != null)
  {
    swapChain.Dispose();
  }

  PresentParameters presentParams = m_presentParams;
  presentParams.BackBufferWidth = viewport.Width;
  presentParams.BackBufferHeight = viewport.Height;

  // check if our buffer is valid
  if(presentParams.BackBufferWidth == 0 || presentParams.BackBufferHeight == 0)
  {
    return;
  }

  presentParams.DeviceWindowHandle = viewport.Handle;

  // save the swap chain in the viewport
  viewport.Tag = new SwapChain(m_device, m_presentParams);
}

private void Render(Viewport viewport)
{
  // get the swap chain of the viewport
  SwapChain swapChain = ((SwapChain)viewport.Tag);

  // safety check
  if (swapChain == null)
    return;

  // retrieve it's back buffer surface where we do the render
  Surface pBackBuffer = swapChain.GetBackBuffer(0);

  // set it as our current back buffer
  m_device.SetRenderTarget(0, pBackBuffer);

  // clear our back buffer
  m_device.Clear(ClearFlags.Target, Color.DarkSlateBlue, 1.0f, 0);

  // we don't need the surface anymore, get rid of it
  pBackBuffer.Dispose();

  // swap the screen buffer and back buffer to present our new data
  swapChain.Present();
}

And this is the end of another tutorial. I’ll add the C++/CLI code shortly (only C# for now), and other updates I might do to the code. Next time I’ll show you… well, I don’t know what I’ll do so stay tuned ;) Your feedback is appreciated, you can leave comments at the end of the page. See ya till next time! ;)

Download Files:

C# version
You’re free to use this code as long as you don’t blame me for any changes/bugs that were introduced and don’t demand anything from me. Only download it if you accept these conditions.

References:

http://msdn.microsoft.com/directx/ – Microsoft DirectX Development Center
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_m/directx/directx9m.asp – DirectX 9.0 for Managed Code
http://forums.microsoft.com/ – MSDN Forums

Managed DirectX Tutorial 1 – Creation of a Direct 3D device and GDI+ rendering

SETTING UP A DIRECTX DEVICE

Here you’ll learn how to setup a simple directx device in C# (and C++/CLI). If you have any experience with C++ or DirectX it will be just a warm up for you, if you don’t, it will be a simple tutorial that you read in 15 minutes.
If it was easy to setup a device in C++, in managed code it’s even easier to do it. Let’s start by creating a new windows form. Next follows the device setup and creation.
In this part you need to know mainly two things: the presentation parameters and the device. Let’s start by explaining the later one: the device is the main “interface” with the graphics system in your computer. It is responsible for rendering triangles, lines, etc., creating resources, shaders, etc. and other useful stuff; the presentation parameters hold the data that describes the setup of device, such as indicating if an application is running in windowed mode or not, the device’s window and the handle to the respective window, etc.
Let’s start by setting up the PresentParameters structure:

PresentParameters presentParams = new PresentParameters();
presentParams.IsWindowed = true;
presentParams.SwapEffect = SwapEffect.Discard;
  • The first parameter, IsWindowed, indicates that we are running the application in windowed mode, this means that we’re rendering to a window (or part of it) instead of rendering to the screen (full screen mode).
  • The second parameter, SwapEffect, indicates that we do not want DirectX to do any special operation when a swap is made between the back buffer and the actual screen buffer.

Note: When rendering some stuff using DirectX you are not actually drawing it to the screen. DirectX (as well as OpenGL) has the capability of drawing to a back buffer. This buffer will hold the final image and then it will be copied to the screen buffer.

DEVICE CREATION

Since you have some presentation parameters set, let’s have a look at the creation of the device:

device = new Device(0, DeviceType.Hardware, this.Handle, CreateFlags.HardwareVertexProcessing, presentParams);

Here we indicate the physical device (0 is default, the type of the device, the handle that we want to use to render our window, some flags for the creation of the device and our presentation parameters.

Now, here there is one thing that it’s interesting, what handle do we provide to the device?
In the .net framework, all components have the Handle property that returns a handle to that component. Well, if you want to create a device that will use a panel as its render scene, we give the panel’s handle to the device or if we want a form to have our scene, we give the form’s handle. This is very useful if we want to use part of a window as a DirectX scene and other part with contents of a regular windows application with buttons, textboxes and labels.

DEVICE MANIPULATION

The device is created, let’s proceed to some action… let’s clear the contents of our scene:

device.Clear(ClearFlags.Target, Color.DarkSlateBlue, 1.0f, 0);

In this command we’re clearing the render target (that’s what we’re indicating in the first field, think of it as our offscreen buffer for now), with the color provided in the second argument and the other two arguments you can ignore them for now.

LAST STEP

The final step is where we indicate that we want to see what we’ve been doing, that is, we want to see the contents of the back buffer.

device.Present();

Although there might be some unexplained stuff here, you might get the basic idea on how to set a new device.

PUTTING IT ALL TOGETHER

We saw how to create a device and clear it, now we’ll see how to put this all together in a managed application. Step by step:

  • Create a new project.
  • Create a new Form (if you don’t have one already), I named it DXSampleForm:
public partial class DxSampleForm : Form
{
	public DxSampleForm()
	{
		InitializeComponent();
	}
}
  • Add a new private member of type Device:
private Device device;
  • Create a method called InitializeDevice where you create the present parameters structure and the device. Call this method after the InitializeComponent in the constructor:
public DxSampleForm()
{
	InitializeComponent();
	InitializeDevice();
}

public void InitializeDevice()
{
	PresentParameters presentParams = new PresentParameters();
	presentParams.IsWindowed = true;
	presentParams.SwapEffect = SwapEffect.Discard;

	device = new Device(0, DeviceType.Hardware, this.Handle, CreateFlags.HardwareVertexProcessing,
		presentParams);
}
  • Override the OnPainBackground and do the DirectX rendering:
protected override void OnPaintBackground(System.Windows.Forms.PaintEventArgs e)
{
	device.Clear(ClearFlags.Target, Color.DarkSlateBlue, 1.0f, 0);
	device.Present();
}

Why the OnPaintBackground and not the OnPaint you may ask?
Simple, overriding the OnPaintBackground can spare us of some problems in future if we ever need to use GDI+ or if we need to have components in the middle of the area where we’re rendering our scene.

Note: Don’t forget that when you use managed DirectX, you’ll have to reference a(some) DLL(s). For this tutorial you only need the Microsoft.DirectX.DLL.

As you can see it’s very simple to create a DirectX sample using the framework .net. It takes only 12 lines of code made by you to have a rendered window.

FAQ

Q: How can I render to a panel?
A: Use the panel’s handle as the handle when creating the device. Example:

device = new Device(0, DeviceType.Hardware, panel.Handle, CreateFlags.HardwareVertexProcessing, presentParams);

Q: I want to draw some buttons and stuff over my DirectX render scene, is it possible?
A: Sure, just be careful to call the DirectX render methods in the OnPaintBackground instead of the OnPaint. As for the components, just drag them in the designer or add them by hand.
Q: How can I use GDI+ with DirectX?
A: First make sure you specify in your present parameters that you want to be able to lock the back buffer:

presentParams.PresentFlag = PresentFlag.LockableBackBuffer;

After that you need to access the back buffer. To do so, you use the GetBackBuffer method where you indicate the swap chain (0 by default) and the index of the back buffer. From this surface you can create a Graphics object that you can use to do your GDI+ drawings:

Microsoft.DirectX.Direct3D.Surface backbuffer;
backbuffer = device.GetBackBuffer(0, 0);
Graphics graphics = backbuffer.GetGraphics();
graphics.DrawRectangle(Pens.Beige, 10, 20, 50, 60);
backbuffer.ReleaseGraphics();
backbuffer.Dispose();

And we reached the end of this tutorial. Next time I’ll show you how to render to two separate areas in a form. This is usually used in editors. Let me know if you find/have any bugs/questions/comments/suggestions. See ya till next time! ;)

managed_directx1

Download Files:

C# and C++/CLI versions
You’re free to use this code as long as you don’t blame me for any changes/bugs that were introduced and don’t demand anything from me. Only download it if you accept these conditions.

References:

http://msdn.microsoft.com/directx/ – Microsoft DirectX Development Center
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_m/directx/directx9m.asp – DirectX 9.0 for Managed Code
http://pluralsight.com/wiki/default.aspx/Craig.DirectX/Direct3DTutorialIndex.html – Direct3DTutorialIndex

Next Tutorial