Kinect and WP7 interaction

By the same time I did my portfolio I also did another experiment that’s in this post. A few days ago I saw the KinectoPhone project and thought it would be cool to adapt it to another project I was developing with Francisco Campelo. This project allows a desktop application using Kinect to communicate with another application on the Windows Phone 7 synching its state between the two apps by communicating through a server.

I changed a bit the network code and plugged in with other code I had done for the particles and here’s the result:

Kinect and WP7 interaction

The red particles on the big screen are from Kinect’s skeleton and the blue particles are sent from the WP7 through touch screen input. On the WP7 screen, the blue particles are from Kinect’s skeleton and the red/white ones from the user’s input.

This could get many improvements, network and feature wise but I’ll leave that to some other time. I have many ideas but the time is short so I’ll be moving to another one soon.

THANKS TO

  • My parents for helping me out testing and recording the video.
  • Anabela Faria for the video editing.
  • Francisco Campelo for the original idea which led me to the Kinect version.

Interactive portfolio using Kinect

I like to explore different technologies that relate with computers and that’s why I own a Lego Mindstorms NXT . More recently I bought a Kinect to transform some ideas into real applications.

I’m currently looking for job in software development in the Toronto (Canada) area and it occurred to me that I could present my portfolio in a fun and entertaining way. So I decided to create my first Kinect application to be this interactive portfolio. Although I have some years of experience in business applications, portals, intranets, etc., I decided to compile only my experience in game development and interactive 3d projects.

If you don’t own a Kinect you can watch the video below to see how it works. I encourage you to skip the video in case you want to try the application or you’ll ruin the experience.

Interactive portfolio

The code uses the following technologies: C#, .NET, XNA and WPF. I also used two open source libraries to help with the gestures and the XNA integration with WPF.

DOWNLOAD

Software requirements:

Kinect application:

Note: Please make sure you install the software requirements or you won’t be able to install the application.

THANKS TO

  • My parents for helping me out recording the video.
  • Anabela Faria for the video editing.
  • Paulo Silva for the animated silhoutte.

SilverMenu library for WP7

Inspired by the menus of WP7 game Puzzle Jumble, I decided to write a library that would help me and others to build menus for WP7 games easily. The Puzzle Jumble’s GUI code is a bit of a mess because it wasn’t thought for this kind of interface from the beginning so I decided to write SilverMenu from scratch.

Puzzle Jumble’s GUI

 

SilverMenu aims to bring some of the Silverlight user interface that is now present on WP7 to the XNA. To make it simple to use you can build simple animated menus with just a few lines of XML. It has only one week of development but you can see some progress already:

First SilverMenu sample.

 

With time I hope it grows into something more complete and useful. I’ll still update my blog about this project but if you want to follow the project closely and keep up to date, as well as, download the last build, you can do it at http://silvermenu.codeplex.com. I hope you find it useful and I appreciate all the feedback I can get. Thanks!

Windows Phone 7 – Tips for programming games

Puzzle Jumble started as an experiment to test and learn a little bit about XNA and Windows Phone 7. I decided to write some tips to help those who like me started experimenting with XNA and WP7.

Stencil

The stencil was used to cut the hole for doors so that pieces would appear to be sliding over the doors but under the board. The stencil was active in every frame which lowered the frame rate quite a lot (18-20 FPS). The solution was to activate the stencil only when a piece is going to leave through a door, so be careful when using the stencil buffer on the WP7.

DXT

GPUs like the DXT format and the WP7 is no exception, but unlike desktop’s GPUs the WP7 processor only loads DXTs that are power of two. My regret is not using DXT from the beginning when I started working on Puzzle Jumble. When I changed all my textures to DXT I had to change a lot of code, all the draw calls where I was using a position to draw a sprite had to be changed to a rectangle because the textures had a different resolution (power of two).
But the change was very useful:
– I gain some FPS;
– the memory footprint was reduced from 95MB (total peak) to about 45MB;
– the XAP file was reduced from 25MB to 9MB;
– thus, as a consequence reduced the loading times.

Floats (culture)

When parsing floats it’s easy to add bugs in your application if you ignore the culture format. Make sure to use the correct format provider or at least use the culture independent System.Globalization.NumberFormatInfo.InvariantInfo.

Float precision

Another problem that I found when using floats was that this comparison: if(a < float.Epsilon) (when a = 0) would work everywhere except on the WP7 device itself where it would always return false.

HTML page requests

This is not a WP7 specific problem but I thought it was worth mentioning it. Since Microsoft didn’t provide us with an API for leader boards I implemented an online score system using our servers to keep track of the users’ scores and best times.
The system was working fine to post data online but there was a problem when retrieving it. The first time the data was retrieved everything was fine but the subsequent retrievals would always be the same as if the data was cached somewhere.
After searching for a while I found out that the problem was that the HTTP request was being cached since the URL didn’t change, it would cache the result for that URL. The solution was to add a random attribute to the URL so that the the request couldn’t be cached. Not pretty but it was the only way I found to make it work.

And those were the problems that I found during the Puzzle Jumble’s development phase. I hope this useful for you!

VSX: How to detect when Property Pages (Project Properties) window is closed

I posted a thread on the MSDN forums to seek for help in detecting when the property pages window (accessible through project->properties) in Visual Studio is closed. You can follow up the thread here: Detect when project properties window is closed. Ziwei Chen was really helpful and could find me a way to achieve this in C# projects but to my surprise, to achieve the same results for C++ projects proved to be harder than I thought.

I decided to write this short tutorial to help those who have the same problem in future.

First of we need to detect when the user click in any project->properties menu selection. Put this code where you initialize your add-in:

m_commandEventsAfterExecuteProperties1 =
    m_applicationObject.DTE.Events.get_CommandEvents("{5EFC7975-14BC-11CF-9B2B-00AA00573819}", 396);
m_commandEventsAfterExecuteProperties1.AfterExecute +=
    new _dispCommandEvents_AfterExecuteEventHandler(CommandEvents_AfterExecuteProperties);

m_commandEventsAfterExecuteProperties2 =
    m_applicationObject.DTE.Events.get_CommandEvents("{5EFC7975-14BC-11CF-9B2B-00AA00573819}", 397);
m_commandEventsAfterExecuteProperties2.AfterExecute +=
    new _dispCommandEvents_AfterExecuteEventHandler(CommandEvents_AfterExecuteProperties);

These command events ids were found using the technique described in this post: How to find Visual Studio command bars. When registering the event handlers be careful with this: Visual Studio .NET events being disconnected from add-in. So declare these as member attributes and not as method variables:

private CommandEvents m_commandEventsAfterExecuteProperties1;
private CommandEvents m_commandEventsAfterExecuteProperties2;

Now, lets dig into some rather awful code to get our property pages handle. Property pages in .NET projects are document windows but, unfortunately, if we try to call the ActiveDocument property directly through the ActiveEnvDTE library, it will throw an exception. So, with the precious help of Chen, this is the code came up with:

// For C# projects the property pages is a document window, lets try to find it.
// I was unable to get a handle using the EnvDTE library which could have eased things a bit.
bool propertyPagesFound = false;

IVsWindowFrame[] frames = new IVsWindowFrame[1];
uint numFrames;

IEnumWindowFrames ppenum;
m_uiShell.GetDocumentWindowEnum(out ppenum);

while (ppenum.Next(1, frames, out numFrames) == VSConstants.S_OK && numFrames == 1)
{
	m_frame = frames[0] as IVsWindowFrame;
	object title;
	m_frame.GetProperty((int)__VSFPROPID.VSFPROPID_Caption, out title);

	// I really don't like the way I'm using to retrieve the property pages handle.
	//TODO: CHANGE IF BETTER SOLUTION IS FOUND.
	if ((title as String).ToLowerInvariant() == GetStartupProject().Name.ToLowerInvariant())
	{
		propertyPagesFound = true;
		((IVsWindowFrame2)m_frame).Advise(this, out m_cookieWindowFrameNotify);
	}
}

First, all document windows are retrieved to the ppenum variable and next we go one by one and test its caption name to see if it’s equal to the project’s name. After we got our desired window frame we register the IVsWindowNotifyFrame event handler.

In the C++ case, things are a bit different. The property pages window isn’t an instance of document or frame, it’s a dialog window. To catch the dialog window events we use an auxiliary class that listens for the destroy message in the WndProc method.

// If we didn't find any property pages before then we need to go and search for a C++ one since they're
// different from C# property pages.
if (!propertyPagesFound)
{
	//NOTE: This might not work well if we happen to have two Visual Studio instances with the same
	//      project names.
	//TODO: TRY TO FIND A WAY TO FIX IT.
	//HACK: Go through all windows to find the project properties dialog window that matters.
	FindWindowLike.Window[] list =
	    FindWindowLike.Find(IntPtr.Zero, GetStartupProject().Name.ToLowerInvariant() + " property pages", "");
	
	// Release any handle we might have.
	if (m_subclassedMainWindow != null)
	{
		m_subclassedMainWindow.ReleaseHandle();
		m_subclassedMainWindow = null;
	}

	if (m_subclassedMainWindow == null && list != null && list.Length >= 1)
	{
		m_subclassedMainWindow = new SubclassedWindow(list[0].Handle);
	}
}

Apart from that FindWindowLike that we’ll check later, I think the code is pretty simple to understand. I really don’t like to have to search through all windows in the system but apparently the dialog window couldn’t be found under the dte’s MainWindow.HWnd hierarchy.

 

As for the event implementations, here is the C# first:

public int OnShow(int fShow)
{
	switch (fShow)
	{
		case (int)__FRAMESHOW.FRAMESHOW_TabDeactivated:
			// Property pages lost focus, put your code here!
			((IVsWindowFrame2)m_frame).Unadvise(m_cookieWindowFrameNotify);
			m_cookieWindowFrameNotify = 0;
			break;

		default:
			break;

	}

	return VSConstants.S_OK;
}

We need to implement the IVsWindowFrameNotify interface and in the OnShow you can check when the tab is closed. I used the FRAMESHOW_TabDeactivated because it is called when the tab loses focus and also when the tab closes.

Now for the C++ version:

/// 
/// Auxiliary window to intercept messages sent to the property pages dialog window.
/// 
public class SubclassedWindow : NativeWindow
{
	private const int WM_NCDESTROY = 0x0082;

	public SubclassedWindow(IntPtr hWnd)
	{
		AssignHandle(hWnd);
	}

	protected override void WndProc(ref Message m)
	{
		if (m.Msg == WM_NCDESTROY)
		{
			// Property pages window was closed, put your code here!
		}
		base.WndProc(ref m);
	}
}

As explained before, in this case we have an auxiliary window to catch the destroy event for the dialog window.

I found this script over at FindWindowLike for C# and changed it a bit to fit my needs. I modified the code that compared the window title; created a new class and changed p/invoke methods’ signature just to keep VS quiet about some warnings.

/// <summary>
/// Now we have to declare p/invoke methods in a class called NativeMethods (or similar) to
/// shut up some VS warnings.
/// </summary>
internal static class NativeMethods
{
	[DllImport("user32")]
	internal static extern IntPtr GetWindow(IntPtr hwnd, int wCmd);

	[DllImport("user32")]
	internal static extern IntPtr GetDesktopWindow();

	[DllImport("user32", EntryPoint = "GetWindowLongW")]
	internal static extern int GetWindowLong(IntPtr hwnd, int nIndex);

	[DllImport("user32")]
	internal static extern IntPtr GetParent(IntPtr hwnd);

	[DllImport("user32", EntryPoint = "GetClassNameW", CharSet = CharSet.Unicode)]
	internal static extern int GetClassName(IntPtr hWnd, [Out] StringBuilder lpClassName, int nMaxCount);

	[DllImport("user32", EntryPoint = "GetWindowTextW", CharSet = CharSet.Unicode)]
	internal static extern int GetWindowText(IntPtr hWnd, [Out] StringBuilder lpString, int nMaxCount);
}

// FROM: http://www.experts-exchange.com/Programming/Languages/C_Sharp/Q_21611201.html
// Just changed the way it checked strings.
internal class FindWindowLike
{
	internal class Window
	{
		public string Title;
		public string Class;
		public IntPtr Handle;
	}

	private const int GWL_ID = (-12);
	private const int GW_HWNDNEXT = 2;
	private const int GW_CHILD = 5;

	public static Window[] Find(IntPtr hwndStart, string findText, string findClassName)
	{
		ArrayList windows = DoSearch(hwndStart, findText, findClassName);

		return (Window[])windows.ToArray(typeof(Window));

	} //Find


	private static ArrayList DoSearch(IntPtr hwndStart, string findText, string findClassName)
	{
		ArrayList list = new ArrayList();

		if (hwndStart == IntPtr.Zero)
			hwndStart = NativeMethods.GetDesktopWindow();

		IntPtr hwnd = NativeMethods.GetWindow(hwndStart, GW_CHILD);
		while (hwnd != IntPtr.Zero)
		{
			// Recursively search for child windows.
			list.AddRange(DoSearch(hwnd, findText, findClassName));

			StringBuilder text = new StringBuilder(255);
			int rtn = NativeMethods.GetWindowText(hwnd, text, 255);
			string windowText = text.ToString();
			windowText = windowText.Substring(0, rtn);

			StringBuilder cls = new StringBuilder(255);
			rtn = NativeMethods.GetClassName(hwnd, cls, 255);
			string className = cls.ToString();
			className = className.Substring(0, rtn);

			if (NativeMethods.GetParent(hwnd) != IntPtr.Zero)
				rtn = NativeMethods.GetWindowLong(hwnd, GWL_ID);

			if (windowText.Length > 0 &&
				windowText.ToUpperInvariant().Contains(findText.ToUpperInvariant()) &&
				(className.Length == 0 ||
					className.ToUpperInvariant().Contains(findClassName.ToUpperInvariant())))
			{
				Window currentWindow = new Window();

				currentWindow.Title = windowText;
				currentWindow.Class = className;
				currentWindow.Handle = hwnd;

				list.Add(currentWindow);
			}

			hwnd = NativeMethods.GetWindow(hwnd, GW_HWNDNEXT);
		}

		return list;
	} //DoSearch
} //Class

And that’s it! I’d really appreciate if someone could tell me how to improve some of this code, mostly the parts where I have TODOs and/or HACK.
Thank you and I hope you find this useful!

Thanks to:

I wanted to thank Ziwei Chen for the excellent help at the forums.

References

VSSDK IDE Sample: Combo Box
Detect when project properties window is closed
I need FindWindowLike for C#
How to find Visual Studio command bars
Visual Studio .NET events being disconnected from add-in

Links:

Visual Studio Extensibility Forum
VSX Home on Code Gallery
Visual Studio on MSDN

Visual Studio 2005/2008 Free Utilities

Here are some free add-ins/tools/utilities that you can use in Visual Studio 2005 (or 2008):

AnkhSVN

http://ankhsvn.open.collab.net/
Do you use subversion version control system? TortoiseSVN? This add-in is great and integrates very well with all VS versions since 2002.

CLIArgs Made Easy

http://www.ricardosabino.com/?pag=vs_addin1
CLI Args Made Easy (Command Line Arguments Made Easy) is an add-in that allow you to change the command line arguments of the startup project very easily by adding a combobox in the toolbar. It also saves all the arguments you insert for future use.

Microsoft Visual Studio 2005 IDE Enhancements

http://www.microsoft.com/downloads/details.aspx?FamilyID=CD7C6E48-E41B-48E3-881E-A0E6E97F9534&displaylang=en
Set of Visual Studio extensions that improve productivity: Source code outliner, Visual C++ code snippets, indexed find, super diff utility, event toaster utility.

MPCL

http://www.todobits.es/mpcl.html
This plug-in allow us to compile the source code files of a Visual Studio 2005 project in parallel. This is extremely useful for processors with more than one core to reduce the compile time.

Power Toys for Visual Studio 2005

http://msdn2.microsoft.com/en-us/vstudio/aa718340.aspx
Pack with various tools for VS 2005.

Project Line Counter

http://www.wndtabs.com/download/Download/LineCounter/
Reports statistics about the files in projects. It’s automatic, has a filter and you can check its source code.

Refactor! For Visual Studio 2005/2008

http://msdn2.microsoft.com/en-us/visualc/bb737896.aspx
Free add-in that refactors (simplifies and re-structures) source code to make it easier to read and less costly to maintain.It has more than 15 different features.

RockScroll

http://www.hanselman.com/blog/IntroducingRockScroll.aspx
This add-in extends the Visual Studio’s scroll bar showing the code with syntax highlight. It is extremely useful to go to where you want within long source code files.

SlickEdit Gadgets

http://www.slickedit.com/content/view/441
Free useful gadgets for VS 2005: Editor gadgets – line ruler, indentation guide, auto-copy selection, editor graphic; command spy, file explorer, data object analyzer and SLOC report.

VSFileFinder 2005/2008

http://www.zero-one-zero.com/vs/
VSFileFinder helps you find files quickly in your projects by typing a few letters from any part of the file name which is much faster than using the solution explorer in large projects.

There is also a version with some improvements (wildcards, regular expressions, personalized tooltips, project filter) and compatible with Visual Studio 2008 available here: VSFileFinder2008.

Visual Studio Add-Ins Every Developer Should Download Now

http://msdn.microsoft.com/msdnmag/issues/05/12/VisualStudioAddins/
MSDN page with a collection of free add-ins mostly for .net languages but not only.

Programming FAQ

– How do I call a method from another class?
– How do I solve linking errors?
– My header files depend on one another, how do I solve this circular dependency?
– What’s wrong with the C++ hello world?
– Why do I need header files?
– Why does my application runs in debug mode but not in release?
– How can I debug my release build?

– How do I call a method from another class?

Static methods can be called directly, i.e., without having to instantiate the class but these don’t allow us to change any data that isn’t static so this is probably not what you want to do. There are several ways to call a method other than static, here is described the most common way. You need to have an object (instance) of that class and call the method through that object. Next is the code in 3 different languages, C++, C++ CLI and C#:

A.h

class B;
class A
{
public:
    A(B* b);
    void Func();
    void Print();

private:
    B* m_b;
};
<p>

A.cpp

#include <iostream>
#include "A.h"
#include "B.h"

A(B* b)
{
    m_b = b;
}

void A::Func()
{
    m_b->Func();
}

void A::Print()
{
  std::cout << "A";
}

B.h

class A;
class B
{
public:
    B(A* a);
    void Func();
    void Print();

private:
    A* m_a;
};

B.cpp

#include <iostream>
#include "B.h"
#include "A.h"

B(A* a)
{
    m_a = a;
}

void B::Func()
{
    m_a->Func();
}

void B::Print()
{
  std::cout << "B";
}

A.h

ref class B;
ref class A
{
public:
    A(B^ b);
    void Func();
    void Print();

private:
    B^ m_b;
};

A.cpp

#include <iostream>
#include "A.h"
#include "B.h"

A(B^ b)
{
    m_b = b;
}

void A::Func()
{
    m_b->Func();
}

void A::Print()
{
  std::cout << "A";
}

B.h

ref class A;
ref class B
{
public:
    B(A^ a);
    void Func();
    void Print();

private:
    A^ m_a;
};

B.cpp

#include <iostream>
#include "B.h"
#include "A.h"

B(A^ a)
{
    m_a = a;
}

void B::Func()
{
    m_a->Func();
}

void B::Print()
{
  std::cout << "B";
}

A.cs

class A
{
public A(B b)
{
    m_b = b;
}

public void Func()
{
    m_b.Print();
}

public void Print()
{
    System.Diagnostics.Debug.WriteLine("A");
}

private B m_b;
}

B.cs

class B
{
public B(A a)
{
    m_a = a;
}

public void Func()
{
    m_a.print();
}

public void Print()
{
    System.Diagnostics.Debug.WriteLine("B");
}

private A m_a;
};

– How do I solve linking errors?

Linking/linkage errors happen when you are not linking your project with a library that contains the definition of a function you’re using from that library. They may also happen when you have a declaration of a function but you don’t implement it and then you use it somewhere. It will give a link error with your function’s name. If you’re using the Visual Studio IDE you may choose to which libraries you want to link by going to the project properties->linker->input->additional dependencies and add the correct respective libraries. If you’re using some functions from the windows api, you’ll probably need one or more of the following libraries: shell32.lib, kernel32.lib, user32.lib.

– My header files depend on one another, how do I solve this circular dependency?

You can solve this by either changing the logic of your program and removing the dependency from one class in the other (having only one class depending instead of both) or you can use forward declarations to solve this circular dependency.

A.h

class B;

class A
{
private:
    B* m_b;
};

B.h

class A;

class B
{
private:
    A* m_a;
};

Since A depends on B and B on A, if we try to include B.h in A.h and vice versa, we’ll get a compiler error. To overcome this you have to use forward declarations shown in the example and make sure that the class isn’t being used (this includes holding an instance of a class instead of a pointer or reference to an object of the class). The forward declaration tells the compiler that there’s a type named A (or B depending of the file) but doesn’t give the compiler any information about that type and that’s the reason why you can’t call any methods or use a forward declared class. In the respective source files (cpp files) you may include both A.h and B.h and use the classes as you wish. Note that there’s no problem when you include both header files in the .cpp files because there’s no circular dependency.

– What’s wrong with the C++ hello world?

#include <iostream>

int main()
{
    cout << "Hello World!";
    return 0;
}

I have crossed with many people learning C++ that can’t compile the hello world in some books. If you’re one of those that have problems compiling this code then you should better double check if the book/tutorial is worth your time. The code doesn’t compile because cout isn’t accessible this way. All types/objects/functions of the C++ standard library are in a namespace called std. Namespaces allow you to group types/objects/functions under a name. So to correct your hello world you could do one of the following:

#include <iostream>
using namespace std;
int main()
{
    cout << "Hello World!";
    return 0;
}
#include <iostream>
using std::cout;
int main()
{
    cout << "Hello World!";
    return 0;
}
#include <iostream>

int main()
{
    std::cout << "Hello World!";
    return 0;
}

– Why do I need header files?

You don’t but for large projects it’s advisable that you use header files. Usually header files contain declarations that later may be used in different files by including this header file. By separating declarations from the actual implementations you get some advantages:

  • The interface (declarations in the header file) is separated from the implementation (definitions in the source files).
  • Better structured files for better reading of interfaces.
  • It helps reusing / maintaining the code.

– Why does my application runs in debug mode but not in release?

Debug and release modes don’t actually “exist”, they are just compiler settings which cause the application to behave differently. In debug mode the compiler sets/clears/initializes the variables so it can track memory allocations/writings and this doesn’t happen (unless you changed the configurations) when you compile in release mode. These uninitialized variables may cause a crash in your program. This is not the only cause of the crashes in release mode but is one very common.
Note: Also be careful with code that is only ran in one or other configuration settings either by using macros (#ifdef #endif) and/or assert’s.
One remote possibility is the compiler causing the crash when optimizing your code. Optimizations are usually turned on in release configurations to speed up our application and due to these changes the compiler may introduce a bug in your code.

– How can I debug my release build?

Yes, it is possible to debug release builds. Under Visual Studio go to:

Project Properties->C/C++->Debug Information Format->Program Database
Project Properties->Linker->Debugging->Generate Debug Info->Yes

Now you can debug your program but be aware that if you have optimizations turned on then you may not see the instruction pointer (yellow arrow) in the right place because the optimizations make the code change or even disappear (whenever it’s not needed) from the final build.

Four-Way Splitter Control

Hi, for some time now I wanted a control that would allow me to have four panels separated with splitters which could be resized horizontally, vertically or all at the same time and that didn’t redraw the panels when resizing. I searched for a solution but didn’t find one that fitted my needs either because it would redraw the panels or it wouldn’t draw a shadow splitter like the default behaviour of the windows’ splitters, thus, I decided to make one.

THE IMPLEMENTATION

Since the normal splitter control from the .Net framework always draws the shadow splitter when you try to move it, I had to make a splitter that wouldn’t do this since I didn’t want it to happen in certain occasions. So this SimpleSplitter derives from the System.Windows.Forms.Splitter and overrides some methods that would draw the shadow splitter: OnMouseDown, OnMouseUp, OnSplitterMoved and OnSplitterMoving but I wanted to still be able to catch these events so what I did was to call the base event but changing the number of clicks because in the base class (the Splitter) will only draw the shadow splitter if the number of clicks in the splitter is one (Thanks to Hans Passant):

Simple Splitter code:

protected override void OnMouseDown(MouseEventArgs e)
{
    MouseEventArgs ebase = new MouseEventArgs(e.Button, 2, e.X, e.Y, e.Delta);
    base.OnMouseDown(ebase);
}
protected override void OnMouseUp(MouseEventArgs e)
{
    MouseEventArgs ebase = new MouseEventArgs(e.Button, 2, e.X, e.Y, e.Delta);
    base.OnMouseUp(ebase);
}
protected override void OnSplitterMoved(SplitterEventArgs e)
{
}
protected override void OnSplitterMoving(SplitterEventArgs e)
{
}

Other possible solution was was to create new events to catch these: MouseDown2 and MouseUp2:

public delegate void MouseEventHandler(object sender, MouseEventArgs e);
public event MouseEventHandler MouseDown2;
public event MouseEventHandler MouseUp2;
protected override void OnMouseDown(MouseEventArgs e)
{
    if (MouseDown2 != null)
        MouseDown2(this, e);
}
protected override void OnMouseUp(MouseEventArgs e)
{
    if (MouseUp2 != null)
        MouseUp2(this, e);
}

Next, I’ll describe the algorithm that defines this control works.

  • Event handlers for MouseDown/MouseUp/MouseMove/MouseEnter/MouseLeave events for every splitter.
  • The MouseDown handlers start by checking the mouse button (only the left button is accepted) and check if the cursor is in the middle (function MouseInCenter).
private bool MouseInCenter(object sender)
{
  Point p = this.PointToClient(MousePosition);
  // are we in the intersection?
  if ((Math.Abs(splitterHorLeft.SplitPosition - p.Y) < m_centerDelta &&
    Math.Abs(splitterVertical.SplitPosition - p.X) < m_centerDelta))
  {
    // yup, let's change the cursor to SizeAll.
    Cursor = Cursors.SizeAll;
    return true;
  }
  else
  {
    // Let's see to which cursor we have to change depending on the splitter we're on.
    Splitter splitter = sender as Splitter;
    if (splitter != null)
    {
      if (splitter.Dock == DockStyle.Left || splitter.Dock == DockStyle.Right)
      {
        Cursor = Cursors.VSplit;
      }
      else if (splitter.Dock == DockStyle.Top || splitter.Dock == DockStyle.Bottom)
      {
        Cursor = Cursors.HSplit;
      }
    }
    return false;
  }
}

This function is easy to understand (as any other function in this code), converts the mouse coordinates to the control’s local coordinates and checks if we’re not apart from the center more than a delta distance. If this is not the case, checks if we’re on top of any splitter and if so, change the cursor to the right shape.

  • Save the contents of the panels so we can restore them when we move the splitter(s) (function SaveGraphicContents).
private void SaveGraphicContents()
{
  // save the graphics contents before we move the splitter
  for (int i = 0; i < m_numPanels; i++)
  {
    // save top panels
    using (Graphics graphics = topPanels[i].CreateGraphics())
    {
      topImgs[i] = new Bitmap(topPanels[i].ClientSize.Width, topPanels[i].ClientSize.Height, graphics);
      Graphics g = Graphics.FromImage(topImgs[i]);
      IntPtr hdc = graphics.GetHdc();
      IntPtr memdc = g.GetHdc();
      BitBlt(memdc, 0, 0, topPanels[i].ClientSize.Width, topPanels[i].ClientSize.Height, hdc, 0, 0,
        RasterOperations.SRCCOPY);
      graphics.ReleaseHdc(hdc);
      g.ReleaseHdc(memdc);
    }
    // save bottom panels
    using (Graphics graphics = bottomPanels[i].CreateGraphics())
    {
      bottomImgs[i] = new Bitmap(bottomPanels[i].ClientSize.Width, bottomPanels[i].ClientSize.Height,
        graphics);
      Graphics g = Graphics.FromImage(bottomImgs[i]);
      IntPtr hdc = graphics.GetHdc();
      IntPtr memdc = g.GetHdc();
      BitBlt(memdc, 0, 0, bottomPanels[i].ClientSize.Width, bottomPanels[i].ClientSize.Height, hdc, 0, 0,
        RasterOperations.SRCCOPY);
      graphics.ReleaseHdc(hdc);
      g.ReleaseHdc(memdc);
    }
  }
}

Here a bitmap is created for each panel and the panel’s surface is copied to the bitmap through the function BitBlt. Notice that this function is not available in .Net so we have to access it through the unmanaged dll gdi32.dll using DllImport.

  • Still in the MouseDown handlers, we hide the child controls of the panels. This is done because if we didn’t hide the controls, the shadow splitters would get drawn under the controls and we wouldn’t see the shadow splitters (Thanks to David Júlio):
private void ShowHidePanelContents(bool show)
{
  for (int i = 0; i < MAX_NUM_PANELS; i++)
  {
    foreach(Control c in m_topPanels[i].Controls)
    {
      c.Visible = show;
    }
    foreach (Control c in m_bottomPanels[i].Controls)
    {
      c.Visible = show;
    }
  }
}

This function goes through every panel in the splitter control and hide all the controls inside each panel. It’s also used to show the controls once we’re done with the drawing.

  • The MouseMove handler is the one that draws the splitters in the right place when we’re moving them. First the old panel’s images are drawn and then auxiliary functions draw a shadow splitter (it’s just a rectangle with a hatchbrush at 50%) in the right place.
  • Finally we have the MouseUp handler that finishes the movement by setting the splitters and their new position, shows the child controls again and refreshes/redraws all panels and, consequently, their child controls.
  • The MouseEnter and MouseLeave handlers only change the shape of the cursor if necessary.

THE DESIGNER

Since we want to be able to add controls to each panel, we have to make those panels available for the designer. First we create a new control designer and then we open the access to the panels by calling the EnableDesignMode function on each panel.

public class FourWaySplitterDesigner : ControlDesigner
{
  public override void Initialize(IComponent comp)
  {
    base.Initialize(comp);
    FourWaySplitter usercontrol = (FourWaySplitter)comp;
    EnableDesignMode(usercontrol.TopLeftPanel, "TopLeftPanel");
    EnableDesignMode(usercontrol.TopRightPanel, "TopRightPanel");
    EnableDesignMode(usercontrol.BottomLeftPanel, "BottomLeftPanel");
    EnableDesignMode(usercontrol.BottomRightPanel, "BottomRightPanel");
  }
}

And that’s it! Pretty simple uh?! :)
Keep in mind that with time, I’ll add more functionalities to this control, most of them will be improve its behaviour in the VS designer so, keep checking this page for updates.

You’re free to use this code as long as you don’t blame me for any changes/bugs that were introduced and don’t demand anything from me. Just download it here if you accept these conditions.

Any bugs/questions/comments/suggestions are always welcome. Enjoy! ;)

Changelog:

v0.1.2

  • Changed the way how we handle the OnMouseDown and OnMouseUp events. (Thanks to Hans Passant.)
  • Now shadow splitters are drawn even if the panels in the control contain any controls inside them. (Thanks to David Júlio.)
  • Fixed bug in the designer that didn’t add other controls to the splitter panels.

References:

http://msdn.microsoft.com – MSDN Home Page

Managed DirectX Tutorial 3 – Cameras (Part 1)

VIEWPORT’S CAMERAS

In this tutorial I’ll explain how you can setup a scene with different cameras for each viewport. This will be a very short tutorial but in the following ones we’ll improve our camera system.
So, how does these cameras work in editors like 3D Studio Max? Just like in the movies, you have a scene (your 3D scene, in this case I used a cube) surrounded by cameras and each one of these cameras is filming your scene and presenting the result in different places (viewports).

A SIMPLE MESH

Before we setup the cameras, we need something to show so I decided to create a simple mesh, a cube (I used the sample that comes in the DirectX documentation). A mesh is just a bunch of lines, triangles, polygons that define the basic structure of a 3D model.

m_mesh = new Mesh(numberVerts * 3, numberVerts, MeshFlags.Managed,
  CustomVertex.PositionColored.Format, m_device);

In the creation of the mesh we indicate:

  • the number of faces, since it’s a cube, it has 24 faces/tris.
  • the number of vertices, a cube has 8 vertices.
  • some mesh flags, we indicate that vertex and index buffers are managed by DirectX.
  • the format of the vertices, our contains position and color information.
  • finally we indicate the device.

Now two important things here are the vertex and index buffers. A vertex buffer is nothing more than a collection (an array) of vertices and an index buffer stores how these vertices are used to draw a triangle. This can be very useful to save space when we render something and in this case you can see clearly how we can save space, just imagine that you have to send the vertices for every triangle in the cube, you’d have to repeat each vertex several times since the vertices are shared by different triangles. This way we send all the vertices we need then we just have to indicate how we build a triangle from these vertices.

short[] indices =
{
  0,1,2, // Front Face
  1,3,2, // Front Face
  4,5,6, // Back Face
  6,5,7, // Back Face
  0,5,4, // Top Face
  0,2,5, // Top Face
  1,6,7, // Bottom Face
  1,7,3, // Bottom Face
  0,6,1, // Left Face
  4,6,0, // Left Face
  2,3,7, // Right Face
  5,2,7 // Right Face
};

using (VertexBuffer vb = m_mesh.VertexBuffer)
{
  GraphicsStream data = vb.Lock(0, 0, LockFlags.None);

  data.Write(new CustomVertex.PositionColored(-5.0f, 5.0f, 5.0f, Color.Red.ToArgb()));
  data.Write(new CustomVertex.PositionColored(-5.0f, -5.0f, 5.0f, Color.Green.ToArgb()));
  data.Write(new CustomVertex.PositionColored(5.0f, 5.0f, 5.0f, Color.Red.ToArgb()));
  data.Write(new CustomVertex.PositionColored(5.0f, -5.0f, 5.0f, Color.Green.ToArgb()));
  data.Write(new CustomVertex.PositionColored(-5.0f, 5.0f, -5.0f, Color.Blue.ToArgb()));
  data.Write(new CustomVertex.PositionColored(5.0f, 5.0f, -5.0f, Color.Blue.ToArgb()));
  data.Write(new CustomVertex.PositionColored(-5.0f, -5.0f, -5.0f, Color.Yellow.ToArgb()));
  data.Write(new CustomVertex.PositionColored(5.0f, -5.0f, -5.0f, Color.Yellow.ToArgb()));

  vb.Unlock();
}

using (IndexBuffer ib = m_mesh.IndexBuffer)
{
  ib.SetData(indices, 0, LockFlags.None);
}

Here we set the mesh vertex and index buffers. Check the indices variable, it contains the indices of the vertices that build up a triangle of the cube. When we add a vertex, we indicate a color so if we’re indicating different colors for each vertex, what will be the color of a triangle? DirectX does a smooth transition between the vertex colors. I added different colors so we can easily see that the cameras are pointing to different sides of the cube.

VIEWPORT SETUP

To support different cameras in each viewport, some minor things were modified in our viewport. If you remember, in the last tutorial we saved the swap chain in the tag field of the viewport, but now, we have also the camera to save in each viewport:

class ViewportInfo
{
  public SwapChain swapChain;
  public CameraInfo camInfo;

  public ViewportInfo()
  {
  }

  public ViewportInfo(SwapChain nswapChain, CameraInfo ncamInfo)
  {
    swapChain = nswapChain;
    camInfo = ncamInfo;
  }
}

In the previous tutorial we already had the swap chain, but now we add a new attribute, the camera info. This CameraInfo, for now, will only save simple data to identify our viewport’s cameras.

class CameraInfo
{
  public Vector3 position;
  public Vector3 lookAt;
  public Vector3 up;

  public bool ortho;
  public float scale;
  public float zNearPlane;
  public float zFarPlane;

  public CameraInfo(Vector3 nposition, Vector3 nlookAt, Vector3 nup, bool northo)
  {
    position = nposition;
    lookAt = nlookAt;
    up = nup;
    ortho = northo;

    zNearPlane = 1f;
    zFarPlane = 100f;
    scale = 0.2f;
  }
}

These structures may (will, almost for sure) suffer changes in future, to accommodate new features.

CAMERA SETUP

How do we setup these cameras? Easy, just setup the look at and projection matrices.
What are these matrices? The look at matrix is also known as view matrix and this matrix transforms our world points into camera space, that is, all objects are relocated around our camera point. The projection matrix is another transformation matrix used to project three dimensional points (from the camera space) onto a two dimensional plane (our computer screen). I’m not going to enter in detail with these matrices, if you want to see how they work or how they’re built, check the DirectX documentation and search for Transformations.

Let’s see how we set these matrices in DirectX:

if (camInfo.ortho)
{
  m_device.Transform.Projection = Matrix.OrthoLH(viewport.Width * camInfo.scale,
    viewport.Height * camInfo.scale, camInfo.zNearPlane, camInfo.zFarPlane);

  m_device.Transform.View = Matrix.LookAtLH(camInfo.position, camInfo.lookAt, camInfo.up);
}
else
{
  m_device.Transform.Projection = Matrix.PerspectiveFovLH((float)Math.PI / 4,
    (float)viewport.Width / (float)viewport.Height, camInfo.zNearPlane, camInfo.zFarPlane);

  m_device.Transform.View = Matrix.LookAtLH(camInfo.position, camInfo.lookAt, camInfo.up);
}

We use two types of cameras, the orthogonal and the perspective.

Orthogonal:

  • Orthogonal cameras map the objects into a 2D view.
  • We create an orthogonal matrix using Matrix.OrthoLH (LH stands for left handed, DirectX uses left handed system) and set the projection matrix. Why is that scale multiplication by width and height? Because the width and height of the viewport are very large when compared to the size of the object, so we zoom a bit.
  • Create a look at matrix with Matrix.LookAtLH and set the view matrix. We indicate our camera’s position, the poitnt where we’re looking at and the camera’s up vector.

Perspective/Projection:

  • These cameras map the objects into a 3D view.
  • Create a perspective matrix from Matrix.PerspectiveFovLH and set the projection matrix. We define a FOV of 90º (degrees), the viewport aspect ratio width/height and the near and far planes that define the boundaries of what we can see (we can see everything that’s between these planes.)
  • Set the view matrix like in orthogonal cameras.

OTHER CHANGES

Viewport has been improved and I decided to add a title to know which camera we have in each viewport. Another thing that was missing in the viewport was the ability to see it rendered in the designer, when used in our Four way splitter control and this has also been fixed. Some other minor improvements here and there were done.

And we reached the end of another tutorial. I’ll add the C++/CLI code shortly (only C# for now), and other updates I might do to the code. Next time we’ll continue with cameras to help us move through the scenario. Feel free to contact me about anything wrong or good with the tutorial, you have my e-mail at the end of the page. See ya till next time! ;)

managed_directx3

Download Files:

C# version
You’re free to use this code as long as you don’t blame me for any changes/bugs that were introduced and don’t demand anything from me. Only download it if you accept these conditions.

References:

http://msdn.microsoft.com/directx/ – Microsoft DirectX Development Center
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_m/directx/directx9m.asp – DirectX 9.0 for Managed Code
http://forums.microsoft.com/ – MSDN Forums

Previous Tutorial

Managed DirectX Tutorial 2 – Introduction to swap chains: Rendering to two panels at the same time

VIEWPORTS AND SWAP CHAINS

In this tutorial I’ll explain how you can use swap chains to create different views of the same scene. This tutorial should be fairly simple for you to understand and follow.

A swap chain is basically the process that DirectX uses when presenting a scene to the user. It draws the scene to an off-screen buffer (back buffer) and when it’s ready, it presents the image to the user swapping the back buffer (that contains the current frame) with the front buffer (that contains the last frame). Many swap chains can be created depending on the memory you have available. These are useful when you want to render a scene in more than one place. Think of an editor or a 3D program like 3D Studio Max, it has different views (using different cameras and projections) of the same scene. By default, DirectX creates one swap chain whenever you create the device.
The creation of a new swap chain is very easy. Let’s see what’s needed:

SwapChain sc = new SwapChain(device, presentParams);

All we need is the device a present parameters structure. So, if we only need these two things to create a new swap chain, how do we indicate to where we will render it? When creating a new device we indicate the handle to the control where we’re going to render our scene but in the case of swap chains we already have the device created, so, we use a field in the present parameters structure to indicate the handle:

presentParams.Window = panel.Handle;

4 WAY SPLIT VIEWPORTS

Now let’s draw something in 4 different viewports. I built a new control for this purpose, the 4 way splitter. To continue we need to take some special measures to assure that our viewports are rendered/updated correctly. What happens if we resize one or more viewports? we need to create new swap chains since viewport’s area has changed. I created a new control to help us on this task, the viewport control. This is a very simple control (for now, we might add some stuff to it later), it has two function callbacks: one for the render and one for the creation of the swap chain. This two callbacks are called whenever the control is painted or resized, respectively.

Viewport code

public delegate void RenderFuncDelegate(Viewport viewport);
private RenderFuncDelegate m_renderFunc;

public delegate void SwapChainFuncDelegate(Viewport viewport);
private SwapChainFuncDelegate m_swapChainFunc;

public RenderFuncDelegate RenderFunc
{
	[Browsable(false)]
	set { m_renderFunc = value; }
}

public SwapChainFuncDelegate SwapChainFunc
{
	[Browsable(false)]
	set { m_swapChainFunc = value; }
}

protected override void OnPaint(PaintEventArgs e)
{
	// render our scene
	if (m_renderFunc != null)
		m_renderFunc(this);
}

protected override void OnResize(EventArgs e)
{
	base.OnResize(e);

	// create new swap chain
	if (m_swapChainFunc != null)
		m_swapChainFunc(this);
}

Our main window is composed by a four way splitter with four viewports in each corner. We do some initializations with the device (see managed directx tutorial (part 1)) and create our swap chain and render functions so they can process requests for the viewports:

public void InitializeViewports()
{
  // save our viewports in an array
  m_viewports[0] = viewportTopLeft;
  m_viewports[1] = viewportTopRight;
  m_viewports[2] = viewportBottomLeft;
  m_viewports[3] = viewportBottomRight;

  // for each viewport register the delegate functions
  foreach(Viewport viewport in m_viewports)
  {
    viewport.SwapChainFunc = new WindowsControls.Viewport.SwapChainFuncDelegate(CreateSwapChain);
    viewport.RenderFunc = new WindowsControls.Viewport.RenderFuncDelegate(Render);
    CreateSwapChain(viewport);
  }
}

First we initialize an array with our viewports and for each viewport we associated the delegate functions that will handle the creation of the swap chain and render of the screen. Next our two delegate functions:

private void CreateSwapChain(Viewport viewport)
{
  // get the swap chain of the viewport
  SwapChain swapChain = ((SwapChain)viewport.Tag);

  // if we already have a swap chain created, get rid of it
  if(swapChain != null)
  {
    swapChain.Dispose();
  }

  PresentParameters presentParams = m_presentParams;
  presentParams.BackBufferWidth = viewport.Width;
  presentParams.BackBufferHeight = viewport.Height;

  // check if our buffer is valid
  if(presentParams.BackBufferWidth == 0 || presentParams.BackBufferHeight == 0)
  {
    return;
  }

  presentParams.DeviceWindowHandle = viewport.Handle;

  // save the swap chain in the viewport
  viewport.Tag = new SwapChain(m_device, m_presentParams);
}

private void Render(Viewport viewport)
{
  // get the swap chain of the viewport
  SwapChain swapChain = ((SwapChain)viewport.Tag);

  // safety check
  if (swapChain == null)
    return;

  // retrieve it's back buffer surface where we do the render
  Surface pBackBuffer = swapChain.GetBackBuffer(0);

  // set it as our current back buffer
  m_device.SetRenderTarget(0, pBackBuffer);

  // clear our back buffer
  m_device.Clear(ClearFlags.Target, Color.DarkSlateBlue, 1.0f, 0);

  // we don't need the surface anymore, get rid of it
  pBackBuffer.Dispose();

  // swap the screen buffer and back buffer to present our new data
  swapChain.Present();
}

And this is the end of another tutorial. I’ll add the C++/CLI code shortly (only C# for now), and other updates I might do to the code. Next time I’ll show you… well, I don’t know what I’ll do so stay tuned ;) Your feedback is appreciated, you can leave comments at the end of the page. See ya till next time! ;)

Download Files:

C# version
You’re free to use this code as long as you don’t blame me for any changes/bugs that were introduced and don’t demand anything from me. Only download it if you accept these conditions.

References:

http://msdn.microsoft.com/directx/ – Microsoft DirectX Development Center
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_m/directx/directx9m.asp – DirectX 9.0 for Managed Code
http://forums.microsoft.com/ – MSDN Forums