UVideos Xbox Application

I moved to Canada last summer by the end of July to join Digiflare. As soon as I arrived I was assigned to work as a Lead Developer on my first Xbox project. The project had to be ready for Black Friday (Nov 23rd) so it had to be launched on November the 20th because new Xbox application roll out on Tuesdays.

It was a tough project but we we’re able to deliver successfully on time.

 

You can find more information on the project in this page: http://www.xbox.com/en-US/live/partners/univision

It has been very interesting to learn the insides of a Lakeview application and I’ll keep at it in the near future.

Geo Reminders for WP7

I wanted to build this app since I bought my WP7 but unfortunately it wasn’t possible to have background tasks running on the phone. It was great to know that Mango would bring this functionality to the Windows Phone and I started working on the app right away.

Since I program these WP7 apps outside work it takes a while to get them done. Geo Reminders allows us to program alarms or reminders depending on our location. This is useful to set a reminder for the next time you visit a place for example. Or imagine that you left a book at work and you want to bring it home, you set the reminder that will only go off when you arrive at work so that way you can grab the book and put it near your stuff when you leave.

Here are some screen shots:

georeminders_1georeminders_2georeminders_3

georeminders_4georeminders_5georeminders_6

georeminders_7georeminders_8

The application is available since early March in the marketplace. You can download it here.

Kinect Calibration

This program adjusts Kinect to better see the user, this way the user doesn’t have to move back or forward (unless Kinect’s angles aren’t enough to see the user). I think this is a useful thing to have when you start your Kinect application.

The code is very simple to follow. Use Kinect’s elevation angle to change the Kinect’s “view” and track the user in sight. For each angle count how many joints Kinect is tracking and save that angle as the best if we reach a new maximum number of tracked joints.

void KinectSkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
{
	// Get skeleton information
	using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
	{
		if (skeletonFrame != null && m_skeletonData != null)
		{
			skeletonFrame.CopySkeletonDataTo(m_skeletonData);
		}
	}

	m_curTime = m_watch.ElapsedMilliseconds;

	if (m_calibrationState == CalibrateState.GoingDown)
	{
		// If the sensor reach its lowest angle, let's do a full scan from min angle to max angle.
		if (m_curTime >= WaitTime)
		{
			m_watch.Reset();
			m_watch.Start();

			m_bestAngle = m_kinect.MinElevationAngle;
			m_maxNumTracked = 0;

			m_calibrationState = CalibrateState.GoingUp;
			m_kinect.ElevationAngle = m_angles[m_curAngleIndex++];
		}
	}
	else if (m_calibrationState == CalibrateState.GoingUp)
	{
		if (m_curTime >= WaitTimeGoingUp)
		{
			m_watch.Reset();
			m_watch.Start();

			// If we scanned all the angles, lets adjust kinect to the best angle.
			if (m_curAngleIndex > m_angles.Length - 1)
			{
				m_calibrationState = CalibrateState.GoingBest;
				m_kinect.ElevationAngle = m_bestAngle;
				return;
			}

			m_kinect.ElevationAngle = m_angles[m_curAngleIndex++];
		}

		// For each skeleton, count the number of tracked joints and save the best
		// angle when the number of tracked joints is greater than previous values.
		foreach (Skeleton skeleton in m_skeletonData)
		{
			if (skeleton == null)
				continue;

			if (skeleton.TrackingState == SkeletonTrackingState.Tracked)
			{
				//TODO: Improve algorithm by using the number of inferred joints.
				int numTracked = 0;
				int numInferred = 0;
				foreach (Joint joint in skeleton.Joints)
				{
					if (joint.TrackingState == JointTrackingState.Tracked)
						numTracked++;
					else if (joint.TrackingState == JointTrackingState.Inferred)
						numInferred++;
				}

				if (numTracked >= m_maxNumTracked)
				{
					m_maxNumTracked = numTracked;
					m_bestAngle = m_kinect.ElevationAngle;
				}
			}
		}
	}
	else if (m_calibrationState == CalibrateState.GoingBest)
	{
		// Just wait until kinect adjusts itself to match the best angle.
		if (m_curTime >= WaitTime)
		{
			m_watch.Reset();

			m_calibrationState = CalibrateState.Idle;
			m_kinect.SkeletonFrameReady -= KinectSkeletonFrameReady;

			// Reset Kinect state.
			DisableStreams();
			EnableStreams();

			// Signal that we finished the calibration.
			if (OnCalibrationComplete != null)
				OnCalibrationComplete();
		}
	}
}

You may notice two things about the code:

  • First is that I’m using time to control between Kinect movements. The reason for this is that if we try to compare Kinect with the angle we set, we might not get the same value for different reasons (Kinect sensor is not 100% accurate or it couldn’t physically rotate the sensor to that exact angle).
  • And second is the fact that I’m only scanning the skeleton for some angles. I don’t know if this is a Kinect’s limitation or if I did something wrong in the code but I couldn’t track any joints while Kinect was moving. So what I did was move the sensor between some angles and wait there a bit to count the bones for each one of those angles.

Known limitations:

  • it only supports 1 user at this moment;
  • the scanning process is not ideal.

Download:

Kinect Calibration (13Kb)

Kinect Slide Tutorial

Part of my Kinect Portfolio presentation had slides controlled by gestures. This short tutorial explains how I made it work. As I said in the other post, I used a Gesture Library to help with the gesture recognition, Kinect SDK Dynamic Time Warping (DTW) Gesture Recognition. This is a great project that is available on CodePlex and uses a cool algorithm called Dynamic Time Warping which is pretty cool for pattern recognition such as gestures or voice, among others. If you’re curious like me and want to know how this algorithm works, take a look at this link.

But onto the slide tutorial. I started by taking the core files to handle the gesture recognition and plug it in with my project. After a couple of little changes mostly to call an event handler when a gesture is recognized and to work with the final Kinect SDK 1.0., I used the original program to generate the gestures file which contains the information needed to detect gestures. I actually recorded two different gestures for each “SlideLeft” and “SlideRight” gesture just to make sure that different gestures done by different people would be caught by the algorithm.

The next step was to figure out how to animate the slides. I didn’t have experience with WPF animations but I arrived at this solution which works but might not be the best.

VisualStates and their transitions

I created 3 visual states to handle the slide left and slide right: previous (VisualState 1), current (VisualState 2) and next (VisualState 3). There are 2 visual transitions (left and right) between each 2 states and every time we change from one visual state to another we update the “next” target with new slides.

To automatize the transitions I decided to create a SlideManager that starts by reading all the “slides” (Grid elements) under a parent element and initializing the visual states by adding the event handlers. I created a simple structure to keep track of slides and transitions between slides because initially I wanted to include vertical transitions by ended up not doing them.

// Initialize slides.
for (int i = 0; i < mainSlideContainer.Children.Count; i++)
{
	Grid slide = mainSlideContainer.Children[i] as Grid;
	if(slide == null)
		continue;

	Slide s = new Slide()
				            {
				                Name = slide.Name,
				                GUIElement = slide,
				            };

	m_slides.Add(s);
}

for (int i = 0; i < m_slides.Count; i++)
{
	Slide nSlide = null;
	Slide pSlide = null;

	if (i < m_slides.Count - 1)
		nSlide = m_slides[i + 1];
 	if (i > 0)
		pSlide = m_slides[i];

	Slide s = m_slides[i];
	s.Transitions = new Dictionary();
	if(nSlide != null)
		s.Transitions.Add(Transition.RightToLeft, nSlide);
	if (pSlide != null)
		s.Transitions.Add(Transition.LeftToRight, pSlide);
}

There are two other important methods that change slides: MoveToNextSlide() and MoveToPrevSlide(). Each method checks if the current slide has the transition we want to perform, updates the current slide and visual state attribute members and calls the method that actually performs the transition.

public void MoveToNextSlide()
{
	if(m_inTransition)
		return;

	Slide nextSlide;
	if (m_curSlide.Transitions.TryGetValue(Transition.RightToLeft, out nextSlide))
	{
		m_prevSlideIndex = m_curSlideIndex;

		m_curSlideIndex = Math.Min(m_curSlideIndex + 1, m_slides.Count - 1);
		m_curSlide = m_slides[m_curSlideIndex];
		m_curVSIndex = (m_curVSIndex + 1) % m_visualStates.Count;
		m_curVS = m_visualStates[m_curVSIndex];

		m_inTransition = true;
		ExtendedVisualStateManager.GoToElementState(m_vsManagerContainer, m_curVS.Name, true);
	}
}

Another part of the code that is worth mentioning is the two event handlers at the end of this manager: Storyboard_Completed and VSGCurrentStateChanged.

private void Storyboard_Completed(object sender, EventArgs e)
{
	m_slides[m_curSlideIndex].GUIElement.Opacity = 1;
	m_slides[m_curSlideIndex].GUIElement.Visibility = Visibility.Visible;

	if (m_prevSlideIndex != m_curSlideIndex)
	{
		m_slides[m_prevSlideIndex].GUIElement.Opacity = 0;
		m_slides[m_prevSlideIndex].GUIElement.Visibility = Visibility.Collapsed;
	}
}

The visual state transitions do the slide movement and the storyboard controls the opacity and visibility of the element in that state. Whenever we change from one state to the other we have to reset the opacity and visibility properties so that the next time we do the transitions the new slides have the correct properties.

private void VSGCurrentStateChanged(object sender, VisualStateChangedEventArgs e)
{
	VisualState nvs = m_curVS;

	// update the next visual state time line (only if necessary)
	if (m_curSlideIndex < m_slides.Count - 1)
	{
		Timeline tl = FindObjectByName(nvs.Storyboard.Children, nvs.Name + "_next" + TLOpacName);
		if (tl != null)
			tl.SetValue(Storyboard.TargetNameProperty, m_slides[m_curSlideIndex + 1].Name);

		tl = FindObjectByName(nvs.Storyboard.Children, nvs.Name + "_next" + TLVisName);
		if (tl != null)
			tl.SetValue(Storyboard.TargetNameProperty, m_slides[m_curSlideIndex + 1].Name);
	}

	m_inTransition = false;
}

The same goes for the storyboard target name property, whenever we change to a new state we have to update the next slide. I also used a flag to prevent it from starting a new transition while we’re still doing the animation from the current transition.

The KinectManager is a simple class to handle the Kinect’s initialization that I adapted from the samples. It’s far from complete as it is but it serves well for this tutorial.

You may ask yourself why go through all this trouble when there is PowerPoint and Kinect applications to do the slide animation controls? Firstly, because it is fun to explore and learn new things and secondly because there are parts of the complete interactive portfolio that weren’t so easy to do in PowerPoint such as the particle system or the pong game.

Note: The transitions between texts were programmed in a similar way but instead of using 3 Visual States I only used 2, one for display the text and the other and to hide it, then it’s just a matter of setting the right text when you do the transitions.

Download
Kinect Slide Tutorial (requires: .Net4, Kinect SDK 1.0, VS2010 and up)

Back To The Future as Code

David Amador twitted about this cool site Movies as Code and I decided to make a simple entry for Back To The Future as C++. Here is my entry: http://moviesascode.net/adventure/back-to-the-future-2/

The code doesn’t appear correctly on that site so here it is the full version:

#include <future>
#include <iostream>

int main()
{
  auto future = std::async([] { return "Future"; });
  std::cout << "Back To The " << future.get() << std::endl;
  return 0;
}

For those who didn’t get it: C++11 added a new feature that is used in the sample called “future” that will hold the result of an async function somewhere in the future. It was just to play around with the name of that feature :)

Kinect and WP7 interaction

By the same time I did my portfolio I also did another experiment that’s in this post. A few days ago I saw the KinectoPhone project and thought it would be cool to adapt it to another project I was developing with Francisco Campelo. This project allows a desktop application using Kinect to communicate with another application on the Windows Phone 7 synching its state between the two apps by communicating through a server.

I changed a bit the network code and plugged in with other code I had done for the particles and here’s the result:

Kinect and WP7 interaction

The red particles on the big screen are from Kinect’s skeleton and the blue particles are sent from the WP7 through touch screen input. On the WP7 screen, the blue particles are from Kinect’s skeleton and the red/white ones from the user’s input.

This could get many improvements, network and feature wise but I’ll leave that to some other time. I have many ideas but the time is short so I’ll be moving to another one soon.

THANKS TO

  • My parents for helping me out testing and recording the video.
  • Anabela Faria for the video editing.
  • Francisco Campelo for the original idea which led me to the Kinect version.

CLIArgsMadeEasy2010 Update

I just updated my CLIArgsMadeEasy2010 Visual Studio add-in. It fixes a bug that prevented it from working correctly on solutions that included projects inside project folders.

I’ve also decided to update it with a small feature that saves a lot of time. If you’re like me and have a few add-ins installed, you might have a context menu like mine that fits the whole screen at 1080 and it becomes painful to find the “set as starting project” option. So CLIArgsMadeEasy2010 v1.2 now includes a neat combo box with the current startup project and all the other projects which allow you pick another project as the startup project very easily by choosing it from this combo box.

If you don’t use any of the features you can hide them by clicking in the arrow beside the add-in toolbar box.

Grab the add-in here.

Interactive portfolio using Kinect

I like to explore different technologies that relate with computers and that’s why I own a Lego Mindstorms NXT . More recently I bought a Kinect to transform some ideas into real applications.

I’m currently looking for job in software development in the Toronto (Canada) area and it occurred to me that I could present my portfolio in a fun and entertaining way. So I decided to create my first Kinect application to be this interactive portfolio. Although I have some years of experience in business applications, portals, intranets, etc., I decided to compile only my experience in game development and interactive 3d projects.

If you don’t own a Kinect you can watch the video below to see how it works. I encourage you to skip the video in case you want to try the application or you’ll ruin the experience.

Interactive portfolio

The code uses the following technologies: C#, .NET, XNA and WPF. I also used two open source libraries to help with the gestures and the XNA integration with WPF.

DOWNLOAD

Software requirements:

Kinect application:

Note: Please make sure you install the software requirements or you won’t be able to install the application.

THANKS TO

  • My parents for helping me out recording the video.
  • Anabela Faria for the video editing.
  • Paulo Silva for the animated silhoutte.

Places I’ve Seen for WP7

This summer I decided to build something for WP7 in Silverlight to learn something new. Places I’ve Seen was the result of my learning experience. It was fun to use Silverlight, it’s very easy and the Visual Studio Editor for Silverlight/WPF is amazing (didn’t try Express Blend yet) and the data binding of controls saves a lot of boring work.

Based on Where I’ve Been, Places I’ve Seen allow you to save a list of your visited locations around the globe, as well as a list of locations to visit (like a wish list). There are three ways to add items to your lists:

  • by typing the name and choosing from a list;
  • by picking a image from your library (only works if you have geo tagging on);
  • or by adding the current location using GPS (this only adds to the list of visited places).

In addition I thought it would be cool to view images from each place so I added that functionality that searches images on Flickr from a place of your choosing.

Here are a few screen shots of what it looks like on the phone:

pis_1pis_2pis_3

pis_4pis_5pis_6

pis_7

 

It’s available on zune here.

SilverMenu library for WP7

Inspired by the menus of WP7 game Puzzle Jumble, I decided to write a library that would help me and others to build menus for WP7 games easily. The Puzzle Jumble’s GUI code is a bit of a mess because it wasn’t thought for this kind of interface from the beginning so I decided to write SilverMenu from scratch.

Puzzle Jumble’s GUI

 

SilverMenu aims to bring some of the Silverlight user interface that is now present on WP7 to the XNA. To make it simple to use you can build simple animated menus with just a few lines of XML. It has only one week of development but you can see some progress already:

First SilverMenu sample.

 

With time I hope it grows into something more complete and useful. I’ll still update my blog about this project but if you want to follow the project closely and keep up to date, as well as, download the last build, you can do it at http://silvermenu.codeplex.com. I hope you find it useful and I appreciate all the feedback I can get. Thanks!