Windows Phone and Microsoft Research Hawaii

Microsoft Research is a part of Microsoft dedicated to conducting research in computer science and software engineering, collaborating with the world’s best academic, government and industrial researchers. Existence of such a program enables innovation, an important part of Microsoft products, to help solve some of the toughest problems in the world and improve technology and the world in the years to come. It’s not rare that they work on ideas and projects which seem many years ahead of us, and sometimes even impossible because of the technological limitations we’re facing today. But many projects can be used today and unfortunately don’t get the attention they deserve. One of those genius projects is the project Hawaii, which works great with Windows Phone. What is the project Hawaii?

Hawaii is a project which enables students, developers, researchers and academic community developing Windows Phone applications and research based on cloud technology using innovative cloud services and Windows Azure. It’s a student focused initiative for exploration of how cloud based services can be used to enhance the mobile experience. The goal of the project is to foster the creation of a set of cloud enabled mobile applications and associated cloud services to help them understand the systems and networking infrastructure needed to create the next generation of applications.

It’s based on the fact that the smartphones we use today are getting more and more powerful, affordable to people and acceptable for both professional and private use. Also, because of the improvements made to wireless technology, we live in increasingly connected world which lets people use cloud services as vital part of applications. I find the following three services the most interesting:

  • Relay service – mobile service providers often don’t give static IPs but dynamic ones to their users, so there’s no way for devices to connect and communicate directly. Relay service provides a relay point in the cloud that mobile applications can use to communicate.
  • OCR – Optical Character Recognition – uses a photo that the user has on the device to find the text on it and returns that text to the application for further use
  • Speech to Text service takes a spoken phrase and returns text (works only with English at the moment)

Other services include

  • Rendezvous service – a mapping service from well-known human-readable names to endpoints in the Hawaii Relay Service. Those names may be used as stable rendezvous points that can be compiled into apps
  • Computation – uses Windows Azure to provide compute servers in the cloud
  • Storage – uses Windows Azure to provide virtual SQL databases in the cloud
  • Mapping – uses Virtual Earth to provide maps for given latitude and longitude coordinates
  • Identification – uses Windows Live ID to identify visitors to web sites

In order to understand how to use the services, you can find the detailed instructions on the web site of the project

If you wish to start using the services, you need to download and install the Hawaii SDK

and to be able to develop Windows Phone apps, you need the Windows Phone SDK.

What comes with the Hawaii SDK? You get:

Clients offers a set of client libraries which you use with the Windows Phone applications to be able to use the cloud services. Documentation gives you many how-tos and the documentation in compiled HTML form. Samples bring you the fully working examples showing you how to use the services with fully functional Windows Phone applications. One thing is missing though – it’s your personal Hawaii Application ID which you get through the site Project Hawaii Signup where you use your LiveID to get the Hawaii App ID.

OCR sample

After you get the Hawaii App ID, you’ll probably want to test one of the samples. I chose the OCR sample because I can show you a nice screenshot of the results. When you open the sample, you’ll find the following files and folders in the Solution Explorer.

You add the Hawaii App id in the HawaiiClient.cs class, otherwise the sample won’t work:

public static class HawaiiClient  
        /// <summary>
        /// The Hawaii Application Id.
        /// Please go to,
        /// obtain a Hawaii Application Id and set HawaiiApplicationId to that value.
        /// </summary>
        public const string HawaiiApplicationId = "";

You can either choose to take a new picture (great if you’re testing the sample on the device), or take one from the library (which is what I will do to test on emulator and show you the results).

I chose the lower icon and opened the photo from one of my older blog posts.

The photo gets resized if it’s too big and sent to the cloud. After a moment or two, you get back the text, and even the details about every recognized word.

When you look at the code in MainPage.xaml.cs, you'll notice the two, in my opinion, most important methods called StartOcrConversion and OnOcrCompleted. StartOcrConversion starts the conversion on a photo stream from either the library or camera. OnOcrCompleted gets called when the recognition is completed and returns you the OcrServiceResult object which holds the text in string format. You can use that text in the rest of your application.

There are many research examples of applications, which you can find here. It’s hard to pick personal favourites, but if I had to, those would be Blind Helper, Classroom7, QUEST7 and Illinois Phone.

If you are looking for free services to use with your commercial app, Hawaii project is not for you! It is for you if you are a student, a researcher wanting to use it for research, teaching, and other academic uses. Services are great and well documented, samples easy to understand, and Windows Phone a great platform to build applications for!

About Igor Ralic

Software engineer at Microsoft. Running for Office. Passionate about making an impact with great apps & services. Stays close to coffee and away from coriander. Opinions expressed here are my own.