Universal Controls

Delivering an ecosystem of interacting controls that embody usability, functionality and aesthetics.

A unified OS, Windows 10 had been announced and the next device to join the new OS was Xbox. Our team’s task was to deliver a set of controls that by default would preserve the look and feel of Xbox but behind the scenes, utilize the richness and power of Windows 10. This way developers making apps for Xbox would have to put in minimum effort in styling and customizing. It was no easy task yet it was incredibly challenging and fulfilling. This is by far the most complex project I have worked on till date.

The initial immersion phase started by listening & visualizing. We would have representatives from engineering, PM, designers from Shell teams from both Xbox and Windows as well as numerous experts give us their thoughts and feedback. We met every day at 9am for 1-2 hrs, capturing and constructing what would later become the framework of our approach to this wicked problem. We would scratch and mark up the mistakes/new ideas and reprint the designs for a next day critique.

The complexity of what we were dealing with is illustrated in this diagram. Windows 10 offers a wide array of input support that no other OS can remotely match. But supporting such an extensive set has direct implications on user behaviors as it influences the interaction, visual and motion of each control. I created this chart as a way to synthesize & visualize this challenge (complex matrix) before I could exploit its power.

Here is a collection of some of the wires that were made during this exercise. These sheets are humongous as they were designed to be printed on a plotter. Each proposal is laid across different devices in order to understand the concept of being “universal”. Even though, we were designing only for Xbox, we took the time to understand how the controls work for phone to PC. We also took upon ourselves of becoming experts to be able to predict the challenges that Microsoft would face for Surface Hub, HoloLens, Wearables and IoT devices. We were literally drinking from the firehose.

With time we crafted our story and value prop. Imagine these controls as building blocks. The building blocks come together to make patterns. One can then create pages which are essentially a collection of patterns. With minimum effort, developers can quickly build their apps. Since the defaults look great and give some styling options, devs can use the given or can take it to the next level by customizing them. Ultimately, when you build for one device, its easy to scale to others as Windows takes care of the background magic. Our team was also responsible for delivering this message at //build2015, Microsoft’s annual developer conference.

Xbox was inheriting about 46 controls. Some of these were new to the platform while others were not even in the mix, like the xbox single line search. Here’s a control cloud showcasing all the controls that were worked upon.

The following illustration shows the breadth of work undertaken. Each control was audited across its XAML and WINJS versions. First part apps on Xbox are built in XAML while all 3rd party apps were in WINJS. The Scale Factor decides the magnification of the screen and the first step was to choose an appropriate Scale Factor for Xbox. We then needed to understand the concept of Effective Pixels (epx) and how it would affect the existing 4px grid on Xbox. At the very minimum, each control had to work across the following inputs: mouse, keyboard, touch, game controller, media remote and chatpad. We then looked at colors, motion and sound. This included supporting TV safe colors, support for light and dark themes as well as a high contrast mode for accessibility. None of which was on Xbox before. Motion curves needed to be optimized for 10 feet and sounds needed to be added. At the time, controls for 2ft experience did not have any sound support. We also looked at the 10 feet type ramp and how it differed from its 2ft counterpart. Lastly, great effort was taken to preserve the Xbox focus state and allow the code to support it.

Since all the information was baked into the control, it was difficult to stylize them. We proposed a new framework that would support great defaults while at the same time allow developers to easily customize these. Concurrently, Xbox was evolving it’s UI and we carefully needed to study the Panama look. We also needed to support our very own voice mode which was unique to Xbox. Every interaction state of every control was audited across interaction, visual and motion. On the left, you see what Windows had. On the right was Xbox’s new look Panama. And in the middle, was the happy medium- the default.

One of the most complex controls is the Media Player or the Media Transport Control. It’s a collection of smaller controls and is supports an incredible amount of modes and permutations. It also changes based on the device. Our team took the ownership of this control and designed it for the phone, tablet, PC and Xbox. Here are some shots of the MTC both across interaction and visual.

If you have an Xbox, then you are interacting and experiencing the work we did here. 😉