Leap Into VR/AR Design

Design thinking in three dimensions.

Virtual reality is no longer in the realm of science fiction. In the last few years, the virtual and augmented reality (VR/AR) market has been heating up with big tech companies like Microsoft, Google, HTC, Facebook, and even Apple jumping in. And the technology is no longer just for gamers but is being used for everything from car sales to corporate training. Forbes predicts VR and AR headset sales will hit $80 million by 2021 and TechCrunch is making a case that AR could become the primary driver of a $108 billion AR/VR market by 2021.

Right now, the VR/AR landscape looks a bit like the wild west. Not many standards or guidelines exist, and many people are still not entirely sure what VR/AR is, and why it matters. For designers, virtual reality design and augmented reality design offers an opportunity to be in on the ground floor of creating and designing an entirely new industry — developing the design standards, rather than following preset rules outlined by someone else.

Since most designers have spent years honing their craft on flat screens: computers, mobile devices, or even print design, it may seem daunting to jump into 3D design. It may be surprising to learn that many of the skills you already have apply to this new frontier.

As with any design project, it’s essential to know and understand as much as possible about the platform, the target audience, and what you (or the client) are trying to achieve. Be realistic. This is a new frontier, so don’t underestimate how long it will take to grasp the technology —– prepare to make mistakes and learn from them.

Getting Started: Immerse Yourself

The best way to get started is to experience VR/AR for yourself if you haven’t already, and if you’ve tried it before, it’s time for total immersion. Since this space is so new, it’s impossible to start designing without getting as much first-hand experience as possible.

For beginners on a budget, start by downloading VR/AR apps from the Google and Apple App stores onto your smartphone or tablet. Many apps are still free, and there’s a pretty wide variety to choose from. Pick up Google Cardboard which is the lowest cost way to start experiencing VR.

If you’re feeling a little more adventurous and have a little more money to throw around, Google Daydream and Samsung Gear VR offer more integrated and higher quality experiences on the lower end.

For those with bigger pocketbooks, Oculus, HTC Vive, and Sony offer higher-end platforms that focus more on gaming. These are not for the faint of heart, and costs can be high, from $949 for a complete PlayStation VR bundle to several thousand dollars for a Vive or Oculus system with a high-end PC. These systems also require quite a bit of set-up and tech savvy.

As you explore, make notes on what works and doesn’t work from a UX/UI standpoint. Make sure you take into account technical limitations and whether the hardware is helping or hindering the user experience. Also, make a note of what elements you’re drawn to, do you like the total immersion of VR or do you prefer the mixed reality of AR? This will help you determine where to spend your time learning.

First Person vs. Flat Design

VR/AR is all about three dimensions, and the user interface is no longer confined to a flat rectangular screen or surface. So, it’s essential to understand spatial design and spatial UI elements a little better.

Types of spatial UI elements:

• Follow-along – UI elements follow the user as they move through the environment and are not fixed. A real-world example of this would be the dashboard of your car.

• World-locked – UI elements have a specifically assigned space within the environment and do not move. In the real world, examples of this are street signs, traffic lights or crosswalk indicators.

Start to think about how design in the real world can apply to the virtual world. Make sure you think about sight lines and font sizes and be prepared to do some rapid prototyping in VR to test your assumptions. Sometimes you can make do with a hotlink on an object for simple interaction; for more complex interactions you’ll need to create dynamic menus with secondary or even tertiary interactions. Gamers are already familiar with many of these concepts and grasp the importance of each type of spatial UI element.

New Inputs, New Challenges

For the first time, users are being asked to interact with a UI without tactile interaction. No more mouse, touchpad, or touchscreen. Although some methods will mimic that tactile interaction, in most cases the user will be interacting with a virtual environment through some unfamiliar inputs, so it’s important to understand how this impacts UI design.

Types of inputs:

• Gesture
• Gaze (eye tracking)
• Voice
• Hardware inputs:
  • Platform specific
  • Touch & click controllers
  • Game controllers
  • Haptic

Each platform or device will have their own unique set of input methods and may have all, or only some of the methods explored above; therefore it’s essential to understand the platform you are designing for.

Back to Basics: The Process of Virtual Design

Every great design begins the same way, no matter what the end product. Once you have a good grasp of the project and goals, start collaborating and brainstorming. Use whatever you currently use for early design concepts: whiteboards, pen and paper, or software. Collect feedback and iterate as possible to refine and narrow the focus before moving on.

After initial concepts are refined, the next step is getting into the mindset of the user by creating user flows. By tracing the path that a user would take through the UI, we’re able to make sure that the path is simple and easy for them to understand.

The final stage is wireframes. Designing the way that the UI will be laid out and accessible to the user.

Start Small: Rapid Prototyping

Don’t be afraid to take sketches or wireframes into rapid VR/AR prototypes at an early stage. It will help you work out any unforeseen issues with your ideas before getting too deep into the development woods.

Some simple tools can be used to test out some of these early concepts quickly:

• Sketch to VR – a plug-in that enables designers to use Sketch to create Google VR enabled demos. An excellent tool for creating and showing early prototypes to stakeholders.
• Vizor – a little more advanced and allows designers to create photo- or 3D scene-based interactive demos. Storyscape walks through how to get started.
• WebVR – an open platform that lets you experience VR in a web browser.

You can also use software such as Sketchup, Blender, and Cinema 4D to mockup and prototype VR/AR interfaces and scenes. Alternatively, if you want to jump right into one of the primary tools for design and development in VR/AR, go directly to Unity and set up a free account. It’s got a bit more of a learning curve, but there are plenty of tutorials out there to get you started.

Visualizing the Virtual

As with traditional UI design, excellent visual design is critical. Apply what you know and don’t try to reinvent the wheel or change your aesthetic too dramatically. Use tools you’re already familiar with to avoid unnecessary learning curves (Adobe CC, Sketch, Framer, etc.).

Keep in mind the device the end user will use to experience your UI. Higher resolution screens and headsets (tablet AR, Oculus, Vive, PlayStation VR) allow for more detailed designs, and smaller, more elegant fonts. Lower end or small screen devices (smartphone VR/AR, Cardboard, Daydream, Gear VR) will need to use larger, thicker fonts and less detailed UI elements.

Keeping It Real

Guide users through your interface using hints, and always provide feedback with each interaction so the user gets oriented quickly. Stick with UX/UI best practices and create clean, simple, user-friendly graphics, allow your users to come to grips with virtual experiences by giving them familiar reference points and visual metaphors.

Believe it or not, after all these years moving away from skeuomorphism in design, VR/AR is bringing it back. Using a gesture to turn a dial or pick up an object feels natural to users. Just ask hardcore gamers, they’ve been interacting with virtual worlds for years.

Jump in Now!

There are no limits to where creative people can go in VR/AR. It’s an open field with loads of new opportunities. It’s rare that designers can get in on the ground floor of something utterly new, and with so many possibilities. Immersive technology desperately needs designers to jump in, get involved, take risks, and make something new.