Microsoft’s HoloLens helping NASA get to Mars

New augmented reality software developed by Microsoft is being used by NASA to help out on some of the biggest projects on – and off – the planet…

Microsoft’s new augmented reality system, HoloLens, is helping organisations such as NASA digitally transform the way they collaborate, communicate and manage projects at the International Space Station, as well as helping it develop the next Mars Rover mission scheduled for 2020.

By wearing a headset, users can experience a mixed-reality environment that overlays the real world with digital content, enabling them to interact with holograms and collaborate with other users.

“HoloLens is the starting point of being able to blend physical and digital reality together,” says Leila Martine, director of new devices at Microsoft. “Customers are telling us it is transformational.”

The space agency has been using this mobile device to help with projects at the International Space Station, providing virtual aid to astronauts. Using the system’s Remote Expert Mode, for example, the astronauts can use Skype to allow a ground operator to see what they see, and provide them with realtime guidance on the tasks they are undertaking.

“Astronauts are smart, but they do not know how to do everything,” says Martine. “If they are fixing things up on the International Space Station, they can call down to ground control to a Skype call. Ground control can see what the astronauts are looking at through the HoloLens environment. They then can walk them through how to solve it.”

The ground control staff can “circle” parts that need fixing on the overlay, and the same pen marks will show up on the astronaut’s view of the situation.

When communication difficulties with the station arise, Procedure Mode gives the astronauts direct access to procedures and manuals which may be needed in an operation.

The software’s flexibility is also helping out on with projects on the ground – NASA is employing it in the design of the Mars Rover 2020, where the software is helping to identify potential errors in the system before launch, and is also being used to train staff who will be involved in the launch in three years’ time.

Microsoft is seeing a growing demand for HoloLens from companies undertaking projects in 3D modelling, says Martine. “HoloLens has been a really big starting point for any organisation that has been dealing with 3D modelling.

“Design companies, architecture, engineering and construction were using different technology to figure out efficiencies, but now [they have a] new tool to do this.”

As the holographic projections are transparent, Martine explains, people on a project can physically walk around the design, collaborate about their ideas and get approval from a client on the project without having to be in the same actual space.

For example, technology company Trimble has been using the HoloLens in 3D modelling to help improve collaboration and efficiency in the construction industry.

Trimble’s proprietary SketchUp 3D modelling software has been integrated with HoloLens so architectural models of buildings can be visualised in three dimensions using a hologram. Staff working on a project can look at the design, analyse the structure, share ideas about it and resolve issues in real time.

According to Martine, once a company has started using it for one project and understand how it can help them, they start using it for others. HoloLens also has huge potential in training – Japan Airlines is already using it to train engine crews.

“It is really expensive to train engine staff,” says Martine. “When an airline needs to train mechanics on an engine, they have to wait until a plane is available in a hanger. They then have to fly in maintenance experts from around the world.”

Instead, the HoloLens mechanics can learn the engine structure through the simulation in the hologram, giving them practical experience of fixing it without the expense of waiting until a hangar is available.

“If you can create an environment which is natural and you can have a conversation with someone, even if they are in San Francisco or China like they were in this case, that is a phenomenal advancement in technology,” says Martine.


Get our latest features in your inbox

Join our community of business leaders