Spaceflight Insider

Microsoft and JPL create virtual Mars with HoloLens

nasa-microsoft-hololens-mars image posted on SpaceFlight Insider

Microsoft's OnSight software simulates life on Mars. Image Credit: NASA

NASA is partnering with Microsoft Corporation to allow scientists to interact with the planet Mars in a more personal, hands-on way than by simply watching images on a computer screen. OnSight is a software that creates a three dimensional Mars environment using data from the Curiosity rover. Using a wearable device called Microsoft HoloLens, scientists will be able to walk around in the simulated Martian environment and examine it up close.

Dave Lavery, program executive for the Mars Science Laboratory mission at NASA Headquarters in Washington, explained how it works. “OnSight gives our rover scientists the ability to walk around and explore Mars right from their offices. It fundamentally changes our perception of Mars, and how we understand the Mars environment surrounding the rover.”

Dave Lavery. Photo Credit: Matt McClain/The Washington Post

Dave Lavery. Photo Credit: Matt McClain/The Washington Post

OnSight was developed at Pasadena’s Jet Propulsion Laboratory. It uses holographic computing, which blends a camera view of the physical world with computer-generated imagery, to overlay into the user’s field of view not only visual information, but rover data such as distances, readings, and sensor information. When a member of the Curiosity mission team dons the Microsoft HoloLens device, he or she is surrounded by images from the Curiosity rover’s Martian field site. Then they can simply walk around the landscape as if they were actually there.

Jeff Norris, JPL’s OnSight project manager, said, “We believe OnSight will enhance the ways in which we explore Mars and share that journey of exploration with the world….Previously, our Mars explorers have been stuck on one side of a computer screen. This tool gives them the ability to explore the rover’s surroundings much as an Earth geologist would do field work here on our planet.”

In an interview with Mashable, Norris said, “Controlling a robot on another planet is a huge challenge. Trying to understand the environment the rover is on from just a bunch of pictures, doesn’t engage the full ability of a human to understand an environment.”

Dr. Jeff Norris as seen on Spaceflight Insider

Dr. Jeff Norris. Photo Credit:

Norris discussed OnSight in a YouTube video. “What we learned is that fieldwork, being present in an environment on Earth, is of great importance to them because they learn so much just by looking around, just by walking around in the environment that they’re trying to explore. We want to give them some of that same sensation while they’re exploring Mars. Not because it’s a gimmick and not because it’s fun, but because it will help them to reach scientific insight more quickly and more confidently than peering at images on the screen, like they do today.”

Instead of watching two-dimensional views on a computer screen, team members can examine rocks, craters, or other items of interest in a realistic three-dimensional setting before instructing the rover on what to do next. They will be able to do this simply by pointing at an object within the simulation and selecting menu items with hand gestures.

“I think I’m most excited about this because I think this is a new way for us to think about exploration,” Norris said. “We want to physically go everywhere, and I think that’s our destiny, that we will send human explorers throughout our Solar System and beyond. But until we’re able to do that, we’ve sent our robots ahead. By building tools that make us feel more connected to those robots and the environments they’re exploring, we can change our experience of exploration in a very fundamental and exciting way.”

JPL plans to begin testing OnSight with actual Curiosity mission operations by July of 2015. After that, potential uses include the Mars 2020 rover, which is based on the Curiosity design and is slated for launch in 2020.

Video courtesy of NASA


Welcome to SpaceFlight Insider! Be sure to follow us on Facebook: SpaceFlight Insider as well as on Twitter at: @SpaceflightIns






Collin R. Skocik has been captivated by space flight since the maiden flight of space shuttle Columbia in April of 1981. He frequently attends events hosted by the Astronaut Scholarship Foundation, and has met many astronauts in his experiences at Kennedy Space Center. He is a prolific author of science fiction as well as science and space-related articles. In addition to the Voyage Into the Unknown series, he has also written the short story collection The Future Lives!, the science fiction novel Dreams of the Stars, and the disaster novel The Sunburst Fire. His first print sale was Asteroid Eternia in Encounters magazine. When he is not writing, he provides closed-captioning for the hearing impaired. He lives in Atlantic Beach, Florida.

⚠ Commenting Rules

Post Comment

Your email address will not be published. Required fields are marked *