NASA announces collaborative project with Microsoft at "HoloLens" device that combines reality and virtual

"Microsoft announced in January 2015"HoloLens"Is a head-mounted display (HMD) that displays 3D images like a hologram by overlaying the 3D image in the real world just by mounting it," VR headset "Oculus Rift"Or"MorpheusIt is a bit different device. Use that HoloLens in outer space "Sidekick"A joint project by Microsoft and NASA was announced.

NASA, Microsoft Collaborate to Bring Science Fiction to Science Fact | NASA
http://www.nasa.gov/press-release/nasa-microsoft-collaborate-to-bring-science-fiction-to-science-fact

The following article explains in detail in what "HoloLens" can do by projecting a hologram in real space.

"HoloLens" demo that projects holograms in reality space is too extensive and must-see level - GIGAZINE


At the start of Sidekick, NASA plans to deliver two HoloLens on SpaceX on 28th June to the International Space Station (ISS) with a SpaceX supply ship. For projects actually being conducted in outer space, "Remote Expert Mode"When"Procedure ModeThere are two operation modes:

In Remote Expert Mode, the video that the astronaut wearing HoloLens watches is streamed by Skype and the ground operator gives instructions in real time. In addition to being able to instruct by voice by the operator, it is also possible to advise by astronauts displaying text in the space that HoloLens sees. Procedure Mode is a mode in which HoloLens is installed and the manual of work is displayed with holographic images in the space the astronauts are working on. According to Procedure Mode, NASA saves time spent on training, there is a possibility that communication may be delayed and it may be useful for difficult tasks.

According to NASA, the two HoloLens delivered to the ISS will test the software and the hardware and will send two more HoloLens to the ISS. After that, functional test and network test will be carried out, and it is expected that the work actually using HoloLens will be started by the end of FY 2015. In addition, you can check the state of the test of the project done beforehand from the following movie.

NASA Testing Project Sidekick - YouTube


The test was done in a zero gravity space made in the aircraft.


An operator who operates the software installed in HoloLens and displayed in space.


This worker is drawing something in space.


Operate the software even with voice control. I am very wondering what is visible to the worker with HoloLens.


Microsoft, which develops HoloLens that can be used in outer space, is a developer event "three days from April 29, 2015"Build 2015"Holographic Academy" which introduces the tool for HoloLens and the method of application development, and carried out a movie containing the appearance. In the course we not only experienced HoloLens but also that we could actually develop applications using prepared code.

Microsoft HoloLens: Holographic Academy and Project Origami - YouTube


Towards developing applications, five courses of "GAZE" "GESTURE" "VOICE" "SPATIAL SOUND" "SPATIAL MAPPING" were introduced in the course.


GAZE is to let the user's eyes be recognized by HoloLens and display the line of sight with a cursor in space.


It is GESTURE to manipulate the objects and software displayed in space with the movements of fingers and hands.


VOICE is a function to control objects displayed in space by voice.


Talk to the floating ball "Drop down!" ... ...


Feeling that the ball falls with stone.


SPATIAL SOUND reproduces sound in space.


The developer here talks with excitement that "When moving in the space left and right, it sounds like the sound is moving to the left and right!"


"SPATIAL MAPPING" is a function to recognize what is in the real space and capture it in the virtual space. One of the developers who took part in the course said, "I experienced HoloLens, but I can hardly believe it is a reality, it was such a shock," said the impression.


Microsoft is a game trade fair held in June 2015 "E3 2015Supported by HoloLensMinecraft DemoAnd you can feel the new possibility of the game from the demo. However, narrowness of vision was pointed out by users who actually experienced HoloLens,Microsoft has no plans to improve its vision... apparently ...