Definition
Mixed reality is a term for devices that display 3D content through a dedicated headset. By definition, this term is a mix between augmented reality and virtual reality. Although it is closer to augmented reality, it is different because of the technology it uses.
According to our experts at Artefacto, the main difference between augmented reality and mixed reality is the ability of the headset to take into account real space, thanks to the sensors integrated into the headset. The user's position is also calculated in real time. This is an enhanced version of augmented reality. In addition, the set allows the user to physically interact with the 3D elements through a series of gestures.
More simply, a layer of virtual is placed on top of the real.
"The virtual embraces the real like a second skin only visible to the wearer of a mixed reality headset". - Stéphane Chaplain, Technical lead R&D at Artefacto
The history of mixed reality
Unlike augmented and virtual reality, mixed reality is not an old technology. The term itself was coined by Paul Milgramm and Fumio Kishino in 1994 in their work on the concept of the virtuality continuum. At the time, the term was used to describe everything that falls between augmented and virtual reality, but which does not precisely fall into these two categories.
More recently, it was Microsoft, which, at the time of the release of its Hololens headset on 21 January 2015, used this term to define its headset. Indeed, as it does not fit into virtual reality and does not meet the exact definition of augmented reality, it was categorised as mixed reality.
Mixed reality: a hybrid technology
This new kind of headset was developed in collaboration with NASA, which itself had already produced a few prototypes in the 1990s. According to Microsoft, the Hololens is defined as an "autonomous holographic computer". The device contains a series of processors and several sensors that can accurately detect the user's movements, space and position. The 3D remains fixed and takes into account the layout of the room. The helmet is completely autonomous and does not require any connection.
Warning: It is important not to confuse the term "mixed reality" used for some virtual reality headsets. They are so called because they are developed on the "Windows Mixed Reality" platform but are much closer to the Oculus Rift and the HTC Vive. Indeed, they are totally closed and do not superimpose virtual on real.
Today mixed reality is used to refer to all devices that are similar to the Microsoft Hololens headset, as is the case with the Meta headset, its main competitor on the market. The Meta mixed reality headset differs in that it needs to be connected to a PC. In return, it offers a better viewing angle.
A tool for professionals
During the first demonstration of the Hololens, Microsoft illustrated the capabilities of its headset with video games such as Minecraft, or by showing the Windows interface in a 3D that perfectly matches the space in a room. It was therefore initially thought that mixed reality would be deployed in homes and become a consumer tool for accessing applications with a virtual overlay.
But with a price tag of several thousand euros per helmet, it was the professional market that was targeted, at least initially.
Application purposes have marked the orientation of mixed reality. Indeed, professionals very quickly realised the benefits that this type of tool could provide, particularly in the industrial, medical and retail sectors. Microsoft then jumped into the breach by proposing development partnerships with international firms such as Volvo, Ford and Boeing.
Industrialists, the first to equip themselves with mixed reality
Industry was the first to embrace mixed reality. In fact, it was Microsoft itself that took a professional approach by optimising its headset for industrial use. Today, multinationals have adopted it in their processes. The possibility of displaying 3D information superimposed on equipment while allowing the operator total freedom of movement quickly convinced them.
At Ford, for example, the Hololens is used to replace traditional clay models to provide a complete 3D model that is fully scalable. This is easily updated and allows for rapid expansion of design and engineering tests. This is a real time saver for the teams and allows them to have an evolving model. Engineers and designers enrich the 3D model of the vehicle as the project evolves.
Renault Truck is also experimenting with mixed reality, by integrating it into its quality control for the engines of its vehicles. Equipped with the Hololens, the operator checks the conformity of the engine assembly against the details of each part. This is nothing more and nothing less than a precise visual aid that helps the user to check the quality of the assembly.
Retail and mixed reality go hand in hand
In retail, mixed reality is also slowly taking hold. Like augmented reality, the Hololens headset has the advantage of being able to embed interactive 3D elements. A real catalogue, the headset allows you to recreate the sensation of being in front of the product or project.
This is the project of the car manufacturer Volvo, which is developing an experiment on the Hololens mixed reality headset in collaboration with Microsoft. Ultimately, the aim is to offer customers a different way of choosing their vehicle at the dealership. They will be able to select options, engines and paintwork from the headset. All the choices are reflected in the 3D model of the vehicle displayed in front of him. This tool is part of an experiential marketing approach and places the customer in a universe at the crossroads of the real and the virtual.
Mixed reality in real estate and construction
Taking the uses of augmented reality, it is now mixed reality that is making its entry into the real estate sector. By taking into account space, position and interactivity, the Hololens headset allows viewers to move freely around the model and interact with it. The fact that the device is self-contained and takes the form of glasses makes it more comfortable than a tablet or smartphone.
Mixed reality also lends itself to the visualisation of 3D models in the context of BIM. Indeed, this collaborative digital model of the project is becoming a must in real estate development, project management and construction. Mixed reality headsets are therefore an excellent way of visualising 3D elements such as electrical ducts, pipes and foundations. They are displayed as a virtual overlay on the real construction.
An incredible tool in project development
This use is very similar to what was mentioned above with the Ford case in industry. Hololens is increasingly used in the case of project development. Industry, design or architecture are using mixed reality to create a 3D model evolving. Teams can then interact around a common model and test different solutions by updating the model. The time saved is enormous and the interactivity of the device encourages motivation and creation.
Hololens goes into surgery
Although it is still an experiment, the Hololens mixed reality headset is making its way into hospitals. Like the augmented reality glasses Google Glass, Microsoft's headset was recently used in a surgical operation. A surgeon at the Avicenne Hospital in Paris used the Hololens to perform a shoulder replacement. The device visualised 2D and 3D information such as X-rays, MRI and skin thickness. According to the surgeon, this mixed reality device enabled him to achieve a level of precision that would have been impossible without the headset. Thanks to the success of this experiment, mixed reality is likely to be used in operating theatres to support surgeons.
The future of mixed reality?
Although still expensive and experimental, mixed reality is emerging as a tool of the future for a multitude of sectors. Obviously, not all the application possibilities have been developed yet, but every day brings new mixed reality projects.
As for the technology itself, it is currently limited with a restricted angle of view. We can therefore hope for a tool that will allow us to see a room and its virtual overlay over the entire FOV (Field of View). This is what was presented in the Microsoft videos during the presentation of the Hololens, proof that the Redmond firm still has a lot of ambition for its headset. The latest information indicates that Microsoft is currently working on a new, more powerful version of its Hololens headset.
The Magic Leap case
Recently, it was the headset Magic Leap and its Magic Leap One headset, which made a big splash. After years of spectacular teasing and record-breaking fundraising, the American start-up finally released its mixed reality headset in early August.
It is finally a headset very close to the Hololens that was presented. It is however more powerful concerning the field of vision (FOV) and especially, it is proposed at a lower price than its counterpart. It is currently available for sale in 6 cities in the United States and will probably be marketed in the rest of the world in a second phase.
The mixed reality market is expected to be worth $187 million by 2025 according to Transparency Market Research.
Do you want to develop a project using mixed reality technology? Please contact us!
