How Do Artificial Intelligence and Industrial Extended Reality Work Together?

This year’s Hannover Messe explored the industrial metaverse as a new frontier for manufacturing. Panels like the one where NVIDIA and Siemens discussed “How Artificial Intelligence (AI) and Digital Twins Enable the Industrial Metaverse,” alongside all of the Extended Reality (XR) vendors present, highlighted the importance of AI boosting the combination of the real and digital worlds. Once the metaverse hype was taken over by the AI hype, it was inevitable that people would realize the benefits of partnering AI with a tool that required a lot of data and processing speeds to ensure a seamless and tailored experience. While there seemed to be fewer metaverse booths at this year’s Hannover Messe, the show highlighted the importance of partnering AI with the industrial metaverse: AI is an accelerator for the combination of the real and digital worlds.

AI enables machines and computers to develop and imitate functions associated with cognition. With this in mind, the clear connection between AI and the industrial metaverse lies with its importance in building environments and immersive experiences for users. AI has been transforming the industry, and partnerships between semiconductor manufacturers and industrial innovators, such as NVIDIA and Siemens, have enabled bridging between the real world and the Information Technology (IT) world. In this way, AI can be applied to all things that need to be designed, built, and operated. For example, for seamless remote operations, digital twins need to be comprehensive, behaving in the same way the real twin does, applying real physics to the digital world, updating in real time to mirror the real world accurately, and requiring significant data processing. This is harder to accomplish without AI, and arguably impossible when including technologies like Simultaneous Location and Mapping (SLAM) tracking for XR devices.

When discussing XR and AI, most people start to think about generative AI, used to make avatars or to help build virtual environments. However, there are significantly more applications than just generative AI. Robots and automation have a significant potential value increase paired with AI. To discover the potential for automation, a form of cognition must be a part of it. Yet, to use AI to create this intelligence, an abundance of data is required, which is not always feasible to extract from the real world. It can, however, come from a virtual world, such as the industrial metaverse. By creating environments, scenarios, and events virtually, robots can be programmed and trained before physical installation. With XR, digital representations can facilitate this design and programming, with spatially accurate data allowing for precise and reliable digital design. XR can enable the Three-Dimensional (3D) environment where we train robots, using AI and digital representations, to become reliable and trustworthy; and once proven, they can be deployed in the real world, saving resources with reduced iteration and poor design/programming.

To help facilitate AI as a booster for everything XR, Original Equipment Manufacturers (OEMs) and enterprises need to consider the following:

• Data Security and Privacy: This has been a topic of discussion for as long as AI and the metaverse have been topics of conversation. Separately, these topics have encountered concerns regarding their security and privacy measures, and as AI and generative AI will more readily be adopted within the XR industry, the more efforts need to be made to ensure that data are securely passed and saved. Cloud computing, a promising way to reduce device requirements and cost, presents another potential vulnerability. Continuously invest in advanced security solutions and end-to-end AI data encryption for data preparation, training, and inferencing.
• Continuous Learning: AI models for avatar creations and environment creations require continuous learning in order to evolve and meet the preferences and the needs of users. For this to occur, maintenance and a significant amount of content, data, and resources are required to keep up with demands. Generative AI and AI models can help expand content creation output; without the pairing of AI and XR, meeting the demands of the public will become increasingly difficult as user numbers grow. This is especially true because XR is often advertised for its unique selling point of tailoring experiences to the user’s requirements, necessitating constant and unique content creation.
• Reliability and Performance of Infrastructure: In order to create a seamless user experience, AI plays a crucial role for the infrastructure of XR to enhance a virtual world’s appearance. With tools such as NVIDIA’s GANverse 3D, content creation processes and transformation of photographs into virtual environments is possible. Deep Learning (DL) techniques can achieve real-time processing and high accuracy or 3D objects; for example Meta’s PyTorch3D, a library for DL with 3D data, provides reusable and efficient components for 3D computer vision, and can support the development of XR and virtual environments.

These recommendations are a few of many that will accelerate XR. AI will be a significant booster to key areas in content creation, digital twins, network infrastructure, and end-user devices. Greater investment in both the breadth and depth of AI applications and capabilities will enable XR to reach its full potential across segments and use cases.