Haptics and the 3D Internet
What haptics means for simulation-based training, remote operation, and advanced content design. Haply lays out the three main areas where touch-enabled devices like the Inverse3 will shape our future, and invites readers to its co-founder's panel session at NVIDIA GTC.
Originally published on haply.co — the canonical version of this article.
Why haptics, and why now?
The recent exponential increase in content creation for the 3D internet is ensuring a vast and rich diversity of experiences to meet the endless needs of those of us seeking to not only see the virtual world, but to touch it too. Such is the function of haptic technology — the electromechanical stimulation of the sense of touch.
This technology has enabled Haply Robotics to create exciting applications in areas like the metaverse, industrial robotics, automation and supervision, and VR simulation. New technologies like enterprise 3D displays, simulation engines, and 5G are creating a wealth of opportunities to build transformative technologies. Together with the developments surrounding the metaverse and VR, the tech ecosystem is ready for an advanced 3D mouse that enables users to interact with the growing 3D internet in ways never before possible. We anticipate the main areas of application for haptic devices to include:
- Simulation-based skills training
- New ways to create and consume 3D content
- Remote device operation
For the past year, we have used NVIDIA’s Omniverse™ technology and NVIDIA accelerated computing systems such as the RTX A6000 to build touch-enabled applications for the Mediverse™, or the medical metaverse. We invite you to join our panel session at the virtual NVIDIA GTC on September 21 at 11 AM PDT, where our cofounder, Colin Gallacher, and other thought leaders in medical imaging, genomics, biopharma, and medical devices, will discuss healthcare applications of robotics, AR technologies and more.
Simulation-Based Skills Training
Training surgeons is difficult, time consuming, and expensive, but there’s no better teacher than experience. This is why a simulation-based approach to skills training is a highly effective way to learn: simulated training scenarios allow students to run through high-stakes procedures with close to zero risk.
Haptic simulation training takes that a step further, facilitating skill mastery by replicating the exact sensations of real-life scenarios and stimulating deep focus through a more natural and immersive learning environment than traditional simulators. The more authentic the simulation environment, the better the results of training — particularly for skills that require precise, tactile feedback and movements, such as virtual medical training. The Haply Inverse3 is a state-of-the-art force-feedback haptic controller designed to precisely emulate complex sensations, such as incising soft tissue with a scalpel or drilling through bone tissue, empowering students, surgeons, and other healthcare professionals to hone life-saving procedures before ever setting foot in an operating room.
Other applications include dangerous task training. A haptic controller coupled with a VR headset can transport trainees to highly realistic environments without endangering anyone or the cost of building a physical training space. For this reason, haptic simulators are commonly used for military training, pilot training, firefighter training, and manufacturing task training — accurately simulating the force feedback of routine hands-on operations.
New Ways to Create and Consume 3D Content
Unity, Epic Games Unreal Engine, Dassault Systèmes SOLIDWORKS and CATIA, Maxon ZBrush, Autodesk Maya, AutoCAD, Inventor, Blender, McNeel Rhino — the list goes on. Since their introduction in the 1960s, 3D modeling tools have become central to workflows in engineering, architecture, gaming, and industrial design.
Gone are the days where architects had to carry heavy portfolios of hand-drawn plans and animators had to manually draw each frame of a cartoon movie. Companies like Meta, Apple, and HP are investing billions into next-generation displays, and the future will allow users to comfortably blend tasks performed in both 2D and 3D environments.
Haply believes that enabling a seamless transition from 2D to 3D content is vital for any input device. Haptic content will allow users to naturally navigate spatial environments and feel what they digitally produce, transforming product development systems and speeding up processes across industries. The future of content creation and consumption will focus on making virtual experiences as photorealistic and immersive as possible — and the rising popularity of haptic controllers signals a major step in that direction.
Remote Device Operation
The arrival of 5G means more than faster internet: it enables increased capacity for connected devices and remote performance, such as VR streaming on NVIDIA’s CloudXR platform. With 5G’s reduced latency, reliable wireless communication supporting haptic technology is finally possible.
Teleoperation in robotics combines robotic manipulation with human decision-making abilities. Two-way teleoperation provides haptic feedback that allows operators to feel interaction forces and intuitively control remotely operated robots — feedback that is essential for delicate tasks involving fragile objects, dangerous environments, and high-precision operations. Examples include inspecting construction sites with drones, controlling surgical robots, supervising industrial robots remotely, and conducting rescue operations in mines.
Remote-controlled robots, vehicles, and devices are not new, but adding haptic feedback creates exciting possibilities for advanced remote operations, providing a natural and intuitive way to supervise machinery through touch as an additional sensory channel.
Designed and built for a 3D world
5G networks are the infrastructure replacing highways in the modern digital universe. Just as 2D mice and tablets replaced paper and pen, Haply’s haptic technologies blend the advantages of traditional input devices with the spatial requirements of the 21st century. 11.2 million AR/VR headsets were sold last year alone, showing increasing demand for simulated experiences and realistic virtual environments — and haptics adds an important sensory component with benefits across 3D design, advanced training, telepresence, and remote manipulation.
At Haply, the focus is on precision and ergonomics within enterprise environments. Using the Inverse3 — the world’s first lightweight, reconfigurable, and completely portable haptic interface that replicates the sense of touch for specialized tasks — companies will create revolutionary solutions to train skills using simulations, create and consume 3D content, and operate remote devices.
To learn more about the future of this technology and how NVIDIA supports advanced graphics, simulation, and healthcare applications through NVIDIA Clara, join our panel session at NVIDIA GTC — Wednesday, September 21 at 11 AM PDT.