To better understand the tool that we developed, reader needs to understand the concept of hand tracking technology, what preceded it, and why is it a more immersive and realistic way of interaction in VR, at least in our humble opinion.

Hand tracking is slowly but surely becoming an integral feature of recent generations of immersive virtual reality devices. It allows users to interact with all sorts of virtual elements without the need to use controllers. Sensors on VR headsets track users' hands and then use the data they gather to create a real-time virtual embodiment of them. 

Before, hand tracking technology in VR was introduced through the first and second iterations of the Oculus Quest, and interactions with the virtual environment were usually performed with hand-held controllers. The user's hands were visualized in the applications in a limited array of poses based on finger position, assumed from contact with triggers and buttons on these controllers. 

Hand tracking can be used for more than just picking up objects or firing weapons at enemies: it also makes interactions with other people feel more real as well as it allows for faster communication between users. With hand tracking enabled on your device, all kinds of things become possible ranging from simple gestures like clapping hands together ("high five") or even communicating in sign language.

In comparison with controllers, hand tracking technology offers three significant advantages:

1. Increased immersion and presence
Research has shown that we have an almost supernatural sense of the positions and shape of our hands when they are shielded or out of sight, and when our hands are removed from our visual worlds, it is a reminder of our disembodiedness. We spend most of our time during various daily tasks moving and twisting our hands, so removing them from the visual scene or not showing the exact movement of our hands in a virtual environment (which is the case when using controllers in VR) has several consequences for our visuomotor behavior.

2. More effective interaction
The main goal of virtual reality is to enable the user to interact with a computer-generated environment in a natural way. An example of such an interaction in its simplest form can be moving the user’s head to experience a wide visual world. But more modern VR experiences usually involve some form of manual interaction like pushing a button or carrying an object. Accurate hand tracking potentially allows for far more precise and better interactions compared to controllers, adding not only to the user’s immersion but even the accuracy of his movements, which seems particularly crucial in the training context.

3. More effective communication
Recent research has shown that individuals have difficulty recognizing many key emotions in body images that have had their hands removed, emphasizing how important information about the shape of the hand is in communication experiences. The value of hand gestures for communication in virtual environments is increasing as real-time face tracking and visualization are technically very difficult due to the extremely high temporal and spatial resolution required to detect and track micro-expressions.

One of the most exciting things about using hands in VR is enabling users to interact with the virtual world more like they do in real life. The core of this is the ability to grab and manipulate objects. We noticed that there is no out-of-the-box solution on the market for quick and easy implementation of tracking and interaction (especially physics) with hands in VR. Without OctoXR, grabbing an object with hands in VR is preceded by weeks or more of development that are now converted to just a few minutes.

OctoXR is a low-code Unity game engine plugin that allows its users to significantly reduce the development time of their realistic and physics-based hand interactions. 

Developing a solution in virtual reality is a time-consuming and complicated process, especially when trying to interact using hands. OctoXR allows the user to facilitate and shorten the process of creating applications within virtual reality using hand tracking. 

The plugin offers a range of features such as physical interactions with objects, a ready-made solution for capturing objects from a distance or proximity, pre-built solutions for moving in virtual reality such as teleportation and continuous movement, inverse kinematics, VR user interface, and gesture recognition that allows the user to further personalize desired interactions. Our plugin includes a library of interaction components that are modular, composable, and scalable that you can integrate into your existing architecture or run standalone. OctoXR takes complicated and undocumented parts of virtual reality and offers them as ready-made solutions which allow a user who does not necessarily have a background in development to turn an idea into reality much faster and easier than ever before.

OctoXR - low code VR hand interaction plugin

1. Kinematic grab
The kinematic grab feature lets you create joint-based objects which collide with the environment or kinematic grabbables for a more game-like feel in just a few clicks! Create grab points with a single click and fine-tune their position in the editor. With the OctoXR interactable module, grabbing stuff has never been easier.

2. Hand physics
This feature lets you easily create and edit a physics-enabled hand rig that makes a VR hand interact with other physically simulated objects in the scene. It will also automatically respond to collisions between any desired hand part and other objects. You can use provided scripts and extend them to make your custom behaviors and interactions based on physical or trigger collisions between hands and other objects.

3. Distance grab
Distance grab lets you detect distant grabbables with raycasting and interact with them from afar using the distance grabbing module or grab what's within the range of your hand using the proximity interactor.

4. Bad hand positioning 
Hand tracking can sometimes act out, with the OctoXR plugin you can stop your hands when things go wrong, preventing your interactions from messing up. Our bad hand tracking detection system allows your hands to stop in place with visual feedback, so your users can know exactly what's going on.

5. Gesture recognition
Making apps with gesture recognition is made easier than ever. You can create gestures with a single click of a button. Created gestures are saved in the projects Asset’s folder. You can then choose which gestures you wish to load in your scene. After you load gestures, with a simple drag and drop technique you attach functionalities to specific gestures. Gestures are recognized by comparing current hand bone positions with the ones saved with every gesture. If a gesture is recognized, the gesture detector fires a recognized event and executes attached functionalities.

6. Locomotion
Developing a good locomotion technique in VR has always been a big issue. Especially using hand tracking. OctoXR offers two readymade solutions: continuous locomotion (or walking) and teleport locomotion. There are a couple of predefined teleportation modes: pinch teleport and gesture teleport. You can create your own gestures you wish to use for teleport and attach them to gesture teleport. But if you feel you need even more freedom to develop your own type of teleport, you can simply inherit the teleport base class and by overriding, a couple of lines of code have your own brand-new teleport.

7. OctoXR UI
We developed the “oculus home like” UI manipulation that all Oculus developers are used to. Simply attach UIPointer to the hand and insert OctoInputModule prefab in the scene and it just works.

8. Inverse kinematics
The inverse kinematics feature provides support for models using the Unity humanoid rig. We integrated head, arm and finger tracking. 

Hand tracking in VR is a game-changer. Many companies are already using it to make their games more immersive, accessible and fun. It's an important part of the future of virtual reality, and OctoXR will help you get there sooner than you think. We believe that by giving people a powerful tool like this, we can make the future of VR so much more exciting and fun. If you have any questions about or feedback on the topic please feel free to reach out. 

For more info about OctoXR make sure to check here.

Create precise and realistic hand interactions in VR with minimal effort

Developing a solution in virtual reality is a time-consuming and complicated process, especially when trying to interact using hands. OctoXR allows the user to facilitate and shorten the process of creating applications within virtual reality using hand tracking. The plugin offers a range of features such as physical interactions with objects, a ready-made solution for capturing objects from a distance or proximity, pre-built solutions for moving in virtual reality such as teleportation and continuous movement, inverse kinematics, VR user interface, and gesture recognition that allows the user to further personalize desired interactions.

A great idea and a strong will to succeed are just not enough in today’s world. If you’re an entrepreneur who’s looking to scale and grow, you will want to look into funding your start-up business. One of the biggest challenges for small business owners is finding the funds and the right partners to grow their businesses. Surveys show that a lack of capital is one of the most common reasons why small businesses fail.

Securing an investor who is committed to your long-term success and has an interest in your industry and mission will add value far beyond capital backing. While capital and cash flow will extend the runaway to success, your business should collect the benefits of business mentors that you got alongside with investment. We were lucky enough to find those qualities in our present partners from Fil Rouge Capital, whose experience and knowledge will drive us in all the right directions.

About a year ago as we were working on some products for our clients, we noticed that we are spending a significant amount of time working on hand interaction features, and that was the time an idea started to form and the OctoXR was born. OctoXR is a Unity 3D plugin that simplifies the development of VR, AR, and MR applications and experiences for developers and users. The tool was conceived as an addition to the Unity 3D development platform and is based on a low-code / no-code mode of visual development. This solves the problem of the need for a high degree of programming knowledge in the development of VR, AR, and MR applications and solutions. By developing its modules and technologies for hand control, physics, multi-user networking, in-display debugging tools, visual programming, and many other features, the product will significantly contribute to the faster and easier development of XR applications in the global market.

A great idea and a strong will to succeed are just not enough in today’s world. If you’re an entrepreneur who’s looking to scale and grow, you will want to look into funding your start-up business.

Corporate training and development programs have followed traditional paths, usually with a combination of written materials, classroom lectures, outdated videos, online courses, and hands-on observation of an instructor regardless of whether the topic is production training, sales pitching, or leadership coaching. But as companies transform their businesses to be more digital, especially due to the corona crisis, the rise of virtual reality (VR) technology provides an opportunity to reshape traditional training methods to better serve a more digitally connected and diverse workforce. By using VR training, it is possible to create scenarios that are not only more engaging and immersive but also more effective and safer than traditional training methods.

Each day companies are faced with the challenge of upgrading existing employee skills to increase production performance and to train new hires to be production-ready with the basic technical, safety, and quality skills. According to go2HR  more than 40% of employees who receive poor job training leave their positions within the first year, so it is not uncommon for companies such as BMW to invest around 373 million Euro a year in employee training.

Huge sums of money are being invested in training across the globe, yet in many cases, the training procedures are not only outdated, slow and many times unsafe but do not clearly outline the complexity of the technical tasks they should teach. Longer training times, employees under-performing, and safety risks can have a hazardous domino effect on the industrial employer. Virtual Reality technology can not only close the gap between outdated training methods but also increase worker production, safety, efficiency, satisfaction, and overall task performance rates in the long run – when compared to training with traditional methods.

According to the Artillery Intelligence report , enterprise VR will grow from $829 million in 2018 to $4.26 billion in 2023, a 39 percent compound annual growth rate. This includes any spending on VR technology from enterprise buyers, including deployed hardware, software running on VR headsets, and VR creation tools (e.g. Unity or Unreal). Employee training will be a key use case in industrial segments and retail.

The benefits of VR for improving learning outcomes has been clear for some time. Companies across many industries have adopted the technology to train their employees better. Enterprises have seen dramatic cuts in costs and time spent training.

Using the headset and controllers or hand gestures, trainees look, speak, and move freely in a 3D virtual space, interacting with simulated real-world tools, machinery, and other trainees and instructors. Virtual reality is enabling workplace scenarios that were once too difficult, expensive, or dangerous to train for, become vastly more practical, cost-effective, and safe in an immersive simulation.  When combined with real-life simulations and learning curriculum developed by experts, the result is trainees learning in the best way possible — through personal experience — significantly improving learning retention, job performance, team collaboration, workplace safety, and cost.

Real-life situations and training scenarios can be created not only to make learning more interactive but also to bring students and trainees together, no matter where the location – improving the efficiency of training and potentially reducing the time required to train staff up to standard.

Another advantage of VR is that it’s more in-depth compared to other training methods. As mentioned earlier, training in real-world scenarios is anecdotal. In contrast, actions in VR can be quantified so you can understand exactly how each action was performed. The result is a data-rich, contextual timeline of all the actions which took place during the session. This provides a reinforced feedback loop that results in continual improvements, making VR simulations an integrated component of the enterprises learning strategy.

One of the biggest advantages of using VR to train employees is the ability to iterate. By understanding which behaviors lead to success, you can encourage and redesign the training programs. This provides additional feedback and a deeper understanding of human behavior. The systematic approach to improving learning ensures that processes are followed to deliver best practices to the employees.

Put another way, let your employees make expensive and dangerous mistakes in VR and not in real-world scenarios where the stakes are much higher.

Corporate training and development programs have followed traditional paths, usually with a combination of written materials, classroom lectures, outdated videos, online courses, and hands-on observation of an instructor regardless of whether the topic is production training, sales pitching, or leadership coaching.

Augmented reality is a powerful tool that consumers no longer just desire, but expect. Brands who have implemented AR in advertising, sales, education, product design, customer entertainment are now experiencing performance benefits and seeing increases in conversion rates.

Augmented reality is a powerful tool that consumers no longer just desire, but expect. Brands who have implemented AR in advertising, sales, education, product design, customer entertainment are now experiencing performance benefits and seeing increases in conversion rates. Through the use of AR, brands can offer their shoppers detailed product information in a more interesting and engaging way, allowing consumers to see more of the benefits that a product can provide.

Based on a Zappar report around the brain analysis of people using AR, brand awareness can increase by up to 70% through the use of augmented reality. No wonder brands are implementing AR campaigns like never before. In order to build a sense of loyalty and a lasting impression on their customers, brands have found that AR is able to do this in an effective and long-term way. AR ads are more successfully remembered by consumers because of their interactiveness and ability to be shared.

To define what we mean by AR advertising, these are paid media placements that allow consumers to activate their smartphone camera and superimpose branded animations in the real world. This is usually to promote or visualize a given advertiser’s products.

Advertising is one of the most lucrative AR subsectors, on pace to reach $1.41 billion this year and $8.02 billion by 2024, with a 54,43% compound annual growth rate.

With numbers like these, it’s no wonder that that ad revenue for AR has reached well over the billion-dollar mark. Brands are recognizing the value of this kind of advertising and taking advantage of its campaign performance.

As is the case across the global economy, mobile AR sectors will be impacted unevenly by ongoing global lockdowns. Given that software mostly fares well in shelter-in-place life, the impact on mobile AR will be mostly positive.

For example, quarantine-friendly consumer AR like social lenses are trending up; and social distancing compels enterprise remote-AR support. These factors will boost near-term adoption while exposing the technology… which in-turn supports its longer-term sustained adoption.

Brand advertisers are also attracted to AR’s business case that continues to be validated in ROI metrics and one of the best metrics is conversion rate. Higher conversion rates indicate that consumers are satisfied with a large part of the marketing strategy and content that is being presented to them. The main reason why AR advertisements achieve high conversion rates is immersive product try-on that encourages informed purchases. Although every ‘high’ conversion rate is unique to each industry, a twenty to eighty percent, which was measured conversion rate on AR ads, is over ten times higher than the average conversion rate across all industries on the search network.

As mentioned, campaigns such as L’Oréal’s AR showcases have seen increased conversion rates, and have continued to work with brands such as Poplar in order to successfully create even more. Plus, brands like the pizza chain Papa Johns have joined in and gained a 25% conversion rate with their AR campaigns.

The proof of AR is in its conversions. While many brands are still using AR to engage their followers and increase brand awareness, the benefits of AR as a conversion tool are only increasing.

As more and more brands continue to report on successful conversion rates, we can expect the adoption of AR to move away from simple engagement into an active channel in the marketer’s toolkit to drive higher conversions and sales.

Activities within the `Emergency medical training 3D` project focused on early-stage innovation development to conduct technical and commercial validation of an innovative concept for 3D emergency education training to minimize potential technical and commercial risks. Through the conducted research, the applicant realized that virtual and augmented reality, due to their immersiveness, represents a decisive opportunity to create new or improve existing skills in employees. Through this project, comprehensive commercial and technical verification and validation of research results with commercial potential were undertaken.

The planned duration of the project is 12 months, within which an attempt was made to prove that technological risks associated with the product can or cannot be solved or whether the technological risk is such that the proposed technology can or cannot function in the environment. The project was implemented through 6 key points to try to demonstrate technical feasibility as follows: code development, 3D modeling and testing, interface design, video animation, and optimization and testing of video call module and multi-user interaction functionality. In addition to the technical feasibility demonstration activities, a CBA, an intellectual property protection plan, and a development concept and strategy will be developed.

The total value of the project is HR 556,904.48, of which HRK 389.751,32 is financed from grants and HRK 167.153,16 represents the applicant’s own participation.

After the project ended, Spectre d.o.o. started working with the Institute of Emergency Medicine Osijek – baranja county on developing virtual reality applications for staff training.

Spectre d.o.o. plans to continue the development of the project by applying to the ERDF to fund innovation and R&D projects.

Activities within the `Emergency medical training 3D` project focused on early-stage innovation development to conduct technical and commercial validation of an innovative concept for 3D emergency education training to minimize potential technical and commercial risks. 

Hear our thoughts

OctoXR - when VR development gets easy

SpectreXR partners with FRC

Virtual Reality Training

Augmented Reality Advertising

Emergency medical training 3D