Redefining AR/VR Interaction

by Joachim Samuelsson
Entrepreneur and Chairman of the Board of Crunchfish

Virtual Reality (VR) and Augmented Reality (AR) are, without doubt, the two megatrends in tech that will have the largest impact on our lives in the near future. Crunchfish may be the best kept secret in the industry as it is still relatively unknown that we have ready technology components that address several key issues in the rapidly emerging AR/VR industries. This note is an attempt to explain how Crunchfish’s software technology components may redefine AR/VR interaction.

VR and its 360 media provides the modern format of photography and film and will come into play in all areas where photos and video are used today as well as in new areas made possible by this new format. It will affect us all at a very fundamental level. Questioning the use of VR is similar to questioning the use of photography or video. Tomorrow’s children won’t understand how we put up with the limited view of the surroundings decided by the film-maker or photographer. In contrast, VR gives the viewer freedom of a display in 360 degrees. You are invited to experience any environment as if you were present, there and then. 

Media Development
VR will show a breakthrough in all areas and 360 media will become the modern format of photography and video. By photographing or filming with a 360 camera, the entire room is captured, even backwards. Many have already seen pictures of the “360” symbol in their Facebook feeds. When it comes to video in the 360 format, virtual reality is best enjoyed with a VR headset, as one can seamlessly look around in the virtual real world via that the headset detects head movement. 

AR, on the other hand, provides people with superpowers in their daily lives, allowing them to discover information about everyone and everything around them. You may even be able to communicate with your surroundings as you could be connected automatically. The dynamic experience with your mobile phone will be a far cry from the static internet experience on your computer at work or home. This highly contextual experience is the next mobile frontier to be conquered, enabling people to understand, and even connect with, the vicinity around them, here and now.

Connected AR

During the past three years, Crunchfish has pioneered an additional technology, which has been implemented in the software product aBubbl®. The technology is a patent pending AR platform as it augments the reality of the mobile user in several ways. It may easily be activated in any application or mobile operating system to allow its mobile users to detect and communicate with other mobile devices in the vicinity.

AR/VR will have its breakthrough sooner rather than later. During the last two years, over 4 billion USD has been invested in Silicon Valley startups[1], with more startups and investors arriving daily. San Francisco has periodically been fueled by industry gold rushes and the latest tech bonanza is certainly AR/VR. A few recent significant launches will accelerate the growth of these industries.

• Google released their VR headset, Daydream View, two weeks ago and their latest Android release Nougat is very VR-friendly. It is now possible to experience our favorite apps in VR in an Android device that fulfill the Daydream specifications. This is the trigger for mobile manufacturers and application developers to get serious with VR now. 

• Facebook was early in acquiring VR helmet company, Oculus Rift. They have announced hand control hardware that is expected to cost more than 200 dollars[2]. In parallel, they are actively encouraging Facebook users to capture and publish content in the new 360 format.

• Niantec’s Pokémon Go showcased what appetite the general public could have for AR. This has also excited the industry and there may be as many as 100 other AR games released before the end of this year1.

Today’s mobile apps are developed for interaction by touch. As touchscreens are typically not accessible for AR/VR users, new interaction paradigms need to emerge and mature. Crunchfish believes that gesture control is the most natural way to interact with AR/VR and will become its killer application. There are a few alternatives, such as remote-, headset- or voice-control, but all have drawbacks either in terms of cost, convenience or precision that will prevent them for becoming anything but peripheral interaction complements to gesture control. The proven Crunchfish touchless gesture control solution Touchless A3D® has been perfected for mobile users since 2011 and is ready to provide interaction to any AR/VR enabled app in the world, regardless if it is pre-installed by the hardware manufacturer or downloadable from a software application provider.

Touchless gesture control in VR 

If you have used a VR-headset to watch a video in your social media feed, you have probably recognized the need for better interaction methods when navigating and controlling the content. Without a screen to touch, we see gesture control as a natural and the obvious way to interact with and experience VR content.

Crunchfish started to invest in C2C beacon technology back in 2014 when the rest of the industry was busy implementing B2C beacon solutions mostly for retail use. All mobile operating systems provide beacon support that essentially allows your mobile phone to wake up and register itself if it detects a beacon signal from an app that you have downloaded and opted to listen to. Mobiles may not only listen to beacons, they are also able to broadcast a beacon signal themselves which allows us to offer a dynamic personal communication system around each mobile user that we refer to as “a bubble”. Within this bubble any mobile user may discover who and what is around them and even start communicating the surroundings, i.e. we provide an augmented reality to each mobile user that even offer the possibility of communication links. What’s more is that Crunchfish Connected AR solution aBubbl® is ready to activate AR in any app in the world today to provide their users with such superpowers.

Connected AR

Connected AR is a research area at Crunchfish focused on Augmented Reality, that facilitate social or other interaction between mobile users in close proximity, via beacon-based ad-hoc temporary networks. The aBubbl® product software product supports Apple’s iOS (iBeacon) and Android as well as cross-OS communication. 

Crunchfish develops software that creates new possibilities for interaction with consumer electronics. Our gesture recognition product Touchless A3D® is applicable to all devices containing a camera, which includes everything from mobile phones to Smart TVs. The consumer electronics market that Crunchfish addresses is estimated to reach over USD 6 billion by 2020[3]. The strong growth of gesture recognition in consumer electronics is expected to accelerate from 2018 and will be driven both by gesture recognition for AR/VR as well as when 3D cameras reach a wider distribution within consumer electronics. Crunchfish gesture solution is already tried and tested and is currently available in millions of devices from global players such as Lenovo, OPPO and Vodafone. It is expected that gesture control will have its breakthrough in 2018 regardless of the massive opportunities in AR/VR. These megatrends support and secure the estimate of 1.6 billion devices with gesture control in 2020[3]. 

Crunchfish’s other product aBubbl® is by its very nature a form of AR. The rate of activation of Connected AR in apps will have a major impact on realizing the full potential of the company. aBubbl® is also an important component in VR solutions as it enables VR users to connect with each other in their vicinity, allowing a social VR experience where interaction is possible and content may be shared between users. It is extremely difficult to estimate the size of the market opportunity for this technology but the shared prediction of Apple’s CEO Tim Cook[4] and Crunchfish’s CEO Joakim Nydemark is that AR is likely to become an even greater industry than VR. As aBubbl® may activate Augmented Reality in any app, rather than “just” deliver the necessary gesture based interaction of AR/VR, it is likely that AR will have a huge impact on Crunchfish’s future potential.

Crunchfish’s software components applied to AR/VR

Both Touchless A3D® and aBubbl® are key components of the rapidly emerging AR/VR industries. Touchless gesture control is expected to be the way users interact with AR/VR. aBubbl® provides VR users the possibility to be social by interacting with each other. aBubbl® is by nature a form of AR and able to activate AR in any application or mobile operating system and augment the reality for its users.  

As an entrepreneur and Chairman of the Board of Crunchfish I have high expectations for the future. I have taken lead in every financing round since the seed round in 2012. In this public round I am signing up for yet another million USD as our excellent position serving the two current tech megatrends AR and VR simply calls for it.

[1] LinkedIn blogpost by Sheridan Tatsuno, October 10, 2016, San Francisco’s VR Gold Rush Heats Up
[2] Ars Technica, September 2016, Oculus Touch controllers cost £190 in the UK
[3] echnavio Insights, 2016, Global Gesture Recognition Market 2016-2020 
[4] The Verge, September 14, 2016, Tim Cook says augmented reality will be bigger than virtual reality