How Snap's AR Tech Measures Distance for Virtual Button Presses

A method for mobile devices to calculate exactly when a user's finger or controller touches a virtual object in augmented reality by measuring precise distances from the camera.

What it covers

This patent describes a mathematical system for mobile devices to determine the depth and position of physical surfaces in the real world. It uses a specific formula involving two camera lenses to calculate the distance to a point on a surface. Once the device knows where that surface is, it renders a virtual button or control there. The system tracks the user's finger or controller and triggers an action when the user's movement crosses a specific distance threshold from that virtual object. It also includes a dynamic adjustment feature where the threshold for 'touching' the button gets tighter as the interaction continues, allowing for more precise control.

What it doesn't cover

• —Does not cover interaction methods that rely solely on 2D screen coordinates without depth calculation.
• —Does not cover systems that do not use a dual-camera (stereo) setup for distance calculation.
• —Does not cover non-visual methods of interaction like voice commands or haptic-only inputs.
• —Does not cover general object recognition that does not involve triggering a virtual control.

The clever bit

The system dynamically shrinks the activation threshold (the 'hitbox') based on the user's previous interactions, effectively 'locking' the user onto the virtual control once they have successfully engaged with it.

Why it matters

As augmented reality moves from simple filters to interactive tools, the 'tap' is the most critical interface challenge. This patent addresses the 'ghost touch' problem where users struggle to know if they are actually interacting with a virtual object. It provides a technical framework for making virtual buttons feel responsive on mobile hardware.

Real-world examples

• 1.Snapchat AR Lenses with interactive buttons
• 2.Mobile augmented reality shopping apps
• 3.Virtual control panels in AR environments