# How to Combine Wide-Angle LiDAR Depth with Standard Camera Images > This patent describes a method for combining a wide-angle depth map from a LiDAR sensor, which has unevenly spaced pixels, with a standard camera image by first correcting the depth map's pixel distribution and then merging the two for a complete view. - **Patent:** US 11543533 - **Original title:** Systems and methods for wide-angle LiDAR using non-uniform magnification optics - **Owner:** Leddartech - **Granted:** 2023 - **Status:** Active - **Times cited:** 1 - **Field:** automotive, telecommunications, software, consumer_electronics, ai_ml ## What it does This patent outlines a method to create a detailed composite image by merging data from two different sensors. First, it captures a "first image" of a scene using a sensor like LiDAR, which measures depth and has pixels distributed unevenly, often non-linearly in the vertical direction (Claim 1, 2). This unevenness means some areas have more detail than others. It also captures a "second image" from a different sensor, such as a regular digital camera (Claim 1, 9), which typically has a more uniform pixel distribution. The key step is "correcting the first image" (Claim 1) based on its non-linear distribution function to produce a "third image," potentially by adding more pixels through interpolation (Claim 6, 7). Finally, it combines this corrected depth image (the third image) with the camera image (the second image) to create a "composite image" (Claim 1), such as an RGBD image (Claim 15), providing both color and accurate depth across a wide field of view (Claim 12). For example, an autonomous vehicle could use this to get a wide, detailed 3D view of its surroundings by merging a wide-angle LiDAR's depth data with a standard camera's color feed. ## What it does NOT cover - Does not cover combining images where both sensors already have a linear or uniform pixel distribution. - Does not cover systems that combine images without first correcting the non-linear pixel distribution of one sensor. - Does not cover systems that only use a single sensor to generate a composite image. - Does not cover combining images where neither the first nor the second image is a depth map. - Does not cover combining images from two sensors that do not have at least one common field of view. ## The clever bit The novelty lies in specifically addressing the challenge of integrating wide-angle sensors, like LiDAR, that inherently produce images with non-uniform pixel distribution – meaning some areas have more detail than others – by correcting this distortion *before* merging it with a standard, uniformly distributed camera image. This ensures accurate alignment and a consistent data representation across the entire wide field of view. ## Real-world examples 1. Autonomous vehicle perception systems 2. Robotics navigation and obstacle avoidance 3. Advanced Driver-Assistance Systems (ADAS) 4. 3D mapping and surveying 5. Augmented reality depth sensing ## Why it matters This technology is important for applications like autonomous driving and robotics, where understanding the environment in 3D is critical. By effectively combining wide-angle depth information from LiDAR with high-resolution color data from cameras, systems can gain a more complete and accurate perception of their surroundings. This allows for better object detection, tracking, and navigation, especially in complex and dynamic environments requiring wide fields of view. ## Frequently asked questions ### What does How to Combine Wide-Angle LiDAR Depth with Standard Camera Images cover? This patent describes a method for combining a wide-angle depth map from a LiDAR sensor, which has unevenly spaced pixels, with a standard camera image by first correcting the depth map's pixel distribution and then merging the two for a complete view. ### Who owns patent US 11543533? Leddartech owns this patent, granted in 2023. ### When does this patent expire? This patent is expected to expire on March 28, 2042, when the invention enters the public domain. ### What is patent US 11543533 cited by? This patent has been cited by 1 later patents that build on its ideas. ### What problem does this patent solve? This technology is important for applications like autonomous driving and robotics, where understanding the environment in 3D is critical. By effectively combining wide-angle depth information from LiDAR with high-resolution color data from cameras, systems can gain a more complete and accurate perception of their surroundings. This allows for better object detection, tracking, and navigation, especially in complex and dynamic environments requiring wide fields of view. ### What does this patent NOT cover? Does not cover combining images where both sensors already have a linear or uniform pixel distribution. **Full plain-English explainer:** https://patentbrief.org/patent/us/11543533/systems-and-methods-for-wide-angle-lidar-using-non-uniform-magnification-optics **Original patent:** https://patents.google.com/patent/US11543533 --- _Source: PatentBrief — https://patentbrief.org. Patent facts are from public records; the plain-English explanation is PatentBrief's._ ## Related patents Semantically similar inventions in the PatentBrief corpus: - [How Self-Driving Cars Find Clear Paths Between Objects](https://patentbrief.org/patent/us/10733420/systems-and-methods-for-free-space-inference-to-break-apart-clustered-objects-in) — This patent describes a method for vehicles, especially self-driving ones, to use lidar data and a 3D grid to precisely identify open spaces between obstacles, helping them navigate safely. - [How Car Factories Calibrate LIDAR Sensors for Self-Driving Cars](https://patentbrief.org/patent/us/11391826/vehicle-lidar-sensor-calibration-system) — This patent describes a system for car factories to precisely adjust a vehicle's LIDAR sensors using special floor and wall targets, ensuring they accurately map the world for features like self-driving. - [How Car Safety Systems Turn Off When Sensors Disagree](https://patentbrief.org/patent/us/11702088/vehicular-driving-assist-system-with-sensor-offset-correction) — This patent describes a car safety system that uses both a camera and a radar sensor to detect objects, and if their detections don't match up, it can disable parts of the driving assistance for safety. - [How Autonomous Cars Process Sensor Data for Driving](https://patentbrief.org/patent/us/20220161815/autonomous-vehicle-system) — Intel's 2020 patent describes a system for autonomous vehicles that cleans and standardizes data from various sensors before using it to perceive the environment and make driving decisions. - [How Eye Tracking Systems Adapt When They Can't See Eye Glints](https://patentbrief.org/patent/us/10878237/systems-and-methods-for-performing-eye-gaze-tracking) — This patent describes an eye-tracking system that intelligently switches between two methods: one using bright reflections on the eye when available, and another using head position and facial features when those reflections are absent.