# How to Combine Wide-Angle LiDAR Depth with Standard Camera Images > This patent describes a method for combining a wide-angle depth map from a LiDAR sensor, which has unevenly spaced pixels, with a standard camera image by first correcting the depth map's pixel distribution and then merging the two for a complete view. - **Patent:** US 11543533 - **Original title:** Systems and methods for wide-angle LiDAR using non-uniform magnification optics - **Owner:** Leddartech - **Granted:** 2023 - **Status:** Active - **Times cited:** 1 - **Field:** automotive, telecommunications, software, consumer_electronics, ai_ml ## What it does This patent outlines a method to create a detailed composite image by merging data from two different sensors. First, it captures a "first image" of a scene using a sensor like LiDAR, which measures depth and has pixels distributed unevenly, often non-linearly in the vertical direction (Claim 1, 2). This unevenness means some areas have more detail than others. It also captures a "second image" from a different sensor, such as a regular digital camera (Claim 1, 9), which typically has a more uniform pixel distribution. The key step is "correcting the first image" (Claim 1) based on its non-linear distribution function to produce a "third image," potentially by adding more pixels through interpolation (Claim 6, 7). Finally, it combines this corrected depth image (the third image) with the camera image (the second image) to create a "composite image" (Claim 1), such as an RGBD image (Claim 15), providing both color and accurate depth across a wide field of view (Claim 12). For example, an autonomous vehicle could use this to get a wide, detailed 3D view of its surroundings by merging a wide-angle LiDAR's depth data with a standard camera's color feed. ## What it does NOT cover - Does not cover combining images where both sensors already have a linear or uniform pixel distribution. - Does not cover systems that combine images without first correcting the non-linear pixel distribution of one sensor. - Does not cover systems that only use a single sensor to generate a composite image. - Does not cover combining images where neither the first nor the second image is a depth map. - Does not cover combining images from two sensors that do not have at least one common field of view. ## The clever bit The novelty lies in specifically addressing the challenge of integrating wide-angle sensors, like LiDAR, that inherently produce images with non-uniform pixel distribution – meaning some areas have more detail than others – by correcting this distortion *before* merging it with a standard, uniformly distributed camera image. This ensures accurate alignment and a consistent data representation across the entire wide field of view. ## Real-world examples 1. Autonomous vehicle perception systems 2. Robotics navigation and obstacle avoidance 3. Advanced Driver-Assistance Systems (ADAS) 4. 3D mapping and surveying 5. Augmented reality depth sensing ## Why it matters This technology is important for applications like autonomous driving and robotics, where understanding the environment in 3D is critical. By effectively combining wide-angle depth information from LiDAR with high-resolution color data from cameras, systems can gain a more complete and accurate perception of their surroundings. This allows for better object detection, tracking, and navigation, especially in complex and dynamic environments requiring wide fields of view. ## Frequently asked questions ### What does How to Combine Wide-Angle LiDAR Depth with Standard Camera Images cover? This patent describes a method for combining a wide-angle depth map from a LiDAR sensor, which has unevenly spaced pixels, with a standard camera image by first correcting the depth map's pixel distribution and then merging the two for a complete view. ### Who owns patent US 11543533? Leddartech owns this patent, granted in 2023. ### When does this patent expire? This patent is expected to expire on March 28, 2042, when the invention enters the public domain. ### What is patent US 11543533 cited by? This patent has been cited by 1 later patents that build on its ideas. ### What problem does this patent solve? This technology is important for applications like autonomous driving and robotics, where understanding the environment in 3D is critical. By effectively combining wide-angle depth information from LiDAR with high-resolution color data from cameras, systems can gain a more complete and accurate perception of their surroundings. This allows for better object detection, tracking, and navigation, especially in complex and dynamic environments requiring wide fields of view. ### What does this patent NOT cover? Does not cover combining images where both sensors already have a linear or uniform pixel distribution. **Full plain-English explainer:** https://patentbrief.org/patent/us/11543533/systems-and-methods-for-wide-angle-lidar-using-non-uniform-magnification-optics **Original patent:** https://patents.google.com/patent/US11543533 --- _Source: PatentBrief — https://patentbrief.org. Patent facts are from public records; the plain-English explanation is PatentBrief's._ ## Related patents Semantically similar inventions in the PatentBrief corpus: - [LiDAR System for Seeing Through Shiny Objects](https://patentbrief.org/patent/us/11041957/systems-and-methods-for-mitigating-effects-of-high-reflectivity-objects-in-lidar) — Toyota's patent on a LiDAR system that can ignore blinding reflections from shiny surfaces to better see what's around a vehicle. - [How Autonomous Vehicles Use Different Camera Lenses for Better Vision](https://patentbrief.org/patent/us/12231783/sensor-layout-for-autonomous-vehicles) — This patent describes a specific arrangement of cameras for self-driving vehicles, using multiple forward-facing cameras with different focal lengths and a pair of backward-facing cameras with identical focal lengths to achieve comprehensive vision. - [Efficiently Combining Data from Different Sensors](https://patentbrief.org/patent/us/12737436/systems-and-methods-of-sensor-data-fusion) — This patent describes a system for combining information from many different types of sensors more efficiently by carefully selecting and processing the data, reducing the computer power needed. - [How a Smart LIDAR System Adapts to Noise for Better Distance Sensing](https://patentbrief.org/patent/us/20240045038/noise-adaptive-solid-state-lidar-system) — This patent describes a LIDAR system that uses multiple lasers and detectors, and intelligently adjusts the electrical settings of specific detectors to reduce noise and improve distance measurements in different lighting conditions. - [How Self-Driving Cars Detect Dirty Lidar Sensors](https://patentbrief.org/patent/us/12130389/methods-and-systems-for-detecting-sensor-occlusions) — Waymo's 2024 patent on a self-driving car system that uses two scans from a rotating Lidar sensor to spot dirt or obstructions on its lens.