Agfdhyk

Lidar-derived image of Marching Bears Mound Group, Effigy Mounds National Monument.

Play media

Animation of a satellite collecting digital elevation map data over the Ganges and Brahmaputra River basin using lidar.

A FASOR used at the Starfire Optical Range for lidar and laser guide star experiments is tuned to the sodium D2a line and used to excite sodium atoms in the upper atmosphere.

This lidar may be used to scan buildings, rock formations, etc., to produce a 3-D model. The lidar can aim its laser beam in a wide range: its head rotates horizontally; a mirror tilts vertically. The laser beam is used to measure the distance to the first object on its path.

Play media

Flying over the Brazilian Amazon with a LIDAR instrument.

Play media

An airplane collecting treetop data over a Brazilian rainforest.

Play media

In this view, the viewer flies down to the rainforest canopy and flies through the virtual leaves.

This visualization shows an airplane collecting a 50 kilometer swath of lidar data over the Brazilian rainforest. For ground level features, colors range from deep brown to tan. Vegetation heights are depicted in shades of green, where dark greens are closest to the ground and light greens are the highest.

Lidar (also called LIDAR, LiDAR, and LADAR) is a surveying method that measures distance to a target by illuminating the target with pulsed laser light and measuring the reflected pulses with a sensor. Differences in laser return times and wavelengths can then be used to make digital 3-D representations of the target. The name lidar, now used as an acronym of light detection and ranging^[1] (sometimes light imaging, detection, and ranging), was originally a portmanteau of light and radar.^[2][3] Lidar sometimes is called 3D laser scanning, a special combination of a 3D scanning and laser scanning. It has terrestrial, airborne, and mobile applications.

Lidar is commonly used to make high-resolution maps, with applications in geodesy, geomatics, archaeology, geography, geology, geomorphology, seismology, forestry, atmospheric physics,^[4]laser guidance, airborne laser swath mapping (ALSM), and laser altimetry. The technology is also used in control and navigation for some autonomous cars.^[5]

History and etymology

Lidar originated in the early 1960s, shortly after the invention of the laser, and combined laser-focused imaging with the ability to calculate distances by measuring the time for a signal to return using appropriate sensors and data acquisition electronics. Its first applications came in meteorology, where the National Center for Atmospheric Research used it to measure clouds.^[6] The general public became aware of the accuracy and usefulness of lidar systems in 1971 during the Apollo 15 mission, when astronauts used a laser altimeter to map the surface of the moon.

Although now most sources treat the word "lidar" as an acronym,^[1] the term originated as a portmanteau of "light" and "radar". The first published mention of lidar, in 1963, makes this clear: "Eventually the laser may provide an extremely sensitive detector of particular wavelengths from distant objects. Meanwhile, it is being used to study the moon by 'lidar' (light radar) ..."^[3] The Oxford English Dictionary supports this etymology.^[2]

The interpretation of "lidar" as an acronym ("LIDAR" or "LiDAR") came later, beginning in 1970,^[7] based on the assumption that since the base term "radar" originally started as an acronym for "Radio Detection And Ranging", "LIDAR" must stand for "Light Detection And Ranging",^[8] or for "Laser Imaging, Detection And Ranging".^[9] Although the English language no longer treats "radar" as an acronym and printed texts universally present the word uncapitalized, the word "lidar" became capitalized as "LIDAR" or "LiDAR" in some publications beginning in the 1980s. Currently no consensus exists on capitalization, reflecting uncertainty about whether or not "lidar" is an acronym, and if it is an acronym, whether it should appear in lower case, like "radar". Various publications refer to lidar as "LIDAR", "LiDAR", "LIDaR", or "Lidar". The USGS uses both "LIDAR" and "lidar", sometimes in the same document;^[10] the New York Times predominantly uses "lidar" for staff written articles,^[11] although contributing news feeds such as Reuters may use Lidar;^[12]

General description

Lidar uses ultraviolet, visible, or near infrared light to image objects. It can target a wide range of materials, including non-metallic objects, rocks, rain, chemical compounds, aerosols, clouds and even single molecules.^[4] A narrow laser beam can map physical features with very high resolutions; for example, an aircraft can map terrain at 30-centimetre (12 in) resolution or better.^[13]

The essential concept of Lidar was originated by EH Synge in 1930, who envisaged the use of powerful searchlights to probe the atmosphere.^[14][15] Indeed, Lidar has since been used extensively for atmospheric research and meteorology. Lidar instruments fitted to aircraft and satellites carry out surveying and mapping – a recent example being the U.S. Geological Survey Experimental Advanced Airborne Research Lidar.^[16]NASA has identified lidar as a key technology for enabling autonomous precision safe landing of future robotic and crewed lunar-landing vehicles.^[17]

Wavelengths vary to suit the target: from about 10 micrometers to the UV (approximately 250 nm). Typically light is reflected via backscattering, as opposed to pure reflection one might find with a mirror. Different types of scattering are used for different lidar applications: most commonly Rayleigh scattering, Mie scattering, Raman scattering, and fluorescence.^[4] Suitable combinations of wavelengths can allow for remote mapping of atmospheric contents by identifying wavelength-dependent changes in the intensity of the returned signal.^[18]

Design

Click image to see animation. A basic lidar system involves a laser range finder reflected by a rotating mirror (top). The laser is scanned around the scene being digitized, in one or two dimensions (middle), gathering distance measurements at specified angle intervals (bottom).

The two kinds of lidar detection schemes are "incoherent" or direct energy detection (which principally measures amplitude changes of the reflected light) and coherent detection (best for measuring Doppler shifts, or changes in phase of the reflected light). Coherent systems generally use optical heterodyne detection. This is more sensitive than direct detection and allows them to operate at a much lower power, but requires more complex transceivers.

Both types employ pulse models: either micropulse or high energy. Micropulse systems utilize intermittent bursts of energy. They developed as a result of ever-increasing computer power, combined with advances in laser technology. They use considerably less energy in the laser, typically on the order of one microjoule, and are often "eye-safe", meaning they can be used without safety precautions. High-power systems are common in atmospheric research, where they are widely used for measuring atmospheric parameters: the height, layering and densities of clouds, cloud particle properties (extinction coefficient, backscatter coefficient, depolarization), temperature, pressure, wind, humidity, and trace gas concentration (ozone, methane, nitrous oxide, etc.).^[4]

Components

Lidar systems consist of several major components.

Laser

600–1000 nm lasers are most common for non-scientific applications. Maximum power is limited by the need to make them eye-safe (in applications that operate around people).

One common alternative, 1550 nm lasers, are eye-safe at relatively high power levels since this wavelength is not strongly absorbed by the eye, but the detector technology is less advanced and so these wavelengths are generally used at longer ranges with lower accuracies. They are also used for military applications because 1550 nm is not visible in night vision goggles, unlike the shorter 1000 nm infrared laser.

Airborne topographic mapping lidars generally use 1064 nm diode pumped YAG lasers, while bathymetric (underwater depth research) systems generally use 532 nm frequency doubled diode pumped YAG lasers because 532 nm penetrates water with much less attenuation than does 1064 nm. Laser settings include the laser repetition rate (which controls the data collection speed). Pulse length is generally an attribute of the laser cavity length, the number of passes required through the gain material (YAG, YLF, etc.), and Q-switch (pulsing) speed. Better target resolution is achieved with shorter pulses, provided the lidar receiver detectors and electronics have sufficient bandwidth.^[4]

Solid state lasers

Solid-state electronics by definition have no moving parts.^[19]

Flash lidar

Flash uses a single light source that illuminates the field of view in a single pulse.^[19]

Seeing at a distance requires a powerful burst of light. The power is limited to levels that do not damage human retinas. Wavelengths must not affect human eyes. However, low cost silicon imagers do not read light in the eye-safe spectrum. Instead, gallium-arsenide imagers are required, which can boost costs to $200,000.^[19]

Phased arrays

A phased array can illuminate any direction by using a microscopic array of individual antennas. Controlling the timing (phase) of each antenna steers a cohesive signal in a specific direction.

Phased arrays have been used in radar since the 1950s. The same technique can be used with light. On the order of a million optical antennas are used to see a radiation pattern of a certain size in a certain direction. The system is controlled by timing the precise flash. A single chip (or a few) replace a $75,000 electromechanical system, drastically reducing costs.^[19]

The control system can change the shape of the lens to enable zoom in/zoom out functions. Specific sub-zones can be targeted at sub-second intervals.^[19]

Electromechanical lidar lasts for between 1,000 and 2,000 hours. By contrast, solid-state lidar can run for 100,000 hours.^[19]

Micro electromechanical mirrors

Micro electromechanical mirrors (MEMS) are not entirely solid-state. However, their tiny form factor provides many of the same cost benefits. A single laser is directed to a single mirror that can be reoriented to view any part of the target field. The mirror spins at a rapid rate. However, MEMS systems generally operate in a single plane (left to right). To add a second dimension generally requires a second mirror that moves up and down. Alternatively another laser can hit the same mirror from another angle. MEMS systems can be disrupted by shock/vibration and may require repeated calibration.^[19]

Scanner and optics

Image development speed is affected by the speed at which they are scanned. Options to scan the azimuth and elevation include dual oscillating plane mirrors, a combination with a polygon mirror and a dual axis scanner. Optic choices affect the angular resolution and range that can be detected. A hole mirror or a beam splitter are options to collect a return signal.

Photodetector and receiver electronics

Two main photodetector technologies are used in lidar: solid state photodetectors, such as silicon avalanche photodiodes, or photomultipliers. The sensitivity of the receiver is another parameter that has to be balanced in a lidar design.

Position and navigation systems

Lidar sensors mounted on mobile platforms such as airplanes or satellites require instrumentation to determine the absolute position and orientation of the sensor. Such devices generally include a Global Positioning System receiver and an Inertial Measurement Unit (IMU).

Sensor

Lidar uses active sensors that supply their own illumination source. The energy source hits objects and the reflected energy is detected and measured by sensors. Distance to the object is determined by recording the time between transmitted and backscattered pulses and by using the speed of light to calculate the distance traveled.^[20]

3-D imaging can be achieved using both scanning and non-scanning systems. "3-D gated viewing laser radar" is a non-scanning laser ranging system that applies a pulsed laser and a fast gated camera. Research has begun for virtual beam steering using Digital Light Processing (DLP) technology.

Imaging lidar can also be performed using arrays of high speed detectors and modulation sensitive detector arrays typically built on single chips using complementary metal–oxide–semiconductor (CMOS) and hybrid CMOS/Charge-coupled device (CCD) fabrication techniques. In these devices each pixel performs some local processing such as demodulation or gating at high speed, downconverting the signals to video rate so that the array can be read like a camera. Using this technique many thousands of pixels / channels may be acquired simultaneously.^[21] High resolution 3-D lidar cameras use homodyne detection with an electronic CCD or CMOS shutter.^[22]

A coherent imaging lidar uses synthetic array heterodyne detection to enable a staring single element receiver to act as though it were an imaging array.^[23]

In 2014, Lincoln Laboratory announced a new imaging chip with more than 16,384 pixels, each able to image a single photon, enabling them to capture a wide area in a single image. An earlier generation of the technology with one fourth that number of pixels was dispatched by the U.S. military after the January 2010 Haiti earthquake; a single pass by a business jet at 3,000 meters (10,000 ft.) over Port-au-Prince was able to capture instantaneous snapshots of 600-meter squares of the city at a resolution of 30 centimetres (12 in), displaying the precise height of rubble strewn in city streets.^[24] The Lincoln system is 10x faster. The chip uses indium gallium arsenide (InGaAs), which operates in the infrared spectrum at a relatively long wavelength that allows for higher power and longer ranges. In many applications, such as self-driving cars, the new system will lower costs by not requiring a mechanical component to aim the chip. InGaAs uses less hazardous wavelengths than conventional silicon detectors, which operate at visual wavelengths.^[25]

Types

Based on orientation

Lidar can be oriented to nadir, zenith, or laterally.
For example, lidar altimeters look down, an atmospheric lidar looks up, and lidar-based collision avoidance systems are side-looking.

Based on platform

Lidar applications can be divided into airborne and terrestrial types.^[26] The two types require scanners with varying specifications based on the data's purpose, the size of the area to be captured, the range of measurement desired, the cost of equipment, and more.
Spaceborne platforms are also possible.

Airborne

Airborne lidar (also airborne laser scanning) is when a laser scanner, while attached to an aircraft during flight, creates a 3-D point cloud model of the landscape. This is currently the most detailed and accurate method of creating digital elevation models, replacing photogrammetry. One major advantage in comparison with photogrammetry is the ability to filter out reflections from vegetation from the point cloud model to create a digital surface model which represents ground surfaces such as rivers, paths, cultural heritage sites, etc., which are concealed by trees. Within the category of airborne lidar, there is sometimes a distinction made between high-altitude and low-altitude applications, but the main difference is a reduction in both accuracy and point density of data acquired at higher altitudes. Airborne lidar can also be used to create bathymetric models in shallow water.^[27]

The main constituents of airborne lidar include digital elevation models (DEM) and digital surface models (DSM). The points and ground points are the vectors of discrete points while DEM and DSM are interpolated raster grids of discrete points. The process also involves capturing of digital aerial photographs. In order to interpret deep seated landslides for example, under the cover of vegetation, scarps, tension cracks or tipped trees air borne lidar is used. Air borne lidar digital elevation models can see through the canopy of forest cover, perform detailed measurements of scarps, erosion and tilting of electric poles.^[28]

Airborne lidar data is processed using a toolbox called Toolbox for Lidar Data Filtering and Forest Studies (TIFFS)^[29] for lidar data filtering and terrain study software. The data is interpolated to digital terrain models using the software. The laser is directed at the region to be mapped and each point's height above the ground is calculated by subtracting the original z-coordinate from the corresponding digital terrain model elevation. Based on this height above the ground the non-vegetation data is obtained which may include objects such as buildings, electric power lines, flying birds, etc. The rest of the points are treated as vegetation and used for modeling and mapping. Within each of these plots, lidar metrics are calculated by calculating statistics such as mean, standard deviation, skewness, percentiles, quadratic mean, etc.^[29]

Airborne Lidar Bathymetric Technology-High-resolution multibeam lidar map showing spectacularly faulted and deformed seafloor geology, in shaded relief and coloured by depth.

Airborne lidar bathymetry

The airborne lidar bathymetric technological system involves the measurement of time of flight of a signal from a source to its return to the sensor. The data acquisition technique involves a sea floor mapping component and a ground truth component that includes video transects and sampling. It works using a green spectrum (532 nm) laser beam.^[30] Two beams are projected onto a fast rotating mirror, which creates an array of points. One of the beams penetrates the water and also detects the bottom surface of the water under favorable conditions.

The data obtained shows the full extent of the land surface exposed above the sea floor. This technique is extremely useful as it will play an important role in the major sea floor mapping program. The mapping yields onshore topography as well as under water elevations. Sea floor reflectance imaging is another solution product from this system which can benefit mapping of underwater habitats. This technique has been used for three dimensional image mapping of California's waters using a hydrographic lidar.^[31]

LIDAR scanning performed with a multicopter UAV.

Drones are now being used with laser scanners, as well as other remote sensors, as a more economical method to scan smaller areas.^[32] The possibility of drone remote sensing also eliminates any danger that crews of a manned aircraft may be subjected to in difficult terrain or remote areas.

Terrestrial

Terrestrial applications of lidar (also terrestrial laser scanning) happen on the Earth's surface and can be either stationary or mobile. Stationary terrestrial scanning is most common as a survey method, for example in conventional topography, monitoring, cultural heritage documentation and forensics.^[26] The 3-D point clouds acquired from these types of scanners can be matched with digital images taken of the scanned area from the scanner's location to create realistic looking 3-D models in a relatively short time when compared to other technologies. Each point in the point cloud is given the colour of the pixel from the image taken located at the same angle as the laser beam that created the point.

Mobile lidar (also mobile laser scanning) is when two or more scanners are attached to a moving vehicle to collect data along a path. These scanners are almost always paired with other kinds of equipment, including GNSS receivers and IMUs. One example application is surveying streets, where power lines, exact bridge heights, bordering trees, etc. all need to be taken into account. Instead of collecting each of these measurements individually in the field with a tachymeter, a 3-D model from a point cloud can be created where all of the measurements needed can be made, depending on the quality of the data collected. This eliminates the problem of forgetting to take a measurement, so long as the model is available, reliable and has an appropriate level of accuracy.

Terrestrial lidar mapping involves a process of occupancy grid map generation. The process involves an array of cells divided into grids which employs a process to store the height values when lidar data falls into the respective grid cell. A binary map is then created by applying a particular threshold to the cell values for further processing. The next step is to process the radial distance and z-coordinates from each scan to identify which 3-D points correspond to each of the specified grid cell leading to the process of data formation.^[33]

Applications

This mobile robot uses its lidar to construct a map and avoid obstacles.

There are a wide variety of applications for lidar, in addition to the applications listed below, as it is often mentioned in National lidar dataset programs.

Agriculture

Lidar is used to analyze yield rates on agricultural fields.

Agricultural robots have been used for a variety of purposes ranging from seed and fertilizer dispersions, sensing techniques as well as crop scouting for the task of weed control.

Lidar can help determine where to apply costly fertilizer. It can create a topographical map of the fields and reveal slopes and sun exposure of the farm land. Researchers at the Agricultural Research Service used this topographical data with the farmland yield results from previous years, to categorize land into zones of high, medium, or low yield.^[34] This indicates where to apply fertilizer to maximize yield.

Another application is crop mapping in orchards and vineyards, to detect foliage growth and the need for pruning or other maintenance, detect variations in fruit production, or count plants.

Lidar is useful in GPS-denied situations, such as nut and fruit orchards, where foliage blocks GPS signals to precision agriculture equipment or a driverless tractor. Lidar sensors can detect the edges of rows, so that farming equipment can continue moving until GPS signal is reestablished.

Plant species classification

Controlling weeds requires identifying plant species. This can be done by using 3-D lidar and machine learning.^[35] Lidar produces plant contours as a "point cloud" with range and reflectance values. This data is transformed, and features are extracted from it. If the species is known, the features are added as new data. The species is labelled and its features are initially stored as an example to identify the species in the real environment. This method is efficient because it uses a low-resolution lidar and supervised learning. It includes an easy-to-compute feature set with common statistical features which are independent of the plant size.^[35]

Archaeology

Lidar has many uses in archaeology, including planning of field campaigns, mapping features under forest canopy, and overview of broad, continuous features indistinguishable from the ground.^[36] Lidar can produce high-resolution datasets quickly and cheaply. Lidar-derived products can be easily integrated into a Geographic Information System (GIS) for analysis and interpretation.

Lidar can also help to create high-resolution digital elevation models (DEMs) of archaeological sites that can reveal micro-topography that is otherwise hidden by vegetation. The intensity of the returned lidar signal can be used to detect features buried under flat vegetated surfaces such as fields, especially when mapping using the infrared spectrum. The presence of these features affects plant growth and thus the amount of infrared light reflected back.^[37] For example, at Fort Beauséjour – Fort Cumberland National Historic Site, Canada, lidar discovered archaeological features related to the siege of the Fort in 1755. Features that could not be distinguished on the ground or through aerial photography were identified by overlaying hill shades of the DEM created with artificial illumination from various angles. Another example is work at Caracol by Arlen Chase and his wife Diane Zaino Chase.^[38] In 2012, lidar was used to search for the legendary city of La Ciudad Blanca or "City of the Monkey God" in the La Mosquitia region of the Honduran jungle. During a seven-day mapping period, evidence was found of man-made structures.^[39][40] In June 2013, the rediscovery of the city of Mahendraparvata was announced.^[41] In southern New England, lidar was used to reveal stone walls, building foundations, abandoned roads, and other landscape features obscured in aerial photography by the region's dense forest canopy.^[42][43][44] In Cambodia, lidar data were used by Demian Evans and Roland Fletcher to reveal anthropogenic changes to Angkor landscape ^[45] In 2016, Lidar was used to map ancient Maya causeways in northern Guatemala, discovering 17 elevated roads linking the ancient city of El Mirador to other sites.^[46][47] In 2018, archaeologists using lidar discovered more than 60,000 man-made structures in the Maya Biosphere Reserve, a "major breakthrough" that showed the Maya civilization was much larger than previously thought.^[48]

Autonomous vehicles

Uber self-driving car with lidar system on roof

Forecast 3-D Laser System using a SICK LMC lidar sensor

Autonomous vehicles may use lidar for obstacle detection and avoidance to navigate safely through environments, using rotating laser beams.^[49] Cost map or point cloud outputs from the lidar sensor provide the necessary data for robot software to determine where potential obstacles exist in the environment and where the robot is in relation to those potential obstacles. Singapore's Singapore-MIT Alliance for Research and Technology (SMART) is actively developing technologies for autonomous lidar vehicles.^[50] Examples of companies that produce lidar sensors commonly used in robotics or vehicle automation are Sick^[51] and Hokuyo.^[52] Examples of obstacle detection and avoidance products that leverage lidar sensors are the Autonomous Solution, Inc. Forecast 3D Laser System^[53] and Velodyne HDL-64E.^[54] Lidar simulation models are also provided in autonomous car simulators.^[55]

The very first generations of automotive adaptive cruise control systems used only lidar sensors.

Object detection for transportation systems

It has been suggested that this section be split out into another article titled Lidar for collision detection in transportation. (Discuss) (September 2018)

In transportation systems, to ensure vehicle and passenger safety and to develop electronic systems that deliver driver assistance, understanding vehicle and its surrounding environment is essential. Lidar systems play an important role in the safety of transportation systems. Lots of electronic systems which add to the driver assistance and vehicle safety such as Adaptive Cruise Control (ACC), Emergency Brake Assist, Anti-lock Braking System (ABS) depend on the detection of a vehicle's environment to act autonomously or semi-autonomously. Lidar mapping and estimation achieve this.

Basics overview: Current lidar systems use rotating hexagonal mirrors which split the laser beam. The upper three beams are used for vehicle and obstacles ahead and the lower beams are used to detect lane markings and road features.^[56] The major advantage of using lidar is that the spatial structure is obtained and this data can be fused with other sensors such as radar, etc. to get a better picture of the vehicle environment in terms of static and dynamic properties of the objects present in the environment.

Below mentioned are various approaches of processing lidar data and using it along with data from other sensors through sensor fusion to detect the vehicle environment conditions.

GRID based processing using 3-D lidar and fusion with radar measurement

In this method, proposed by Philipp Lindner and Gerd Wanielik, laser data is processed using a multidimensional occupancy grid.^[57] Data from a four-layer laser is pre-processed at the signal level and then processed at a higher level to extract the features of the obstacles. A combination two- and three-dimensional grid structure is used and the space in these structures is tessellated into several discrete cells. This method allows a huge amount of raw measurement data to be effectively handled by collecting it in spatial containers, the cells of the evidence grid. Each cell is associated with a probability measure that identifies the cell occupation. This probability is calculated by using the range measurement of the lidar sensor obtained over time and a new range measurement, which are related using Bayes' theorem. A two dimensional grid can observe an obstacle in front of it, but cannot observe the space behind the obstacle. To address this, the unknown state behind the obstacle is assigned a probability of 0.5. By introducing the third dimension or in other terms using a multi-layer laser, the spatial configuration of an object could be mapped into the grid structure to a degree of complexity. This is achieved by transferring the measurement points into a three-dimensional grid. The grid cells which are occupied will possess a probability greater than 0.5 and the mapping would be color coded based on the probability. The cells that are not occupied will possess a probability less than 0.5 and this area will usually be white space. This measurement is then transformed to a grid coordinate system by using the sensor position on the vehicle and the vehicle position in the world coordinate system. The coordinates of the sensor depends upon its location on the vehicle and the coordinates of the vehicle is computed using egomotion estimation, which is estimating the vehicle motion relative to a rigid scene. For this method, the grid profile must be defined. The grid cells touched by the transmitted laser beam are calculated by applying Bresenham's line algorithm. To obtain the spatial extended structure, a connected component analysis of these cells is performed. This information is then passed on to a rotating caliper algorithm to obtain the spatial characteristics of the object. In addition to the lidar detection, RADAR data obtained by using two short range radars is integrated to get additional dynamic properties of the object, such as its velocity. The measurements are assigned to the object using a potential distance function.

Advantages and disadvantages

The geometric features of the objects are extracted efficiently, from the measurements obtained by the 3-D occupancy grid, using rotating caliper algorithm. Fusing the radar data to the lidar measurements give information about the dynamic properties of the obstacle such as velocity and location of the obstacle with respect to the sensor location which helps the vehicle or the driver decide the action to be performed in order to ensure safety. The only concern is the computational requirement to implement this data processing technique. It can be implemented in real time and has been proven efficient if the 3-D occupancy grid size is considerably restricted. But this can be improved to an even wider range by using dedicated spatial datastructures that manipulate the spatial data more effectively, for the 3-D grid representation.

Fusion of 3-D lidar and color camera for multiple object detection and tracking

The framework proposed in this method by Soonmin Hwang et al.,^[58] is split into four steps. First, the data from the camera and 3-D lidar is input into the system. Both inputs from lidar and camera are parallelly obtained and the color image from the camera is calibrated with the lidar. To improve the efficiency, horizontal 3-D point sampling is applied as pre-processing. Second, the segmentation stage is where the entire 3-D points are divided into several groups per the distance from the sensor and local planes from close plane to far plane are sequentially estimated. The local planes are estimated using statistical analysis. The group of points closer to the sensor are used to compute the initial plane. By using the current local plane, the next local plane is estimated by iterative update. The object proposals in the 2-D image are used to separate foreground objects from background. For faster and accurate detection and tracking Binarized Normed Gradients for Objectness Estimation at 300fps is used.^[59] BING is a combination of normed gradient and its binarized version which speeds up the feature extraction and testing process, to estimate the objectness of an image window. This way the foreground and background objects are separated. To form objects after estimating the objectness of an image using BING, the 3-D points are grouped or clustered. Clustering is done using DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm which could be robust due to its less-parametric characteristic. Using the clustered 3-D points, i.e. 3-D segment, more accurate region-of-interests (RoIs) are generated by projecting 3-D points on the 2-D image. The third step is detection, which is broadly divided into two parts. First is object detection in 2-D image which is achieved using Fast R-CNN^[60] as this method doesn't need training and it also considers an image and several regions of interest. Second is object detection in 3-D space that is done by using the spin image method.^[61] This method extracts local and global histograms to represent a certain object. To merge the results of 2-D image and 3-D space object detection, same 3-D region is considered and two independent classifiers from 2-D image and 3-D space are applied to the considered region. Scores calibration^[62] is done to get a single confidence score from both detectors. This single score is obtained in the form of probability. The final step is tracking. This is done by associating moving objects in present and past frame. For object tracking, segment matching is adopted. Features such as mean, standard deviation, quantized color histograms, volume size and number of 3-D points of a segment are computed. Euclidean distance is used to measure differences between segments. To judge the appearance and disappearance of an object, similar segments (obtained based on the Euclidean distance) from two different frames are taken and the physical distance and dissimilarity scores are calculated. If the scores go beyond a range for every segment in previous frame, the object being tracked is considered to have disappeared.

Advantages and disadvantages

The advantages of this method are using 2-D image and 3-D data together, F l-score (which gives a measure of test's accuracy), average precision (AP) are higher than that when only 3-D data from lidar is used. These scores are conventional measurements which judge the framework. The drawback of this method is the usage of BING for object proposal estimation as BING predicts a small set of object bounding boxes.

Obstacle detection and road environment recognition using lidar

This method proposed by Kun Zhou et al.^[63] not only focuses on object detection and tracking but also recognizes lane marking and road features. As mentioned earlier the lidar systems use rotating hexagonal mirrors that split the laser beam into six beams. The upper three layers are used to detect the forward objects such as vehicles and roadside objects. The sensor is made of weather-resistant material. The data detected by lidar are clustered to several segments and tracked by Kalman filter. Data clustering here is done based on characteristics of each segment based on object model, which distinguish different objects such as vehicles, signboards, etc. These characteristics include the dimensions of the object, etc. The reflectors on the rear edges of vehicles are used to differentiate vehicles from other objects. Object tracking is done using a 2-stage Kalman filter considering the stability of tracking and the accelerated motion of objects^[56] Lidar reflective intensity data is also used for curb detection by making use of robust regression to deal with occlusions. The road marking is detected using a modified Otsu method by distinguishing rough and shiny surfaces.^[64]

Advantages

Roadside reflectors that indicate lane border are sometimes hidden due to various reasons. Therefore, other information is needed to recognize the road border. The lidar used in this method can measure the reflectivity from the object. Hence, with this data road border can also be recognized. Also the usage of sensor with weather-robust head helps detecting the objects even in bad weather conditions. Canopy Height Model before and after flood is a good example. Lidar can detect high detailed canopy height data as well as its road border.

Lidar measurements help identify the spatial structure of the obstacle. This helps distinguish objects based on size and estimate the impact of driving over it.^[57]

Lidar systems provides better range and a large field of view which helps detecting obstacles on the curves. This is one major advantage over RADAR systems which have a narrower field of view. The fusion of lidar measurement with different sensors makes the system robust and useful in real-time applications, since lidar dependent systems can't estimate the dynamic information about the detected object.^[57]

It has been shown that lidar can be manipulated, such that self-driving cars are tricked into taking evasive action.^[65]

Biology and conservation

Lidar imaging comparing old-growth forest (right) to a new plantation of trees (left)

Lidar has also found many applications in forestry. Canopy heights, biomass measurements, and leaf area can all be studied using airborne lidar systems. Similarly, lidar is also used by many industries, including Energy and Railroad, and the Department of Transportation as a faster way of surveying. Topographic maps can also be generated readily from lidar, including for recreational use such as in the production of orienteering maps.^[66]

In addition, the Save the Redwoods League has undertaking a project to map the tall redwoods on the Northern California coast. Lidar allows research scientists to not only measure the height of previously unmapped trees, but to determine the biodiversity of the redwood forest. Stephen Sillett, who is working with the League on the North Coast lidar project, claims this technology will be useful in directing future efforts to preserve and protect ancient redwood trees.^{[67][full citation needed]}

Geology and soil science

High-resolution digital elevation maps generated by airborne and stationary lidar have led to significant advances in geomorphology (the branch of geoscience concerned with the origin and evolution of the Earth surface topography). The lidar abilities to detect subtle topographic features such as river terraces and river channel banks, to measure the land-surface elevation beneath the vegetation canopy, to better resolve spatial derivatives of elevation, and to detect elevation changes between repeat surveys have enabled many novel studies of the physical and chemical processes that shape landscapes.^[68]
In 2005 the Tour Ronde in the Mont Blanc massif became the first high alpine mountain on which lidar was employed to monitor the increasing occurrence of severe rock-fall over large rock faces allegedly caused by climate change and degradation of permafrost at high altitude.^[69]

In geophysics and tectonics, a combination of aircraft-based lidar and GPS has evolved into an important tool for detecting faults and for measuring uplift. The output of the two technologies can produce extremely accurate elevation models for terrain – models that can even measure ground elevation through trees. This combination was used most famously to find the location of the Seattle Fault in Washington, United States.^[70] This combination also measures uplift at Mt. St. Helens by using data from before and after the 2004 uplift.^[71] Airborne lidar systems monitor glaciers and have the ability to detect subtle amounts of growth or decline. A satellite-based system, the NASA ICESat, includes a lidar sub-system for this purpose. The NASA Airborne Topographic Mapper^[72] is also used extensively to monitor glaciers and perform coastal change analysis.
The combination is also used by soil scientists while creating a soil survey. The detailed terrain modeling allows soil scientists to see slope changes and landform breaks which indicate patterns in soil spatial relationships.

Atmosphere

Initially based on ruby lasers, lidar for meteorological applications was constructed shortly after the invention of the laser and represent one of the first applications of laser technology. Lidar technology has since expanded vastly in capability and lidar systems are used to perform a range of measurements that include profiling clouds, measuring winds, studying aerosols, and quantifying various atmospheric components. Atmospheric components can in turn provide useful information including surface pressure (by measuring the absorption of oxygen or nitrogen), greenhouse gas emissions (carbon dioxide and methane), photosynthesis (carbon dioxide), fires (carbon monoxide), and humidity (water vapor). Atmospheric lidars can be either ground-based, airborne or satellite depending on the type of measurement.

Atmospheric lidar remote sensing works in two ways –

1. by measuring backscatter from the atmosphere, and


2. by measuring the scattered reflection off the ground (when the lidar is airborne) or other hard surface.

Backscatter from the atmosphere directly gives a measure of clouds and aerosols. Other derived measurements from backscatter such as winds or cirrus ice crystals require careful selecting of the wavelength and/or polarization detected. Doppler lidar and Rayleigh Doppler lidar are used to measure temperature and/or wind speed along the beam by measuring the frequency of the backscattered light. The Doppler broadening of gases in motion allows the determination of properties via the resulting frequency shift.^[73][74] Scanning lidars, such as the conical-scanning NASA HARLIE LIDAR, have been used to measure atmospheric wind velocity.^[75] The ESA wind mission ADM-Aeolus will be equipped with a Doppler lidar system in order to provide global measurements of vertical wind profiles.^[76] A doppler lidar system was used in the 2008 Summer Olympics to measure wind fields during the yacht competition.^[77]

Doppler lidar systems are also now beginning to be successfully applied in the renewable energy sector to acquire wind speed, turbulence, wind veer, and wind shear data. Both pulsed and continuous wave systems are being used. Pulsed systems use signal timing to obtain vertical distance resolution, whereas continuous wave systems rely on detector focusing.

The term, eolics, has been proposed to describe the collaborative and interdisciplinary study of wind using computational fluid mechanics simulations and Doppler lidar measurements.^[78]

The ground reflection of an airborne lidar gives a measure of surface reflectivity (assuming the atmospheric transmittance is well known) at the lidar wavelength, however, the ground reflection is typically used for making absorption measurements of the atmosphere. "Differential absorption lidar" (DIAL) measurements utilize two or more closely spaced (<1 nm) wavelengths to factor out surface reflectivity as well as other transmission losses, since these factors are relatively insensitive to wavelength. When tuned to the appropriate absorption lines of a particular gas, DIAL measurements can be used to determine the concentration (mixing ratio) of that particular gas in the atmosphere. This is referred to as an Integrated Path Differential Absorption (IPDA) approach, since it is a measure of the integrated absorption along the entire lidar path. IPDA lidars can be either pulsed^[79][80] or CW^[81] and typically use two or more wavelengths.^[82] IPDA lidars have been used for remote sensing of carbon dioxide^[79][80][81] and methane.^[83]

Synthetic array lidar allows imaging lidar without the need for an array detector. It can be used for imaging Doppler velocimetry, ultra-fast frame rate (MHz) imaging, as well as for speckle reduction in coherent lidar.^[23] An extensive lidar bibliography for atmospheric and hydrospheric applications is given by Grant.^[84]

Scheimpflug principle

Another lidar technique for atmospheric remote sensing has emerged. It is based on Scheimpflug principle, referred to as Scheimpflug lidar (slidar).

"The implication of the Scheimpflug principle is that when a laser beam is transmitted into the atmosphere, the backscattering echo of the entire illuminating probe volume is still in focus simultaneously without diminishing the aperture as long as the object plane, image plane and the lens plane intersect with each other".^[85] A two dimensional CCD/CMOS camera is used to resolve the backscattering echo of the transmitted laser beam.

Thus as in the case of conventional lidar technologies continuous wave light sources such as diode lasers can be employed for remote sensing instead of using complicated nanosecond pulse light sources.^[85] The SLidar system is also a robust and inexpensive system based on compact laser diodes and array detectors.^[85]

Law enforcement

Lidar speed guns are used by the police to measure the speed of vehicles for speed limit enforcement purposes.^[86] Additionally, it is used in forensics to aid in crime scene investigations. Scans of a scene are taken to record exact details of object placement, blood, and other important information for later review. These scans can also be used to determine bullet trajectory in cases of shootings.^[87]

Military

Few military applications are known to be in place and are classified (such as the lidar-based speed measurement of the AGM-129 ACM stealth nuclear cruise missile), but a considerable amount of research is underway in their use for imaging. Higher resolution systems collect enough detail to identify targets, such as tanks. Examples of military applications of lidar include the Airborne Laser Mine Detection System (ALMDS) for counter-mine warfare by Areté Associates.^[88]

A NATO report (RTO-TR-SET-098) evaluated the potential technologies to do stand-off detection for the discrimination of biological warfare agents. The potential technologies evaluated were Long-Wave Infrared (LWIR), Differential Scattering (DISC), and Ultraviolet Laser Induced Fluorescence (UV-LIF). The report concluded that : Based upon the results of the lidar systems tested and discussed above, the Task Group recommends that the best option for the near-term (2008–2010) application of stand-off detection systems is UV-LIF ,^[89] however, in the long-term, other techniques such as stand-off Raman spectroscopy may prove to be useful for identification of biological warfare agents.

Short-range compact spectrometric lidar based on Laser-Induced Fluorescence (LIF) would address the presence of bio-threats in aerosol form over critical indoor, semi-enclosed and outdoor venues such as stadiums, subways, and airports. This near real-time capability would enable rapid detection of a bioaerosol release and allow for timely implementation of measures to protect occupants and minimize the extent of contamination.^[90]

The Long-Range Biological Standoff Detection System (LR-BSDS) was developed for the U.S. Army to provide the earliest possible standoff warning of a biological attack. It is an airborne system carried by a helicopter to detect synthetic aerosol clouds containing biological and chemical agents at long range. The LR-BSDS, with a detection range of 30 km or more, was fielded in June 1997.^[91]
Five lidar units produced by the German company Sick AG were used for short range detection on Stanley, the autonomous car that won the 2005 DARPA Grand Challenge.

A robotic Boeing AH-6 performed a fully autonomous flight in June 2010, including avoiding obstacles using lidar.^[92][93]

Mining

For The calculation of ore volumes is accomplished by periodic (monthly) scanning in areas of ore removal, then comparing surface data to the previous scan.^[94]

Lidar sensors may also be used for obstacle detection and avoidance for robotic mining vehicles such as in the Komatsu Autonomous Haulage System (AHS)^[95] used in Rio Tinto's Mine of the Future.

Physics and astronomy

A worldwide network of observatories uses lidars to measure the distance to reflectors placed on the moon, allowing the position of the moon to be measured with millimeter precision and tests of general relativity to be done. MOLA, the Mars Orbiting Laser Altimeter, used a lidar instrument in a Mars-orbiting satellite (the NASA Mars Global Surveyor) to produce a spectacularly precise global topographic survey of the red planet.

In September, 2008, the NASA Phoenix Lander used lidar to detect snow in the atmosphere of Mars.^[96]

In atmospheric physics, lidar is used as a remote detection instrument to measure densities of certain constituents of the middle and upper atmosphere, such as potassium, sodium, or molecular nitrogen and oxygen. These measurements can be used to calculate temperatures. Lidar can also be used to measure wind speed and to provide information about vertical distribution of the aerosol particles.^[97]

At the JET nuclear fusion research facility, in the UK near Abingdon, Oxfordshire, lidar Thomson Scattering is used to determine Electron Density and Temperature profiles of the plasma.^[98]

Rock mechanics

Lidar has been widely used in rock mechanics for rock mass characterization and slope change detection. Some important geomechanical properties from the rock mass can be extracted from the 3-D point clouds obtained by means of the lidar. Some of these properties are:

• Discontinuity orientation ^{[99][100][101]}
  


• Discontinuity spacing and RQD ^{[101][102][103]}
  


• Discontinuity aperture


• Discontinuity persistence ^{[101][103][104]}
  


• Discontinuity roughness ^[103]
  


• Water infiltration

Some of these properties have been used to assess the geomechanical quality of the rock mass through the RMR index. Moreover, as the orientations of discontinuities can be extracted using the existing methodologies, it is possible to assess the geomechanical quality of a rock slope through the SMR index.^[105] In addition to this, the comparison of different 3-D point clouds from a slope acquired at different times allows researchers to study the changes produced on the scene during this time interval as a result of rockfalls or any other landsliding processes.^{[106][107][108]}

THOR

THOR is a laser designed toward measuring Earth's atmospheric conditions. The laser enters a cloud cover^[109] and measures the thickness of the return halo. The sensor has a fiber optic aperture with a width of 7.5 inches that is used to measure the return light.

Robotics

Lidar technology is being used in robotics for the perception of the environment as well as object classification.^[110] The ability of lidar technology to provide three-dimensional elevation maps of the terrain, high precision distance to the ground, and approach velocity can enable safe landing of robotic and manned vehicles with a high degree of precision.^[111] Lidar are also widely used in robotics for simultaneous localization and mapping and well integrated into robot simulators.^[112] Refer to the Military section above for further examples.

Spaceflight

Lidar is increasingly being utilized for rangefinding and orbital element calculation of relative velocity in proximity operations and stationkeeping of spacecraft. Lidar has also been used for atmospheric studies from space. Short pulses of laser light beamed from a spacecraft can reflect off of tiny particles in the atmosphere and back to a telescope aligned with the spacecraft laser. By precisely timing the lidar 'echo,' and by measuring how much laser light is received by the telescope, scientists can accurately determine the location, distribution and nature of the particles. The result is a revolutionary new tool for studying constituents in the atmosphere, from cloud droplets to industrial pollutants, which are difficult to detect by other means."^[113][114]

Surveying

This TomTom mapping van is fitted with five lidars on its roof rack.

Airborne lidar sensors are used by companies in the remote sensing field. They can be used to create a DTM (Digital Terrain Model) or DEM (Digital Elevation Model); this is quite a common practice for larger areas as a plane can acquire 3–4 km wide swaths in a single flyover. Greater vertical accuracy of below 50 mm can be achieved with a lower flyover, even in forests, where it is able to give the height of the canopy as well as the ground elevation. Typically, a GNSS receiver configured over a georeferenced control point is needed to link the data in with the WGS (World Geodetic System).^[115]

LiDAR are also in use in hydrographic surveying. Depending upon the clarity of the water LiDAR can measure depths from 0.9m to 40m with a vertical accuracy of 15 cm and horizontal accuracy of 2.5m.^[116]

Forestry

Lidar systems have also been applied to improve forestry management.^[117] Measurements are used to take inventory in forest plots as well as calculate individual tree heights, crown width and crown diameter. Other statistical analysis use lidar data to estimate total plot information such as canopy volume, mean, minimum and maximum heights, and vegetation cover estimates.

Transport

Lidar has been used in the railroad industry to generate asset health reports for asset management and by departments of transportation to assess their road conditions. CivilMaps.com is a leading company in the field.^[118] Lidar has been used in adaptive cruise control (ACC) systems for automobiles. Systems such as those by Siemens, Hella, and Cepton use a lidar device mounted on the front of the vehicle, such as the bumper, to monitor the distance between the vehicle and any vehicle in front of it.^[119] In the event the vehicle in front slows down or is too close, the ACC applies the brakes to slow the vehicle. When the road ahead is clear, the ACC allows the vehicle to accelerate to a speed preset by the driver. Refer to the Military section above for further examples. A lidar-based device, the Ceilometer is used at airports worldwide to measure the height of clouds on runway approach paths.^{[citation needed]}

Wind farm optimization

Lidar can be used to increase the energy output from wind farms by accurately measuring wind speeds and wind turbulence.^[120][121] Experimental lidar systems^[122][123] can be mounted on the nacelle^[124] of a wind turbine or integrated into the rotating spinner^[125] to measure oncoming horizontal winds,^[126] winds in the wake of the wind turbine,^[127] and proactively adjust blades to protect components and increase power. Lidar is also used to characterise the incident wind resource for comparison with wind turbine power production to verify the performance of the wind turbine^[128] by measuring the wind turbine's power curve.^[129] Wind farm optimization can be considered a topic in applied eolics. Another aspect of Lidar in wind related industry is to use computational fluid dynamics over Lidar-scanned surfaces in order to assess the wind potential,^[130] which can be used for optimal wind farms placement.

Solar photovoltaic deployment optimization

Lidar can also be used to assist planners and developers in optimizing solar photovoltaic systems at the city level by determining appropriate roof tops ^[131][132] and for determining shading losses.^[133] Recent airborne laser scanning efforts have focused on ways to estimate the amount of solar light hitting vertical building facades,^[134] or by incorporating more detailed shading losses by considering the influence from vegetation and larger surrounding terrain.^[135]

Video games

Recent simulation racing games such as iRacing, Assetto Corsa and Project CARS increasingly feature race tracks reproduced from 3-D point clouds acquired through Lidar surveys, resulting in surfaces replicated with millimeter precision in the in-game 3-D environment.

The 2017 exploration game Scanner Sombre, by Introversion Software, uses Lidar as a fundamental game mechanic.

Other uses

The video for the song "House of Cards" by Radiohead was believed to be the first use of real-time 3-D laser scanning to record a music video. The range data in the video is not completely from a lidar, as structured light scanning is also used.^[136]

Alternative technologies

Recent development of Structure From Motion (SFM) technologies allows delivering 3-D images and maps based on data extracted from visual and IR photography. The elevation or 3-D data is extracted using multiple parallel passes over mapped area, yielding both visual light images and 3-D structure from the same sensor, which is often a specially chosen and calibrated digital camera.^{[citation needed]}

See also

References

Further reading

• Heritage, E. (2011). 3D laser scanning for heritage. Advice and guidance to users on laser scanning in archaeology and architecture. Available at www.english-heritage.org.uk. [2]
  


• Heritage, G., & Large, A. (Eds.). (2009). Laser scanning for the environmental sciences. John Wiley & Sons.
  ISBN 1-4051-5717-8
  


• Maltamo, M., Næsset, E., & Vauhkonen, J. (2014). Forestry Applications of Airborne Laser Scanning: Concepts and Case Studies (Vol. 27). Springer Science & Business Media.
  ISBN 94-017-8662-3
  


• Shan, J., & Toth, C. K. (Eds.). (2008). Topographic laser ranging and scanning: principles and processing. CRC press.
  ISBN 1-4200-5142-3
  


• Vosselman, G., & Maas, H. G. (Eds.). (2010). Airborne and terrestrial laser scanning. Whittles Publishing.
  ISBN 1-4398-2798-2
  

External links

Wikimedia Commons has media related to LIDAR.

• 
  The USGS Center for LIDAR Information Coordination and Knowledge (CLICK) – A website intended to "facilitate data access, user coordination and education of lidar remote sensing for scientific needs."


• Free online lidar data viewer


• Lidar Industry News and Education