Methane detection limits, emission rate quantification accuracy, and potential cross-species interference are assessed for Bridger Photonics’ Gas Mapping LiDAR (GML) system utilizing data collected during laboratory testing and single-blind controlled release testing. Laboratory testing identified no significant interference in the path-integrated methane measurement from the gas species tested (ethylene, ethane, propane, n-butane, i-butane, and carbon dioxide). The controlled release study, comprised of 650 individual measurement passes, represents the largest dataset collected to date to characterize GML with respect to point-source emissions. Binomial regression is utilized to create detection curves illustrating the likelihood of detecting an emission of a given size under different wind conditions and for different flight altitudes. Wind-normalized methane detection limits (90% detection rate) of 0.25 (kg/h)/(m/s) and 0.41 (kg/h)/(m/s) are observed at a flight altitude of 500 feet and 675 feet above ground level, respectively. Quantification accuracy is also assessed for emissions ranging from 0.15 to 1,400 kg/h. When emission rate estimates were generated using wind from high-resolution rapid refresh (HRRR) model (the primary wind source that Bridger uses for their commercial operations), linear regression indicates bias of 8.1% (R2 = 0.89). For 95% of controlled releases above Bridger’s stated production-sector detection sensitivity (3 kg/h with 90% probability of detection), the accuracy of individual emission rate estimates produced using HRRR wind ranged from −64.1% to +87.0%. Across all controlled releases, 38.1% of estimates had error within ±20%, and 87.3% of measurements were within a factor of two (−50% to +100% error). At low wind speed (less than 2 m/s) and low emission rates (less than 3 kg/h), emission estimates are biased high, however when removed do not impact the regression significantly. The aggregate quantification error including all detected emission events was +8.2% using the HRRR wind source. The resulting detection curves and quantification accuracy illustrate important implications that must be considered when using measurements from GML or other remote emission measurement techniques to inform or validate inventory models or to audit reported emission levels from oil and gas systems.

Bridger Photonics’ (2021) Gas Mapping LiDAR (GML) is a remote sensing system deployed by aircraft to detect and quantify methane emissions. The solution uses the absorption of light emitted from a tuned laser to produce a measurement of the path-integrated methane gas concentration between the aircraft and the ground. As the aircraft travels forward, the GML laser beam is scanned to produce a map of the path-integrated gas concentration. Aerial digital photography and topographical LiDAR are also acquired and used to provide attribution of measured emissions and processing tasks. Emission quantification estimates are produced using proprietary processing techniques, which take into account the measured gas concentrations, height information of the plume, and a local wind profile estimated by high-resolution rapid refresh (HRRR) and/or North American Mesoscale Forecast System (NAM12) (Benjamin et al., 2016; National Centers for Environmental Information, n.d.). In this study, we assess potential cross-species interference, methane detection limits, and quantification accuracy of the GML system.

Changes in regulatory requirements, reporting standards, and emerging certified gas programs are driving the oil and gas industry to consider new methods for detection and quantification of emissions, including continuous monitoring methods, mobile solutions, and remote sensing systems by aircraft and satellite. For example, the U.S. Environmental Protection Agency (2021) has proposed updates to the fugitive monitoring requirements allowing operators to use advanced leak detection technologies that can meet a rigorous minimum detection threshold; Cheniere Energy, the largest U.S. producer of liquefied natural gas, recently announced its quantification, monitoring, reporting, and verification program, which drives emission monitoring requirements up the company’s supply chain to midstream and upstream operations (Cheniere Energy, Inc., 2021); Certified gas programs, such as MiQ, Project Canary’s TrustWell Certification, and GTI’s Veritas program, seek to incentivize environmentally responsible gas production and each includes a set of methane monitoring and measurements standards (Pro, 2021; Ver, 2021; MiQ, n.d.); finally, methane emissions are also receiving international attention with the recent launch of the International Methane Emissions Observatory by the UN Environment Programme and the OGMP2.0 reporting framework (UN Environment Programme, 2021; OGMP, n.d.).

Remote emission detection methods provide an effective way to rapidly screen many facilities for emissions. Even though these systems may not be capable of detecting emission sources as small as a traditional leak detection and repair (LDAR) inspection, there is evidence that some remote sensing systems detect greater overall emissions than traditional LDAR inspections with equivalent monitoring frequency (Tyner and Johnson, 2021). Rapid screening of many facilities also potentially enables greater emissions mitigation by detecting and repairing large emission sources faster (Fox et al., 2019; Schwietzke et al., 2019; Sherwin et al., 2021). To understand the effectiveness of these systems relative to traditional LDAR methods, it is critical to understand their detection limits. Additionally, there is interest in using the quantification estimates of detected emission sources to prioritize repairs and inform emissions inventories from oil and gas systems. This requires the quantification accuracy of these solutions to be well characterized including uncertainty of individual measurements and potential bias across many measurements.

Detection limits and quantification accuracy of the GML have been investigated previously by Johnson et al. (2021) and Bridger Photonics (2021). Johnson et al. performed a total of 29 controlled releases (11 true positive [TP] detections, 10 false negative [FN] detections, and 8 other releases excluded from the analysis) during fully blind testing of the gas mapping LiDAR system. The study identified a minimum detection limit as a function of wind speed measured 3 m above ground level (AGL): $MDL [kg/h]=0.56×WindSpeed[m/s]+0.31$. A linear correlation between the accuracy of quantification estimates with the accuracy of the underlying wind estimate is also reported. Bridger Photonics’ (2021) evaluation focuses on quantification accuracy of high emission rate sources (100–550 kg/h). The study completed a total of 70 measurements (all TP detections) and reports a measurement bias (regression slope) of 3%–13% depending on the wind data source with an aggregate bias across the set of measurements of ≤4%.

This study is motivated by the need for a robust, independent assessment of the detection limits and quantification accuracy of the GML system. Interference of nonmethane emissions on the instrument’s column-integrated methane signal is investigated in a laboratory experiment, and 650 individual measurement passes over single-blind controlled releases are used to develop a robust detection curve and quantification assessment.

Interference testing experiments were conducted in a controlled setting to assess the impact of other gas species on methane measurements made by the GML system. The GML instrument was installed in a stationary fixture trained on a target at a distance of 125 m. A 1-m-long sample tube was introduced within the beam path (see SI— Interference Testing). For each test, the sample tube was vacuum-evacuated and then filled to 1 atm by various blends of sample gas and nitrogen determined by partial pressures. Additionally, an independent measure of absorbance was made for each test by monitoring the GML beam power before and after the sample cell. Experiments were conducted using methane at 0.5–5% by volume in nitrogen, ethylene at 2–20% by volume in nitrogen, and ethane, propane, n-butane, i-butane, and carbon dioxide at 20–50% by volume in nitrogen and at 100% by volume.

The background-corrected, path-integrated methane concentration (ppm-m) measured by the GML instrument was compared to the volume fraction of the test gas within the sample cell using ordinary least-squares regressions and Pearson’s correlation coefficient. Very high, positive correlations were found for methane, ethylene, and n-butane. The fit for methane showed good agreement between measured and test values (slope = 0.97 and R2 = 1.0). Although well-correlated, ethylene and n-butane showed only very weak responses “as methane” in the GML measurements with slope = 0.00179 and R2 = 0.97 for ethylene and slope = 0.000049 and R2 = 0.89 for n-butane. These responses are 500 times and 20,000 times weaker, on average, than the methane response of the GML instrument for ethylene and n-butane, respectively, which makes significant interference with methane measurements very unlikely.

During controlled release testing, a methane release rate of 453 SCFH was confirmed by the GML system at 424 SCFH, with a path-integrated methane concentration reading of 4,438 ppm-m at a flight altitude of 200 m. This is similar to the lab test case of methane at 0.5% by volume. If, somehow, the ethylene volume fraction in the column of air sampled was four times greater than methane, the GML reading would be biased high by <1% by the presence of ethylene. If the n-butane volume fraction in the column of air sampled was 200 times greater than methane, the methane reading would be biased high by <0.05%. Conversely, propane showed a very high negative correlation, and its presence in the column of air sampled at 200 times greater than methane would lead to a low bias in the methane measurement of <0.5%. In summary, the GML system seems largely insensitive to interference from the sample gas species tested. Ethylene shows the greatest level of interference, though the degree of interference is unlikely to have any meaningful impact on the results of real-world measurements made in an oil and gas production region.

Single-blind controlled release testing was performed following the survey test protocol developed by the Methane Emission Technology Evaluation Center at Colorado State University and adapted by the research team to remote sensing aircraft systems (Bell and Zimmerle, 2022). During this testing, the GML system repeated aerial passes over a controlled release system at a known location. Although the Bridger team was aware of the release location, the timing and emission rate of all controlled releases were held confidential by the study team. Bridger was instructed to conduct tests in a manner as similar to commercial operations as possible.

This approach to controlled testing allows data from many discreet passes, each under different environmental conditions and controlled release conditions to be collected in an efficient manner and is conducive to the data analysis with only a single emission source present in each pass. However, the testing approach does not perfectly match operational deployments, where emission sources may be more complex (e.g., multiple overlapping plumes, intermittent sources, varied composition, or buoyant exhaust gases) and the opportunity for optimizing the flight path over many repeated passes is limited. In this testing, the pilot maintained a figure-eight flight path for the duration of experiments and passes were not optimized to align with the wind direction or incrementally improve successive measurements.

Data from two series of controlled release experiments are included in this work.

1. Experiments in Midland, TX. In this series, the release system was overseen and operated by personnel from Colorado State University. A total of 536 passes over the release point were conducted across 5 days of testing from October 4 through October 8, 2021. These experiments were performed in Midland, TX, in an attempt to represent Bridger’s operations in the Permian basin and to provide more comparable conditions to Bridger’s internal quantification measurements (Bridger Photonics, 2021).

2. Experiments in Ehrenberg, AZ. In this series, the release system was overseen by Stanford University personnel and operated by Rawhide Leasing. A total of 114 passes over the release point were conducted across 2 days of testing from November 3 through November 4, 2021. These experiments were performed in Ehrenberg, AZ, to simplify access to high release rate equipment, and in conjunction with similar measurements performed by other airborne remote sensing instruments, the results of which will be presented elsewhere.

Descriptions of the controlled release system setup at both locations can be found in SI—Controlled Release Systems. Gas compositions from the Compressed Natural Gas trailer and Liquified Natural Gas system were analyzed by gas chromatograph (92.16% and 96.2% methane by volume, respectively, SI—Gas Composition Analyses) and were applied to all flow data to report methane flow rates in kg/h. Local wind speed and wind direction were measured and logged at 1 Hz during the release experiments using a sonic anemometer. In TX, the sonic anemometer was positioned at 4.3 m AGL. In AZ, the sonic anemometer was positioned at 10 m AGL. Wind data are extrapolated from the height of measurement to the reported plume height and to 10 m AGL assuming a logarithmic wind profile with zero-displacement plane at 0.066 m and ground roughness of 0.01 m as described by Johnson et al. (2021).

Design of experiments

Two objectives guided the selection of release rates during the experiments.

1. Assess the detection limits of the GML system at nominal flight altitudes of 500 ft (≍152 m) AGL and 675 ft (≍205 m) AGL under different wind conditions.

2. Evaluate the accuracy of quantification estimates from the GML system.

Table 1 summarizes the aircraft passes during controlled release testing. Each pass was classified as TP, FN, false positive, or not established (NE). Further demographics of the release experiments including histograms of flight altitude, release rates, and wind speeds can be seen in SI—Controlled Release Demographics. Note, while zero false positive detections were observed in the experiments, only 4 measurement passes were performed with no controlled release present, which limits the significance of any interpretation regarding a false positive rate. Experiments conducted in TX were primarily focused on assessing the detection limits of the GML system, and therefore, the majority of data were collected at low emission rates (420 passes below 5 kg/h) and included both nominal flight altitudes. The TX experiments also included 104 passes with release rates from 5 to 172 kg/h to assess quantification accuracy at 675 ft AGL nominal flight altitude. The AZ experiments focused on characterizing quantification accuracy across a larger span of emission rates and included experiments from 3.9 to 1,428 kg/h at 675 ft AGL nominal flight altitude. By combining data from experiments conducted in both TX and AZ, the accuracy of quantification estimates from the GML system is assessed over TP detections with emission rates from 0.17 to 1,428 kg/h. Dates of the experiments were selected considering historical wind speed at the release locations in an attempt to characterize performance in wind speeds from 0 to 10 m/s. However, actual wind speed during experiments (10 m AGL) ranged from 0.48 to 8.0 m/s.

Table 1.

Aircraft passes during controlled releases

Number of Passes by ClassificationNumber of Passes by Release Rate (kg/h)Release Rates (kg/h)
LocationTrue PositiveFalse NegativeFalse PositiveNot EstablishedTotalm = 00 < m < 55 ≤ mNAMin.Max.
TX 356 158 22 536 420 104 12 0.08 172
AZ 110 114 11 103 3.9 1,428
Total 466 158 23 650 431 207 12 0.08 1,428
Number of Passes by ClassificationNumber of Passes by Release Rate (kg/h)Release Rates (kg/h)
LocationTrue PositiveFalse NegativeFalse PositiveNot EstablishedTotalm = 00 < m < 55 ≤ mNAMin.Max.
TX 356 158 22 536 420 104 12 0.08 172
AZ 110 114 11 103 3.9 1,428
Total 466 158 23 650 431 207 12 0.08 1,428

Bridger detection reports

After the controlled release experiments, Bridger analyzed each pass of the aircraft independently and reported data to the study team. The reported data included the timing of the pass, the altitude of the pass, and whether or not a plume was detected. For each pass where a plume was detected, Bridger also provided the height of the detected plume, maximum concentration observed, an emission rate estimate, and the windspeed used to produce the emission rate estimate. In addition, a KMZ file was provided by Bridger showing the flight path, lidar scan coverage area (scan swath), and integrated path concentration (ppm-m) observed during each pass.

Bridger processed flight data using each of three different wind data sources; therefore, three emission rate estimates are associated with each detection. Initially, Bridger reported an emission rate estimate using wind data from the HRRR and NAM12 weather observations. After these initial two estimates were provided to the study team, the sonic anemometer data collected near the release system were provided to Bridger. A third emission estimate was produced using these local wind measurements.

Analysis methods

Data from each pass reported by Bridger were paired with controlled release and meteorological data measured by the study team using the timestamps. Controlled release data and meteorological data for each pass are reported as the average for 65 s preceding each detection time reported by Bridger, where the detection time was unavailable (e.g., FN detections), and data were averaged instead for a 65-s window centered on the flight feature time (a timestamp reported by Bridger indicating the start of a 30-s “flight feature” within which the aircraft passed over the controlled release location). An averaging period of 65 s was selected assuming a 1-m/s wind would carry the plume from the center to the edge of a 130-m detection swath in 65 s. In some cases, the plume observed by the GML system may correspond to a longer time frame (e.g., if the plume is oriented parallel to the flight path), and in other cases, the plume observed may correspond to a shorter time frame (e.g., if the wind speed is higher than 1 m/s and the plume is oriented perpendicular to the flight path).

Passes were excluded from the analysis if the emission plume was NE for a sufficient time prior to each detection. These criteria were assessed by integrating the wind speed as measured by the sonic anemometer from the moment the controlled release started until the detection time reported by Bridger to estimate a plume length (m), where the detection time was unavailable (e.g., FN detections), the flight feature time was utilized when computing the plume length instead. The plume was considered NE and excluded from further analysis if the estimated distance the gas had traveled since the start of the release did not exceed 150 m.

Detection curves

Detection limits were assessed for nominal flight altitudes of 500 ft AGL and 675 ft AGL using binomial logit regression (Table 2 and Figures 1 and 2). Data are presented in terms of the release flowrate normalized by the windspeed at 10 m AGL to collapse variability in wind speed and flowrate into a single independent parameter, consistent with the analysis performed by Sherwin et al. (2021); 90.0% detection rates of 0.25 and 0.41 (kg/h)/(m/s) are observed at 500 ft AGL and 675 ft AGL, respectively. Bridger’s stated detection sensitivity is 3 kg/hr (90% probability of detection under typical conditions).

Table 2.

Probability of detection—Logistic regression results

Nominal Altitude ft (m)Logistic
$PD = 11+ e−(B0 + B1x)$
P ValueMinimum Detection Limit at PD = 90% (kg/h)/(m/s)
B0B1
500 (152) $PD = 11+ e5.434−30.286x$ 6.6 × 10−10 5.9 × 10−11 0.25
675 (205) $PD = 11+ e5.222−17.903x$ 5.2 × 10−11 1.4 × 10−11 0.41
Nominal Altitude ft (m)Logistic
$PD = 11+ e−(B0 + B1x)$
P ValueMinimum Detection Limit at PD = 90% (kg/h)/(m/s)
B0B1
500 (152) $PD = 11+ e5.434−30.286x$ 6.6 × 10−10 5.9 × 10−11 0.25
675 (205) $PD = 11+ e5.222−17.903x$ 5.2 × 10−11 1.4 × 10−11 0.41
Figure 1.

Detection sensitivity at 500 ft above ground level (AGL) nominal flight altitude. The data are presented in terms of the controlled release rate normalized by the wind speed extrapolated to 10 m AGL. A histogram in the upper panel shows the count of true positive and false negative detections underlying the binomial regression shown in the lower panel. True positive and false negative data points are shown at y = 1 and y = 0, respectively. Bootstrapping is used to resample from the data and repeat the regression to show the confidence bounds of the result. A 90.0% detection rate is observed at 0.25 (kg/h)/(m/s).

Figure 1.

Detection sensitivity at 500 ft above ground level (AGL) nominal flight altitude. The data are presented in terms of the controlled release rate normalized by the wind speed extrapolated to 10 m AGL. A histogram in the upper panel shows the count of true positive and false negative detections underlying the binomial regression shown in the lower panel. True positive and false negative data points are shown at y = 1 and y = 0, respectively. Bootstrapping is used to resample from the data and repeat the regression to show the confidence bounds of the result. A 90.0% detection rate is observed at 0.25 (kg/h)/(m/s).

Close modal
Figure 2.

Detection sensitivity at 675 ft above ground level (AGL) nominal flight altitude. The data are presented in terms of the controlled release rate normalized by the wind speed extrapolated to 10 m AGL. A histogram in the upper panel shows the count of true positive and false negative detections underlying the binomial regression shown in the lower panel. True positive and false negative data points are shown at y = 1 and y = 0, respectively. Bootstrapping is used to resample from the data and repeat the regression to show the confidence bounds of the result. A 90.0% detection rate is observed at 0.41 (kg/h)/(m/s).

Figure 2.

Detection sensitivity at 675 ft above ground level (AGL) nominal flight altitude. The data are presented in terms of the controlled release rate normalized by the wind speed extrapolated to 10 m AGL. A histogram in the upper panel shows the count of true positive and false negative detections underlying the binomial regression shown in the lower panel. True positive and false negative data points are shown at y = 1 and y = 0, respectively. Bootstrapping is used to resample from the data and repeat the regression to show the confidence bounds of the result. A 90.0% detection rate is observed at 0.41 (kg/h)/(m/s).

Close modal

To check the assumption that detection sensitivity is a function of emission rate normalized by wind speed, data are filtered by windspeed and the binomial logit regression is repeated using the controlled release rate as the independent variable. This exercise results in detection limits similar to those predicted at the wind speed used to filter the data (SI—Detection Curves).

Quantification accuracy

Quantification accuracies of TP detections were assessed for emission rate estimates calculated by Bridger using each wind source (HRRR, NAM12, and Sonic; see Table 3). The HRRR data, which is the primary wind source used by Bridger in their commercial operations, produced the most accurate emission estimates in this study with an average bias of +8.1% (R2 = 0.89) as identified by linear regression with a fixed zero intercept (Figure 3, top panel). Quantification error of individual measurements ranges from −58.8% to +161.4% of the metered release rate (95% CI). Figure 3 (middle and lower panels) shows the dependence of quantification error on emission rate. In general, higher fractional error is observed at lower release rates; however, these data have little impact on the linear regression since errors at higher release rates contribute a disproportionate fraction of total residuals. This increase in quantification error may be attributed to decreasing gas concentration measurement fidelity and fewer data points per gas plume for the lower emission rate releases. It is important to note that while increased fractional error is observed, these low release rates are below Bridger’s stated detection sensitivity.

Table 3.

Quantification regression results

Wind Data SourceLinear RegressionR2
High-resolution rapid refresh y = 1.081 × x 0.89
North American Mesoscale Forecast System y = 0.798 × x 0.87
Sonic y = 1.342 × x 0.96
Wind Data SourceLinear RegressionR2
High-resolution rapid refresh y = 1.081 × x 0.89
North American Mesoscale Forecast System y = 0.798 × x 0.87
Sonic y = 1.342 × x 0.96
Figure 3.

Accuracy of emission estimates produced with high-resolution rapid refresh data versus release rate. Linear regression with no intercept (Measured = 1.081 × Metered) indicates a bias of 8.1% high across the data (upper panel). Error of individual measurements is shown as a percentage of the metered release rate (center panel). Marker colors represent the average windspeed preceeding the measurement. A box and whisker plot (lower panel) illustrates the mean (dashed line), median (solid line), inner quartiles (box), and 95% confidence (whiskers) for data within each order of magnitude. Edges of boxes are shown at the maximum and minimum metered release rate of data points within each range. Fliers are shown for data outside the 95% CI of each boxplot. Note that the y-limits of the center and bottom panels are set to a maximum of 250% error to improve visibility of the figures although 3 data points exceed 250% error. Only true positive detections are included in the figures and regression.

Figure 3.

Accuracy of emission estimates produced with high-resolution rapid refresh data versus release rate. Linear regression with no intercept (Measured = 1.081 × Metered) indicates a bias of 8.1% high across the data (upper panel). Error of individual measurements is shown as a percentage of the metered release rate (center panel). Marker colors represent the average windspeed preceeding the measurement. A box and whisker plot (lower panel) illustrates the mean (dashed line), median (solid line), inner quartiles (box), and 95% confidence (whiskers) for data within each order of magnitude. Edges of boxes are shown at the maximum and minimum metered release rate of data points within each range. Fliers are shown for data outside the 95% CI of each boxplot. Note that the y-limits of the center and bottom panels are set to a maximum of 250% error to improve visibility of the figures although 3 data points exceed 250% error. Only true positive detections are included in the figures and regression.

Close modal

Figure 4 illustrates how the emission rate quantification results are impacted by wind speed. The top panel shows the reported emission rate (using HRRR data) versus wind speed. Increased fractional error is observed at low wind speeds (Figure 4, middle and lower panels) with data below 2 m/s biased high. The bias may be attributed to increased column densities (ppm-m) measured by the GML system from pooling, swirling, and otherwise inconsistent flow of the emission plume under low, unstable wind conditions. Reproducing Figure 3 and excluding these regions of higher fractional error (wind speed less than 2 m/s and emission rate less than 3 kg/h) does not change the regression results significantly (bias = 7.7%, R2 = 0.85; SI—Quantification Accuracy). By inspecting the center panel of Figure 4, it is clear that for wind speeds greater than 3 m/s, the upper whiskers in the box and whisker plot are driven by estimates of controlled releases with low emission rates, consistent with the findings in Figure 3.

Figure 4.

Accuracy of emission estimates produced with high-resolution rapid refresh data versus windspeed. Marker colors represent the release rate. Error of individual measurements is shown as a percentage of the metered release rate (center panel). A box and whisker plot (lower panel) illustrates the mean (dashed line), median (solid line), inner quartiles (box), and 95% confidence (whisker) for data within each m/s wind category. Edges of boxes are shown at the maximum and minimum wind speed of data points within each range. Note that the y-limits of the center and bottom panels are set to a maximum of 250% error to improve visibility of the figures although 3 data points exceed 250% error. Only true positive detections are included in the figures and regression.

Figure 4.

Accuracy of emission estimates produced with high-resolution rapid refresh data versus windspeed. Marker colors represent the release rate. Error of individual measurements is shown as a percentage of the metered release rate (center panel). A box and whisker plot (lower panel) illustrates the mean (dashed line), median (solid line), inner quartiles (box), and 95% confidence (whisker) for data within each m/s wind category. Edges of boxes are shown at the maximum and minimum wind speed of data points within each range. Note that the y-limits of the center and bottom panels are set to a maximum of 250% error to improve visibility of the figures although 3 data points exceed 250% error. Only true positive detections are included in the figures and regression.

Close modal

The NAM12 data produced estimates with a low bias of −20.2% (R2 = 0.87; SI—Quantification Accuracy). When wind as measured by the sonic anemometer at the release site was utilized the quantification estimates had less scatter (R2 = 0.96) but an increased bias of 34.2% (SI—Quantification Accuracy). Both estimates produced with NAM12 and sonic wind show a higher average fractional error accompanied by a wider confidence interval at emission rates less than 1 kg/h. The NAM12 results also exhibit increased fractional error and wider confidence interval at low wind speeds, similar to that seen in the HRRR results. Using the sonic wind measurements improved the fractional error at low wind speed, reducing the bias and confidence interval of emissions estimates below 2 m/s wind (SI—Quantification Accuracy). However, using onsite wind measurements to improve emission rate estimates is limited in practicality since oil and gas infrastructure are widely distributed and local, site-specific wind measurements are generally unavailable.

Many field studies have identified a skewed distribution of emission rates, where the largest sources contribute a disproportionately large portion of the total emissions measured during the study (Brandt et al., 2014; Zavala-Araiza et al., 2017; Zimmerle et al., 2020; Chen et al., 2022). These skewed distributions suggest an effective emissions management strategy may be to target the “fat-tail,” whereby eliminating the largest emission sources will mitigate the majority of emissions at a lower cost than spending the time and effort to detect and mitigate all of the sources. Additionally, several studies have shown “bottom-up” emissions inventories underestimate total emissions from onshore oil and gas producing regions in the United States relative to “top-down" measurements which assess total regional emissions (Alvarez et al., 2018; Rutherford et al., 2021). This has been largely attributed to underestimating the frequency, duration, and emission rate of large emission sources or “superemitters.” Improving our understanding of these emission sources has proven difficult using traditional LDAR techniques and field measurements due to (1) the relatively slow pace of traditional LDAR techniques, resulting in only a few facilities inspected and measured each day; (2) the rarity of these events, resulting in a relatively low chance of being at the “right place at the right time" to capture a large emitter; and (3) limitations in performing direct measurements of large emission sources due to accessibility, safety, and measurement equipment. Remote sensing techniques have been identified as a potential method to rapidly scan many facilities and inform bottom-up inventory models to help reconcile these differences.

Understanding the detection limits of the proposed monitoring solutions is a critical step to evaluating their suitability to support this type of screening methodology. Results from this study indicate GML have a ≥90% probability of detecting emission sources ≥1 kg/h when flying at 500 ft AGL and ≥2 kg/h when flying at 675 ft AGL, under the environmental conditions present during this series of experiment (average wind speed of 3.9 m/s). When flown under low wind conditions, the detection limits improve, likely due to emissions pooling instead of being advected away from the source, thereby resulting in larger column integrated ppm-m measurements. These detection limits are well below the “fat-tail" of emission rates typically noted in recent field measurement campaigns and demonstrate the potential of data collected by GML to contribute to improving activity estimates of smaller emission sources as well.

For example, the table in SI—Example Detections from Another Study compares the detection limits to categories of emissions measured in a recent large study of gathering compressor stations, not considering large emitters, which were handled separately in that study (Zimmerle et al., 2020). For this analysis, we assume the sources measured in that campaign were released as clearly defined point sources, suitable for recovery by GML. All of the emitter categories analyzed are “cold sources” (i.e., not combustion sources with dilute methane, high temperatures, or co-emitted combustion products), where concentrations near the source location are high, and dispersion is governed primarily by local winds. Other source types such as diffuse (underground, still vents, etc.) or low-concentration (e.g., combustion slip, heater vents, flare slip, etc.) may be less likely to detect. In 16% or 4% of emission categories analyzed, GML would detect more than 25% of emitters at 1 or 2 kg/h, respectively. However, even detecting a small number of emitters captures the majority of emissions. At 1 kg/h, GML would capture more than half the emissions in 52% of categories; at 2 kg/h in 40% of categories.

A recent study showed discrepancies in emissions detected and measured by GML and those detected and measured during OGI surveys at the same facilities, where GML found fewer sources but greater emissions overall than recorded by OGI (Tyner and Johnson, 2021). The study identified many sources that are not the subject of traditional LDAR screening, or are difficult or impossible to measure using other methods, were identified by GML and estimated to be significantly higher than current inventory estimates. This highlights the potential for remote sensing systems like GML to improve inventories and emissions models, by characterizing the emission rate of these sources, but requires a clear understanding of the quantification accuracy of these systems. It is important to note the difference between lower limits for detection and that for accurate quantification when considering use cases. In the context of prioritized and directed repairs, the quantification accuracy of individual measurements as illustrated in Figures 3 and 4 is critical to understand. However, in the context of emissions accounting, it is important to understand the bias in the cumulative emission estimate. Figure 5 shows cumulative error when emission sources are summed from smallest to largest. The high bias at low emission rates is reflected in the cumulative error from the smallest releases (far left of lower panel). However, these small sources contribute very little to the total cumulative emission rate and hence very little to total cumulative error. As the release rate increases, the error in cumulative emission rates quickly falls for all wind sources, where the quantification error of larger emission sources dominates the cumulative error. The bias in cumulative emission estimates across the full dataset is 8.2%, −17%, and 28% for the HRRR, NAM12, and sonic wind sources, respectively.

Figure 5.

Cumulative error in emission rate estimates. All data are ordered by the controlled release rate from smallest to largest. Only true positive detections are included. The controlled release rate of each pass (black marker) and cumulative sample count (blue line) are shown versus the cumulative release rate in the upper panel. The center panel illustrates the cumulative emission estimates diverging from the cumulative metered emission rate. Error in the cumulative emission estimates is shown in the lower panel as a percentage of the cumulative metered release rate. The y-axis limits are set to a maximum of 40% error for legibility, although high-resolution rapid refresh (HRRR), North American Mesoscale Forecast System (NAM12), and sonic estimates reach maximum of 212%, 326%, and 167%, respectively, on the left edge of the figure, where release rates are small and contribute little to the total cumulative emission rate. Bias in cumulative emission estimates across the full measurement set is 8.2%, −17%, and 28% for the HRRR, NAM12, and Sonic wind data sets, respectively.

Figure 5.

Cumulative error in emission rate estimates. All data are ordered by the controlled release rate from smallest to largest. Only true positive detections are included. The controlled release rate of each pass (black marker) and cumulative sample count (blue line) are shown versus the cumulative release rate in the upper panel. The center panel illustrates the cumulative emission estimates diverging from the cumulative metered emission rate. Error in the cumulative emission estimates is shown in the lower panel as a percentage of the cumulative metered release rate. The y-axis limits are set to a maximum of 40% error for legibility, although high-resolution rapid refresh (HRRR), North American Mesoscale Forecast System (NAM12), and sonic estimates reach maximum of 212%, 326%, and 167%, respectively, on the left edge of the figure, where release rates are small and contribute little to the total cumulative emission rate. Bias in cumulative emission estimates across the full measurement set is 8.2%, −17%, and 28% for the HRRR, NAM12, and Sonic wind data sets, respectively.

Close modal

It is important to note that the detection limits identified in this study are the representative of clearly defined “point sources,” which result in a concentrated emission plume (e.g., an outdoor component leak, tank thief hatch, vent stack, compressor exhaust, or flare exhaust). It is possible that the detection limit would be higher for emission sources, which are located inside a structure (e.g., fugitive components in a compressor house, enclosed separator, or enclosed dehydrator unit) since GML will need to detect the gas leaving the structure, where the methane gas concentration is lower due to dispersion within the structure itself. While the specific location chosen for this study may be qualitatively similar to other topography in the Permian basin, no effort was made in this study to treat the effect of surface reflectivity on the results. Moreover, the wind speeds during the testing were lower than normal for the time of year of the study in Midland, TX. Lower wind speeds are more favorable conditions for detection sensitivity performance but are more difficult conditions for emission rate quantification performance.

All data required to repeat the analysis are included as supplementary material.

1. SI.pdf - Supplementary information

We acknowledge the Bridger Photonics team including Pete Roos, Mike Thorpe, Chris Wilson, and Cameron Murdock for their support preparing and executing the testing programs.

This research was funded in part by:

1. The Environmental Partnership (TEP) of the American Petroleum Institute (API) via the Methane Emission Technology Evaluation Center (METEC) at Colorado State University.

2. ExxonMobil and the Stanford Natural Gas Initiative, an industry consortium that supports independent research at Stanford University.

The authors have no competing interests to declare.

Contributed to conception and design: CB, DZ, TV, JR, AB, ES.

Contributed to acquisition of data: CB, DZ, TV, JR, AB, ES.

Contributed to analysis and interpretation of data: CB, DZ, TV, JR, AB, ES.

Approved the submitted version for publication: CB, DZ, TV, JR, AB, ES.

Alvarez
,
RA
,
Zavala-Araiza
,
D
,
Lyon
,
DR
,
Allen
,
DT
,
Barkley
,
ZR
,
Brandt
,
AR
,
Davis
,
KJ
,
Herndon
,
SC
,
Jacob
,
DJ
,
Karion
,
A
,
Kort
,
EA
,
Lamb
,
BK
,
Lauvaux
,
T
,
Maasakkers
,
JD
,
Marchese
,
AJ
,
Omara
,
M
,
Pacala
,
SW
,
Peischl
,
J
,
Robinson
,
AL
,
Shepson
,
PB
,
Sweeney
,
C
,
Townsend-Small
,
A
,
Wofsy
,
SC
,
Hamburg
,
SP.
2018
.
Assessment of methane emissions from the U.S. oil and gas supply chain
.
Science
361
(
6398
):
186
188
. DOI: http://doi.org/10.1126/science.aar7204.
Bell
,
C
,
Zimmerle
,
D.
2022
.
METEC controlled test protocol: Survey emission detection and quantification
.
Available at
https://mountainscholar.org/handle/10217/235363.
Accessed 01 June 2022
.
Benjamin
,
SG
,
Weygandt
,
SS
,
Brown
,
JM
,
Hu
,
M
,
Alexander
,
CR
,
Smirnova
,
TG
,
Olson
,
JB
,
James
,
EP
,
Dowell
,
DC
,
Grell
,
GA
,
Lin
,
H
,
Peckham
,
SE
,
Smith
,
TL
,
Moninger
,
WR
,
Kenyon
,
JS
,
Manikin
,
GS.
2016
.
A North American hourly assimilation and model forecast cycle: The rapid refresh
.
Monthly Weather Review
144
(
4
):
1669
1694
.
Available at
http://journals.ametsoc.org/doi/10.1175/MWR-D-15-0242.1.
Brandt
,
AR
,
Heath
,
GA
,
Kort
,
EA
,
O’Sullivan
,
F
,
Pétron
,
G
,
Jordaan
,
SM
,
Tans
,
P
,
Wilcox
,
J
,
Gopstein
,
AM
,
Arent
,
D
,
Wofsy
,
S
,
Brown
,
NJ
,
,
R
,
Stucky
,
GD
,
Eardley
,
D
,
Harriss
,
R.
2014
.
Methane leaks from North American natural gas systems
.
Science
343
(
6172
):
733
735
.
Available at
https://www.science.org/doi/full/10.1126/science.1247045.
Bridger Photonics
.
2021
.
Performance of gas mapping LiDAR for quantification of very high methane emission rates
.
Available at
https://www.bridgerphotonics.com/blog/performance-gas-mapping-lidar-quantification-high-methane-emission-rates.
Accessed 05 April 2022
.
Chen
,
Y
,
Sherwin
,
ED
,
Berman
,
ES
,
Jones
,
BB
,
Gordon
,
MP
,
Wetherley
,
EB
,
Kort
,
EA
,
Brandt
,
AR
.
2022
.
Quantifying regional methane emissions in the New Mexico Permian Basin with a comprehensive aerial survey
.
Environmental Science & Technology
56
(
7
):
4317
4323
. DOI: http://doi.org/10.1021/acs.est.1c06458.
Cheniere Energy, Inc
.
2021
.
Cheniere announces collaboration with natural gas suppliers and academic institutions to quantify, monitor, report and verify GHG emissions
.
Available at
https://lngir.cheniere.com/news-events/press-releases/detail/220/cheniere-announces-collaboration-with-natural-gas-suppliers.
Accessed 18 April 2022
.
Fox
,
TA
,
Barchyn
,
TE
,
Risk
,
D
,
Ravikumar
,
AP
,
Hugenholtz
,
CH.
2019
.
A review of close-range and screening technologies for mitigating fugitive methane emissions in upstream oil and gas
.
Environmental Research Letters
14
(
5
):
053002
. DOI: http://doi.org/10.1088/1748-9326/ab0cc3.
GTI Energy
.
2021
.
Veritas: A GTI differentiated gas measurement and verification initiative
.
Available at
https://www.gti.energy/veritas-a-gti-differentiated-gas-measurement-and-verification-initiative/.
Accessed 05 April 2022
.
Johnson
,
MR
,
Tyner
,
DR
,
Szekeres
,
AJ.
2021
.
Blinded evaluation of airborne methane source detection using Bridger Photonics LiDAR
.
Remote Sensing of Environment
259
:
112418
. DOI: http://doi.org/10.1016/j.rse.2021.112418.
Available at
https://www.sciencedirect.com/science/article/pii/S003442572100136X.
MiQ
.
n.d.
MiQ methane certification—Homepage
.
Available at
https://miq.org/.
Accessed 05 April 2022
.
National Centers for Environmental Information
.
n.d.
North American Mesoscale forecast system (NAM) [12 Km]
.
Available at
Accessed 15 August 2022
.
OGMP
.
n.d.
OGM partnership
.
Available at
https://www.ogmpartnership.com/.
Accessed 01 June 2022
.
Project Canary
.
2021
.
Project Canary—Trustwell standards and certification for ESG
.
Available at
https://www.projectcanary.com/private/trustwell-and-rsg-definitional-document/.
Accessed 05 April 2022
.
Rutherford
,
JS
,
Sherwin
,
ED
,
Ravikumar
,
AP
,
Heath
,
GA
,
Englander
,
J
,
Cooley
,
D
,
Lyon
,
D
,
Omara
,
M
,
Langfitt
,
Q
,
Brandt
,
AR
.
2021
.
Closing the methane gap in US oil and natural gas production emissions inventories
.
Nature Communications
12
(
1
):
4715
.
Available at
https://www.nature.com/articles/s41467-021-25017-4.
Schwietzke
,
S
,
Harrison
,
M
,
Lauderdale
,
T
,
Branson
,
K
,
Conley
,
S
,
George
,
FC
,
Jordan
,
D
,
Jersey
,
GR
,
Zhang
,
C
,
Mairs
,
HL
,
Pétron
,
G
,
Schnell
,
RC
.
2019
.
Aerially guided leak detection and repair: A pilot field study for evaluating the potential of methane emission detection and cost-effectiveness
.
Journal of the Air & Waste Management Association
69
(
1
):
71
88
. DOI: http://doi.org/10.1080/10962247.2018.1515123.
Sherwin
,
ED
,
Chen
,
Y
,
Ravikumar
,
AP
,
Brandt
,
AR.
2021
.
Single-blind test of airplane-based hyperspectral methane detection via controlled releases
.
Elementa: Science of the Anthropocene
9
(
1
):
00063
. DOI: http://doi.org/10.1525/elementa.2021.00063.
Tyner
,
DR
,
Johnson
,
MR.
2021
.
Where the methane is—Insights from novel airborne LiDAR measurements combined with ground survey data
.
Environmental Science & Technology
55
(
14
):
9773
9783
. DOI: http://doi.org/10.1021/acs.est.1c01572.
UN Environment Programme
.
2021
.
IMEO action
.
Available at
http://www.unep.org/explore-topics/energy/what-we-do/methane/imeo-action.
Accessed 05 April 2022
.
U.S. Environmental Protection Agency
.
2021
.
EPA proposes new source performance standards updates, emissions guidelines to reduce methane and other harmful pollution from the oil and natural gas industry
.
Available at
https://www.epa.gov/controlling-air-pollution-oil-and-natural-gas-industry/epa-proposes-new-source-performance.
Accessed 05 April 2022
.
Zavala-Araiza
,
D
,
Alvarez
,
RA
,
Lyon
,
DR
,
Allen
,
DT
,
Marchese
,
AJ
,
Zimmerle
,
DJ
,
Hamburg
,
SP.
2017
.
Super-emitters in natural gas infrastructure are caused by abnormal process conditions
.
Nature Communications
8
(
1
):
14012
. DOI: http://doi.org/10.1038/ncomms14012.
Available at
https://www.nature.com/articles/ncomms14012.
Zimmerle
,
D
,
Vaughn
,
T
,
Luck
,
B
,
Lauderdale
,
T
,
Keen
,
K
,
Harrison
,
M
,
Marchese
,
A
,
Williams
,
L
,
Allen
,
D
.
2020
.
Methane emissions from gathering compressor stations in the U.S
.
Environmental Science & Technology
54
(
12
):
7552
7561
. DOI: http://doi.org/10.1021/acs.est.0c00516.

How to cite this article: Bell, C, Rutherford, J, Brandt, A, Sherwin, E, Vaughn, T, Zimmerle, D. 2022. Single-blind determination of methane detection limits and quantification accuracy using aircraft-based LiDAR. Elementa: Science of the Anthropocene 10(1). DOI: https://doi.org/10.1525/elementa.2022.00080

Domain Editor-in-Chief: Detlev Helmig, Boulder AIR LLC, Boulder, CO, USA

Guest Editor: Stefan Schwietzke, Environmental Defense Fund, Boulder, CO, USA

Knowledge Domain: Atmospheric Science

Part of an Elementa Special Forum: Oil and Natural Gas Development: Air Quality, Climate Science, and Policy

This is an open-access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. See http://creativecommons.org/licenses/by/4.0/.