Single-blind detection, localization, and quantification of methane emissions using continuous path-integrated column measurements

Methane (CH₄) is the primary constituent of natural gas, a major source of heating and electricity generation around the world. CH₄ is also a potent greenhouse gas with a global warming potential 81–84 times that of carbon dioxide over a 20-year period (International Energy Agency, 2021). The largest industrial sources of CH₄ in the United States are oil and natural gas systems, responsible for an estimated 28% of U.S. CH₄ emissions in 2022 (United States Environmental Protection Agency, n.d.). Accurate characterization of CH₄ emissions in complex environments, such as many oil and gas facilities, is a challenging measurement problem due to limited access, potentially high emission rates creating safety concerns, a large number of potential emission locations, and high temporal variability in emissions (Vaughn et al., 2018; Riddick et al., 2022). In such environments, emissions can occur at many locations and at any time of day and year. Periodic leak detection and repair programs utilizing handheld optical gas imaging sensors, typically performed semiannually or quarterly, may be effective at finding persistent leaks in expected locations (at equipment prone to leaking or failure), but they do not adequately address the problems of intermittent leaks not emitting at the time of the leak survey or leaks occurring in unexpected locations that may be omitted from the survey area (Lyon et al., 2015; Nathan et al., 2015; Zavala-Araiza et al., 2015). Periodic airborne leak surveys can have more complete spatial coverage but are generally less sensitive to small leaks, limited to midday periods when the boundary layer is well mixed, and expensive to operate (National Academies of Sciences, Engineering, and Medicine, 2018). Furthermore, leaks may persist for weeks or months before being detected by the next periodic survey, potentially resulting in significant total emissions of CH₄ and prohibiting the quantification of such emissions due to the uncertainty in duration. Continuous, wide-area monitoring over entire facilities has the potential to overcome these measurement gaps and to provide rapid detection of leaks with more accurate estimates of total time-integrated gas emissions from oil and gas production, distribution, and storage facilities.

In this single-blind study, a continuous monitoring approach was evaluated based on its ability to detect, locate, and quantify CH₄ emissions with no prior knowledge of emission location, time, rate, or duration. The GreenLITE™ gas concentration measurement system employs continuous laser-absorption-spectroscopy-based, open-path, integrated column measurements in conjunction with a tomographic reconstruction and an inverse dispersion model to locate and estimate CH₄ emission rates. While other open-path continuous monitoring methods require specific knowledge of potential leak locations to pre-position equipment such as reflective targets (Alden et al., 2019), the GreenLITE™ system utilized in this study (consisting of two sensor units) is capable of identifying and localizing emissions from anywhere within the monitoring area. The ability of GreenLITE™ to detect, locate, and quantify CH₄ emissions was evaluated through the use of a set of performance metrics defined in the Continuous Monitoring Protocol (CMP) (Zimmerle, 2020) developed under the Advancing Development of Emissions Detection (Colorado State University [CSU] Energy Institute, n.d.) program run by the Methane Emissions Technology Evaluation Center (METEC) at CSU, Fort Collins, CO. By focusing on top-level metrics such as probability of detection (PD) and accuracy of localization and quantification rather than sensor-specific metrics such as sensitivity and signal-to-noise ratio, the CMP aims to provide a means for regulatory agencies and oil and gas operators to directly compare the utility of different monitoring solutions.

Evaluation of other continuous monitoring solutions has been performed at the METEC facility, designed and built specifically for evaluation of CH₄ leak detection and quantification systems. The facility contains decommissioned oil and gas equipment representative of conventional and small nonconventional gas production facilities, namely wellheads, separation equipment, and small liquid storage tank batteries. While METEC provides realistic site topography, experimental design is still critical in evaluating the long-term expected performance of monitoring solutions at operational oil and gas facilities. For example, in prior experiments conducted at METEC in 2018 and described by Alden et al. (2019), for any given test, the pad containing the controlled emission was known to the sensing system operators, and the sensing system only needed to determine which equipment within the pad the emission source was located on. A total of 17 releases were performed, each lasting 1.5–5.5 h, with the release beginning and end times known by the sensing system operators. Emission rate and location within the pad were the sole blind aspects of these tests. The experiment provided only a single time window of about 4.25 h during which no gas was released and false positive (FP) detections were possible. All other results provided were from known controlled releases, limiting assessment of the actual FP detection performance of the system being tested. More recent testing at METEC utilizing the CMP has addressed some of these methodological issues, with participating solutions fully blind, greater numbers of controlled releases, and more continuous operation allowing for greater opportunities for FP detections (Bell et al., 2023).

The GreenLITE™ experiment detailed here applies the same CMP performance metrics utilized by METEC to a different test site while also attempting to avoid some of the shortcomings of similar monitoring technology evaluations. This study allowed for releases anywhere within 20 grid boxes covering a total area of over 0.18 km² at any time within a 10-h daily window on any of 27 days. The longest release lasted 1 h, and the average release was 22.8 min in duration. Both point releases and diffuse (area) releases were performed, with diffuse releases occurring at ground level and point releases occurring either at ground level or 1 m above the surface. The GreenLITE™ system operated for more than 314 h, with releases being conducted during only 5.1% of that time, leaving the rest of that time available for the possibility of FP detections. This experimental design allowed for a more representative assessment of full-time continuous emissions monitoring capabilities over a monitoring area large enough to potentially include multiple operational oil and gas production facilities or represent a much larger single facility such as a refinery or tank farm (Lyon et al., 2016).

GreenLITE™ is a laser absorption-based gas measurement system that consists of one or more optical transceiver units and some number of retroreflectors arranged such that a clear line of sight exists between each transceiver and each reflector. Backend processing and analytics convert measured optical depth values to dry air volume mixing ratios (henceforth, mixing ratios) in near-real time and generate 2D distributions of CH₄ mixing ratios (Dobler et al., 2015; Dobler et al., 2017; Zaccheo et al., 2019). GreenLITE™ measurements are interfaced with an inverse dispersion model to estimate CH₄ emission rates and locations. The measurement approach and emission retrieval scheme are described in detail by Pernini et al. (2022).

While GreenLITE™ may be used to measure the mixing ratio over a single atmospheric path, the more common system configuration involves the transceiver scanning to multiple reflectors to measure an area. The transceiver optical head is scanned to sequentially point at each reflector for a period that typically spans 10–30 s depending on the application, measuring the path-integrated mixing ratio of the target gas along the straight-line path (“chord”) from the transceiver to the reflector. If two transceivers are arranged such that their measurement chords intersect one another (as seen in Figure 1, for example), a 2D reconstruction of the distribution of the gas mixing ratio over an area that can span up to 25 km² can be obtained using a sparse tomographic approach (Dobler et al., 2015; Dobler et al., 2017).

Prior to this experiment, GreenLITE™ has previously been successfully tested and deployed in several environments for CO₂ and CH₄ monitoring, including a 6-month study to monitor for CO₂ leaks from an underground carbon storage facility (Blakley et al., 2020), a 1-year study to monitor urban CO₂ emissions over central Paris, France (Dobler et al., 2017; Lian et al., 2019; Zaccheo et al., 2019), a 1-week limited evaluation of the CH₄ and CO₂ leak quantification ability of GreenLITE™ through a series of blind and unblind controlled releases from a fixed single known source location at an oil and gas facility (Watremez et al., 2018), and multiple campaigns monitoring fugitive CH₄ and CO₂ diffuse surface emissions from an oil sands open-pit mine and tailings pond (Pernini et al., 2022). Only the testing described in Watremez et al. (2018) involved controlled releases, but there was no localization required, and the monitoring area was only 0.02 km². In contrast, controlled releases in this study were conducted at unknown locations within a more relevant 0.35 km² monitoring area.

GreenLITE™ is theoretically capable of detecting, quantifying, and locating multiple simultaneous emission sources, but test equipment availability limited this experiment to a single emission source at any one time. Future studies will be required to assess the performance of the system in the presence of multiple simultaneous emission sources.

The experiment detailed in this article was designed to simulate the area of a typical oil and gas storage tank farm. The chosen test site was a farm located in a rural area at the edge of a more suburban area. The site was mostly flat with few obstructions. While no known sources of CH₄ were present in the immediate vicinity of the test site, highways, shopping centers, housing developments, railroad tracks, and several active farms were present within a 2-km radius. While testing at a known experimental site such as METEC was considered, a site near the GreenLITE™ manufacturer was selected for these first blind tests based on experimental design consideration and time and cost constraints. Though the METEC facility more realistically represents infrastructure at a typical oil and gas facility, GreenLITE™ is well suited for monitoring areas much larger than the approximately 0.03 km² METEC site. A GreenLITE™ deployment could monitor an area large enough to include multiple oil and gas production sites or a much larger single facility such as a refinery or tank farm. Furthermore, longer path lengths benefit the GreenLITE™ differential absorption measurement via higher signal-to-noise ratio due to the chosen wavelengths. The selected test site and methodology were reviewed by a METEC representative involved in the development of the CMP. By performing the experiment on a much larger scale than what is possible at METEC, this work demonstrates that the CMP is transferrable to any potential test location, enabling adaptation for a wider range of developing technologies. To the authors’ knowledge, prior to this work, the CMP has not been applied to controlled release testing anywhere outside the METEC facility.

Detection, quantification, and localization performance metrics for the GreenLITE™ system (or any continuous monitoring sensor) may be expected to vary in different environments due to differences in area, topography, meteorology, and on-ground infrastructure, and the authors acknowledge that results from similar testing performed at an operational oil and gas facility would likely differ from the results of this work. Specifically, the lack of oil and gas equipment on the test site used for this work significantly reduces the turbulent mixing that occurs in the presence of such equipment. The GreenLITE™ emissions measurement approach is robust in that measurement lines of sight and heights can be readily adapted to a given site by optimizing transceiver and reflector locations and heights for the site-specific application. A site with a high density of obstructions may result in fewer chords being attainable, although the 2D concentration reconstruction algorithm can utilize shortened chords that do not span the entire monitored area (e.g., a chord from a transceiver to a reflector located in front of an obstruction). Chord endpoints located higher above the ground present no challenges in the retrieval of concentration from measured differential optical depth but will likely impact the PD and both quantification and localization accuracy. Whether the impacts are positive or negative depends on the emission source location and atmospheric conditions. Further testing using more varied system layouts and turbulence-inducing infrastructure is required to better understand the impacts of system component placement and localized turbulence on system performance. A priori knowledge of likely source locations can be used to constrain and improve localization estimation even when chord placement is less than ideal.

Testing was performed in the time period from January to April 2022 over a range of weather conditions. Meteorological conditions experienced during this study were representative of the region but did not cover the full range of conditions that would be expected at a majority of oil and gas facilities around the world. While performance evaluation of GreenLITE™ with controlled releases has only minimally been performed at one other location (Watremez et al., 2018), GreenLITE™ has demonstrated successful operation through temperatures ranging from −25°C to 42°C and wind speeds in excess of 20 m s⁻¹. See Section S1 of the Supplemental Material for a more detailed description of the meteorological conditions experienced during testing.

GreenLITE™ was installed at the farm test site as shown in Figure 1. The layout included two GreenLITE™ transceivers and 29 retroreflectors arranged to provide 54 measurement chords covering an area of 0.35 km². The number of chords in this experiment is representative of the number that would be used in an operational deployment. Previous deployments of the system in the mapping configuration (two transceivers with intersecting chords) have utilized as few as 31 chords and as many as 60 chords. The number of chords is sometimes limited by site topography or obstructions, with fewer chords resulting in coarser spatial resolution. Evaluation of the impact of chord density on emissions quantification accuracy would require further investigation using controlled releases or truth data under different deployment configurations. Because of the flat site topography and lack of obstructions at the farm test site, all chord endpoints (transceiver optical heads and retroreflectors) were simply placed 2 m above ground level, with optical heads mounted on custom pipe-frame structures and retroreflectors mounted on tripods. Images of a GreenLITE™ transceiver and a reflector as installed at the farm test site are provided in Section S2 of the Supplemental Material.

Reflector locations were selected to create consistent angular spacing between chords and to minimize regions with large gaps between chords. Because the eventual release locations were unknown, reflector/chord placement was not informed by expected emission source location, as it would likely be in an installation at a real oil and gas facility. The system configuration is extremely flexible and adaptable to each unique site, with the only hard requirements being the need for lines of sight between transceivers and reflectors. Configuration variations include the use of a third transceiver to improve coverage in the presence of many obstructions and the use of additional reflectors at multiple heights to gain knowledge of vertical transport and mixing, both with the trade-off of increased hardware cost. Increasing the number of reflectors will also increase the time required to obtain a measurement from the entire set of chords, likely increasing Detection Time by seconds or minutes but also improving localization accuracy and, potentially, PD.

Chords measured 216–817 m in length. The measurement area was subdivided into 44 potential emission location boxes. Releases could be conducted in any location other than the outermost boxes (for a total of 20 potential release boxes equaling 0.18 km²). The buffer boxes around the perimeter ensure that emissions within the site are not missed regardless of wind direction. The grid boxes do not have to be identical in shape and size; however, the number of boxes in any 2D reconstruction may not exceed the number of measurement chords in order to provide a unique solution. In a past deployment of GreenLITE™ to an operational oil and gas facility, where likely emission source locations were known a priori, the boxes were defined such that likely sources were not located near or on the boundaries between boxes, and larger boxes were defined in areas where no potential emitters existed. Since no likely source locations were predefined in this experiment, the boxes were laid out in a grid with identical box sizes. It is entirely coincidental that most of the releases were located away from grid box boundaries. Releases performed near grid box boundaries may have resulted in reduced quantification and localization accuracy, depending on wind direction. While GreenLITE™ is able to monitor much larger areas (Dobler et al., 2017; Pernini et al., 2022), the area monitored in this experiment was primarily limited by property lines.

Meteorological data were provided by an onsite ATMOS 14 (measuring temperature, pressure, and relative humidity) and an ATMOS 22 (measuring wind speed and direction) from METER Group installed near transceiver T01 (green marker near lower left of Figure 1).

The system was installed in a nonpermanent manner to avoid disruption to ongoing farming activities. Consequently, frequent ground freeze–thaw cycles experienced throughout the winter–spring testing period impacted optical alignment between the transceivers and reflectors. This was expected but unavoidable given the installation constraints. While GreenLITE™ is tolerant of very minor disturbances in optical alignment, larger disturbances result in the transmitted laser missing the reflector and causing insufficient optical power to be collected by the transceiver. Regular realignment was performed via the system’s remote interface on all testing days, including both days when releases were performed and days when no releases were performed. Realignment entailed making minor adjustments to the programmed scanner positions that are used to point the sensor optical head at each of the reflectors to compensate for the platform movement. Such realignment is not needed when GreenLITE™ is installed in a semipermanent or permanent fashion, as was done in the deployments described by Dobler et al. (2017), Pernini et al. (2022), and Blakley et al. (2020).

A set of unblind releases was conducted in mid-January to evaluate the release procedure and verify expected performance. High-purity gas consisting of 99%–99.5% pure CH₄ was used for all releases. Section S3 of the Supplemental Material contains more information about the release system. A second round of unblind releases was conducted in late January, allowing the detection sensitivity of the system to be coarsely evaluated and adjusted while providing guidance regarding the range of release rates that would be appropriate for the ensuing blind release tests. These unblind releases were performed at rates of 0.17–2.15 kg h⁻¹. While storage tanks are known to be frequent emitters of CH₄ (Lyon et al., 2016), very little literature exists regarding true leak emission rates from oil and gas storage tanks due to the challenges associated with direct measurement of such leaks (Johnson et al., 2022). The release rates tested in this experiment are expected to be smaller than true emission rates from storage tanks and serve as an evaluation of the lower detection thresholds for the GreenLITE™ system. In theory, no upper detection limit exists for GreenLITE™, but quantification performance at higher emission rates may differ from that at the lower rates tested in this work.

Prior to conducting any blind releases, the test plan and test setup were audited onsite by a representative from METEC, whose recommendations were incorporated into the final test and execution.

Recent evaluation of continuous monitoring systems at METEC included hundreds of individual releases, some lasting up to 8 h, at emission rates up to 6.39 kg h⁻¹ (Bell et al., 2023). In this study, releases were limited to rates ≤2.15 kg h⁻¹ and durations ≤60 min due to limitations of the release system hardware and the prohibitive cost of large quantities of CH₄ gas. Initial unblind test results indicated that high release rates were not needed to identify the minimum detection threshold and that short durations would provide ample information on detection thresholds to determine the 90% PD emission rate. Blind releases ranged in duration from 12 min to 1 h, with the average release lasting 22.8 min. No releases were performed at night for reasons of personnel safety.

Personnel involved in this experiment were split into two groups: the test center team, responsible for planning, executing, and documenting the controlled releases; and the performer team, responsible for maintaining system operation and identifying, locating, quantifying, and reporting emission events. The test center team provided no information to the performer team regarding test times, durations, locations, emission rates, or on which days within the 6-week testing period releases would be conducted. The performer team knew that the releases would not be performed in any of the outermost grid boxes and that all releases would occur within the daily 10-h time window. The aforementioned optical realignment was performed daily outside this 10-h time window. The performer team also knew that the maximum possible release rate allowed by the release system was 2.15 kg h⁻¹, although this information was not used to cap or constrain any emissions rate estimates. Lastly, the performer team knew that both point and diffuse releases were possible but had no knowledge of when a given release type might occur. The performer team executed all tasking and analyses offsite via remote data communication and transfer mechanisms. This arrangement ensured that the performer team remained blind to the release schedule and details to allow for an unbiased assessment of the emission detection, localization, and quantification capabilities of GreenLITE™.

For each release event, the test center team recorded all relevant information about the release. Releases occurred on 6 separate days dispersed over a 6-week period containing 27 potential release days—30 weekdays minus 2 days for power outage and 1 day for equipment maintenance. All releases occurred between 8:00 AM and 6:00 PM local time to ensure system alignment and to minimize the impact of frost heaving. Releases were conducted at 10 unique locations indicated in Figure 1, with releases being repeated at two of those locations on different days.

A histogram of the releases as a function of release rate is provided in Figure 2 (left). The right plot of Figure 2 shows the number of GreenLITE™ scans as a function of release duration and rate. A complete scan producing mixing ratio measurements on all 54 chords took approximately 4 min. More releases were performed at lower emission rates to improve confidence in the PD evaluated at those rates, and higher emission rates were conducted for shorter release periods to limit gas consumption.

If any emission events were detected, emission rates and locations were estimated for each detection or group of detections. All detections and localization and quantitative estimates were computed by the single-blind performer team at that time and reported for comparison to the test center team release record upon completion of blind testing.

The chord mixing ratio measurements were passed through a detection algorithm to identify likely emission events. The median of all chord mixing ratio measurements over a 4-min rolling time window was used to correct for natural variations in atmospheric background CH₄ mixing ratio. Median chord-to-chord differences were removed through a flat-fielding correction prior to the identification of chord mixing ratio measurements exceeding the detection threshold. The detection threshold was dynamically adjusted independently for each chord based on the degree of variability in the concentration measurements from that chord. The detection threshold was determined by multiplying a sensitivity factor by the median absolute deviation of the chord concentration measurements over the entire dataset from that day, and any concentration value that exceeded the background-subtracted chord median plus the detection threshold was classified as a detection. The sensitivity factor remained constant for this entire study but can be adjusted to trade detection sensitivity for FP rate. For each detection, the sample time, chord ID, and measured chord mixing ratio were recorded in the detections file for that day.

An emission end time was defined as the time occurring 2 min (one-half of a scan period) after the last detection after which two complete 4-min scans produced no additional detections. For example, if the last detection in a series of detections occurred at time t, and no further detections occurred in the time window from t to t + 8 min, the end of the emission was determined to be at time t + 2 min. While this method could result in a single emission event being classified as two separate events if a gap greater than 8 min occurs between consecutive detections, this was not observed in this experiment. Conversely, this method could result in two separate emission events being classified as a single emission if the second emission event begins less than 8 min after the first ends. This also did not occur in this experiment, as a minimum time of 13 min (>3 full scans) was allowed between emission events to allow residual CH₄ gas to clear from the test site to avoid confounding results between emission events.

The integrated column mixing ratio measurements were combined with local wind information to create 2D estimates of mixing ratios within the plane defined by the chords and their intersecting horizontal area using a sparse tomographic approach that minimizes the error between an analytical model of the field and the observed chord mixing ratio values (Pernini et al., 2022). The 2D mixing ratio field is modeled as a set of rectangular subregions (“boxes”), matching the red grid boxes shown in Figure 1.

The approach adopted in this study was to combine box retrieval data with a standard emissions modeling framework called Second-order Closure Integrated puff model with Chemistry (SCICHEM; Chowdhury et al., 2015). In this application, SCICHEM is used in an iterative scheme to provide emission estimates in the box sectors depicted with red lines in Figure 1.

Detection Times were used to determine which time periods to process for emissions estimation. Emission rates and locations were computed for each 10-min time period in which an emission event had been identified by the detection algorithm. If an emission event spanned multiple 10-min periods, resulting in multiple estimates of emission rate and location, only the last estimated emission rate and location for that emission event were used in the computation of CMP metrics. Since the mixing ratio values associated with the 2D box reconstructions are used in the iterative inverse dispersion model emission retrieval scheme, emission locations were estimated to the fidelity of the red boxes shown in Figure 1, with each emission event assigned to a numbered box. Emissions for each reconstruction box were computed, and the largest emission value in a non-edge box was provided as the estimated emission rate and the box as the emission location. The estimated emissions results, including the time window, emission rate, emission location, and average wind speed and direction over the time window, were recorded in the emissions file for that day. Emission rates were estimated in units of grams per hour (g h⁻¹) in accordance with the CMP, but results are presented here in kg h⁻¹.

Upon completion of all controlled releases, the reported detections and emissions results were compiled and compared to the test center release log data. Performance metrics were computed as defined in the CMP. Full results from each blind controlled release are provided in Section S4 of the Supplemental Material.

The primary CMP metrics related to detection are PD, false positive fraction (FPF), false negative fraction (FNF), and Detection Time. PD is simply the number of true positive (TP) detections divided by the sum of the number of TP and false negative (FN) detections. PD represents the fraction of emission events that were correctly identified and is calculated as a function of release rate.

The distribution of releases and detections as a function of release rate is shown in Figure 3. A logistic regression is used to create a smooth, continuous PD curve from the discrete data samples, while a bootstrapping technique provides an indication of uncertainty in the result if experiments were repeated. Figure 4 shows bootstrapped logistic-regression PD curves based on all detections from each release. This approach considers the number of releases at a given rate and the length of the releases. The bootstrapping technique envelopes the expected detection performance of the system under test and is performed by randomly selecting N detection samples out of N actual detections (with replacement) prior to performing the logistical regression. This process is repeated 100 times to build the gray curves seen in the left panel of Figure 4, and the blue curve is the logistic regression fit to the estimated emissions.

The right panel of Figure 4 shows the PD curves, binned by average wind speed during the release, using a logistic regression without bootstrapping of the data. Note that the binning was chosen due to the limited number of samples at lower and higher wind speeds. The right panel of Figure 4 indicates that one of the primary drivers of uncertainty in the PD curve is wind speed. To evaluate this further, a logistic regression with bootstrapping was performed using the release rate normalized by wind speed, consistent with analysis by Bell et al. (2022) and Sherwin et al. (2021) in the evaluation of other remote sensing systems. The metered release rate for each release was divided by the average measured wind speed observed during the release period to generate the independent variable in the logistic regression. The result is illustrated in Figure 5 and shows a narrower confidence interval seen as a tighter grouping of the 100 bootstrap results.

Not captured in the previous figures is the number of FP detections and the FPF. FPF is the number of FP detections divided by the total number of reported detections. By definition, an FP detection is a reported detection that occurred at a time when no controlled release was being conducted. However, due to the uncontrolled nature of the test site and the fact that it is surrounded by a variety of potential CH₄ sources (e.g., homes, farms, forest, roads and highways, railroad tracks, shopping centers), a reported detection outside of a controlled release time may in fact be a detection of a true elevated CH₄ mixing ratio from a source outside the monitored area. The CMP instructs that detections classified as originating from outside the facility are to be omitted from the classification process and calculation of FPF. Section S5 of the Supplemental Material contains details about 2 days on which off-site emissions sources were observed.

The total number of reported FP detections during the 6-week testing period was 31. These 31 FP detections occurred during a total non-release monitoring time of 298 h and 17 min. The breakdown of FP detections between days with and without controlled releases (Table 1) shows no meaningful difference in FP rate between release and non-release days.

The CMP defines FPF as the number of FP detections divided by the total number of reported detections (FP + TP). The drawback to this definition is that the FPF is influenced as much by the number of releases and resulting TP detections as it is by the number of FP detections. A test consisting of many controlled releases and a short time period in which no releases are conducted will result in a lower FPF than one in which only a few releases are performed over a long period of operational time (as in the case of this experiment).

As defined by the CMP, the GreenLITE™ FPF for this single-blind test was 0.261 (31 FP of 119 total reported detections). Computing FPF in a manner similar to how FNF is computed may be more meaningful. FNF is defined as the number of FN detections divided by the number of controlled releases, which represents the fraction of controlled release periods not containing at least one TP detection. Conversely, dividing the number of FP detections by the number of non-release periods represents the fraction of non-release periods containing at least one FP detection. We define this alternative metric as FPF′. A non-release period is defined as a period of time equal to the average duration of all controlled releases but containing no controlled release.

The average release duration for this experiment was 22.8 min. The total non-release time divided by the average release period yields 785 time periods 22.8 min in length during which no controlled releases were conducted and results in an FPF′ of 0.039. A more standardized definition of this metric might use 1-h non-release time periods, for which the GreenLITE™ FPF′ would be 0.104 in this experiment.

The FNF for this experiment is 0.40 for all release rates. Note that the FNF is influenced by the distribution of emission rates included in the experiment and is therefore more meaningful when considered as a function of release rate, as shown in Figure 6. With the objective being to characterize the low-end sensitivity of the GreenLITE™ system, the vast majority of releases that did not produce detections were at relatively low rates (<0.65 kg h⁻¹). The single FN detection at a rate greater than 0.65 kg h⁻¹ (see Figure 3) was associated with a 16.7-min release at 1.51 kg h⁻¹ during which the average wind speed was 5.6 m s⁻¹ (the highest of all controlled releases).

Detection Time is defined as the elapsed time between the start of a release and the time when the release is first reported. Detection Time values in this experiment ranged from 46 s to just under 23 min and exhibited an inverse relationship with metered release rate. Detailed Detection Time results are given in Section S6 of the Supplemental Material.

The primary metrics related to emission localization defined by the CMP are localization precision and localization accuracy. The CMP requires that a detection report specify the emission location by identifying the Equipment Unit ID to which the emission is attributed. If an emission is believed to have originated from outside the testing area, it may be classified as Off Facility and excluded from the calculation of metrics. Because this experiment was performed in an agricultural setting and not on an operational or simulated oil and gas production facility, the test site included no equipment to which emissions could be attributed. Instead, each grid box (see Figure 1) was assigned a unique Cell Unit identifier to which an emission event believed to have originated from within that area could be attributed. The boxes immediately adjacent to a given box were defined as the Cell Group. Each detected emission location was reported to the Cell Unit level (i.e., to a specific box), and the reported locations were then compared to the true locations to compute the localization precision and accuracy metrics. The terminology used here was chosen to correspond to the language used by the CMP for computing localization accuracy metrics while also denoting the distinction between an Equipment Unit (such as a valve or separator) and a Cell Unit (an area 109 m × 83 m in size) or an Equipment Group (such as a tank battery) and a Cell Group (a 3 × 3 grouping of Cell Units).

Localization precision is simply a count of the number of detected releases that were identified to the correct Cell Unit, Cell Group, and Facility levels. Of the 25 detected blind controlled releases, 9 were correctly localized to the Cell Unit level (box), 9 others to the Cell Group level (adjacent box), and the remaining 7 to the Facility level (elsewhere within the measurement footprint). A subset of the Cell Group level detections included true release locations that were near the box boundary of the nearest downwind neighboring box. For these releases, wind likely carried the gas plume to a downwind measurement chord that passed through the neighboring box which was subsequently estimated to be the release box. In future work, both the proximity of the chord for which a detection occurred to respective box boundaries and wind speed/direction will be accounted for when localizing leaks, which should improve localization accuracy.

The localization accuracy metric represents the fraction of all detections (both TP and FP) that were correctly identified at the Unit, Group, and Facility levels of precision. A detection correctly localized to the Unit level also counts as a detection at the correct Group and Facility levels. The GreenLITE™ localization accuracy metrics were also calculated with FP detections omitted from the denominator, as was done by Bell et al. (2023), since the number of FP detections is a function of total monitoring time rather than the number of releases. The localization accuracy results are presented in Table 2.

A secondary metric of the CMP is emission rate quantification accuracy. Emission rate quantification accuracy typically benefits from the averaging that longer release durations permit, whereas the short releases executed in this experiment presented a highly challenging scenario with little to no opportunity to average emissions estimates. The overall results of estimated emissions versus metered rates are provided in Figure 7. The figure depicts the linear fit between the estimated and metered emission rates for all detected releases. Horizontal error bars represent the standard deviation of the metered release rate as measured throughout each release. These results indicate a bias of −6.1% from the linear fit with a fixed zero intercept. Relatively high variability is seen for release rates that were repeated multiple times, which may be due to a number of factors including wind conditions and proximity of release location to the nearest downwind measurement chord. Quantification error of individual rate estimates ranges from −89% to +263% of the metered release rate, with 80% of estimation errors falling between −67% and +67%.

The quantification results presented here are representative only of the limited range of emission rates included in this experiment, which were selected to develop confidence in the PD curve as opposed to confidence in the quantification estimates over a much broader operational range. Quantification performance may be better or worse at higher emission rates, and further testing is required to determine the quantification accuracy of GreenLITE™ for larger emission events. Also of note is the apparent difference in quantification accuracy between point and diffuse releases, with some of the point release estimates falling further from the linear fit line than all of the diffuse release estimates. Due to the limited number of releases available, releases were not separated by type for the purposes of reporting and computation of CMP metrics but are separated in Figure 7 to illustrate a likely difference in GreenLITE™ performance that warrants further testing.

The last primary metric defined by the CMP is the Operational Factor, which represents the fraction of time a system is operational relative to the total planned testing time. The total planned testing time was 345.4 h, while the operational time totaled 314.2 h, for an Operational Factor of 0.910. Causes of downtime included laser beam attenuation due to thick fog and heavy rain, failure of a component power supply, and operator error. This calculation does not include the two periods of time when power to the equipment was disrupted due to a local power outage.

The results from this single-blind study show that, with no prior knowledge of potential leak locations, rates, times, or durations, GreenLITE™ can detect CH₄ emissions as small as 0.22 kg h⁻¹ and that, for the 22 controlled releases with average wind speeds <5 m s⁻¹, it demonstrated a 100% detection rate for leaks ≥0.65 kg h⁻¹. Of the 42 releases in total, 25 were detected as emission events. For localization, the correct release box (Cell Unit) was determined for 9 of the detections, an adjacent box (Cell Group) was identified in 9 of the detections, and the remaining 7 were localized elsewhere within the field (Facility). Quantification of the 25 detected emissions exhibited a bias of −6.1%. Prior work (Pernini et al., 2022) demonstrated that primary sources of uncertainty in emission rate estimates are variability in averaged surface meteorology and surface meteorology measurement precision, while chord concentration measurement accuracy and error in dispersion modeling are not significant contributors to uncertainty (estimated to be less than 5%). These results represent an important first step in demonstrating feasibility of the GreenLITE™ system to offer continuous monitoring of relatively large oil and gas facilities to identify, locate, and quantify CH₄ emission sources, including those from arbitrary locations at random times.

Additional testing to further assess the capabilities of GreenLITE™ for use in monitoring oil and gas facilities should include the presence of multiple emissions sources, allowed (known or expected) emission events, and higher emissions rates, as well as testing in a more relevant environment containing oil and gas equipment and infrastructure.

While the CMP metrics are a useful framework for providing a method to compare the key characteristics of continuous monitoring systems that are most of interest to oil and gas operators and regulators, our work illuminates some limitations and suggests some refinements and additions that could be made to further increase the utility of the metrics. Limitations of the CMP methodology to perform classification of detections as TP or FP detections have been identified in other testing programs (Bell et al., 2023), suggesting a need to consider the application or use case of the continuous monitoring system under evaluation and to tailor the methodology as appropriate. Defining FPF such that it is not dependent on the number of controlled releases would allow more meaningful comparison between systems tested under very different experimental designs. One such solution would be to compute FPF as the number of FP detections reported per fixed time period of system operation (e.g., 1 h or 24 h). A modified localization accuracy metric that does not utilize the Unit/Group/Facility classification method would provide more consistency across monitoring systems intended for different use cases, such as large-area, site-wide monitoring versus localized equipment group monitoring. The experiment detailed here was intended to evaluate the ability of GreenLITE™ to detect, quantify, and localize leaks from oil and gas storage tanks rather than from well pads. Thus, a Cell Unit was defined to be much larger in size than it would have been if attempting to localize a leak to a specific valve or compressor. Determination of localization accuracy using some distance-based measure would allow better comparison of results from testing performed at facilities of different scales.

The CMP does not specify that controlled releases must be from a point source or that they may or may not be from a diffuse source, which simulates such emission sources as open-top liquid storage tanks or buried pipeline leaks. Lastly, different test sites vary wildly in terms of their infrastructure layout. Nothing in the CMP requires specific oil and gas equipment to be located on the test site, but the presence of such equipment may increase confidence that the test results will be more representative of field performance. If some basic parameters (such as size dimensions and typical leak rates) were specified to define equipment and components commonly found on oil and gas sites, other test sites may be able to create some approximation of these items to more closely represent a real site or other test sites that contain such equipment.

When comparing CMP-defined metrics for monitoring solutions tested at different times or locations, one must consider the other independent factors such as the range of release rates, duration of the experiments, facility layout and size, environmental conditions during testing, and so on before drawing conclusions regarding the relative performance of the systems. A limited performance comparison of GreenLITE™ to other solutions tested at METEC under the CMP is provided in Section S7 of the Supplemental Material.

The GreenLITE™ emission detection algorithm can be modified to apply a temporal requirement such that multiple detection events must occur within a specified time window before a detection is reported. This change will result in a reduction in FP detections and increased confidence that reported detections are TP detections. Due to the relatively short duration of releases included in this experiment, a quantitative analysis of the potential improvement was not conducted.

Emissions localization and quantification may be improved by more fully utilizing measured wind speed and direction as well as the proximity of measurement chords to box boundaries. For example, a localization constraint limiting possible predicted leak locations to boxes that fall on or upwind of detection chords may further improve localization performance. Emission localization and quantification accuracy may also be improved by using a Gaussian plume reconstruction approach with the emission source modeled as a point emission in the inverse dispersion model rather than the box reconstruction with area emission sources. This is supported by the quantification estimates separated by release type, shown in Figure 7, which show better agreement between estimated rates for diffuse releases than for point releases using the box reconstructions.

Data are available from https://doi.org/10.5281/zenodo.10888088.

The supplemental files for this article can be found as follows:

Supplemental Material.pdf

The authors acknowledge Michael Braun for his assistance in designing and constructing the gas dissemination system used in this study.

This work was supported by the United States Environmental Protection Agency SBIR Program under grant number 68HERC22C0013.

The GreenLITE™ system has been co-developed by Spectral Sensor Solutions, LLC (S3) and Atmospheric and Environmental Research, Inc. (AER). NB, JTD, and DM are employees of S3. TGP and TSZ are employees of AER. Subsequent to the work described in this manuscript, CB began working for bpx energy, headquartered in Denver, Colorado. bpx energy did not participate in the drafting of this article, and the views set forth in the paper do not necessarily reflect those of bpx energy.

Contributed to conception and design: JTD, NB, CB.

Contributed to acquisition of data: JTD, DM.

Contributed to analysis and interpretation of data: NB, TGP, TSZ, CB.

Drafted and/or revised the article: NB, TGP, JTD, TSZ, DM, CB.

Approved the submitted version for publication: NB, TGP, JTD, TSZ, DM, CB.

How to cite this article: Blume, N, Pernini, TG, Dobler, JT, Zaccheo, TS, McGregor, D, Bell, C. 2024. Single-blind detection, localization, and quantification of methane emissions using continuous path-integrated column measurements. Elementa: Science of the Anthropocene 12(1). DOI: https://doi.org/10.1525/elementa.2024.00022

Domain Editor-in-Chief: Detlev Helmig, Boulder AIR LLC, Boulder, CO, USA

Guest Editor: David Lowry, Department of Earth Sciences, Royal Holloway University of London, Egham, UK

Knowledge Domain: Atmospheric Science

Part of an Elementa Special Forum: Oil and Natural Gas Development: Air Quality, Climate Science, and Policy