Limit Sensitivity to Systematic Uncertainties

3.12 Poisson Limit Setting

3.12.7 Limit Sensitivity to Systematic Uncertainties

The final consideration for limit setting was the effect of various systematic un- certainties. Specifically, we wanted to understand how uncertainties on energy resolution, energy calibration, CTII probabilities, and cut efficiency could change the final limits. For absorption processes, the uncertainty on𝜎₁below 3.2 eV (see Section 3.12.2) was also considered.

Figure 3.48: Peak-selection results for dark-photon absorption calculated using 10%

of the science data. (Top) Limits calculated for each𝑒⁻ℎ⁺peak with 90% confidence level. Each limit’s shaded region is bounded by the limits produced under the over- and under-fluctuation assumptions. If a peak’s lower limit is below all other peaks’

upper limits, that peak will be included in the selection. (Bottom) The peaks selected using this method. Colors correspond to the labels in the top plot’s legend.

(Left) The result using 10% data as is. (Right) The result using 10% data with exposure and number of events scaled by 9.

Figure 3.49: The peaks selected (as a function of mass) for dark-photon absorption.

The result of combining both selections from Figure 3.48 by including any peaks selected in either. The number of limits defines the confidence level of individual limits used during the final limit setting (using Eq. 3.36).

As was done in Run 2, we quantified the effect by producing a systematic-uncertainty sensitivity "band" around each model’s official limit result. The upper and lower bounds of the band were set to enclose the central 68.27% of 5000 limits produced with inputs randomly varied based on the input’s uncertainties. Specifically, the following procedure was performed.

1. A probability density function (PDF) was defined for each input parameter with uncertainty. The mean value of each PDF corresponded to the value used for the official limit setting.

2. 5000 sets of input parameters were generated by randomly drawing each parameter from its associated PDF.

3. Individual peak limits were set (for each model) using each set of parameters and the same individual CL values as the official limits.

4. Each parameter set’s limits were combined using the same peak for each mass as the official limit.

5. The distribution of all combined limits was used to calculate the upper and lower bounds on the central 68.27% region (as a function of mass).

The 68.27% value was chosen to emulate one standard deviation in a Gaussian distribution. An example distribution of limits can be seen in Figure 3.50. Such plots were produced for every model and mass considered (too many to include here).

The systematic uncertainties and their assigned PDFs are summarized in Table 3.2.

For detector energy resolution, the PDF was a flat distribution bound by the minimum and maximum measured resolutions (see Section 3.10). This distribution was designed to conservatively consider all measured resolutions. For charge trapping (CT) and impact ionization (II), truncated Gaussian distributions were used to prevent probabilities from going below 0% or above 100%. The II value in Table 3.2 is the probability to produce a specific quasiparticle (0.8% probability to produce an electron, and a separate 0.8% probability to produce a hole). The PDF for cut efficiency was also a Gaussian and theoretically should have been truncated at 0 and 100%. However, the uncertainty on cut efficiency was so small that the probability of drawing a non-physical efficiency (<0% or >100%) was utterly negligible.

log

_!"

$

_#$%

Figure 3.50: The distribution of limits produced by varying the inputs according to their uncertainties for absorption of a 2 eV dark photon. The median limit value is marked with a dashed black line. The upper and lower bounds of the 68.27% region are marked with dashed red lines.

As seen in Section 3.7.5, the uncertainty on energy calibration was a function of energy. We used a unit Gaussian distribution as its PDF. If the drawn value was positive (negative), we multiplied it by the upper (lower) combined calibration uncertainty and added the result to the calibrated energies. Therefore, a draw of 𝜎_{𝑃 𝐷 𝐹}= 0 would return the nominally calibrated energies. A draw of𝜎_{𝑃 𝐷 𝐹}= +(-)1 would return calibrated energies equivalent to using the upper (lower) edge of the combined calibration uncertainty band.

For absorption processes, uncertainty on the real part of the complex conductivity of silicon (𝜎₁) was also energy dependent and generated using a unit Gaussian PDF. This time, positive (negative) drawn values were multiplied by the difference between the upper (lower) and nominal𝜎₁values from [54]. The results were then added to the nominal value. Similarly to the energy calibration, a draw of𝜎_{𝑃 𝐷 𝐹} = 0 would return the nominal 𝜎₁. A draw of 𝜎_{𝑃 𝐷 𝐹} = +(-)1 would return the upper (lower)𝜎₁. Note that the nominal, lower, and upper𝜎₁values are only different for absorption energies <3.2 eV.

Diagnostic plots were also produced to visualize how varying the parameters affected the resulting limits. An example for DPA of mass 2 eV is shown in Figure 3.51. The example results are typical among all models and masses with the exception of the 𝜎₁dependence (which is non-existent for DMe).

parameter PDF shape mean value PDF parameters

energy resolution (𝜎_𝐸) flat 4.26 eV bounds = [3.03,5.49] eV CT probability truncated Gaussian 12.8% 𝜎_{𝑃 𝐷 𝐹} = 1.5%

bounds = [0,100]%

II probability truncated Gaussian 0.8% 𝜎_{𝑃 𝐷 𝐹} = 0.9%

bounds = [0,100]%

cut efficiency Gaussian 90.05% 𝜎_{𝑃 𝐷 𝐹} = 0.054%

energy calibration unit Gaussian 0 𝜎_{𝑃 𝐷 𝐹} = 1

conductivity of silicon (𝜎₁) unit Gaussian 0 𝜎_{𝑃 𝐷 𝐹} = 1

Table 3.2: The various systematic uncertainties considered for the Run 3 experiment.

The PDFs used to calculate the sensitivity to systematic uncertainties are also described for each parameter.

The diagnostic plots agree with our assumption that energy resolution (within the considered range) would not significantly affect limit results. The plots also suggest that calibration and efficiency uncertainties had insignificant effects. Cut efficiency scales the expected number of events and could heavily affect limits, but the Run 3 efficiency uncertainty was very small (0.054%). CT and II had clear effects. Limits assuming higher CT or II rates produced worse (larger) limits. This was due to both effects moving events to between-peak regions that the limit-setting method did not consider. DPA and ALP limits for masses <3.2 eV also showed a strong dependence on complex conductivity. Greater 𝜎₁ produced better (smaller) limits. This was expected since increasing 𝜎₁ increases the expected rate of events (see Eqs. 3.22 and 3.24).

%₎[eV] CT [fraction]

II [fraction]

%_*+,

impact ionization cut efficiency complex conductivity

%_*+,[0.054%] %_*+,

Figure 3.51: Diagnostic plots used to test the effect of various systematic uncertain- ties on the calculated limits. This example is for absorption of a 2 eV dark photon.

Orange (blue) distributions show the parameters that contributed to larger (smaller) than average limits. These results are typical among all models and masses.