Blinding the Spectral Extraction

Chapter IV: UCNA: Fierz Interference Analysis

4.2 The Spectral Extraction Method

4.2.1 The Super-Sum Spectrum

4.2.1.3 Blinding the Spectral Extraction

After the publication of the UCNA 2010 dataset Fierz interference direct spectral extraction (summarized later in section 4.2.5), there was a move towards blinding the data for the subsequent UCNA 2011-2012 and 2012-2013 datasets to ensure a more robust analysis. For an in-depth discussion on blinding data analyses in experimental physics, see [KR05]. In summary, there are several case studies of physics experiments throughout history that show unconscious bias ultimately leading to convergences in experimental measurements. These convergences are non-statistical in nature and their prevalence can be explained by unconscious bias.

Blinding thus was introduced as a method of combating this human factor and has lead to a successful reinterpretation of experimental measurements, producing a distribution that fluctuates within experimental uncertainties. The community as a whole has generally concluded blinding to be a crucial step in validating the integrity of experimental analysis results. With this in mind, we decided to blind the 2011-2012 and 2012-2013 Fierz interference extractions. Below, we describe the blinding methodology for the direct spectral analysis.

Recall from the previous section, section 4.2.1.2, that we use two baseline Monte

Carlo histograms, processed through the GEANT4 simulation and with the detector response model to become “data-like”. We then use these data-like histograms to fit the real data histogram and extract a fractional presence of the 𝑏 =0 histogram and the𝑏 =∞histogram. These fractional percentages directly relate to a𝑏 value.

In order to perform the blinding, there would be no way to reliably modify the data since we only have one true dataset and a priori we do not know what𝑏 value that dataset would extract. Instead, what we do is modify the baseline Monte Carlo histograms. By “mixing in” a percentage of events from the𝑏→ ∞histogram, we distort the 𝑏 = 0 histogram and transform it into a 𝑏 ≠ 0. In particular, when we mix in𝑏→ ∞events into the𝑏 =0 histogram, the baseline𝑏=0 histogram shifts to𝑏 > 0 (the energy peak shifts to lower energy, which physically is what a 𝑏 > 0 distortion induces) and hence when we fit a standard 𝑏 = 0 histogram with these two “blinded” baseline histograms, we would produce a𝑏 < 0 value. We note this is the first “case”: creating a blinded𝑏 < 0 value.

To show this explicitly, we start by taking the results of equation 4.5, 𝑃_𝑏(𝐸_𝑒) =

1+𝑏(^𝑚𝑒

𝐸𝑒) 1+𝑏

D𝑚𝑒 𝐸𝑒

E𝑃_{𝑆 𝑀}(𝐸_𝑒), and that in equation 4.6, 𝑃_𝑏→∞ = 𝑃_{𝑆 𝑀}^𝑚𝑒

𝐸_𝑒

D𝑚𝑒

𝐸_𝑒

E−1

. Also, the fitting is defined in equation 4.7 as 𝑓_{𝑑𝑎𝑡 𝑎}(𝐸_𝑒) =

𝑃_{𝑆 𝑀}(𝐸_𝑒)+𝑏 D𝑚𝑒

𝐸𝑒

E

𝑃_𝑏→∞(𝐸_𝑒) 1+𝑏

D𝑚𝑒 𝐸𝑒

E .

We now introduce a mixing percentage,𝑟, to our 𝑏 = 0 distribution. That is, each event in the𝑏 =0 histogram is sampled and with probability𝑟 that event is replaced by the corresponding numbered event from the 𝑏 → ∞histogram. We note both generated histograms have the same number of events. This results in the following transformation on our baseline SM distribution:

𝑃_{𝑆 𝑀} ↦→ (1−𝑟)𝑃_{𝑆 𝑀} +𝑟 𝑃_𝑏→∞ (4.8) where 𝑃_{𝑆 𝑀} and 𝑃_𝑏→∞ have implicit energy dependence that we suppress for the sake of notation here. Replacing𝑃_{𝑆 𝑀} in equation 4.7 with the result from equation 4.8, we get

𝑓_{𝑑𝑎𝑡 𝑎}(𝐸_𝑒) = 𝑃⁰

𝑆 𝑀(𝐸_𝑒) +𝑏h^𝑚𝑒

𝐸𝑒

i𝑃_𝑏→∞(𝐸_𝑒)

1+𝑏h^𝑚𝑒

𝐸𝑒

i

=

(1−𝑟)𝑃_{𝑆 𝑀}(𝐸_𝑒) +𝑟 𝑃_𝑏→∞(𝐸_𝑒) +𝑏h^𝑚𝑒

𝐸𝑒

i𝑃_𝑏→∞(𝐸_𝑒)

1+𝑏h^𝑚𝑒

𝐸𝑒

i

=

(1−𝑟)𝑃_{𝑆 𝑀}(𝐸_𝑒) + [𝑟+𝑏h^𝑚𝑒

𝐸𝑒

i]𝑃_𝑏→∞(𝐸_𝑒)

1+𝑏h^𝑚𝑒

𝐸_𝑒i

(4.9)

The goal here is to introduce a false𝑏value that then becomes our blinded𝑏value.

In order to determine the𝑏 value introduced by a mixing percentage𝑟, we replace the 𝑓_{𝑑𝑎𝑡 𝑎}(𝐸_𝑒)distribution with a𝑏=0 Monte Carlo distribution. This gives us

𝑃_{𝑆 𝑀}(𝐸_𝑒) =

(1−𝑟)𝑃_{𝑆 𝑀}(𝐸_𝑒) + [𝑟 +𝑏h^𝑚𝑒

𝐸𝑒

i]𝑃_𝑏→∞(𝐸_𝑒)

1+𝑏h^𝑚𝑒

𝐸𝑒

i (4.10)

For equation 4.10 to be true, we can set the coefficient in front of𝑃_{𝑆 𝑀} to 1 and the coefficient in front of𝑃_𝑏→∞(𝐸_𝑒)to 0. Solving both of these gives the same answer since this problem is over-constrained.

∴ 𝑏= −𝑟 h^𝑚𝑒

𝐸𝑒

i (4.11)

𝑏 is the Fierz interference value we are trying to introduce as a blinding and 𝑟 is a percentage of events we are mixing between a SM distribution and a 𝑏 → ∞ distribution.

The second case is producing a blinded 𝑏 > 0 value. We can repeat the same procedure except we mix𝑏 =0 and𝑏 =−1 events together. Due to the form of𝑏in equation 3.1, 𝑏 =−1 is the lowest possible value (any more negative values would induce a negative decay rate in the equation). Hence𝑏 =−1 conceptually represents the opposite of 𝑏 → ∞, an effective 𝑏 → −∞. By mixing in𝑏 = −1 events, the baseline Monte Carlo histogram transform from 𝑏 = 0 to 𝑏 < 0 (the energy peak shifts to a high energy) and hence when we fit a standard 𝑏 = 0 histogram, we produce a𝑏 > 0 value.

To show this explicitly, we again use a mixing percentage, 𝑟, of 𝑏 = −1 events in our𝑏 =0 distribution. This results in the following transformation on our baseline SM distribution:

𝑃_{𝑆 𝑀} ↦→ (1−𝑟)𝑃_{𝑆 𝑀} +𝑟 𝑃_𝑏=−

1 (4.12)

where𝑃_{𝑆 𝑀} and𝑃_𝑏=−

1(and𝑃_𝑏→∞below) have implicit energy dependence that we again suppress for the sake of notation. When we use equations 4.5 and 4.6, we also get

𝑃_𝑏=−

1= 1− ^𝑚𝑒

𝐸𝑒

1− h^𝑚𝑒

𝐸𝑒

i𝑃_{𝑆 𝑀}

= 𝑃_{𝑆 𝑀} 1− h^𝑚𝑒

𝐸𝑒

i −

𝑚𝑒

𝐸𝑒

𝑃_{𝑆 𝑀} 1− h^𝑚𝑒

𝐸𝑒

i

= 1 1− h^𝑚𝑒

𝐸_𝑒i

𝑃_{𝑆 𝑀}− h𝑚_𝑒 𝐸_𝑒

i𝑃_𝑏→∞

(4.13)

Plugging this result into equation 4.12, the new baseline distribution we are fitting with becomes𝑃_𝑏→∞and

𝑃_{𝑆 𝑀} ↦→ (1−𝑟)𝑃_{𝑆 𝑀}+ 𝑟 1− h^𝑚𝑒

𝐸_𝑒i

𝑃_{𝑆 𝑀}− h𝑚_𝑒 𝐸_𝑒

i𝑃_𝑏→∞

(4.14)

Now we collect the terms in front of 𝑃_{𝑆 𝑀} and 𝑃_𝑏→∞ and equate this to a 𝑏 = 0 Standard Model histogram, giving

𝑃_{𝑆 𝑀} =

(1−𝑟) + ^𝑟

1−h^𝑚

𝐸i

1+ h^𝑚

𝐸i 𝑃_{𝑆 𝑀} +

𝑏h^𝑚

𝐸i + ^𝑟h

𝑚 𝐸i 1−h^𝑚

𝐸i

1+ h^𝑚

𝐸i 𝑃_𝑏→∞ (4.15) In order for equation 4.15 to hold, the coefficient in front of𝑃_{𝑆 𝑀} must be 1 and the coefficient in front of𝑃_𝑏→∞must be 0. Again, solving both of these gives the same answer since this problem is over-constrained.

∴ 𝑏= 𝑟 1− h^𝑚

𝐸i (4.16)

where, again,𝑏 is the Fierz interference term we are trying to introduce and𝑟 is a percentage of events we are mixing between a 𝑏 = 0 (SM) spectrum and a𝑏 = −1 (maximal Fierz in positive energy shift for the spectrum peak) spectrum.

Equations 4.11 and 4.16 represent the blinded value of 𝑏 chosen by sampling a percentage of events from a𝑏 → ∞distribution and a𝑏=−1 distribution and using them to replace the same number of events from a𝑏 = 0 distribution. Afterwards, we tested equations 4.11 and 4.16 by fitting a known input𝑏value with equation 4.7 and confirmed that they produced a blinded𝑏value when fitting a𝑏 =0 spectrum.

We then randomly sampled which formula to use, corresponding to whether we blinded with𝑏 < 0 or𝑏 > 0, and then chose an unknown value of𝑟 corresponding to a blinding value𝑏 ∈ [−0.075,0.075].