Chapter II: Physics with the CMS detector
2.3 The trigger system
Figure 2.8: Diagram of the CMS detector layout, with the barrel (HB), endcap (HE), outer (HO), and forward (HF) components of the HCAL labeled. The pur- ple stripes in the outer regions of the detector indicate the placement of the muon chambers [17].
are useful in ensuring that muon hits are assigned to the correct bunch crossing.
The layout of the muon system is indicated in Figure 2.8.
For muons withpT lower than 200-300 GeV, scattering in the inner detector material limits the momentum resolution of the muon system. Thus, measurement of muon momentum is performed mainly by the inner tracker. For very high-pT muons, however, the effect of scattering is small and the momentum measurement benefits from the incorporation of muon chamber hit information.
imately 100 kHz of events. The HLT further reduces this to ~1 kHz of physics events, which are saved to disk for offline processing.
The L1 Trigger
The L1 Trigger is implemented in hardware using custom ASIC chips and FPGA cards. Within a latency of 4 µs, it must receive the event data from the detector, perform basic reconstruction of physics objects, and make a trigger decision. Due to the limits of the CMS readout electronics, the L1 output event rate must be no more than 100 kHz.
A diagram of the L1 event processing logic is shown in Figure 2.9. Physics object reconstruction is performed using data from the calorimeters (ECAL and HCAL) and from the muon chambers (DTs, CSCs, and RPCs). Calorimeter data is aggre- gated by a two-stage system consisting of a Regional Calorimeter Trigger (RCT) and a Global Calorimeter Trigger (GCT). The output of the GCT is a set of candi- date electron, photon, and jet objects. Data from the muon chambers is processed by the Global Muon Trigger (GMT), which outputs a set of muon candidates.
Figure 2.9: Diagram of the L1 trigger [19].
The final trigger decision is made by the L1 Global Trigger (GT), which receives as input the physics objects reconstructed by the GCT and the GMT. The GT consists
of a number of separate trigger paths, which compare the multiplicity and momenta of reconstructed physics objects against a set of predefined selection cuts.
The L1 system was upgraded in 2016 to feature more modern FPGAs and high- bandwidth optical links for communication between trigger cards [20]. Where previously the calorimeter hit data was coarse-grained before being processed by the RCT, the upgraded system allows the calorimeter triggers access to the full ECAL and HCAL granularity. This improvement allows physics objects to be re- constructed at the L1 with higher resolution, yielding more performant triggers that have lower cut thresholds yet still respect the 100 kHz rate limit. The upgrade also significantly increases the power and flexibility of the GT. It relaxes the hard limit of 128 trigger paths that existed previously, and enables more complex trigger logic involving, for example, the invariant masses of reconstructed particles.
The HLT
The HLT is implemented as software running on a processor farm at LHC Point 5.
It consists of a few hundred trigger paths, each selecting for a particular physics signature. The paths execute in parallel for each physics event selected by the L1 Trigger, and an event is accepted if it meets the requirements of any path.
HLT paths are divided into modules, each of which is either a ‘producer’ (which performs some reconstruction task) or a ‘filter’ (which applies a cut and rejects the event if it does not pass). An HLT path can in principle execute arbitrary code when making its trigger decision, using the raw CMS event data as input. The main constraint is the total compute time needed to process an event. Modified versions of the offline particle reconstruction algorithms described in Section 2.4, including the particle flow algorithm, have been written for use at the HLT. These algorithms yield physics objects similar to those built by the full reconstruction software, but trade accuracy for speed where necessary.
Some HLT paths areprescaled, meaning that they are blind to a certain fraction of events. A prescaled trigger is assigned a prescale factor that is an integerN > 1, and the trigger only runs on one in everyN events entering the HLT. Prescaled triggers are used to perform measurements that do not require the full statistics of the data, in cases where the unprescaled path would have prohibitively high rate.
To keep up with the rate of events selected by the L1 Trigger, the average time to run the full suite of HLT paths should not exceed a certain threshold, which is on
the order of hundreds of milliseconds. The exact CPU time budget varies from year to year as the HLT farm is upgraded with new machines. The distribution of event times is long-tailed: most events can be accepted or rejected very quickly, while a few require more intensive processing before the decision can be made. This long- tailed distribution is shown in Figure 2.10. Also shown is the mean execution time of the menu as a function of the instantaneous luminosity. In case the luminosity is too high for the full menu to run within the time constraint, the HLT can be dynamically switched to a more restricted menu where some paths are prescaled or disabled.
Figure 2.10: Left: distribution of simulated HLT event processing times for two different instantaneous luminosity scenarios at 13 TeV [21]. Right: average HLT event processing time as a function of instantaneous luminosity in 2016 [22]. The red line represents the limit imposed by the HLT farm.
The most time-intensive task performed at the HLT is track and vertex reconstruc- tion. The HLT particle flow algorithm makes use of an iterative track reconstruction algorithm similar to that used offline. Some iterations of the tracking sequence are omitted to save time; this limits the HLT’s ability to reconstruct displaced tracks, such as those from hadrons that undergo a nuclear interaction within the tracker material. The combinatorics of track seed extension are also alleviated by retaining fewer track candidates at each layer (in the offline algorithm, up to five candidates are considered at each layer).
A simpler track reconstruction procedure is also available that only considers hits in the pixel tracker, ignoring the strip tracker. These ‘pixel tracks’ are used as seeds for the full tracking algorithm, and as input to the HLT vertex identification algorithm.
Some applications make use of an even faster vertexing algorithm (‘fast primary vertex finding’) that identifies vertices compatible with selected high-pT jets by projecting pixel hits along the jet axis [23]. This algorithm reduces the running time of some HLT paths by a factor of 4 to 6.
Streams and primary datasets
HLT paths are organized intostreams, which are groups of paths sharing a similar purpose and having the same output event content. For example, ‘physics’ streams contain HLT paths selecting events for offline physics analysis. Events entering these streams are marked for prompt reconstruction, and their full raw data is sent to the CMS Tier-0 computing center for storage and further processing.
Other data streams contain HLT paths used for specialized tasks, such as detector calibration. These tasks may not require the full raw event data, and therefore some parts of the raw data may be discarded for events in these streams in order to save storage space. For example, a dedicated data stream is used for the calibration of the ECAL usingπ0decays to photons [24]. Only ECAL data from crystals near π0 candidates is needed for this calibration task, so the raw data from all other parts of the detector is discarded for events in this stream. The raw CMS data is formed from the output of ~600 Front-End Driver (FED) boards, each associated with a particular subdetector. Each FED’s data is a modular unit that can be individually saved or discarded, which facilitates the process of reducing the event size in these streams.
Streams are further subdivided into primary datasets (PDs). All PDs in a stream are processed in the same way and have the same event content; the subdivision is mainly to avoid excessively large file sizes.