Limits on tau Lepton-Flavor Violating Decays in three charged leptons

A search for the neutrinoless, lepton-flavor violating decay of the tau lepton into three charged leptons has been performed using an integrated luminosity of 468 fb^-1 collected with the BABAR detector at the PEP-II collider. In all six decay modes considered, the numbers of events found in data are compatible with the background expectations. Upper limits on the branching fractions are set in the range (1.8-3.3) X 10^-8 at 90% confidence level.

A search for the neutrinoless, lepton-flavor violating decay of the lepton into three charged leptons has been performed using an integrated luminosity of 468 fb À1 collected with the BABAR detector at the PEP-II collider. In all six decay modes considered, the numbers of events found in data are compatible with the background expectations. Upper limits on the branching fractions are set in the range ð1:8-3:3Þ Â 10 À8 at 90% confidence level. DOI Lepton-flavor violation (LFV) involving charged leptons has never been observed, and stringent experimental limits exist [1][2][3]. The experimental observation of neutrino oscillations [4] implies that, within the standard model (SM), there are amplitudes contributing to LFV in the charged sector, although their effects must be well below the current experimental sensitivity [5]. Many descriptions of physics beyond the SM predict enhanced LFV in decays over decays with branching fractions within present experimental sensitivities [6][7][8]. An observation of LFV in decays would be a clear signature of new physics, while improved limits will further constrain models.
This paper reports the latest results from BABAR on the search for LFV in the neutrinoless decay . All six lepton combinations consistent with charge conservation are considered. The analysis is based on data recorded by the BABAR [10] detector at the PEP-II asymmetric-energy e þ e À B factory operated at the SLAC National Accelerator Laboratory. The data sample is provided by an integrated luminosity of 426 fb À1 recorded at a center-of-mass (c.m.) energy ffiffi ffi s p ¼ 10:58 GeV, and of 42 fb À1 recorded at about ffiffi ffi s p ¼ 10:54 GeV. With these conditions, the expected cross section for -pair production is ¼ 0:919 AE 0:003 nb [11], corresponding to a data sample of about 430 Â 10 6 -pairs.
Charged-particle (track) momenta are measured with a 5-layer double-sided silicon vertex tracker and a 40-layer helium-isobutane drift chamber inside a 1.5 T superconducting solenoid magnet. An electromagnetic calorimeter consisting of 6580 CsI(Tl) crystals is used to measure electron and photon energies, a ring-imaging Cherenkov detector is used to identify charged hadrons, and the instrumented magnetic flux return (IFR) is used to identify muons. About half of the data sample under study was recorded with the IFR instrumented with resistive plate chambers (RPC). During the second half of the data taking period most RPCs were replaced by limited streamer tubes in the barrel section of the IFR.
A Monte Carlo (MC) simulation of lepton-flavor violating decays is used to estimate the signal efficiency and optimize the search. Simulated -pair events including higher-order radiative corrections are generated using KK2F [12] with one decaying to three leptons with a uniform three-body phase-space distribution, while the other decays according to measured rates [13] simulated with TAUOLA [14]. Final-state radiative effects are simu-lated for all decays using PHOTOS [15]. The detector response is simulated with GEANT4 [16].
The signature for À ! ' À 1 ' þ 2 ' À 3 is a set of three charged particles, each identified as either an e or a , with an invariant mass and energy equal to that of the parent lepton. Events are preselected requiring four reconstructed tracks and zero net charge, selecting only tracks pointing toward a common region consistent with þ À production and decay. The polar angles of all four tracks in the laboratory frame are required to be within the calorimeter acceptance range, to ensure good particle identification. The event is divided into two hemispheres in the e þ e À c.m. frame using the plane containing the interaction point and perpendicular to the thrust axis, as calculated from the observed tracks and neutral energy deposits. The signal hemisphere must contain exactly three tracks (3prong) with an invariant mass less than 3:5 GeV=c 2 , while the other hemisphere must contain exactly one (1-prong) track, and may contain also neutral energy deposits. In order to reduce backgrounds coming from photon conversions we require that the two couples of oppositely charged tracks in the 3-prong hemisphere have an invariant mass, calculated using electron mass hypothesis for the tracks, larger than 20 MeV=c 2 , or 30 MeV=c 2 for e À e þ e À and e À þ À .
With respect to our previous result [17], this analysis relies on significantly improved particle identification (PID) techniques for both AE and e AE . Electrons are identified applying an algorithm based on error correcting output code technique [18] which uses as input the ratio of calorimeter energy to track momentum ðE=pÞ, the ionization energy loss in the tracking system ðdE=dxÞ, and the shape of the shower in the calorimeter. Muon identification exploits a bagged decision tree (BDT) [19] algorithm, which uses as input the number of hits in the IFR, the number of interaction lengths traversed, and the energy deposition in the calorimeter. Since AE with momenta less than 500 MeV=c do not penetrate enough into the IFR to provide useful information, the BDT also uses information obtained from the inner trackers to maintain a very low À misidentification probability with high selection efficiencies. The electron and muon identification efficiencies are measured to be 91% and 77%, respectively. The probability for a AE to be misidentified as an e AE in 3-prong decays is 2.4%, while the probability to be misidentified as a AE is 2.1%.
The quantity ÁE E ?
rec À E ? beam is defined, where E ? rec is the total energy of the system observed in the 3-prong hemisphere and E ? beam is the beam energy (the superscript ? indicates quantities measured in the c.m. frame). We define ÁM ec M ec À m with M 2 ec E ?2 beam =c 4 À jp ? 3l j 2 =c 2 , where jp ? 3l j 2 is the squared momentum of the 3-prong system, m ¼ 1:777 GeV=c 2 is the mass [13], and the energy constrained momentum of the 3-prong system, jp ? 3l j, is obtained from a kinematic fit: the fit requires the energy measured in the c.m. to be ffiffiffiffiffiffiffi ffi s=2 p , taking into account the errors on the reconstructed track parameters and the beam energy measurement.
The signal distributions in the ðÁM ec ; ÁEÞ plane (see Fig. 1) are broadened by detector resolution and radiative effects. In all decay modes, the radiation of photons from the incoming e þ e À particles and from the outgoing decay products leads to a tail at low values of ÁE. Radiation from the final-state leptons, which is more likely for electrons than for muons, produces a tail at high values of ÁM ec as well. Signal regions (SR) in the ðÁM ec ; ÁEÞ plane are optimized in order to obtain the smallest expected upper limit (UL) when no LFV signal is present. The expected ULs are estimated using MC simulations and data control samples, instead of candidate signal events. The upper right corner of the signal region in the ðÁM ec ; ÁEÞ plane, in units of ðMeV=c 2 ; MeVÞ, is fixed at (30, 50) for À e þ e À and e À þ À and at (30, 100) for the other four channels. The lower left corner is at ðÀ30; À300Þ for the e À e þ e À , À e þ e À , and e À þ À decay modes, ðÀ30; À350Þ for þ e À e À and e þ À À , and ðÀ25; À200Þ for À þ À . Figure 1 shows the observed data in the ðÁM ec ; ÁEÞ plane, along with the signal region boundaries and the expected signal distributions. To avoid biases, a blind analysis procedure was followed, with the number of events in the SR remaining unknown until the selection criteria were finalized and all cross-checks were performed.
Each track present in the signal hemisphere must be identified as either a muon or an electron, depending on the channel under study. For the channels where two tracks of the same charge sign can be either an electron or a muon (i.e. À e þ e À and e À þ À ), it is possible that both tracks satisfy both electron and muon PID selectors: in these rare cases we measure ÁM ec and ÁE in both mass hypotheses. For all events showing this behavior only one of the two combinations falls in the large box (LB) of the ðÁM ec ; ÁEÞ plane, defined as the region lying between À600 and 400 MeV=c 2 in ÁM ec and À700 and 400 MeV in ÁE.
The PID requirements strongly suppress background, but further selection is applied: for all decay modes, the momentum of the 1-prong track is required to be less than 4:8 GeV=c in the c.m. frame. The 1-prong side mass is approximately reconstructed from the 4-momentum obtained by adding the 1-prong track, the neutral energy deposits in the 1-prong hemisphere, and the missing 3momentum of the event, assuming a zero mass as is appropriate if just a single neutrino is missing. This invariant mass is required to be in the range 0:2-3:0 GeV=c 2 for e À e þ e À , À e þ e À , and e À þ À , and in the range of 0:1-3:5 GeV=c 2 for þ e À e À , e þ À À , and À þ À . To suppress Bhabha backgrounds we reject events where any oppositely charged track pair has an invariant mass compatible with a photon conversion when assigning the electron mass to the two tracks. M e þ e À is required to be >200 MeV=c 2 for all channels except for e þ À À where M e þ e À > 300 MeV=c 2 is required. For the e À e þ e À and e À þ À decay modes, the charged particle in the 1-prong hemisphere is required to be matched to an energy deposit in the calorimeter inconsistent with an electron, and must not be identified as an electron, while for the þ e À e À , À e þ e À , and À þ À decay modes this track must not be identified as a muon. For the e À e þ e À and e À þ À decay modes, the missing momentum of the event should be greater than 300 MeV=c, for e À þ À and À þ À this should be more than 200 MeV=c and for þ e À e À and e þ À À this lower limit is set at 100 MeV=c. For the e À e þ e À and þ e À e À channels the cosine of the angle between the direction of the sum of the three signal track momenta and the direction of the 1-prong track momentum ( 13 ), is required to satisfy cosð 13 Þ > À0:995 and cosð 13 Þ > À0:997 respectively, to further reduce Bhabha contributions.
The backgrounds still contaminating the sample have been identified in three broad categories: low multiplicity q " q events (comprising both continuum light quark pairs and c " c pairs), QED events (Bhabha or þ À depending on the particular channel), and SM þ À events. These three background classes have distinctive distributions in the ðÁM ec ; ÁEÞ plane. The q " q events tend to populate the plane uniformly, while QED backgrounds fall in a narrow band at positive values of ÁE, and þ À backgrounds are restricted to negative values of both ÁE and ÁM ec due to the presence of at least one undetected neutrino. The possible background contribution arising from two-photon processes has been studied on a data control sample, as discussed in the following, and it is found to be negligible.
The expected background rates for each decay mode are determined by fitting a set of probability density functions (PDFs) to the observed data in the grand sideband (GS) region of the ðÁM ec ; ÁEÞ plane as was done in the previous published analysis [17]. The GS region covers the same region as the LB but does not include the SR. The functional forms of the PDFs are the same as in [17]. For the q " q background, a two-dimensional PDF is constructed from the product of two PDFs, P M 0 and P E 0 , where P M 0 ðÁM 0 Þ is a bifurcated Gaussian and P E 0 ðÁE 0 Þ ¼ ð1 À x= ffiffiffiffiffiffiffiffiffiffiffiffiffiffi 1 þ x 2 p Þ Â ð1 þ ax þ bx 2 þ cx 3 Þ with x ¼ ðÁE 0 À dÞ=e. The ðÁM 0 ; ÁE 0 Þ axes have been slightly rotated from ðÁM ec ; ÁEÞ to take into account the observed correlation between ÁE and ÁM ec for the distribution. For the þ À background PDF, the function P M 00 ðÁM 00 Þ is the sum of two Gaussians with common mean, while the functional form of P E 00 ðÁE 00 Þ is the same as that for the q " q PDF. To properly model the wedge-shaped distribution due to the kinematic limit in tau decays, a coordinate transformation of the form ÁM 00 ¼ cos 1 ÁM ec þ sin 1 ÁE and ÁE 00 ¼ cos 2 ÁE À sin 2 ÁM ec is performed.
QED backgrounds represent one of the major sources of backgrounds for e À e þ e À , þ e À e À , À e þ e À , and e À þ À . To study this background category, specially selected control samples obtained from data were produced. Two different methods were used to extract QED control samples: for e À e þ e À , þ e À e À , and e À þ À channels the sample was produced selecting events passing all selection requirements, except the lepton veto in the tag side, and requiring the track in the tag side to be identified as a muon ( þ e À e À channel) or as an electron (for the other two channels). To obtain a large enough sample for the À e þ e À channel we selected events where a muon is present in the tag side, and the reconstructed mass in the tag side is between 0:5 GeV=c 2 and 2:5 GeV=c 2 , and the momentum of the tag-side particle is required to be larger than 4:8 GeV=c. To fit these control samples, an analytic PDF is constructed from the product of a crystal ball function [20] in ÁE 0 and a third-order polynomial in ÁM 0 , where again the ðÁM 0 ; ÁE 0 Þ axes have been rotated slightly from ðÁM ec ; ÁEÞ to fit the observed distribution.
The expected background rate in the SR is obtained by an unbinned maximum likelihood fit to the data in the GS region, with the shapes of the three background PDFs fixed by making an unbinned likelihood fit to the MC and the control samples. The PDF shape determinations and background fits are performed separately for each of the six decay modes. Cross-checks of the background estimation are performed by considering the numbers of events expected and observed in sideband regions immediately neighboring the signal region for each decay mode.
The efficiency of the selection for signal events is estimated with the MC simulation of signal LFV events. The efficiency of signal MC passing preselection requirements varies between 45% and 49%. The total efficiency for signal events to be found in the signal region is shown in Table I for each decay mode and ranges from 6.4% to 12.7%. This efficiency includes the 85% branching fraction for 1-prong decays. With respect to the previous analysis, improvements in particle ID, in tracking algorithms and in selection criteria allowed us to obtain higher signal efficiencies along with a reduction of the expected backgrounds thus improving the UL sensitivity.
Uncertainties in signal efficiency estimation and in the number of the expected events in the SR obtained by the fit affect the final result. The systematic uncertainties from PID dominate the error on the efficiency. They are estimated on data control samples, by measuring the variation of the data and MC efficiencies for tracks with the same kinematic properties. The uncertainty on the efficiency of the electron identification was evaluated using a control sample consisting of both radiative and nonradiative Bhabba events, while the uncertainty for muons was estimated using a radiative di-muon (i.e. e þ e À ! þ À ) control sample. The uncertainty on the pion misidentifica- tion probability, as muon or electron, was investigated using samples of decays into three pions. The uncertainties vary between a relative error of 1.8% for e À e þ e À and 7.8% for À þ À . The modeling of the tracking efficiency contributes an additional 1% relative uncertainty. All other sources of uncertainty in the signal efficiency are found to be smaller than 1.0%, including the statistical limitation of the MC signal samples, the modeling of higher-order radiative effects, track momentum resolution, trigger performance, observables used in the selection criteria, and knowledge of the tau 1-prong branching fractions. The MC signals sere generated assuming flat phasespace distributions. The efficiencies were measured over the Dalitz plot for the six channels, and were found to vary by 10-15% for all channels [21]. No additional uncertainties have been included to account for model-dependent structure in the decays. The systematic uncertainty due to errors in background estimation is determined from fits to data in the GS region. In addition to varying PDF parameters by their uncertainties, alternative functional forms are used to determine the uncertainty on the expected background yield in the SR. The total errors on the background estimates are reported in Table I. Systematics coming from unsimulated background contributions, such as two-photon processes, are checked using background enriched control samples. Twophoton processes are characterized by a small transverse momentum, so the control samples were produced selecting events with a transverse momentum smaller than 0:2 GeV=c, and with the momentum of the tag-side track in the center of mass smaller than 4:0 GeV=c. The uncertainties introduced by two-photon processes and unsimulated backgrounds are found to be negligible.
Background expectations (N bgd ) and the number of observed events (N obs ) are shown in Table I. No events are observed in the SR for any of the modes and we place 90% confidence level (CL) ULs on the branching fractions using UL obs 90 ¼ N 90 UL =ð2"L Þ, where N 90 UL is the 90% CL UL for the number of signal events when N obs events are observed with N bgd background events expected. The values ", L, and are the selection efficiency, luminosity, and þ À cross section, respectively. The uncertainty on the product L Á is 0.9%. The branching fraction ULs are calculated, with all uncertainties included, using the technique of Cousins and Highland [22] following the implementation of Barlow [23]. The expected average upper limit UL 90 exp , defined as the mean UL expected in the background-only hypothesis, is included in Table I. The 90% CL ULs on the À ! ' À 1 ' þ 2 ' À 3 branching fractions are in the range ð1:8-3:3Þ Â 10 À8 . These limits supersede the previous BABAR analysis [17], and are compatible with the latest Belle limits [24].