Search for the decay B+ ->K+ tau-/+ mu+/-

We present a search for the lepton flavor violating decay B+ -->K+ tau-/+ mu+/- using 383 million B Bbar events collected by the BABAR experiment. The branching fraction for this decay can be substantially enhanced in new physics models. The kinematics of the tau from the signal B decay are inferred from the K+, mu, and other B in the event, which is fully reconstructed in one of a variety of hadronic decay modes, allowing the signal B candidate to be fully reconstructed. We observe no excess of events over the expected background and set a limit of B(B+ -->K+ tau mu)<7.7 x 10^-5 at 90% confidence level, where the branching fraction is for the sum of the K+ tau- mu+ and K+ tau+ mu- final states. We use this result to improve a model-independent bound on the energy scale of flavor-changing new physics.

the K + , µ, and other B in the event, which is fully reconstructed in one of a variety of hadronic decay modes, allowing the signal B candidate to be fully reconstructed. We observe no excess of events over the expected background and set a limit of B(B + → K + τ µ) < 7.7 × 10 −5 at 90% confidence level, where the branching fraction is for the sum of the K + τ − µ + and K + τ + µ − final states. We use this result to improve a model-independent bound on the energy scale of flavor-changing new physics.
PACS numbers: 13.25.Hw, 14.40.Nd Lepton flavor violation (LFV) thus far has only been observed in the neutrino sector [1,2,3]. Manifestations of LFV in B meson decays that have final states with charged leptons (e.g., B → Kℓℓ ′ ) are allowed in standard model interactions, if massive neutrinos are included [4], but such processes occur only at the oneloop level and are extremely suppressed by powers of m 2 ν /M 2 W . Branching fractions for lepton flavor violating B decays can be substantially enhanced in many extensions of the standard model [5,6,7,8]. The semileptonic decay B → Kτ µ is likely to have higher sensitivity to new physics, when compared to leptonic decays such as B 0 → τ ℓ ′ , since the latter is both helicity and Cabibbo-Kobayashi-Maskawa quark-mixing matrix (CKM) suppressed by a factor of |V td /V cb | 2 . Some new physics models require flavor changing neutral currents (FCNC's) to occur at the one-loop level, as in the standard model. In other extensions, such as models with a Z ′ or additional Higgs doublets, FCNC's occur naturally at the tree level, unless they are eliminated by imposing an ad hoc discrete symmetry.
A limit on the process B → Kτ µ, which involves the second and third generations of both quarks and leptons, would provide a unique and powerful constraint on model parameters of grand unified theories. Cheng, Sher, and Yuan [5,9] propose that in models with an extended Higgs sector the most natural value of the FCNC Yukawa couplings connecting generations i and j are proportional to √ m i m j /m τ , which implies that FCNC's in these theories should be largest in processes involving the second and third generations. An observation of B → Kτ µ would be an unambiguous sign of physics beyond the standard model. In this paper, we present the results of a search for B → Kτ µ.
We use a data sample of 383 million BB pairs produced by the PEP-II asymmetric-energy e + e − collider, running at the Υ (4S) resonance, collected by the BABAR experiment [10] at SLAC. Charged particles are identified using a Cerenkov radiation detector and dE/dx measurements in the tracking system. Instrumentation embedded within the iron of the flux return for the 1.5 T solenoid aids in the identification of muons. An electromagnetic CsI(Tl) crystal calorimeter (EMC) is used to reconstruct photons and identify electrons.
The analysis strategy is to reconstruct the Υ (4S) → B + B − in the search for B + → K + τ µ [11]. One of the B mesons (B tag ) is fully reconstructed in one of a large num-ber of hadronic final states, B − → D ( * )0 X − [12]. The X − represents a system of charged and neutral hadrons with total charge −1 composed of n 1 π ± , n 2 K ± , n 3 K 0 S , and n 4 π 0 , with n 1 + n 2 ≤ 5, n 3 ≤ 2, and n 4 ≤ 2. The D * 0 is reconstructed in the D 0 π 0 and D 0 γ channels, the D 0 in the K − π + , K − π + π 0 , K − π + π − π + , and K 0 S π + π − channels, and K 0 S in the π + π − channel. We search for the decay B + → K + τ µ using the remaining tracks in the event. The momentum vector of the signal B candidate, p sig , must be equal in magnitude and opposite in direction to that of B tag in the center-of-mass (CM) frame. The τ candidate kinematic variables, E τ and p τ , are fully constrained by p sig , the measured momenta of the K + and µ tracks, and the constraint where E beam is the CM beam energy. The reconstructed τ invariant mass m τ = E 2 τ − p 2 τ peaks sharply at the true τ mass for the signal.
Events are required to contain a B tag candidate with m ES ≡ E 2 beam − p 2 tag > 5.27 GeV/c 2 and E tag consistent with E beam within three standard deviations, where p tag and E tag are the momentum and energy of the B tag candidate in the CM frame. A B tag meson is fully reconstructed in about 0.2% of Monte Carlo events where one B ± decays to K ± τ µ. Even though the τ daughters are not needed to reconstruct m τ , we require the τ in the signal B candidate to be consistent with a "one-prong" (i.e., one-charged-track) τ decay to reject combinatoric background. Therefore we require exactly three charged tracks in the event not associated with B tag and with net charge opposite to that of B tag . Among these three tracks, we require: a kaon candidate with charge opposite B tag , a muon candidate, and a third track (the τ daughter) with charge opposite the muon candidate. The event is rejected if any of the three tracks is consistent with a proton hypothesis or if either of the two non-kaon tracks is consistent with a kaon hypothesis. Signal candidates are divided into three categories based on the properties of the τ -daughter track: electron, muon, and pion.
The kaon, muon, and electron particle identification criteria used in this analysis have momentumdependent efficiencies and misidentification probabilities (fake rates). The kaon candidate must pass loose selection criteria, based on the measured Cerenkov angle and dE/dx in the tracking system. Muon candidates, either from the B decay or from the τ decay, must pass minimum selection criteria that are 85% efficient for muons above 1.5 GeV/c and less than 10% efficient for pions and kaons. Tau daughter electrons must pass minimum electron selection criteria that are 95% efficient for electrons. More stringent electron and muon identification criteria for the tau daughter track are incorporated through a likelihood ratio described below. Tau daughters that do not pass either the electron or muon criteria fall into the pion category.
For muon channel signal B candidates, there are two muons in the final state: one from the B decay (primary muon) and one from a leptonic τ decay. Of the two possible track assignments for the primary muon, we use the one that gives m τ closest to the known τ mass. The bias in the background m τ distribution for the muon channel from using this procedure is found to be negligible.
Semileptonic B decays can produce final states that appear identical to the signal. For example B + →D 0 µ + ν µ followed byD 0 → K + π − produces a K + µ + π − ν final state. If theD 0 decays semileptonically, the final state is K + µ + ℓ − νν. These backgrounds are easily removed by requiring that the invariant mass, m(Kπ), of the kaon candidate and the oppositely-charged, signal-track candidate, when this track is assumed to be a pion, be greater than 1.95 GeV/c 2 . This requirement is greater than 50% efficient for the signal and removes about 99% of the background from BB events.
The B + →D 0 µ + ν µ and B + →D * 0 µ + ν µ , withD 0 → K + π − , backgrounds also form the Dµν control sample, which we use to normalize the signal branching fraction. Events for the Dµν control sample are required to have m(Kπ) in the range [1.845, 1.885] GeV/c 2 (within about three standard deviations of the D 0 mass). The neutrino momentum is calculated from p tag and the three tracks in the Dµν final state. We then compute We use ∆E Dµν rather than m miss = E 2 miss − p 2 ν , similar to our m τ reconstruction, because the expected D 0 µν missing mass is zero. The ∆E Dµν distribution for D 0 µν decays is centered at zero, while for D * 0 µν events, it is shifted by −150 MeV and slightly asymmetric, due to the missing neutral particle from the D * 0 → (π 0 , γ)D 0 decay. We determine the yield of D 0 µν and D * 0 µν events simultaneously in an unbinned maximum likelihood fit of ∆E Dµν (Figure 1).
For the K + τ µ signal, a non-negligible source of BB background in the sample remaining after the m(Kπ) requirement comes from B + → (cc)K + with (cc) → µ + µ − , where (cc) is a charmonium resonance. This background mainly enters in the muon channel, but is also present in the pion channel. For the muon and pion channels this background is removed by requiring the invariant mass of the two non-kaon tracks, when both are assumed to be muons, to be outside of the ranges [3.03, 3.14] GeV/c 2 and [3.60,3.75] GeV/c 2 , which are centered on the J/ψ and ψ(2S) resonances masses respectively. At this stage in the selection, the background is dominated by continuum events (e + e − → qq with q = u, d, s, c). We suppress this background using a likelihood ratio where x is a vector of four discriminating variables (see below) and P s (x i ) (P b (x i )) is the probability density function (PDF), which describes the signal (background) for variable x i . The PDFs are separate for each of the three signal categories. The four discriminating variables are: | cos θ thr | the magnitude of the cosine of the CM angle between the B tag thrust axis and the thrust axis of the rest of the event; E cal the total neutral EMC energy that is not associated with B tag ; the quality of the primary lepton identification; and the quality of the secondary lepton identification (for electron and muon channel candidates). We require a minimum E cal energy of 50 MeV (100 MeV) for clusters in the barrel (forward endcap) to be included in E cal . The | cos θ thr | distribution is flat for signal and peaks near one for continuum. The E cal distribution peaks at zero for signal, while the background distribution is broad, peaking at around 1.5 GeV. The lepton quality is divided into four hierarchical, mutually-exclusive categories with fake rates decreasing with increasing quality rank. For the highest-quality muon candidate rank, the fake rates from pions and kaons are less than 2%. The highest-quality electron candidate rank has a fake rate of less than 0.1% for pions, as high as 3% for low-momentum kaons, and below 0.4% for kaons above 0.8 GeV/c. We fit signal and background Monte Carlo histograms of | cos θ thr | and E cal to define P s (x i ) and P b (x i ) for those variables. We use the relative fractions in the four lepton quality categories in the Monte Carlo samples for P s (x i ) and P b (x i ) for the primary and secondary lepton variables.
We make a minimum L R requirement for each of the three signal categories (electron, muon, and pion) which has been optimized to give the lowest signal branching fraction limit under the assumption of no signal in the data. The signal region in m τ is defined to be [1.65, 1.90] GeV/c 2 , which contains 90% of the signal. The signal selection efficiency (ǫ i ), including the L R requirements, in the m τ signal region is 3.17%, 2.04%, and 2.13% for the electron, muon, and pion channels respectively. The denominator of these ǫ i is the same for all three and includes all tau decays. We have used a simple 3-body phase space decay model to generate our signal Monte Carlo sample. The systematic uncertainty on ǫ i is determined by varying the signal and background PDFs for each L R . Because the signal branching fraction is determined from the ratio of the signal and Dµν yields in the data, many systematic uncertainties associated with tracking, particle identification, and the B tag reconstruction cancel. The amount of background, b i , in the m τ signal region is estimated from the number of events outside the m τ signal region (the m τ sidebands) in the ranges [0, 1.65] and [1.9, 3.5] GeV/c 2 and the signal-to-sideband ratio from the background Monte Carlo sample.
The signal branching fraction for each channel (B i ) is estimated using the relation where n i is the observed number of events in the m τ signal region, b i is the expected background, ǫ i is the signal efficiency, and S 0 is a common sensitivity factor given by where N Dµν , B Dµν , and ǫ Dµν are the fitted yield, total branching fraction, and selection efficiency for the Dµν control sample and ǫ Kτ µ tag and ǫ Dµν tag are the B tag efficiencies for the signal and Dµν samples respectively. The last factor (ǫ Kτ µ tag /ǫ Dµν tag ) is determined from the Monte Carlo samples and close to one (0.922 ± 0.052) since the topology of the events in the signal and Dµν samples is very similar. We find N Dµν = 867 ± 52 with ǫ Dµν = 0.345±0.008 and B Dµν = (3.29±0.22)×10 −3 [13], which gives S 0 = (7.0 ± 0.7) × 10 5 .
The m τ signal region in the data was kept blind during the development of the analysis, to avoid experimenter's bias. After all analysis decisions were made, we "opened the box" and found 1, 0, and 2 events in the m τ signal region for the electron, muon, and pion channels respectively. These totals are consistent with our expectations from background only, which are given in Table I  The central value for the signal branching fraction is B = (0.8 +3.5 −2.3 ) × 10 −5 , where the uncertainties include both statistical and systematic sources. The three channels were combined by maximizing the likelihood, which is defined as the product of three Poisson probabilities in n i , where the mean is given by Equation 3. Uncertainties on the b i , ǫ i , and S 0 parameters, which determine the Poisson mean for each channel, were included by convolving the Poisson PDFs with Gaussians in b i , ǫ i , and S 0 . The uncertainties on B correspond to the points where the log likelihood drops by 0.5 with respect to the maximum log likelihood value. We have verified, with a Monte Carlo study, that this maximum likelihood technique is unbiased and that the uncertainties are a reasonable estimate of one standard deviation. We find a 90% confidence level upper limit on the signal branching fraction of B < 7.7 × 10 −5 using the prescription of Feldman and Cousins [14] for defining the confidence belt. The central value and upper limit on the signal branching fraction are both limited by statistical uncertainties.
In conclusion, we present the first search for the forbidden decay B + → K + τ µ using 383 million BB pairs collected by the BABAR experiment. The observed events are consistent with the background-only hypothesis. We set an upper limit of B < 7.7×10 −5 on the signal branching fraction at 90% confidence level. This result can be used to improve the model-independent bound on the energy scale of new physics in flavor-changing operators reported in [6] from > 2.6 TeV to > 13 TeV.
We are grateful for the excellent luminosity and machine conditions provided by our PEP-II colleagues, and for the substantial dedicated effort from the computing organizations that support BABAR. The collaborating institutions wish to thank SLAC for its support and kind hospitality. This work is supported by DOE   I: The number of events in the mτ sidebands, N sb , for the Monte Carlo sample and the data; the ratio of background events, inside / outside the mτ signal region, BG ratio; the expected number of background events, bi, and number of observed data events, ni, in the mτ signal region; and the signal selection efficiency ǫi for each of the three channels. The first (second) uncertainty on ǫi is statistical (systematic).