Study of Resonances in Exclusive B Decays to Dbar(*)D(*)K

We present a study of resonances in exclusive decays of $B$ mesons to $\bar{D}^{(*)} D^{(*)} K$. We report the observation of the decays $B \to \Dbar^{(*)} D^+_{s1}(2536)$ where the $D^+_{s1}(2536)$ is reconstructed in the $D^{*0}K^+$ and $D^{*+} K^0_S$ decay channels. We report also the observation of the decays $B \to \psi(3770) K$ where the $\psi(3770)$ decays to $\Dbar^0 D^0$ and $D^- D^+$. In addition, we present the observation of an enhancement for the $\Dbar^{*0} D^0$ invariant mass in the decays $B \to \Dbar^{*0} D^0 K$, at a mass of $(3875.1 {}^{+0.7}_{-0.5} \pm 0.5)$ MeV/$c^2$ with a width of $(3.0 {}^{+1.9}_{-1.4} \pm 0.9)$ MeV (the first errors are statistical and the second are systematic). Branching fractions and spin studies are shown for the three resonances. The results are based on 347 fb$^{-1}$ of data collected with the BaBar detector at the PEP-II $B$ factory.

and D * + K 0 S decay channels. We report also the observation of the decays B → ψ(3770)K where the ψ(3770) decays to D 0 D 0 and D − D + . In addition, we present the observation of an enhancement for the D * 0 D 0 invariant mass in the decays B → D * 0 D 0 K, at a mass of (3875.1 +0. 7 −0.5 ± 0.5) MeV/c 2 with a width of (3.0 +1. 9 −1.4 ± 0.9) MeV (the first errors are statistical and the second are systematic). Branching fractions and spin studies are shown for the three resonances. The results are based on 347 fb −1 of data collected with the BABAR detector at the PEP-II B factory.
The D + s1 (2536) resonance is the narrow P-wave D + s meson with J P = 1 + assignment strongly favored. It can be produced in B → D ( * ) D + s1 (2536) decays and should decay dominantly to D * 0 K + and D * + K 0 [1]. Evidence for D + s1 (2536) production in B decays was found by BABAR in the sum of all D ( * ) D * 0 K + final states [2] and more recently in the decay B 0 → D * − D * + K 0 S [3]. For most of the B → D ( * ) D + s1 (2536) modes, only limits have been placed on the branching fractions [2]. We report herein branching fraction measurements of B → D ( * ) D + s1 (2536) decays, through a comprehensive study of both D ( * ) D * 0 K + and D ( * ) D * + K 0 S final states. The ψ(3770) meson is a charmonium state with J P = 1 − , with a mass just above the open charm threshold. This meson is thought to be an admixture of the D-wave and S-wave of the angular momentum eigenstates of cc system. Study of this state in B decays and branching fraction measurements could provide more information on the structure of the ψ(3770) wave function. This resonance decays dominantly to D 0 D 0 and D − D + [1], and was observed in B meson decays by the Belle experiment [4]. We present a study of the DD mass distribution in DDK events.
The X(3872) resonance was discovered by Belle in the invariant mass distribution of J/ψπ + π − produced in B → J/ψπ + π − K decays, and was thereafter confirmed by BABAR, D0 and CDF [5]. This new meson has a mass of 3871.2 ± 0.5 MeV/c 2 and a natural width less than 2.3 MeV at 90% confidence level (C.L.). At present, the quantum numbers compatible with the observations are J P C = 1 ++ or 2 −+ [6]. Recently, Belle showed an excess of events in the D 0 D 0 π 0 invariant mass in the channel B → D 0 D 0 π 0 K, with a mass of 3875.2 ± 0.7 +0.3 −1.6 ± 0.8 MeV/c 2 [7] (where the third error is due to the uncertainty in the neutral D mass). The X(3872) is probably not a charmonium state, given its measured mass and width, and several alternative interpretations have been proposed: D * 0 D 0 molecule, tetraquark state, hybrid or gluonium states [8]. We present a search for X(3872) decays to D * 0 D 0 .
The measurements reported here use 347 fb −1 of data, corresponding to (383±4)×10 6 BB pairs, collected at the Υ (4S) resonance with the BABAR detector at the PEP-II B factory. The BABAR detector is described in detail elsewhere [9]. We use a Monte Carlo (MC) simulation based on GEANT4 [10] to study the relevant backgrounds and estimate the selection efficiencies.
The B 0 and B + mesons are reconstructed in a sample of hadronic events in the 22 possible D ( * ) D ( * ) K modes. The selection criteria are optimized for each final state by maximizing the significance S/ √ S + B, where S and B refer to the expected number of signal and background events, based on MC simulation. The K 0 S candidates are reconstructed from two oppositely charged tracks consistent with coming from a common vertex and having an invariant mass within ±9.5 MeV/c 2 of the nominal K 0 S mass. For some channels, depending on the background level, we apply a requirement on the displacement of the K 0 S vertex in the plane transverse to the beam axis of at least 0.2 cm. The π 0 candidates are reconstructed from pairs of photons with energies E γ > 30 MeV in the laboratory frame that have an invariant mass of 115 < m γγ < 150 MeV/c 2 . We reconstruct D mesons in the modes D 0 → K − π + , K − π + π 0 , K − π + π − π + , and D + → K − π + π + . The K and π tracks are required to originate from a common vertex. Charged kaon identification, based on the Cherenkov angle and the dE/dx measurements, is used for K + from B decays and from most D decays. The invariant masses of the D candidates are required to be within ±2σ of the measured D mass. The measured D mass resolution, σ D , is 13 MeV/c 2 for D 0 → K − π + π 0 and varies from 5.5 to 7 MeV/c 2 for the other modes. The D * candidates are reconstructed in the decay modes D * + → D 0 π + , D * + → D + π 0 , D * 0 → D 0 π 0 , and D * 0 → D 0 γ. The π 0 and the π + must have momentum below 450 MeV/c in the Υ (4S) rest frame, while the γ energy in the laboratory frame must be greater than 100 MeV. The mass difference between the D * and D candidates is required to be within 3 MeV/c 2 of the nominal value [1] for D * + decays (4 MeV/c 2 and 10 MeV/c 2 for D * 0 → D 0 π 0 and D * 0 → D 0 γ, respectively). The mode D * + → D + π 0 is used only in the reconstruction of decays B 0 → D * − D * + K 0 S and B + → D * − D * + K + . B candidates are reconstructed by combining a D ( * ) , a D ( * ) and a K candidate. For most of the modes involving two D 0 mesons, at least one of them is required to decay to K − π + . During the optimization process, we remove D decay modes if the significance for a particular B decay mode improves. In practice, only D * 0 D * + K 0 S decays benefit from this optimization, for which we remove the combination with the first D 0 meson of the decay chain going to K − π + and the second going to K − π + π − π + (and vice-versa). A mass-constrained kinematic fit is applied to the intermediate particles (D * 0 , D * + , D 0 , D + , K 0 S , π 0 ) to improve their momentum resolution. For the D 0 and the D * − D 0 mass differences, we use the recent CLEO measurement of the D 0 mass [11]. To suppress the e + e − → qq (q = u, d, s and c) continuum background, we perform a selection based on the ratio of the second to zeroth Fox-Wolfram moments of the event [12] and on the cosine of the angle between the thrust axis of the candidate decay and the thrust axis of the rest of the event. Signal events have m ES = s/4 − p * 2 B , where p * B is the center-of-mass momentum of the B candidate, compatible with the known B meson mass, and a difference between the candidate energy and the beam energy in the center-of-mass, ∆E, compatible with 0. On average we have about 1.8 signal B candidates per event. If more than one candidate is selected in an event, we retain the one with the smallest |∆E|. In the final selection, we require |∆E| to be less than nσ ∆E , where the resolution σ ∆E varies between 7 and 14 MeV and n is determined for each mode by the optimization procedure (n ranges from 1 to 4). For each mode we define a B signal region m min ES < m ES < m max ES , where m min ES ranges from 5.268 to 5.277 GeV/c 2 , and m max ES from 5.284 to 5.290 GeV/c 2 , and a control region, 5.20 < m ES < 5.26 GeV/c 2 . The signal purity obtained in the signal region ranges from 17 to 77%, depending on the mode.
We consider several sources of systematic errors. From the difference between data and Monte Carlo efficiencies we derive systematic errors of 0.5% per charged track, 2.2% per soft pion from D * decays, 2.5% per K 0 S , 2% per K + , 3% per π 0 and 2% per single photon. As an example, the particle identification efficiency for K + is measured using a D * + → D 0 π + data control sample with D 0 → K − π + . Other sources of systematic error are also taken into account: limited MC statistics (1-3%), the estimate of the total number of B mesons in the data sample (1.1%), uncertainties on the D ( * ) and K 0 S branching fractions (3-8%) and uncertainties on the D ( * ) and K masses (0.5-6%). In addition, there are uncertainties in the fit procedure for the different resonances: when fixing a parameter, we repeat the fit varying the parameter by ±1σ of its error (where σ is the 68% C.L. standard deviation); these uncertainties include also variation of the background parametrization. Using the m ES control region, we check that the combinatorial background events do not contain any significant D + s1 (2536), ψ(3770) or X(3872) signals or additional peaking structures. Furthermore, for the mass and width measurements, we include effects from the energy loss in the tracking system, from the uncertainties on the magnetic field and from the calibration and background of the calorimeter.
We search for D + s1 (2536) decaying to D * + K 0 S and Fig. 1a the D * + K 0 S and D * 0 K + invariant mass distribution for the sum of the eight B modes. The overlaid curve is the result of a unbinned extended maximum likelihood fit. Since the D + s1 (2536) meson is narrow (Γ D + s1 (2536) < 2.3 MeV at 90% C.L.
[1]), the invariant mass peak is described by the convolution of a non-relativistic Breit-Wigner with a Gaussian function (called a Voigtian function) and the background by a threshold function a(m − m 0 ) b × e c(m−m0) , where m is the D * K invariant mass, m 0 is the sum of the D * and K meson masses, and a, b and c are parameters of the fit. To take into account the large tails in the reconstruction of D + s1 (2536) → D * 0 K + decays, the probability density function (PDF) for the D * 0 K + modes is constructed from a sum of a Voigtian function and a Gaussian function, with a common mean. The D + s1 (2536) mass and yield are floating parameters in the fit. The natural width of the D + s1 (2536) is fixed to 1 MeV and varied from 0.1 to 2.0 MeV to estimate systematic errors. The other parameters (the second Gaussian function and the relative contribution of the Voigtian and the Gaussian function) are taken from the simulation. A significant signal is observed in each of the modes separately (see Table I). A fit to the eight B modes gives 182 ± 19 events in the peak with a statistical significance of 11.8σ. We compute an estimate of the statistical significance calculating PROB(2(L 0 − L signal ), N dof ), where L 0(signal) is the minimum of the likelihood without (with) the signal contribution, N dof is the number of free parameters in the signal PDF and PROB is the upper tail probability of a chi-squared distribution, converting this probability into a number of standard deviations.
From the D + s1 (2536) yields, we compute cross-feedcorrected branching fractions, using the signal efficiency and the relative contributions from cross-feed between the different B → D ( * ) D + s1 (2536) channels, as obtained from simulated events. The resulting branching fractions, the efficiencies, including the intermediate branching fractions, and the internal cross-feed contributions are given in Table I. In addition to the effects previously mentioned, systematics in the table include uncertainties from the cross-feed events (0-3%), underestimate of the MC resolution (1-10%) and uncertainty on the D + s1 (2536) natural width (5-18%). Using only modes containing D * + K 0 S in order to minimize the systematic error, we also fit the D + s1 (2536) mass, M (D + s1 (2536)) = (2534.78 ± 0.31 ± 0.40) MeV/c 2 . Our measurement is in good agreement with the world average [1].
The helicity angle distribution for the sum of the four B → DD + s1 (2536) modes, determined by fitting the D * K mass distribution for the D + s1 (2536) yield in ten cos θ D + s1 (2536) bins, is shown in Fig.1b. Here θ D + s1 (2536) is defined as the angle between the D * direction and the B direction in the D + s1 (2536) frame. We fit the helicity distribution to different spin-parity hypotheses (one free parameter for J P = 1 − and 2 + , and two free parameters for J P = 1 + and 2 − ). The fits to J P = 1 + in pure S wave (flat distribution, not shown in the figure), J P =1 + with S-D wave admixture, and J P =1 − are all in good agreement with data, with χ 2 /n.d.f. of 15.9/9, 9.3/8 and 9.6/9 respectively. Fits to J P =2 + and 2 − are disfavored, with χ 2 /n.d.f. of 26.0/9 and 26.0/8 respectively.
We Fig. 1c we show the DD invariant mass for the sum of B + → D 0 D 0 K + and B + → D − D + K + candidates. No significant signal is observed in the DDK 0 S modes. The curve shown is the result of a fit where the ψ(3770) peak is described by a P-wave Breit-Wigner with the mass as a free parameter in the fit and a natural width fixed to 23 MeV [1], while the background is described by the threshold function described previously. We observe 57 ± 11 events, with a statistical significance of 6.4σ, from which we obtain M (ψ(3770)) = (3775.5±2.4±0.5) MeV/c 2 , in good agreement with the recent high precision measurement of the BES collaboration [13]. We obtain the branching fractions, or limits, reported in Table I For the two modes with significant signal, we study the ψ(3770) helicity angle, θ ψ(3770) (Fig. 1d), defined as the angle between the D 0/+ direction and the B direction in the ψ(3770) frame. We confirm the spin 1 assignment of the ψ(3770) (χ 2 /n.d.f.=2.9/9). A spin 0 hypothesis gives χ 2 /n.d.f.=22.0/9.
We search for decays B → X(3872)K, X(3872) We plot the D * 0 D 0 invariant mass distribution for the sum of B 0 and B + candidates in Fig. 1e. Due to the proximity of the threshold and to the fact that the natural width of the X(3872) is comparable to the D * 0 D 0 mass resolution, there is no easy analytic parametrization of the reconstructed X(3872) mass spectrum. To measure the mass and width of the X(3872), we generate and reconstruct high statistics MC samples of B → X(3872)K events with various masses (from 3872 to 3877 MeV/c 2 ) and widths (from 0 to 20 MeV), assuming a pure S-wave decay of a spin 1 resonance. We perform binned extended maximum likelihood fits to the measured D * 0 D 0 invariant mass distributions using these different MC samples as signal PDFs combined with a threshold function for the background. We compare the agreement of each mass and width hypothesis to the data by computing the χ 2 of the fit for the sum of bins below 3.9 GeV/c 2 . Figure 2 shows the interpolated χ 2 − χ 2 min contours versus the simulated masses and widths of the different signal samples, where χ 2 min is the χ 2 value for the best fit. This best fit gives 33 ± 7 events in the X(3872) peak, with a statistical significance of 4.9σ. Mass and width central values are obtained at the minimum of the χ 2 distribution, while the errors are given by the extreme points of the contour in the (mass, width) plane defined at χ 2 min +1. We obtain a mass of (3875.1 +0.7 −0.5 ±0.5) MeV/c 2 and a width of (3.0 +1.9 −1.4 ± 0.9) MeV, where the systematic errors include additional contributions from the choice of the bin width of the invariant mass distribution (0.14 MeV/c 2 and 0.07 MeV respectively on the mass and on the width) and from the fact that in the MC we assume S-wave X(3872) decays to D * 0 D 0 (0.20 MeV/c 2 and 0.80 MeV respectively on the mass and on the width, determined using MC events with P -wave X(3872) decays). Independently of the mass value, the width measurement is 1.8σ away from 0 MeV. The B + and B 0 branching fractions to X(3872)K (reported in Table I) are obtained by fitting the D * 0 D 0 invariant mass spectra, separately for B + and B 0 , choosing the MC sample with M (X(3872)) = 3875 MeV/c 2 and Γ(X(3872)) = 3 MeV, which is found to give the best fit to the data.
We study the helicity angle of the X(3872), θ X(3872) , for the sum of B 0 and B + modes (see Fig. 1f). Here, θ X(3872) is defined as the angle between the D 0 or D * 0 direction and the B direction in the X(3872) frame. Comparing the curves obtained with different spin hypotheses with the data distribution, we obtain the following χ 2 /n.d.f.: 9.8/7 for J P = 1 − , 3.9/7 for 1 + assuming a pure S wave (flat distribution, not shown in the figure), 2.5/6 for 1 + with S-D wave admixture, 5.9/7 for 2 + and 2.7/6 for 2 − . On the basis of this data sample, we cannot distinguish the different spin assignments. However the D * 0 D 0 decay would be suppressed by the angular momentum barrier for J = 2.
The ratio of X(3872) candidates reconstructed in the D 0 D 0 π 0 and D 0 D 0 γ final states is 1.37 ± 0.56 (statistical error only), while we expect 1.30 for a decay that proceeds exclusively via a D * 0 meson. In addition, we measure parameters which can be used to differentiate various theoretical interpretations [8,14]: ∆m, the mass difference between the state seen in B 0 decays and B + decays, as well as R 0/+ , the ratio of branching fractions between B 0 decays and B + decays. Assuming that the signal seen in B 0 decays is not a statistical fluctuation, we obtain ∆m = (0.7 ± 1.9 ± 0.3) MeV/c 2 and R 0/+ = 1.33 ± 0.69 ± 0.43.
In summary, we report the observation of the eight B → D ( * ) D + s1 (2536) decays, with a D + s1 (2536) mass of (2534.78 ± 0.31 ± 0.40) MeV/c 2 . We observe the ψ(3770) resonance in B → DDK decays and measure its mass to be (3775.5±2.4±0.5) MeV/c 2 . We show that an enhancement of data is observed near the limit of phase space for the D * 0 D 0 invariant mass, at a mass of (3875.1 +0.7 −0.5 ± 0.5) MeV/c 2 , with a width of (3.0 +1.9 −1.4 ± 0.9) MeV. This enhancement could be interpreted as the X(3872), although the observed mass is 4.5σ away from the mass measured in the J/ψπ + π − decay mode. Our mass value is in good agreement with the value measured by Belle in the D 0 D 0 π 0 final state.
We are grateful for the excellent luminosity and machine conditions provided by our PEP-II colleagues, and for the substantial dedicated effort from the computing organizations that support BABAR. The collaborating institutions wish to thank SLAC for its support and kind hospitality. This work is supported by DOE    , and statistical significances for B → D ( * ) D + s1 (2536) followed by D + s1 (2536) → D * 0 K + or D + s1 (2536) → D * + K 0 (first part of the table); for B → ψ(3770)K followed by ψ(3770) → D 0 D 0 or ψ(3770) → D − D + (second part of the table); for B → X(3872)K followed by X(3872) → D * 0 D 0 (third part of the table). No branching fraction is given in the absence of signal observation. A 90% C.L. upper limit on the branching fraction is given when the statistical significance is less than or equal to 3σ (these limits use the profile likelihood method [15] and include systematic errors). The first error on B is statistical and the second is systematic.