GLUON-MEDIATED RARE DECAYS OF THE TOP QUARK - ANOMALOUS THRESHOLD AND ITS PHENOMENOLOGICAL CONSEQUENCES

Flavor-changing neutral-current decays of the top quark by one-gluon emission (e.g., t --> cg) and exchange (e.g., t --> cg* --> cqqBAR) are analyzed in the three-generation standard and two-Higgs-doublet models. These rare decays provide a sensitive probe of the analytic properties of (penguin-dominated) loop-induced neutral currents. The occurrence of an anomalous threshold in the amplitude for the one-gluon exchange mode requires a careful description of unstable (or off-mass-shell) external particles, in order to obtain sensible results for physical quantities. The branching fraction SIGMA(q) B(t --> cg* --> cqqBAR) in the standard model turns out to be larger than B(t --> cg) for top quark masses less-than-or-similar-to 180 GeV, due in part to the effects of the anomalous threshold. This is despite a suppression of the exchange mode by a factor of order alpha(s)(m(t)2)/12-pi almost-equal-to 3 x 10(-3), coming from coupling constants and phase space. The largest branching fractions occur in the two-Higgs-doublet models, where B(t --> cg) approximately 10(-5) and SIGMA(q) B(t --> cg* --> cqqBAR) approximately 10(-6) in some regions of parameter space.