Using the near-infrared fluxes of local galaxies derived from Cosmic Background Explorer (COBE)/Diffuse Infrared Background Experiment (DIRBE)(5) J-(1.25 mu m) K - (2.2 mu m), and L-band (3.5 mu m) maps and published Cepheid distances, we construct Tully-Fisher (TF) diagrams for nearby galaxies. The measured dispersions in these luminosity-line width diagrams are remarkably small: sigma(J) = 0.09 mag, sigma(K) = 0.13 mag, and sigma(L) = 0.20 mag. These dispersions include contributions from the intrinsic TF relation scatter and the errors in estimated galaxy distances, fluxes, inclination angles, extinction corrections, and circular speeds. For the J and K bands, Monte Carlo simulations give a 95% confidence interval upper limit on the true scatter in the TF diagram of sigma(J) less than or equal to 0.35 and sigma(K) less than or equal to 0.45. We determine the Milky Way's luminosity and place it in the TF diagram by fitting a bar plus exponential disk model of the Milky Way to the all-sky DIRBE maps. For ''standard'' values of its size and circular speed (Sun-Galactic center distance R(0) = 8.5 kpc and Theta(0) = 220 km s(-1)), the Milky Way lies within 1.5 sigma of the TF relations. We can use the TF relation and the Cepheid distances to nearby bright galaxies to constrain R(0) and Theta(0): 1.63 log (Theta(0)/220 km s(-1)) - log (R(0)/8.5 kpc) = 0.08 +/- 0.03. Alternatively, we can fix the parameters of the Galaxy to their standard values, ignore the Cepheid zero point, and use the TF relation to determine the Hubble constant directly: H-0 = 72 +/- 12 km s(-1) Mpc(-1). We have also tested the TF relation at longer wavelengths, where the emission is dominated by dust. We find no evidence for a TF relation at wavelengths beyond 10 mu M. The tight correlation seen in the L band suggests that stellar emission dominates over the 3.3 mu m polycyclic aromatic hydrocarbon emission.