In this paper and a companion work, we show that a broad class of astrophysical accretion disk is dynamically unstable to axisymmetric disturbances in the presence of a weak magnetic field. Because of the ubiquity of magnetic fields, this result bears upon gaseous differentially rotating systems quite generally. This work presents a linear analysis of the instability. (The companion work presents the results of nonlinear numerical simulations.) The instability is local and extremely powerful. The maximal growth rate is of order the angular rotation velocity and is independent of the strength of the magnetic field, provided only that the energy density in the field is less than the thermal energy density. Unstable axisymmetric disturbances require the presence of a poloidal field component, and are indifferent to the presence of a toroidal component. The instability also requires that the angular velocity be decreasing outward. In the absence of a powerful dissipation process, there are no other requirements for instability. Fluid motions associated with the instability directly generate both poloidal and toroidal field components. We discuss the physical interpretation of the instability in detail. Conditions under which saturation occurs are suggested. The nonemergence of the classical Rayleigh criterion for shear instability in the limit of vanishing field strength is noted and explained. The instability is sensitive neither to disk boundary conditions nor to the constituative fluid properties. Its existence precludes the possibility of internal (noncompressive) wave propagation in a disk. If present in astrophysical disks, the instability, which has the character of an interchange, is very likely to lead to generic and efficient angular momentum transport, thereby resolving an outstanding theoretical puzzle.