We report studies of the analyses of and compensation for group dispersion to improve the axial resolution of high-speed optical coherence tomography (OCT) by acousto-optic modulation (AOM). Theoretical modeling and experiments reveal that the high-order group dispersion induced by acousto-optic crystals broadens the measured coherence length (L-c) and thus degrades the axial resolution of OCT imaging. Based on our experimental studies, we can compensate for the dispersion to less than 50% broadening of the source L-c by adjusting the grating-lens-based optical delay in the reference arm and can further eliminate it by inserting like acousto-optic crystals in the sample arm of the OCT system. The results demonstrate that this ACM-mediated OCT system permits high-performance OCT imaging at A-scan rates of as much as 4 kHz by use of a resonant scanner. Because of its ultrastable direct frequency modulation, this AOM-mediated OCT system can potentially improve the performance of high-speed Doppler OCT techniques. (c) 2005 Optical Society of America.