The response of GaAs metal-semiconductor-metal (MSM) photodiodes at low temperatures has been investigated in the time domain by photoconductive sampling. The dependence of the response time on temperature for T > 50 K can be described by phonon mediated intervalley and intravalley scattering. The pulse width drops from 10.8 ps at 300 K to 5.6 ps at 70 K and then grows rapidly with decreasing temperature below 50 K. These results demonstrate that the response is limited by the electron/hole transport in the semiconductor rather than by external capacitances or inductances.