We report the case of a patient (AZ) with a semantic refractory access dysphasia. On matching-to-sample tests assessing comprehension of the spoken word, AZ shows all the hallmarks of a refractory access disorder, namely inconsistent performance on repeated testing and sensitivity to both presentation rate and the semantic similarity between competing responses. However, on tasks examining her visual knowledge, such as matching two structurally different exemplars of the same item, AZ's performance is quantitatively and qualitatively different. In a series of experiments testing her knowledge of animate and inanimate items, AZ demonstrated significantly worse performance with verbal-visual matching than with visual-visual matching. Furthermore, response accuracy was observed to decrease with successive probing of an item in the verbal conditions but not the visual conditions. We also demonstrate that this discrepancy cannot be explained on the basis of either task difficulty or presentation rate. We attribute our results to a build-up of refractoriness in the systems mediating verbal comprehension whilst those underlying visual comprehension remain unaffected. We argue that our data speak against a unitary amodal semantic system and in favour of at least partially separate verbal and visual semantic processing.