We study analytically the Ising model coupled to random lattices in dimension three and higher. The family of random lattices we use is generated by the large N limit of a colored tensor model generalizing the two-matrix model for Ising spins on random surfaces. We show that, in the continuum limit, the spin system does not exhibit a phase transition at finite temperature, in agreement with numerical investigations. Furthermore we outline a general method to study critical behavior in colored tensor models. (C) 2012 Elsevier B.V. All rights reserved.