A radial basis function approximation has the form [GRAPHIC] where phi: R(d) --> R is some given (usually radially symmetric) function, (y(j))1n are real coefficients, and the centers (x(j))1n are points in R(d). For a wide class of functions phi, it is known that the interpolation matrix A = (phi(x(j) - x(k)))j,k=1n is invertible. Further, several recent papers have provided upper bounds on parallel-to A-1 parallel-to2, where the points (x(j))1n satisfy the condition parallel-to x(j) - x(k)parallel-to2 greater-than-or-equal-to delta, j not-equal k, for some positive constant delta. In this paper we calculate similar upper bounds on parallel-to A-1 parallel-top for p greater-than-or-equal-to 1 which apply when phi decays sufficiently quickly and A is symmetric and positive definite. We include an application of this analysis to a preconditioning of the interpolation matrix A(n) = (phi(j - k))j,k=1n when phi(x) = (x2 + c2)1/2, the Hardy multiquadric. In particular, we show that sup(n)parallel-toA(n)-1parallel-toinfinty is finite. Furthermore, we find that the bi-infinite symmetric Toeplitz matrix E = (phi(j - k))j,k element-of T(d) enjoys the remarkable property that parallel-to E-1 parallel-to(p) = parallel-to E-1parallel-to2 for every p greater-than-or-equal-to 1 when phi is a Gaussian. Indeed, we also show that this property persists for any function phi which is a tensor product of even, absolutely integrable Polya frequency functions.