Diagonalization in reduced Hilbert spaces using a systematically improved basis: Application to spin dynamics in lightly doped ladders

method is proposed to improve the accuracy of approximate techniques for strongly correlated electrons that use reduced Hilbert spaces. As a first step, the method involves a change of basis that incorporates exactly part of the short-distance interactions. The Hamiltonian is rewritten in new variables that better represent the physics of the problem under study. A Hilbert-space expansion performed in the new basis follows. The method is successfully tested using both the Heisenberg model and the t-J model with holes on two-leg ladders and chains, including estimations for ground-state energies, static correlations, and spectra of excited states. An important feature of this technique is its ability to calculate dynamical responses on clusters larger than those that can be studied using exact diagonalization. The method is applied to the analysis of the dynamical spin structure factor S(q, omega) on clusters with 2X16 sites and 0 and 2 holes. Our results confirm previous studies [M. Troyer, H. Tsunetsugu, and T. M. Rice, Phys. Rev. B 53, 251 (1996)] that suggested that the state of the lowest energy in the spin-1 2-hole subspace corresponds to the bound state of a hole pair and a spin triplet. Implications of this result for neutron-scattering experiments both on ladders and planes are discussed. [S0163-1829(98)00141-6].