Quantum limit and anomalous field-induced insulating behavior in eta-Mo4O11

The quasi-two-dimensional metal eta-Mo4O11 undergoes two successive charge-density-wave transitions at 109 and 30 K, with corresponding changes in the electronic structure. Measurements of the magnetoresistance, Hall effect, and magnetization, in magnetic fields of up to 50 T and as a function of temperature and pressure, have been performed to determine the ground-state electronic structure. Several distinct quantum oscillations are observed in the low-field magnetoresistance, corresponding to very small closed Fermi surfaces, all of which reach the quantum limit by 20 T. Analysis of the quantum oscillation amplitudes indicate very low carrier effective masses (m*<0.1m(e)) associated with each of the closed two-dimensional Fermi-surface pockets. At higher fields (B>20 T) we see a crossover from (semi-) metallic to semiconducting behavior, which we associate with a field-induced electron- and hole-band inversion, resulting in a clear gap at the Fermi-energy. Measurements of this energy gap allow an independent determination of the carrier effective masses, which are in excellent agreement with the values obtained from the analysis of the low-field (B<15 T) quantum oscillations. We find that the transport and thermodynamic properties of the field-induced insulating state are highly anomalous. In particular, we discuss the Hall effect and the origin of an additional quantum oscillation at high magnetic fields (B>20 T).