THERMODYNAMIC FORMALISM AND SELBERG'S ZETA FUNCTION FOR MODULAR GROUPS

In the framework of the thermodynamic formalism for dynamical systems [26] Selberg's zeta function [29] for the modular group PSL(2, Z) can be expressed through the Fredholm determinant of the generalized Ruelle transfer operator for the dynamical system defined by the geodesic flow on the modular surface corresponding to the group PSL(2, Z) [19]. In the present paper we generalize this result to modular subgroups Gamma with finite index of PSL(2, Z). The corresponding surfaces of constant negative curvature with finite hyperbolic volume are in general ramified covering surfaces of the modular surface for PSL(2, Z). Selberg's zeta function for these modular subgroups can be expressed via the generalized transfer operators for PSL(2, Z) belonging to the representation of PSL(2, Z) induced by the trivial representation of the subgroup Gamma. The decomposition of this induced representation into its irreducible components leads to a decomposition of the transfer operator for these modular groups in analogy to a well known factorization formula of Venkov and Zograf for Selberg's zeta function for modular subgroups [34].