An Extension of the Thermodynamic Formalism Approach to Selberg’s Zeta Function for General Modular Groups

In the framework of the thermodynamic formalism for dynamical systems [Rue78] Selberg’s zeta function [Sel56] for the modular group PSL(2, ℤ) can be expressed through the Fredholm determinant of the generalized Ruelle transfer operator for the dynamical system defined by the geodesic flow on the modular surface [May91b]. In the present paper we generalize this result to modular subgroups Fof PSL(2, ℤ).with finite index. The corresponding surfaces of constant negative curvature with finite hyperbolic volume are in general ramified covering surfaces of the modular surface for PSL(2, ℤ). Selberg’s zeta function for these modular subgroups can be expressed through the generalized transfer operator for PSL(2, ℤ).belonging to the representation of PSL(2, ℤ) induced by the trivial representation of the subgroup F.The decomposition of this induced representation into its irreducible components leads to a decomposition of the transfer operator for these modular groups in analogy to a well known factorization formula of Venkov and Zograf for Selberg’s zeta function for modular subgroups [VZ83]. The transfer operator can be continued into the entire complex βplane and its Fredholm determinant defines a meromorphic function in this plane.