In this paper we construct, for n≧ 2, arbitrarily large families of infinite towers of compact, orientable Riemannian n-manifolds which are isospectral but not isometric at each stage. In dimensions two and three, the towers produced consist of hyperbolic 2-manifolds and hyperbolic 3-manifolds, and in these cases we show that the isospectral towers do not arise from Sunada's method.