Mountains are a prominent feature of the African continent and, along with climatic and vegetation heterogeneity, have been suggested as major drivers of vertebrate radiations and phylogeographic structuring. Cape rock hyrax is a small mammal species associated with mountains and other rocky terrain. We used coalescent-based analyses of a multi-locus data set (mtDNA, two nuclear intronic loci and microsatellites) to tease apart life history traits and extrinsic factors that have shaped the intraspecific diversity, distribution and demography of this species in southern Africa. Rock hyrax gene trees show evidence of diversification during the late Miocene. Since this epoch, southern African biomes have experienced cyclic patterns of expansion and contraction linked to global and local climatic variability, impacting on small mammal speciation and intraspecific diversity. We confirm the presence of two unique parapatric mitochondrial lineages; however, the nuclear DNA data reveal secondary contact and introgression between these historically separated populations. We supplemented the genetic data with distribution modelling, leading to the identification of several important refugia for rock hyrax. These were identified mainly along the Great Escarpment and its associated mountain ranges. Although no discernible barriers to gene flow were evident from our study, we propose that rock hyrax diversification and population dynamics are driven by the availability of suitable habitat (rock crevices and nearby, abundant food sources), as well as factors such as social dynamics and disease outbreaks. The coalescent approach indicates that there is currently unrecognized diversity within the genus and species delimitation across the distributional range is warranted.