β-Amyloid plaques and tau-containing neurofibrillary tangles are the two neuropathological hallmarks of Alzheimer’s disease (AD) and are thought to play crucial roles in a neurodegenerative cascade leading to dementia. Both lesions can now be visualized in vivo using positron emission tomography (PET) radiotracers, opening new opportunities to study disease mechanisms and improve patients’ diagnostic and prognostic evaluation. In a group of 32 patients at early symptomatic AD stages, we tested whether β-amyloid and tau-PET could predict subsequent brain atrophy measured using longitudinal magnetic resonance imaging acquired at the time of PET and 15 months later. Quantitative analyses showed that the global intensity of tau-PET, but not β-amyloid–PET, signal predicted the rate of subsequent atrophy, independent of baseline cortical thickness. Additional investigations demonstrated that the specific distribution of tau-PET signal was a strong indicator of the topography of future atrophy at the single patient level and that the relationship between baseline tau-PET and subsequent atrophy was particularly strong in younger patients. These data support disease models in which tau pathology is a major driver of local neurodegeneration and highlight the relevance of tau-PET as a precision medicine tool to help predict individual patient’s progression and design future clinical trials.