Some infants experience atypical levels of over-responsivity to sensations, which limit their ability to interact and explore their environment. Yet, little is known about typical development of over-responsivity during infancy or whether the presence of extreme over-responsivity in infancy is a predictor of clinically significant sensory over-responsivity (SOR) at school-age. This study followed a representative sample of children (n = 521, 47% boys) at four time points from infancy (mean ages in months Year 1 = 18.23, Year 2 = 30.39, Year 3 = 39.40) to elementary school-age (mean age = 7.97 years). SOR was measured via parent report. A latent growth curve model predicting SOR at school age from the intercept and slope of Sensory Sensitivity between Years 1–3 showed excellent fit with the data. Both early sensory sensitivities and change in early sensitivities were associated with SOR status at school-age.