The distinctiveness of neural information representation is crucial for successful memory performance but declines with advancing age. Computational models implicate age-related neural dedifferentiation on the level of item representations, but previous studies mostly focused on age differences of categorical information representation in higher-order visual regions. In an age-comparative fMRI study, we combined univariate analyses and whole-brain searchlight pattern similarity analyses to elucidate age differences in neural distinctiveness at both category and item levels and their relation to memory. Thirty-five younger (18-27 years old) and 32 older (67-75 years old) women and men incidentally encoded images of faces and houses, followed by an old/new recognition memory task. During encoding, age-related neural dedifferentiation was shown as reduced category-selective processing in ventral visual cortex and impoverished item specificity in occipital regions. Importantly, successful subsequent memory performance built on high item stability, that is, high representational similarity between initial and repeated presentation of an item, which was greater in younger than older adults. Overall, we found that differences in representational distinctiveness coexist across representational levels and contribute to interindividual and intraindividual variability in memory success, with item specificity being the strongest contributor. Our results close an important gap in the literature, showing that older adults’ neural representation of item-specific information in addition to categorical information is reduced compared with younger adults.