Maintained structural integrity of hippocampal and cortical gray matter may explain why some older adults show rather preserved episodic memory. However, viable measurement models for estimating individual differences in gray matter structural integrity are lacking; instead, findings rely on fallible single indicators of integrity. Here, we introduce multitrait–multimethod methodology to capture individual differences in gray matter integrity, based on multimodal structural imaging in a large sample of 1522 healthy adults aged 60–88 years from the Berlin Aging Study II, including 333 participants who underwent magnetic resonance imaging. Structural integrity factors expressed the common variance of voxel-based morphometry, mean diffusivity, and magnetization transfer ratio for each of four regions of interest: hippocampus, parahippocampal gyrus, prefrontal cortex, and precuneus. Except for precuneus, the integrity factors correlated with episodic memory. Associations with hippocampal and parahippocampal integrity persisted after controlling for age, sex, and education. Our results support the proposition that episodic memory ability in old age benefits from maintained structural integrity of hippocampus and parahippocampal gyrus. Exploratory follow-up analyses on sex differences showed that this effect is restricted to men. Multimodal factors of structural brain integrity might help to improve our biological understanding of human memory aging.