To determine the association between meditation and yoga practice, experienced stress, and amygdala and hippocampal volume in a large population-based study. This study was embedded within the population-based Rotterdam Study and included 3742 participants for cross-sectional association. Participants filled out a questionnaire assessing meditation practice, yoga practice, and experienced stress, and underwent a magnetic resonance scan of the brain. 2397 participants underwent multiple brain scans, and were assessed for structural change over time. Amygdala and hippocampal volumes were regions of interest, as these are structures that may be affected by meditation. Multivariable linear regression analysis and mixed linear models were performed adjusted for age, sex, educational level, intracranial volume, cardiovascular risk, anxiety, depression and stress. 15.7% of individuals participated in at least one form of practice. Those who performed meditation and yoga practices reported significantly more stress (mean difference 0.2 on a 1-5 scale, p < .001) and more depressive symptoms (mean difference 1.03 on CESD, p = .015). Partaking in meditation and yoga practices was associated with a significantly lower right amygdala volume (β = - 31.8 mm3, p = .005), and lower left hippocampus volume (β = - 75.3 mm3, p = .025). Repeated measurements using linear mixed models showed a significant effect over time on the right amygdala of practicing meditation and yoga (β = - 24.4 mm3, SE 11.3, p = .031). Partaking in meditation and yoga practice is associated with more experienced stress while it also helps cope with stress, and is associated with smaller right amygdala volume.