Objectives: Depression is a significant mental health issue for older adults. Rumination is a key risk factor for depressive symptoms, and self-compassion is a protective factor. The aim of the current study was to test the processes by which self-compassion might act as a protective factor among older adults by investigating a mediation model, and whether the model is conditional on gender. It was hypothesised that self-compassion and its six components (self-kindness, common humanity, mindfulness, self-judgement, isolation, and over-identification) would be indirectly associated with depressive symptoms via rumination. Method: A sample of 135 older Australian women and 106 older Australian men aged between 65 and 89 years completed the Centre for Epidemiologic Studies Depression Scale, the Ruminative Thought Style Questionnaire, and the Self-Compassion Scale. Results: Results supported the mediation models for self-compassion for both men and women, however, support for the models involving the individual components of self-compassion varied according to gender. The mediation effects were stronger for women than men. Conclusion: Interventions aimed at increasing self-compassion might have benefits for older adults. Future research investigating whether tailoring interventions according to gender of participants is efficacious is needed.