Based on the premise that bullies are deficient in empathy or even lack it completely, bullying prevention and intervention programs often include empathy training. These programs are not always as effective as they aim to be, which may be caused by a failure to acknowledge the multidimensional nature of empathy as well as its complex association with involvement in bullying. To provide a clear overview of the research on the association between empathy and involvement in bullying, this article systematically reviews 40 studies on the association of cognitive empathy (24 studies) and affective empathy (38 studies) with four categories of involvement in bullying: bullying, victimization, defending, and bystanding. The results showed that bullying was negatively associated with cognitive and—in particular—affective empathy. Victimization was negatively associated with cognitive empathy but not with affective empathy. Defending was consistently positively associated with both types of empathy. Contradictory findings were observed in bystanding, with studies reporting both negative and positive associations with cognitive empathy, and studies reporting negative and no associations with affective empathy. Together, the findings stress the importance of the distinction between cognitive and affective empathy in involvement in bullying and suggest different intervention strategies for the four types of involvement in bullying.