This study compared the role of the parent and grandparent in the adjustment of children who had, and who had not, been left behind when their parents moved to urban areas for work, and tested whether self-esteem mediated the association between these family relationships and child adjustment. The sample included 428 middle school students in central China, 204 of whom were left-behind children who were cared for by grandparents and 224 of whom were not left behind. The students responded to five questionnaires measuring parent–child relationships, grandparent–child relationships, self-esteem, life satisfaction and school engagement. Regression analyses showed that self-esteem mediated the association between parent–child and grandparent–child relationships and children’s adjustment, but there were different patterns of association for the non-left-behind children and left-behind children groups. Furthermore, the unique effect of parent–child relationships was higher than that of grandparent–child relationships for the non-left-behind children, but for the left-behind children, grandparent–child relationships played a more important role than grandparent–child relationships in predicting their school engagement, and parent–child relationships played a more important role in predicting their life satisfaction. Parents who are far away and grandparents who are close at hand can shape different psychological and behavioral development outcomes of left-behind children.