This article looks at the implications of emotion recognition, zooming in on the specific case of the care robot Pepper introduced at a hospital in Toronto. Here, emotion recognition comes with the promise of equipping robots with a less tangible, more emotive set of skills – from companionship to encouragement. Through close analysis of a variety of materials related to emotion detection software – iMotions – we look into two aspects of the technology. First, we investigate the how of emotion detection: what does it mean to detect emotions in practice? Second, we reflect on the question of whose emotions are measured, and what the use of care robots can say about the norms and values shaping care practices today. We argue that care robots and emotion detection can be understood as part of a fragmentation of care work: a process in which care is increasingly being understood as a series of discrete tasks rather than as holistic practice. Finally, we draw attention to the multitude of actors whose needs are addressed by Pepper, even while it is being imagined as a care provider for patients.