Being able to adequately process numbers is a key competency in everyday life. Yet, self-reported negative affective responses towards numbers are known to deteriorate numerical performance. Here, we investigated how physiological threat responses predict numerical performance. Physiological responses reflect whether individuals evaluate a task as exceeding or matching their resources and in turn experience either threat or challenge, which influences subsequent performance. We hypothesized that, the more individuals respond to a numerical task with physiological threat, the worse they would perform. Results of an experiment with cardiovascular indicators of threat/challenge corroborated this expectation. The findings thereby contribute to our understanding of the physiological mechanism underlying the influence of negative affective responses towards numbers on numerical performance.