Previous studies pointed out a selective interaction between different working memory subsystems (i.e., phonological and visuospatial) and arithmetic operations (i.e., multiplication and subtraction). This was interpreted to support the idea that multiplication and subtraction predominantly rely on a phonologically or spatially organized number code, respectively. Here, we investigated this idea in two groups (multiplication and subtraction group) using a dual-task paradigm. Going beyond previous studies, we carefully controlled and balanced the difficulty of both working memory and calculation tasks within and across participants. This allowed us to test the reciprocal impact of calculations on working memory. We observed no selective interaction between different working memory subsystems and arithmetic operations. Instead, both types of arithmetic operations were impaired by both types of concurrent working memory tasks. Likewise, both types of working memory tasks were impaired by both types of concurrent arithmetic. Our findings suggest that multiplication and subtraction depend on both phonological and visuospatial codes and highlight the importance of balancing task demands within and between participants in the context of dual-task studies.