Switching between competing tasks is supported by active inhibition of the preceding task. The level of task stimulus processing at which interference between competing tasks must occur for inhibition to be recruited is still unclear. Here, we investigated whether inhibition is recruited by task conflict occurring at an early or late (semantic) stage of task stimulus processing by dissociating the task stimulus format from its meaning. In two experiments, participants performed three different numerical judgment tasks on numerical stimuli that could be presented as digits or number words (e.g., “6” or “six”) in a cued task-switching procedure. The effects of the change of stimulus format for the inhibition of the previous task were investigated and assessed by the N-2 task repetition cost, an index of the extent to which task representations are inhibited. The N-2 task repetition cost observed in the same stimulus format condition disappeared when target stimuli on task N-1 were presented in a different format from stimuli of task N-2 and N. This occurred both when the format changed from digits to number words stimuli (Experiment 1) as well as when it changed from number words to digits stimuli (Experiment 2). Results indicated that task set inhibition is recruited very early during the stimulus processing stage. They also provided evidence that task inhibition is not tied to task preparation processes but operates as a reactive, rather than proactive mechanism of conflict resolution.