Pitch-height can be represented in a spatial format. Reaction times (RTs) to lower pitch-heights are faster when responses are executed in the lower side of space, whereas RTs to higher pitch-heights are faster when responses are executed in the upper side of space. This effect is called the Spatial-Music Association of Response Codes (SMARC) effect. We investigated how pitch-height and the brightness of a tone’s timbre might contribute in eliciting the SMARC effect as a function of music expertise by comparing the results of 24 musicians with the results we gathered previously (Pitteri et al., 2017) with 24 non-musicians. Three experimental conditions were used: pitch-height varied, brightness varied; pitch-height varied, brightness fixed; pitch-height fixed, brightness varied. We found that the coherent modulation of both pitch-height and brightness elicited the strongest SMARC effect, independently of music expertise. These results add evidence to the hypothesis that the strongest SMARC effect does not belong to pitch-height or brightness, but to pitch-height and brightness together.