A variety of methodological paradigms, including dot probe and eye movement tasks, have been used to examine attentional biases to threat in anxiety disorders. Unfortunately, little attention has been devoted to the psychometric properties of measures from these paradigms. In the current study, participants selected for high and low social anxiety completed a dot probe and eye movement task using angry, disgust and happy facial expressions paired with neutral expressions. Results indicated that dot probe bias scores, eye movement first fixation indices, and eye movement proportions of viewing time in the first 1,500 ms had unacceptably low reliability. However, eye movement indices of attentional bias over the full 5,000 ms time course had excellent reliability. Individuals’ dot probe and eye movement biases were largely uncorrelated across the two tasks and demonstrated little relation with social anxiety scores. Implications for future research are discussed.