Recent work from our group has demonstrated that visual speech can recalibrate auditory speech identification [1]. Repeated exposure to an ambiguous auditory token (one intermediate between /aba/ and /ada/), combined with the sight of a face articulating one of the endpoint tokens (/aba/ or /ada/), increased during subsequent unimodal auditory identification tests the frequency of responses consistent with the preceding visual stimulus. In contrast, exposure to a combination of congruent unambiguous auditory and visual tokens (both /aba/ or both /ada/) reduced the tendency to judge the ambiguous auditory token in accordance with the exposed tokens, revealing selective speech adaptation. In the present experiment, it is shown that these two effects build up at different rates, thus bringing new evidence for the existence of different underlying processes.