Contextual saccadic adaptation is studied with a variant of the double step saccade adaptation paradigm, in which the direction of the intra-saccadic step (ISS) is signaled by two different contexts. This allows to simultaneously inducing two distinct saccadic adaptation states. Importantly, effective contextual adaptation is not always observed: e.g. it occurs when the amplitude of the first step serves as context, but not when using the target color or shape. Our study compares different types of contexts. All experimental sessions are based on the same contextual adaptation procedure, the only difference being the context used. We tested seven different contexts : (1) the duration of a visual stimulus, (2) the lateralization of a sound in space, (3) the pitch of a sound, (4) the statistical regularity across trials, and (5) a symbolic cue, as well as (6) the amplitude of the first step and (7) the target color and shape to compare our results with previous studies. We collected data from 23 participants (out of the planned 90). Fisher's test revealed an effect of the context with the amplitude of the first step condition in all participants, and with the symbolic cue for one participant (out of 4). The Kolmogorov-Smirnov distance confirms a large contextual effect for the amplitude, but not for the other contexts. The lack of contextual learning reveals that predicting the intra-saccadic step is relatively difficult and strongly depends on the nature of the context, even for highly salient contexts that are perfectly correlated with the ISS. The correlation between context and ISS is not sufficient for learning, even for non-visual contexts. A similar effect, termed selective learning or biological constraints on learning, has been previously reported in pavlovian and operant conditioning animal studies.