Long-term effects of cognitive conflict on performance are not as well understood as immediate effects. We used a change detection task to explore long-term consequences of cognitive conflict by manipulating the congruity between a changing object and a background scene. According to conflict-based accounts of memory formation, incongruent trials (e.g., a cow on the street), in spite of hindering immediate performance, should promote stronger encoding than congruent trials (e.g., a cow on a prairie). Surprisingly, across three experiments we show that semantic incongruity actually impairs remembering of the information presented during scene processing. This set of results is incompatible with the frequently accepted hypothesis of conflict-triggered learning. Rather, we discuss the present data and other studies previously reported in the literature in the light of two much older hypotheses of memory formation: the desirable difficulty and the levels of processing principles.