A grammaticality illusion occurs when an ungrammatical sentence is perceived as acceptable (Phillips, Wagers, & Lau, 2011
). One well-known example of such an illusion is the so-called missing-VP effect: When a double center-embedded relative clause structure, as in (1a), is turned into an ungrammatical string by removing the middle verb phrase (VP), as in (1b), its subjective acceptability does not appear to be negatively affected. Indeed, Gibson and Thomas (1999
) found that the grammatical and ungrammatical sentence versions are equally difficult to understand; and other studies even report higher comprehensibility or acceptability ratings for double-embedded structures from which the second VP is missing (Christiansen & MacDonald, 2009
; Gimenes, Rigalleau, & Gaonac’h, 2009
). Gibson and Thomas (1999
) ascribe this missing-VP effect to working-memory limitations that cause the prediction of the second VP to be structurally forgotten when three consecutive noun phrases are processed.
The exciting book that the popular author who the reviewers meticulously criticized very confidently published was missing a number of pages.
The exciting book that the popular author who the reviewers meticulously criticized was missing a number of pages.
The missing-VP effect has also been observed in word-reading times: Vasishth, Suckow, Lewis, and Kern (2010
) found that the final verb and post-verbal region are read faster in the ungrammatical condition than in correct double-embedded sentences, at least in English. Interestingly, this effect is reversed when German native speakers read German double-embedded sentences. In this case, the grammar violation caused a slowdown in reading. Vasishth et al. (2010
) argue that this is because German has verb-final relative clauses, which is to say that the verb is always located at the end of a relative clause. Consequently, speakers of German often encounter sentences where the verb appears late, which may increase their ability to keep verb predictions in working memory so that they are less prone to structural forgetting than English speakers.
Frank, Trompenaars, and Vasishth (2016
) replicated Vasishth et al.’s (2010
) reading-time results in Dutch; a language that, like German, has verb-final relative clauses. Again, there was no missing-VP effect in the verb-final language. However, the effect reappeared when Dutch or German native speakers were tested in English (as a second language), suggesting that the cross-linguistic difference is not due to properties of the speakers (i.e., higher verbal working-memory capacity for Dutch and German speakers compared to English speakers) but is caused by properties of the languages. For example, Dutch and German word order makes consecutive VPs much more common in those languages than in English. Sensitivity to these statistics could speed up the processing of three consecutive VPs in Dutch/German compared to English.
These results do not imply that a missing-VP effect can never arise in verb-final languages. As a case in point, Häussler and Bader (2015
) found longer reading times in grammatical double-embedded German sentences compared to versions without the second VP, if the entire structure was presented as a complement clause (as in: “I believe that the exciting book that ...”). The present study investigates sentences without such a complementizer, that is, sentences like (1a) and (1b), for which the absence of the missing-VP effect in Dutch and German is well established, at least when reading time is the dependent variable.
Neither Vasishth et al. (2010
) nor Frank et al. (2016
) asked participants to rate the sentence stimuli so the question remains whether the cross-linguistic difference between English and German/Dutch also appears in subjective judgments. Häussler and Bader (2015
) claim that categorical grammaticality judgments in German reveal a missing-VP effect, based on the fact that “in a substantial number of cases” (p. 10) the missing-VP sentences are judged to be grammatical. However, the acceptance rate was much higher for grammatical sentences (81 versus 33%, for sentence without an initial complementizer) which stands in stark contrast to previous studies in English and French that found comprehensibility or acceptability ratings to be equal or lower in the grammatical condition (Christiansen & MacDonald, 2009
; Gibson & Thomas, 1999
; Gimenes et al., 2009
). Therefore, it seems premature to conclude that there is a missing-VP effect in German grammaticality judgments.
The current study investigates whether the missing-VP effect occurs in judgments about Dutch equivalents to sentences like (1a) and (1b). If the effect depends on the language’s statistical properties, as Frank et al. (2016
) argue, the cross-linguistic difference may be restricted to reading times, while sentence comprehensibility or acceptability remain relatively unaffected. This is because word-reading times are automatically tuned to the probabilistic (i.e., statistical) information conveyed by each word (e.g., Smith & Levy, 2013
). Although a recent study by Lau, Clark, and Lappin (2017
) has shown that sentence acceptability ratings, too, correlate with the probabilities that follow from the statistics of a language’s word-order patterns, ratings are likely to be less probability sensitive than reading times because they result from conscious, deliberative processes.
In addition, we investigate if the difference between verb-initial and verb-final languages also appears when sentence comprehension is facilitated by semantic support. The sentence items of Vasishth et al. (2010
) and Frank et al. (2016
) contained nouns and verbs that allow for any combination of agent, action, and patient. For example, the sentence “The mother who the daughter who the sister found frightened greeted the grandmother” can only be understood through syntactic analysis because the meaning of the individual words do not provide any cue about who does what to whom. In sentence (1a), on the other hand, it stands to reason that it is the book that was missing a page and that the reviewers did the criticizing, even if word order is ignored. Christiansen and MacDonald (2009
) demonstrated that the missing-VP effect occurs in English irrespective of whether such semantic support is present, but in German and Dutch it has only been investigated on semantically neutral sentences. Possibly, semantic support leads to more shallow parsing (Sanford & Sturt, 2002
) or to prioritizing semantic over syntactic analysis (Townsend & Bever, 2001
). This, in turn, could mean that the language’s word order (a purely syntactic parameter) is no longer relevant to the presence of the missing-VP effect.
We had participants rate sentences on their comprehensibility (as did Gibson & Thomas, 1999
, and Gimenes et al., 2009
) as well as their acceptability (Christiansen & MacDonald, 2009
; Häussler & Bader, 2015
). It is conceivable that effects diverge between these two dependent variables, for example because the reading slowdown on Dutch ungrammatical sentences causes a decreased sense of acceptability without affecting perceived comprehensibility. However, our expectation was that comprehensibility and acceptability show similar patterns because they form merely alternative measures of participants’ underlying sentence-reading experience. If effects on the two measures indeed show similar patterns, this can, therefore, be considered converging evidence for the effect of sentence grammaticality.
In Experiment 1, native Dutch-speaking participants rated Dutch double-embedded sentences, similar to the English items from the Gibson and Thomas (1999
) study. Results showed that there was no missing-VP effect: grammatically correct sentences were rated as more comprehensible and more acceptable than the ungrammatical versions. Experiment 2 is identical to Experiment 1 except that it has English stimuli and participants are native speakers of English. Consistent with Gibson and Thomas (1999
), sentences with a missing second verb phrase were rated as more acceptable and comprehensible than grammatically correct sentences. Finally, Experiment 3 replicates Experiments 1 and 2 in a within-subjects design by presenting both Dutch and English items to native Dutch speakers of English as a second language. This experiment confirmed the findings from the first two experiments: The presence of the missing-VP effect is language dependent.
Vasishth et al. (2010
) and Frank et al. (2016
) demonstrated that the missing-VP effect on reading times is language dependent, in that it appears in English but not in German or Dutch. The current results provide further support for this finding and expand on it in three respects. First, we showed that the interaction between grammaticality (second VP present or missing) and language (Dutch or English) is not limited to reading times but also appears as a subjective illusion in sentence ratings: Dutch sentences with a missing VP were judged to be significantly less comprehensible and acceptable than their grammatical counterparts, whereas the same was not the case (or even reversed) for English sentences.
Second, we found that the cross-linguistic difference is also present for sentences whose propositional content (the “who-does-what-to-whom”) is apparent from the semantic relations between agents, patients, and actions. In contrast, understanding the materials of Vasishth et al. (2010
) and Frank et al. (2016
) required a full syntactic analysis because the nouns and verbs used in these sentences made any agent–action–patient triplet semantically possible.
Third, we compared among three ungrammatical conditions, corresponding to each of the three VPs being removed. Such a comparison was not available to Vasishth et al. (2010
) and Frank et al. (2016
) because the absence of semantic constraints in their stimuli made it impossible to tell which of the three VPs was missing. Our results for English were consistent with Gibson and Thomas (1999
) in that ratings were higher in the V2 condition than when one of the other two VPs was removed. There was no sign of such a difference for Dutch (see Table 2
), further strengthening the conclusion that the missing-VP effect does not arise in that language.4
Unlike the previous missing-VP sentence rating studies (Christiansen & MacDonald, 2009
; Gibson & Thomas, 1999
; Gimenes et al., 2009
; Häussler & Bader, 2015
), we had participants rate both acceptability and comprehensibility of the stimuli. Results were nearly identical for these two measures, which suggests there may be only one underlying cognitive factor at work. Alternatively, the convergence could have been caused by a type of anchoring effect (Tversky & Kahneman, 1974
) in which a participant’s rating on the first scale biases the response on the second.
Explaining the missing-VP effect
Three explanations of the missing-VP effect have been proposed in the literature. First, the structural forgetting account (Gibson & Thomas, 1999
) claims that encountering a double-embedded structure results in working memory overload which leads to one of VP predictions to be forgotten. According to Gibson’s (1998
) Syntactic Prediction Locality Theory, most memory is freed up if it is the prediction of the second VP that is dropped, which explains why only condition V2 of Experiment 2 results in relatively high comprehensibility and acceptability ratings. As explained in the Introduction, speakers of a verb-final language are more accustomed to keeping verb predictions in working memory and, according to Vasishth et al. (2010
), this explains why the missing-VP effect does not occur in such languages.
Second, the interference account (Häussler & Bader, 2015
) claims that the missing-VP effect is not caused by forgetting but by memory interference. More specifically, upon encountering the second verb phrase, there are two possible attachment sites (the first and second noun phrase) and a grammaticality illusion can occur when the first noun phrase instead of the second is incorrectly retrieved from memory. The first VP does not lead to such confusion because, at that point, the final noun phrase, to which it is to be attached, is still active in working memory. According to Häussler and Bader (2015
), the reason why no grammaticality illusion arises when the third VP is deleted is that the first items of a list are more strongly represented in working memory than later items (except for the most recent ones) — this is the primacy effect that is well known from the working-memory literature (e.g., Page & Norris, 1998
). If the second VP is correctly attached to the second noun phrase, the absence of a third noun phrase is detected relatively easily because of the first noun phrase’s primacy advantage. The advantage of the interference account over Gibson and Thomas’s (1999
) structural forgetting account is that the former follows naturally from general properties of working memory rather than from a particular idea about syntactic complexity, such as the Syntactic Prediction Locality Theory.5
Third, according to the language statistics account (Frank et al., 2016
), the language’s statistical word-order patterns are central to the occurrence of the missing-VP effect. This can explain why English behaves differently from Dutch and German, irrespective of the participant’s native language. As explained in the Introduction, the occurrence of consecutive VPs are more probable in verb-final than verb-initial languages. Sensitivity to such statistics can cause the third VP to be highly unexpected in English, while it is much less surprising in Dutch/German. Computational simulations using neural networks and other statistical models have shown that this is indeed a viable explanation of the cross-linguistic difference (Engelmann & Vasishth, 2009
; Frank et al., 2016
; Futrell & Levy, 2017
). When these models have learned the statistical word-order patterns from a large corpus of English texts, they estimate higher word probabilities (corresponding to faster reading; Hale, 2001
; Levy, 2008
) in double-embedded English sentences when one verb phrase is removed. When the models are trained on a Dutch or German corpus, however, they estimate higher word probabilities in the grammatical Dutch/German sentence condition. In short, the models predict that the missing-VP effect on reading times arises in English but not in Dutch or German. Whether they predict the same cross-linguistic difference for a sentence rating task depends on whether such ratings correlate with probabilities. Recent research suggests that they do (Lau et al., 2017
) so our results are in line with the language statistics account.
None of the three accounts on its own can straightforwardly explain all the available empirical data. For structural forgetting to explain why native Dutch and German speakers display the illusion in English, it needs to assume that working memory capacity is language specific, something that is highly unlikely (see the “Discussion
” of Experiment 3, and references therein). Likewise, the interference account is unable to explain why the same sentence structures yield opposite effects in verb-initial (English and French) versus verb-final (German and Dutch) languages. The language statistics account is hard pressed to explain why dropping the first or third VP does not result in a grammaticality illusion in English.
The simplest model that can explain the current results may be a hybrid statistical and working-memory account (see Christiansen & Chater, 2016
, Ch. 7, for a similar view). Primacy and recency effects lead to stronger working memory activation of the first and last noun phrase, so the absence of their verb phrases is easily detected. For grammatical English double-embedded sentences, the low occurrence probability of three consecutive VPs leads to a sense of unacceptability (and slowdown in reading) on the third VP, even if it is syntactically required. In verb-final languages, in contrast, the third VP is less unexpected, and therefore, more acceptable. This acceptability, in turn, can lead to a (false) sense of comprehension, although it should be kept in mind that these very complex sentences are still not considered very acceptable or comprehensible, with average scores of just over 4 on a 7-point scale.
It may appear unrealistic to claim that syntactically correct (three-VP) English sentences receive a lower probability than ungrammatical two-VP structures, but the subjective (and implicit) probabilities assigned by the language comprehension system need not be based on complete and correct syntactic parses. As a case in point, Lau et al. (2017
) found that the probabilities assigned by a context-free grammar display weaker correlation with acceptability ratings than probabilities from a recurrent neural network. The same has been reported for eye-tracking and EEG data (Frank & Bod, 2011
; Frank, Otten, Galli, & Vigliocco, 2015
Ambiguity of Dutch relative clauses
As mentioned in the “Materials
” section of Experiment 1, Dutch relative clauses are ambiguous between subject-relative (SR) and object-relative (OR) readings. That is, the Dutch sentence fragment “Het spannende boek dat de populaire schrijver publiceerde” (lit.: “The exciting book that the popular author published”) can in principle be interpreted to mean that the book published the author. This is because Dutch, unlike German, does not have case marking. We reduced the amount of ambiguity by having each sentence start with an inanimate noun, which leads to an initial preference for the (intended) OR reading (Mak et al., 2002
). Furthermore, only the third noun was plural so that the innermost embedding is unambiguously OR as soon as the plural verb is encountered. Semantically, too, the two relative clauses are more likely to be interpreted as OR. Nevertheless, the syntactic ambiguity of the first relative clause remains, which could lead to increased cognitive (or working memory) load compared to the English sentences in which any ambiguity is already resolved at the first word following the relative pronoun. However, this is unlikely to have caused the cross-linguistic difference because, to the extent that the missing-VP effect is caused by cognitive overload, an increased difficulty due to the Dutch ambiguity should make the grammaticality illusion even stronger. In contrast, what we find is its absence.
In Frank et al.’s (2016
) Dutch sentences, no useful semantic information was present and all nouns were animate. Consequently, these structures could not easily be interpreted as ORs. For this reason, an adjective was inserted after each relative pronoun, which syntactically disambiguates towards an SR reading. Frank et al. (2016
) argue that this could not have caused the reversal of the missing-VP effect compared to English, and our current results indeed confirm that the illusion is also absent in Dutch double-embedded OR clauses.