The IRIS Replication Award

(Download PDF version)

The fifth round of IRIS Replication Award is now open for submissions.

£350 will be awarded every two years to a published (or accepted) self-labeled replication* study that has used (or adapted) materials (i.e., elicitation instruments, stimuli, or data) held on IRIS.

Eligibility for submission:

(1) The study must be labelled with the word ‘replication’ in the title or the abstract. In the case where the study is a computational or analytic reproduction (using the same input data as the initial study), the study should have an equivalently explicit label such as reproduction or re-analysis.

(2) Materials and/or data from the initial study that were used for the replication must be held on IRIS prior to submission of the application. Applications made when materials and data are not already on IRIS at the point of application are not eligible and applicants cannot reapply for the award that year*.

(3) Materials and/or data that emerged out of the replication study must also be held on IRIS prior to submission of the application. Applications made when materials and data are not already on IRIS at the point of application will not be eligible and applicants cannot reapply for the award that year*.

*Priority will be given to authors who refer to (1) and (2) in the body of their published or accepted article.

(4) The submission can be at one of three different stages of publication in the relevant two-year time frame: Accepted, Early View, or In Print (assigned to an issue). This may mean, for example, that some articles could be submitted in one round as an Accepted or Early View version, and could then be submitted again in the next round as an In Print publication.

(5) Priority will be given to submissions that have all possible elements held on IRIS: materials, data, and code.

(6) Priority will be given to pre-registered studies and Registered Reports, as these publication routes provide an additional layer of transparency and openness during the review process itself.

Submissions must include:

i) the article or chapter (with evidence of ‘in press’ status, if not already published);

ii) the links to all materials and/or data and/or analysis code held on IRIS that were used to conduct the replication study, provided in a short cover letter.

iii) the link to the materials and/or data and/or analysis code held on IRIS that emerged out of the replication study. This could include, for example, any adaptations of the original materials, new materials, new datasets, or new analysis code. These materials and adaptations should be uploaded to IRIS prior to application for the award. The record on IRIS should include a note to link it to the earlier study’s materials. Please provide the URL(s) for all relevant materials and data on IRIS.

Review Process

Submissions can be made to iris@iris-database.org at any time. This round will close at midnight (GMT) on 30th November 2026. The study must have been published (or accepted) from 1st December 2023 onwards. All submissions will be reviewed by the IRIS directors and up to two anonymous reviewers. The first award will be announced by 28th February 2027.

Please note that IRIS operates the following system to reduce conflicts of interest:

● If the selected article is from one of the journals in which one of the directors of IRIS is involved as an editor or Board member, that person's evaluations are withdrawn from the final decision-making.

● If the selected article continues to receive the highest evaluations among the remaining evaluations, then the award can go ahead. If the selected article no longer receives the highest evaluations, all the evaluations are considered again but with the evaluation by the reviewer in question for the previously selected article withdrawn from the process.

● If there is a tie in the highest ratings, a new reviewer will be brought in to arbitrate and make a final decision.

Replications using materials developed by the current IRIS Directors (Emma Marsden, Cylcia Bolibaugh, Charlie Nagle) are not eligible.

Criteria for award

Submissions will be welcome from any area of language-related r esearch broadly defined, including multilingualism, psycholinguistics, neuroscience, language education, research methodology. All domains of language-related research, methodological approaches, and theoretical perspectives are welcome.

Submissions will be reviewed against the following criteria:

● Strength of the justification for the replication.

● Soundness and transparency of methods and analysis, including clarity about how these are similar/different to those of the initial study.

● Degree to which the discussion and conclusions are warranted by the methods, data and analysis.

● Integration of the findings from the initial study(ies) into the set-up, analysis and interpretation of the replication study.

● The extent to which the publication has made its process open in terms of the different elements: materials, data, analysis code.

If two or more submissions meet all the above criteria equally, special consideration will then be given to replications with one or more of the following characteristics:

∙ Multi-site replications

∙ Pre-registered studies, including Registered Reports

∙ Direct or partial replications (though we emphasize that conceptual replications, which change more than one significant variable, are also very welcome and strongly encouraged)

Please direct any questions to iris@iris-database.org.

*The study is labeled with the word “replication” (or equivalent for a computational or analytic reproduction study) in the title and/or abstract.

2024 Winner:

Huensch, A. (2024). “Clarifying the role of inhibitory control in L2 phonological processing: A preregistered, close replication of Darcy et al. (2016)”

Thanks to the open research practices of the authors of previous studies, the researcher was able to use materials from Darcy, I., Mora, J. C. & Daidone, D. (2016) already housed on IRIS: https://www.iris-database.org/details/xSmdt-ui4Xg

The study examined the relationship between language abilities and general cognition, or specifically, how inhibitory control might relate to L2 speech perception and production. It was a close replication of Darcy et al. (2016) which examined the relationship between inhibitory control and L2 speech perception and production and unexpectedly reported differential relationships between inhibitory control and perception and production. This close replication followed the procedures of Darcy et al. but incorporated two additional tests of inhibition. Bringing together the findings from DM&D and the current replication, no strong, clear, or consistent relationship emerges between inhibitory control and L2 perception/production skills. These findings lead us to consider that despite inhibition being demonstrated to relate to language processing in bilinguals in lexical access studies, its relationship to phonological processing may be null or weak at best.

This study was published in Studies in Second Language Acquisition. It is a preregistered direct replication and includes open data.

Data and materials from Huensch, A. (2024) are available on IRIS: https://www.iris-database.org/search/?s_publicationAPAInlineReference=Huensch%20(2024) and OSF: https://osf.io/fxzvj/

2022 Winner:

Huensch, A., & Nagle, C. (2021). The effect of speaker proficiency on intelligibility, comprehensibility, and accentedness in L2 Spanish: A conceptual replication and extension of Munro and Derwing (1995a). Language Learning, 71 (3), 626-668.

This study investigated the relationship among intelligibility, comprehensibility, and accentedness in the speech of second language learners of Spanish of varying proficiency in instructed contexts. It conceptually replicated studies by Munro and Derwing (1995a) and Derwing and Munro (1997), who found partial independence among the three speech dimensions but also evidence that proficiency may mediate the relationship between linguistic features of stimuli (e.g., phonemic and grammatical error rates) and speech dimensions. Speech data from 42 second language learners of Spanish recruited from two different universities were elicited via a semispontaneous speaking task: the picture-based narration from the initial study. Amazon Mechanical Turk was used to recruit 80 native Spanish listeners to transcribe and rate extracted utterances. The utterances were coded for grammatical and phonemic errors, goodness of prosody, and speaking rate. Analyses included mixed-effects models that allowed estimation of individual variation across facets of the data, particularly those of listeners.

The materials from Huensch, A., & Nagle, C. (2021) are on IRIS: https://www.iris-database.org/details/Z1wIr-hMCVg

Thanks to the open research practices of the authors of previous studies, the hunter story task from Munro and Derwing (1995a) is on IRIS: https://www.iris-database.org/details/hWgPi-JCEru

And an elicited imitation test (EIT) is on IRIS: https://www.iris-database.org/details/yhzL0-J2jzK

2020 Winner:

Hiver, P. & AI-Hoorie, A. H. (2020). Reexamining the Role of Vision in Second Language Motivation: A Preregistered Conceptual Replication of You, Dörnyei, and Csizér (2016). Language Learning, 70 (1), 48-102.

This study replicated You, Dörnyei, and Csizér (2016). Motivation, vision, and gender: A survey of learners of English in China. Language Learning, 66, 94–123. The objective of this conceptual replication was to test whether language learners’ vision of themselves in the future (their mental imagery) can energize action and help them develop positive learning behaviors in the present. The reviewers found this study to have a strong justification for doing the replication, sound and transparent methods and analysis, thorough explanations and discussion, and careful integration of the findings from the initial study into the discussion of the findings from the replication itself.

The materials from Hiver & AI-Hoorie (2020) are on IRIS: https://www.iris-database.org/iris/app/home/detail?id=york%3a936427&ref=search

Thanks to the open research practices of the authors of the original study, the materials from that study are on IRIS: https://www.iris-database.org/iris/app/home/detail?id=york%3a847528&ref=search

2018 Winner:

Suzuki, Y. (2017). The optimal distribution of practice for the acquisition of L2 morphology: A conceptual replication and extension. Language Learning, 67, 512–545.

This study replicated Suzuki & DeKeyser (2017*). Effects of distributed practice on the proceduralization of morphology. Language Teaching Research, 21, 166-188. The main objective of the study was to investigate further the role of linguistic complexity for different levels of L2 grammar practice distribution. The reviewers found this replication study to have a very thorough and clear methodology, clearly explaining all deviations from the original study with strong rationales for the changes made. The materials from Suzuki (2017) are on IRIS.

* The original study by Suzuki & DeKeyser (2017) was first published online in 2015.