From 468493fe34f51ac48775cdbfe4a3a606569845d8 Mon Sep 17 00:00:00 2001 From: Sterett Mercer Date: Tue, 6 Oct 2026 07:58:59 -0700 Subject: [PATCH 1/8] docs: add 2026 Model 3 study to README --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 6999116..337f11f 100644 --- a/README.md +++ b/README.md @@ -119,6 +119,8 @@ Also see the list of code [contributors](https://github.com/shmercer/writeAlizer ### Journal Articles +Keller-Margulis, M. A., Mercer, S. H., Matta, M., & Duran, B. (2026). *Identifying optimal indicators of performance using automated written-expression curriculum-based measurement: An examination of genre effects* [Preprint]. EdArXiv. https://osf.io/preprints/edarxiv/36dn5_v1/ + Matta, M., Keller-Margulis, M. A., & Mercer, S. H. (2025). Improving written-expression curriculum-based measurement feasibility with automated writing evaluation programs. *School Psychology, 40*(6), 707–717. https://doi.org/10.1037/spq0000691 Matta, M., Mercer, S. H., & Keller-Margulis, M. A. (2023). Implications of bias in automated writing quality scores for fair and equitable assessment decisions. *School Psychology, 38*, 173–181. https://doi.org/10.1037/spq0000517 From 7c294747785b86239162fb3b51a3d21529b95f3f Mon Sep 17 00:00:00 2001 From: Sterett Mercer Date: Tue, 6 Oct 2026 07:59:13 -0700 Subject: [PATCH 2/8] docs: add Keller-Margulis et al. 2026 bibliography entry --- vignettes/references.bib | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/vignettes/references.bib b/vignettes/references.bib index 6f1d653..bad064a 100644 --- a/vignettes/references.bib +++ b/vignettes/references.bib @@ -1,3 +1,12 @@ +@misc{Keller-Margulis2026, + author = {Keller-Margulis, Milena A. and Mercer, Sterett H. and Matta, Michael and Duran, Brenda}, + title = {Identifying optimal indicators of performance using automated written-expression curriculum-based measurement: An examination of genre effects}, + year = {2026}, + type = {Preprint}, + publisher = {EdArXiv}, + url = {https://osf.io/preprints/edarxiv/36dn5_v1/} +} + @article{Keller-Margulis2021, author = {Keller-Margulis, Milena A. and Mercer, Sterett H. and Matta, Michael}, title = {Validity of automated text evaluation tools for written-expression curriculum-based measurement: A comparison study}, From 25440f2b0f44a3b272aa97717858be5417bf8795 Mon Sep 17 00:00:00 2001 From: Sterett Mercer Date: Tue, 6 Oct 2026 07:59:26 -0700 Subject: [PATCH 3/8] docs: cite Model 3 training study and correct grade range --- vignettes/scoring-model-development.Rmd | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/vignettes/scoring-model-development.Rmd b/vignettes/scoring-model-development.Rmd index a72f3d2..3f47bba 100644 --- a/vignettes/scoring-model-development.Rmd +++ b/vignettes/scoring-model-development.Rmd @@ -2053,11 +2053,11 @@ Each column sums to approximately 100, allowing for rounding (values describe re ReaderBench Model 3, recommended for current use, is an ensemble (formed by averaging predicted quality scores) of three genre-specific models, detailed below. -The models were trained on ReaderBench scores from 15 min narrative, expository, and persuasive writing samples from students in Grades 2-5 to predict holistic writing quality on the samples (theta scores calculated from paired comparisons). +The models were trained on ReaderBench scores from 15 min narrative, expository, and persuasive writing samples from students in Grades 3–5 to predict holistic writing quality on the samples (theta scores calculated from paired comparisons) [@Keller-Margulis2026]. Highly correlated ReaderBench metrics (_r_ > |.90|) were excluded during pre-processing (see section on [Scoring Model Development](#scoring-model-development) for more details). -More details on the sample will be provided once peer review is complete on the main study using this model. +See [@Keller-Margulis2026] for additional details on the study sample, writing tasks, human scoring, and model-development procedure. ## ReaderBench Model 3narr @@ -3825,11 +3825,11 @@ Each column sums to approximately 100, allowing for rounding (values describe re Coh-Metrix Model 3, recommended for current use, is an ensemble (formed by averaging predicted quality scores) of three genre-specific models, detailed below. -The models were trained on Coh-Metrix scores from 15 min narrative, expository, and persuasive writing samples from students in Grades 2-5 to predict holistic writing quality on the samples (theta scores calculated from paired comparisons). +The models were trained on Coh-Metrix scores from 15 min narrative, expository, and persuasive writing samples from students in Grades 3–5 to predict holistic writing quality on the samples (theta scores calculated from paired comparisons) [@Keller-Margulis2026]. Highly correlated CohMetrix metrics (_r_ > |.90|) were excluded during pre-processing (see section on [Scoring Model Development](#scoring-model-development) for more details). -More details on the sample will be provided once peer review is complete on the main study using this model. +The training sample, writing tasks, and paired-comparison writing-quality criterion are described in [@Keller-Margulis2026]. ## CohMetrix Model 3narr From 3108b80995e9de2a078e14ad4053ee0a0a335b82 Mon Sep 17 00:00:00 2001 From: Sterett Mercer Date: Tue, 6 Oct 2026 07:59:35 -0700 Subject: [PATCH 4/8] docs: cite Model 3 training sample in getting-started guide --- vignettes/writealizer-getting-started.Rmd | 2 ++ 1 file changed, 2 insertions(+) diff --git a/vignettes/writealizer-getting-started.Rmd b/vignettes/writealizer-getting-started.Rmd index 6a0ef6e..3f4935d 100644 --- a/vignettes/writealizer-getting-started.Rmd +++ b/vignettes/writealizer-getting-started.Rmd @@ -205,6 +205,8 @@ ReaderBench Model 3 keys also accept an explicit `_v2` suffix; the output column | `coh_mod2` | `pred_coh_mod2a` through `pred_coh_mod2c` | `pred_coh_mod2_mean` | | `coh_mod1` | `pred_coh_mod1a` through `pred_coh_mod1f` | `pred_coh_mod1_mean` | +The ReaderBench and Coh-Metrix Model 3 families were trained on the same Grades 3–5 sample of 15-minute narrative, expository, and persuasive writing described in [@Keller-Margulis2026]. + Model 2 is a simplified version of Model 1. Published Model 1 applications include [@Keller-Margulis2021; @Matta2022]. ### GAMET and the offline demonstration From 635e13ffa3c819c6c928e98f427893261dc4c3f3 Mon Sep 17 00:00:00 2001 From: Sterett Mercer Date: Tue, 6 Oct 2026 07:59:45 -0700 Subject: [PATCH 5/8] docs: include Model 3 study in package metadata --- DESCRIPTION | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/DESCRIPTION b/DESCRIPTION index 8d2b1a3..93dafd5 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -10,9 +10,10 @@ Description: Imports variables from 'ReaderBench' (Dascalu et al., 2018), 'Coh-Metrix' (McNamara et al., 2014), and/or 'GAMET' (Crossley et al., 2019) output files; downloads - predictive scoring models described in Mercer & Cannon - (2022) and Mercer et - al.(2021); and generates predicted + predictive scoring models documented in Keller-Margulis et al. + (2026) , Mercer & Cannon + (2022), and Mercer et + al. (2021); and generates predicted writing quality and curriculum-based measurement (McMaster & Espin, 2007) scores. License: MIT + file LICENSE From 1741b6eb49e523dd7ad95a113afbdd4a502478a8 Mon Sep 17 00:00:00 2001 From: Sterett Mercer Date: Tue, 6 Oct 2026 07:59:59 -0700 Subject: [PATCH 6/8] docs: sync codemeta Model 3 citation --- codemeta.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/codemeta.json b/codemeta.json index 686003e..19d61d6 100644 --- a/codemeta.json +++ b/codemeta.json @@ -2,7 +2,7 @@ "@context": "https://doi.org/10.5063/schema/codemeta-2.0", "@type": "SoftwareSourceCode", "identifier": "writeAlizer", - "description": "Imports variables from 'ReaderBench' (Dascalu et al., 2018), 'Coh-Metrix' (McNamara et al., 2014), and/or 'GAMET' (Crossley et al., 2019) output files; downloads predictive scoring models described in Mercer & Cannon (2022) and Mercer et al.(2021); and generates predicted writing quality and curriculum-based measurement (McMaster & Espin, 2007) scores.", + "description": "Imports variables from 'ReaderBench' (Dascalu et al., 2018), 'Coh-Metrix' (McNamara et al., 2014), and/or 'GAMET' (Crossley et al., 2019) output files; downloads predictive scoring models documented in Keller-Margulis et al. (2026) , Mercer & Cannon (2022), and Mercer et al. (2021); and generates predicted writing quality and curriculum-based measurement (McMaster & Espin, 2007) scores.", "name": "writeAlizer: Generate Predicted Writing Quality Scores", "relatedLink": ["https://shmercer.github.io/writeAlizer/", "https://CRAN.R-project.org/package=writeAlizer"], "codeRepository": "https://github.com/shmercer/writeAlizer/", From 95160f450c95e76b429d7cdbcd0786919adaabcc Mon Sep 17 00:00:00 2001 From: Sterett Mercer Date: Tue, 6 Oct 2026 08:01:50 -0700 Subject: [PATCH 7/8] docs: use DOI for Model 3 preprint citation --- DESCRIPTION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/DESCRIPTION b/DESCRIPTION index 93dafd5..8b60191 100644 --- a/DESCRIPTION +++ b/DESCRIPTION @@ -11,7 +11,7 @@ Description: Imports variables from 'ReaderBench' (Dascalu et al., al., 2014), and/or 'GAMET' (Crossley et al., 2019) output files; downloads predictive scoring models documented in Keller-Margulis et al. - (2026) , Mercer & Cannon + (2026), Mercer & Cannon (2022), and Mercer et al. (2021); and generates predicted writing quality and curriculum-based measurement (McMaster & Espin, From d965183d515b07692bd5a2316e080f3b7d916e39 Mon Sep 17 00:00:00 2001 From: Sterett Mercer Date: Tue, 6 Oct 2026 08:01:53 -0700 Subject: [PATCH 8/8] docs: use DOI for Model 3 preprint in codemeta --- codemeta.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/codemeta.json b/codemeta.json index 19d61d6..096d265 100644 --- a/codemeta.json +++ b/codemeta.json @@ -2,7 +2,7 @@ "@context": "https://doi.org/10.5063/schema/codemeta-2.0", "@type": "SoftwareSourceCode", "identifier": "writeAlizer", - "description": "Imports variables from 'ReaderBench' (Dascalu et al., 2018), 'Coh-Metrix' (McNamara et al., 2014), and/or 'GAMET' (Crossley et al., 2019) output files; downloads predictive scoring models documented in Keller-Margulis et al. (2026) , Mercer & Cannon (2022), and Mercer et al. (2021); and generates predicted writing quality and curriculum-based measurement (McMaster & Espin, 2007) scores.", + "description": "Imports variables from 'ReaderBench' (Dascalu et al., 2018), 'Coh-Metrix' (McNamara et al., 2014), and/or 'GAMET' (Crossley et al., 2019) output files; downloads predictive scoring models documented in Keller-Margulis et al. (2026), Mercer & Cannon (2022), and Mercer et al. (2021); and generates predicted writing quality and curriculum-based measurement (McMaster & Espin, 2007) scores.", "name": "writeAlizer: Generate Predicted Writing Quality Scores", "relatedLink": ["https://shmercer.github.io/writeAlizer/", "https://CRAN.R-project.org/package=writeAlizer"], "codeRepository": "https://github.com/shmercer/writeAlizer/",