Abstract
Beaulieu and colleagues (2023) recently reviewed single-case research evaluating behavior-analytic interventions for challenging behavior following traumatic brain injury and found that most cases had medium to large effect sizes via visual analysis but poor methodological rigor when assessed via the Risk of Bias in N-of-1 Trials (RoBiNT) scale. The purpose of the current study was to extend this review by reevaluating the same 45 cases using two different quantitative effect size metrics and a second critical appraisal tool for comparison purposes. We applied a weighted algorithm to classify the RoBiNT scores into categories and compared those classifications to research report strength determinations from the Evaluative Method. We also extracted the raw data from each case and calculated a non-overlap index, the non-overlap of all pairs (NAP), and a mean-based effect size metric, the log response ratio (LRR). We found 48.9% agreement between RoBiNT classifications and Evaluative Method determinations across cases, and the cases with disagreements yielded more rigorous classifications via the RoBiNT in all but one case. We also found the strongest association between the effect sizes determined via visual analysis in the previous review and NAP interpretations, followed by NAP and LRR, and LRR and visual analysis. Possible explanations for these findings are discussed along with recommendations to guide future behavior-analytic research in neurorehabilitation settings.