Agreement between visual inspection and objective analysis methods : a replication and extension
Article [Version of Record]
Abstract(s)
Behavior analysts typically rely on visual inspection of single-case experimental designs to make treatment decisions. However, visual inspection is subjective, which has led to the development of supplemental objective methods such as the conservative dual-criteria method. To replicate and extend
a study conducted by Wolfe et al. (2018) on the topic, we examined agreement between the visual
inspection of five raters, the conservative dual-criteria method, and a machine-learning algorithm
(i.e., the support vector classifier) on 198 AB graphs extracted from clinical data. The results indicated that average agreement between the 3 methods was generally consistent. Mean interrater agreement was 84%, whereas raters agreed with the conservative dual-criteria method and the support
vector classifier on 84% and 85% of graphs, respectively. Our results indicate that both objective
methods produce results consistent with visual inspection, which may support their future use.