1 April 2020: Avoiding pitfalls: Bayes factors can be a reliable tool for post hoc data selection in implicit learning

Mateo Leganes-Fonteneau
Center of Alcohol Studies
Rutgers University–New Brunswick

Research on implicit processes has revealed multiple shortcomings of awareness categorization tools, which are most often based on the failure to score significantly above chance level. Moreover, post-hoc awareness categorizations result in regression to the mean effects, by which aware participants are wrongly categorized as unaware. Using Bayes factors to obtain sensitive evidence for participants’ lack of knowledge could prevent regression to the mean effects. Here I present the reliability analysis of a novel Bayesian awareness categorization procedure, comparing it to traditional approaches based on t-tests.

Following a reward conditioning task, we divided participants in different groups (Aware, Unaware, Insensitive) depending on the results of individual Bayesian analysis ran on an awareness measure. We examined the cumulative awareness scores of each group, and found that Unaware participants did score below chance level, contrary to the predictions of regression to the mean effect. This was further confirmed using a resampling procedure with multiple iterations. Conversely, when categorizing participants using t-tests, Unaware participants scored significantly above chance level on the resampling procedure, showing regression to the mean effects.

The results show that using Bayes factors instead of t-tests as a categorization tool it is possible to avoid regression to the mean effects. The reliability of the Bayesian awareness categorization procedure strengthens previous evidence for implicit reward conditioning. I will discuss how Bayes factor can reduce measurement errors to provide a reliable post-hoc awareness categorization tool, improving the quality of research on implicit processes. The toolbox necessary to perform the categorization procedure is available to be incorporated in future experiments.

View the presentation HERE