The reason frequentist stats don't work (I think) is that in practice there are no realistic situations for it.
First they are only applicable to formal experimental situations where you have worked to remove biases using randomization and controls, most data is observational, packed with plenty of biases so the objectivity of any p-value can be called into question there.
Second, where there is a formal experiment it requires people analyzing the results just to do the pre-decided test, and no messing about. But people with an interest in the outcome are going to mess about with the data, only a very disinterested person is going to do it right.
So the case where frequentist stats works is where a disinterested person is prepared to spend a lot of money and time setting up a formal experiment. Not a situation that occurs often!
Pre-registration can help correct for biases that drive p-hacking imo. I push for RAD scholarship in my discipline (rhetoric, writing, and composition) so orthodox social Justice theorizing/metaphors can be proven/disproven rather than propagated by mere belief and nonreplicable case studies. Every few years a scholar makes the same RAD call…to no change.
But I do teach my writing students. To extend this discussion, I recommend reading Science Fictions (2021) by Stuart Ritchie as a follow-up to this excellent 4-book review. He contends that incentives like publication hype, negligence, and fraud contribute to the replication crisis where studies are rarely replicated, if ever. We need to fund replication and meta-analyses not only genius original research.
I certainly enjoyed Clayton's book too.
The reason frequentist stats don't work (I think) is that in practice there are no realistic situations for it.
First they are only applicable to formal experimental situations where you have worked to remove biases using randomization and controls, most data is observational, packed with plenty of biases so the objectivity of any p-value can be called into question there.
Second, where there is a formal experiment it requires people analyzing the results just to do the pre-decided test, and no messing about. But people with an interest in the outcome are going to mess about with the data, only a very disinterested person is going to do it right.
So the case where frequentist stats works is where a disinterested person is prepared to spend a lot of money and time setting up a formal experiment. Not a situation that occurs often!
Pre-registration can help correct for biases that drive p-hacking imo. I push for RAD scholarship in my discipline (rhetoric, writing, and composition) so orthodox social Justice theorizing/metaphors can be proven/disproven rather than propagated by mere belief and nonreplicable case studies. Every few years a scholar makes the same RAD call…to no change.
But I do teach my writing students. To extend this discussion, I recommend reading Science Fictions (2021) by Stuart Ritchie as a follow-up to this excellent 4-book review. He contends that incentives like publication hype, negligence, and fraud contribute to the replication crisis where studies are rarely replicated, if ever. We need to fund replication and meta-analyses not only genius original research.