Current Events > Scientists Rarely Admit Mistakes. A New Project Wants to Change That.

(+) Yeah! (+)

https://undark.org/article/loss-of-confidence-project-replication-crisis/

...
The aim is to simplify how such statements are reported as opposed to the current process, in which researchers bicker in back-and-forth commentaries and rebuttals, says Julia Rohrer, who studies personality psychology at the International Max Planck Research School on the Life Course in Berlin, Germany, and is one of three researchers working on the project. People will defend their scientific claims until their death, Rohrer said. As scientists, we should be aware that people are often wrong. Carneys move, for example, was generally well-received by psychologists who welcomed her transparency, she noted.

Rohrer and her colleagues, Tal Yarkoni of the University of Texas at Austin and Christopher Chabris, at the Geisinger Health System in Pennsylvania, are currently accepting submissions of loss-of-confidence statements, focusing on psychology studies and with some ground rules: Authors submitting a loss-of-confidence statement, for example, are expected take primary responsibility for methodological or theoretical problems with their paper otherwise, the entry goes into whistleblowing territory and is not eligible for publication. The researchers eventually plan to publish the statements in an academic paper, Rohrer said.
...

---

We know. This isn't news to people who have been outside of academia at some point in their life.
---

Few people regularly admit their mistakes.
---

"We're sorry we put feathers on your T.rex."

REMercsChamp posted...

We know. This isn't news to people who have been outside of academia at some point in their life.

Deliberately missing the point doesn't make you look more informed.
---

FortuneCookie posted...

"We're sorry we put feathers on your T.rex."

It's all I want to hear
---

Psychology isnt that good of an example when its literal founder - Freud - has had a number of his theories proven as faulty over time.

Otherwise, meh, it doesnt really matter imo if a scientist like a physicist or biologist admits they were wrong - their theories are just later disproven and theyre debunked.
---

They don't have to since the scientific method does that for them.
---

Of course they'll refuse to admit their mistakes. That'd mean giving up funding lmao
---

masticatingman posted...

Psychology isnt that good of an example when its literal founder - Freud - has had a number of his theories proven as faulty over time.

Otherwise, meh, it doesnt really matter imo if a scientist like a physicist or biologist admits they were wrong - their theories are just later disproven and theyre debunked.

Thinking this is restricted to psychology is a big mistake. Cancer bio has a reported replication of like 30%, a recent genomic imaging replication attempt only had a 8% replication rate. Cultural incentives in science and the file drawer effect prevent science to truly be self correcting. At least at any reasonable rate.
---

masticatingman posted...

Psychology isnt that good of an example when its literal founder - Freud - has had a number of his theories proven as faulty over time.

Otherwise, meh, it doesnt really matter imo if a scientist like a physicist or biologist admits they were wrong - their theories are just later disproven and theyre debunked.

Everything you posted is false.

masticatingman posted...

Psychology isnt that good of an example when its literal founder - Freud - has had a number of his theories proven as faulty over time.

FYI, this is not the correct way to approach science.

There is a problem with inductive reasoning, so the only way to truly progress knowledge is by ruling out the incorrect ideas and you do that by putting forth falsifiable hypotheses.

Newton's mechanics were "proven wrong" in the sense that you can't apply them to something like GPS satellites and be accurate. That required Einstein's relativity. But Newton's mechanics are still a valid way of approaching something like simple projectile motion.

Freud isn't really the "founder" of Psychology, except to laymen. He was a neurologist who armchair analyzed behavior. His psychodynamics aren't falsifiable, so they're not amenable to scientific examination. The real founders of modern psychology were the behaviorists like Skinner, Watson, and Pavlov.

You're taking it to mean that Freud's "theories" were wrong and thus psychology isn't a valid science. That's wrong.
---

iHuman posted...

They don't have to since the scientific method does that for them.

The scientific method does, but when you incorporate the current academic culture, it's not exactly followed to the ideals of the scientific method.

Authors are incentivized to publish new findings, since a lot of journals won't accept you attempting to replicate another lab's findings or publishing null hypothesis findings.

If you don't publish, and therefore bring in more grant funding to your institution, you won't get tenure.

So really its an issue of economics.
---

I think people overexaggerate things like that and then a certain political party latches on and soon people are claiming scientists are claiming false things for money
---

BlameAnesthesia posted...

Freud isn't really the "founder" of Psychology, except to laymen. He was a neurologist who armchair analyzed behavior. His psychodynamics aren't falsifiable, so they're not amenable to scientific examination. The real founders of modern psychology were the behaviorists like Skinner, Watson, and Pavlov.

Tbh, I'd trace it back to Wundt, Weber, and the other psychophysics people, and then psychoanalysis was just a psychology as a science deadzone, picked back up with behaviorists lol.
---

COVxy posted...

Tbh, I'd trace it back to Wundt, Weber, and the other psychophysics people, and then psychoanalysis was just a psychology as a science deadzone, picked back up with behaviorists lol.

Yeah, that's a good point. The great thing about Wundt was his more physiological way of approaching it and how I've always conceptualized the science.

It pains me seeing how people think of psychology about as critically as fortune cookies.
---

BlameAnesthesia posted...

COVxy posted...

Tbh, I'd trace it back to Wundt, Weber, and the other psychophysics people, and then psychoanalysis was just a psychology as a science deadzone, picked back up with behaviorists lol.

Yeah, that's a good point. The great thing about Wundt was his more physiological way of approaching it and how I've always conceptualized the science.

It pains me seeing how people think of psychology about as critically as fortune cookies.

Because it's impossible to make credible predictions without knowing more about neurology, biology, and chemistry.

It'd be like trying to do astronomy without gravity. You can come up with many logical incorrect explanations.
---

iHuman posted...

Because it's impossible to make credible predictions without knowing more about neurology, biology, and chemistry.

It'd be like trying to do astronomy without gravity. You can come up with many logical incorrect explanations.

That's not true at all, and bridging levels of analysis is actually harder, especially in complex systems.
---

Thats another reason why psychology is seen as a farce. I dunno how less information would help it with a system as complex as human behavior.
---

Idk if you understood what I was saying.
---

COVxy posted...

Idk if you understood what I was saying.

He doesn't. His gimmick is to pretend to be smart(iq of 160, he claims), but it's clear he is of average or below average intelligence. He has a hard time grasping very simple explanations quite often. He's always punching above his weight. And it's very obvious.

COVxy posted...

bridging levels of analysis is actually harder

Don't even need my full iq to decided that
---

iHuman posted...

COVxy posted...

bridging levels of analysis is actually harder

Don't even need my fill iq to decided that

You need something.

A worthy intellectual adversary
---

iHuman posted...

A worthy intellectual adversary

You'd lose an argument with a brick. At least it could hold up to more scrutiny.

Waiting for covxy to explain his supposed hidden message. You should find smarter people to piggyback off of.
---

Fix peer review first lmao.
---

iHuman posted...

Waiting for covxy to explain his supposed hidden message. You should find smarter people to piggyback off of.

Nothing is "hidden". Your response just made it clear that you didn't understand what I meant.

Credible predictions can and are made in psychology, though theory is relatively weak (as it is in biology).

Crossing levels of analysis in complex systems is hard. Lower level systems don't simply predict higher level systems.
---

I haven't seen any credible predictions in psychology

And lower levels precisely predict higher levels. Prove it doesn't.

Reductionism wouldn't have been so effective if it doesn't.
---

science goes to the highest bidder. with enough money you can buy studies that say whatever you want them to, just like big oil has
---

iHuman posted...

I haven't seen any credible predictions in psychology

They must not exist, then?

iHuman posted...

And lower levels precisely predict higher levels. Prove it doesn't.

Reductionism wouldn't have been so effective if it doesn't.

They do, but not in any simple or linear way in complex systems. You've never heard of "the sum is greater than the whole?" Interactions between many smaller elements makes prediction from small to large hard.
---

SpinKirby posted...

Fix peer review first lmao.

---

What are your issues with peer review?
---

That's why peer review exists. Scientists rarely admit mistakes but other scientists want to point out those mistakes. So potential scientific research is looked over by other scientists who was t you to fail and look at every little detail.
---

SpinKirby posted...

Fix peer review first lmao.

What's wrong with peer review?
---

COVxy posted...

What are your issues with peer review?

The only thing it does is help with some errors, assuming the reviewer catches or even understands what's going on.

Everything else is just really bad for publication, and causes people to publish or present information in a form that stops them from getting buttblasted by reviewers. Peer review is really good for suppressing a shit ton of information, at the same time, the quality of what gets through can also be unaffected.

So basically it does nothing, but it's fun to have around.
---

Here's a better explanation about it.

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1420798/
---

SpinKirby posted...

COVxy posted...

What are your issues with peer review?

The only thing it does is help with some errors, assuming the reviewer catches or even understands what's going on.

Everything else is just really bad for publication, and causes people to publish or present information in a form that stops them from getting buttblasted by reviewers. Peer review is really good for suppressing a shit ton of information, at the same time, the quality of what gets through can also be unaffected.

So basically it does nothing, but it's fun to have around.

Right, I'm all for open reviews (and preprints) for this purpose. Encourages people to give proper reviews.
---

On the topic: here's a group that published a flashy finding, collected more data, and then refuted their own flashy finding.

http://blogs.discovermagazine.com/neuroskeptic/2018/07/08/self-correction-small-talk-happiness/

Good on them.
---

frozenshock posted...

Few people regularly admit their mistakes.

This.

Also, once again people treating science as though it were religion. ugh
---

HylianFox posted...

Also, once again people treating science as though it were religion. ugh

Naw, the assumption is mostly that scientists would be unresponsive to the reward structure within their own environment, which too is a silly assumption.
---