Marcus wrote: ↑Thu Jan 05, 2023 3:10 am
Morley wrote: ↑Thu Jan 05, 2023 2:28 am
Res Ipsa doesn't understand probabilities and scientific papers better than the researchers,
Well, he might. He has a practical bent to his thinking that borders on obsessiveness— something that, imnsho, more than a few scientists could benefit from!!
That's very kind of you Marcus, but I don't think I understand papers better than their authors do. I just do my very best to understand what the authors are saying in a given paper. It just so happens that the skills needed to understand what a judge is saying in a legal opinion also apply to understanding what the authors of a scientific paper are saying. And, once I dig into a paper, I am obsessive about doing what needs to be done to understand what the authors are actually saying as opposed to what I'd like them to say.
I've found there are a few handy rules of thumb that help us non-science folks understand an academic paper:
1. Don't rely on just an abstract. Find and read the whole paper if you can. If you can't, be extremely cautious about drawing conclusions from the abstract alone.
2. Understand what the paper is about -- what is the question the authors are asking?
3. Make sure you understand the definitions of important terms. Don't assume that key terms are being used in their ordinary sense.
4. Understand the author's assumptions, which are commonly found in the introductory section of the paper. "Assumptions" include reliance on other published papers. Sometimes you need to take a look at those papers to understand the context of the paper you are trying to understand.
5. Understand what the authors are not saying. Sometimes the paper has a separate limitations section, which is really handy for lay folks like me. Other times the limitations aren't in a separate section, but appear in or following the conclusion.
6. Understand the conclusion. If you are going to use a paper as an authority for a proposition you are arguing, you need to understand what the authors actually conclude. Using a paper as an authoritative source using something other than the author's actual conclusion creates a significant risk of being flat-assed wrong.
All those steps get you is a good shot at understanding what the authors are saying in the paper. And I think that's as far as I've gone in the referenced thread. Critiquing a paper is an entirely different process. The papers discussed in the referenced thread us methodologies that are far beyond my capacity to make any sort of independent judgment. The methodology section of that Bayesian model paper made me break out in a cold sweat. For me, determining whether a paper is "good" or "bad" involves reading lots of other papers or critiques of the paper I'm interested in.
