Science and Research for Non-Scientists
In the past year, distrust and denial of science have grown dramatically. This is of great concern to us at Ascent, not only because of the immediate, COVID-related consequences, but for the long-term impact on society.
Everything we do at Ascent is anchored in scientific research; indeed, as individuals we support research in various fields. As a research-trained scientist myself who has spent much of his career "translating" from science to practice, it occurred to me that many scientists are not good at communicating what they do, and this is an obstacle to those who want to explore a complex issue themselves without understanding the process - they may not know where to begin, and who to trust. So we have put together a guide that explains how scientists work and communicate in plain English. We hope this is helpful.
A HOPEFULLY INFORMATIVE GUIDE TO "RESEARCH."
by Stephen P. Kelner, Jr., PhD
Why are so many scientists incensed about people talking about how they "did research"? Can’t anyone look around and research a topic? Isn’t that one of the joys of the Internet, to fall down a rabbit hole on an interesting or obscure subject? What if I am reading books on history, or art? Isn’t that research?
Yes, it is, and they should. Where scientists object (including me) is when people equate this kind of research with SCIENTIFIC research, for purposes of arguing with scientists.
It’s not about “elitism,” if by that you mean scientists are acting as gatekeepers to information - most scientists I know are more than willing to talk about their work ad infinitum, beyond all bounds of boredom. Indeed, this is a unique time in history, when you can genuinely find out about real and current scientific research being done: you can find out about peer-reviewed and published papers and read their abstracts for free or even buy the journal article; you can read blogs and posts about ongoing research; you can even find places where groundbreaking scientists are posting their actual data online! (This has been true for a number of climate researchers, for example, for YEARS now.)
So much for gatekeeping. Then what's the big deal?
Here's the big deal: there ARE some requirements to do RELIABLE research, even of the non-scientific variety:
Finding Reliable Sources
In other words, does the content you are reading meet scientific standards? Acceptable standards include:
Publications in nonprofit, peer-reviewed academic journals. These must go through a gauntlet of edits and challenges from people in the same field, who therefore know all the pitfalls and traps that may lie in wait for unwary researchers, and can help ensure the results are both valid (it works!) and reliable (it will repeat!). There are phony journals that charge to publish, or pretend to be scientific journals but are in fact home to people selling a particular point of view. To be sure, you can check major academic societies (e.g., I belong to the Association for Psychological Science) who sponsor top journals, or even the publishers of such journals, such as Wiley or Elsevier. Wikipedia can be a guide as to where to look. You can also ask someone who knows the field. If it’s in the Journal of Personality or Psychological Science, I'd trust it. Likewise, Nature or Science are good sources, and written for a more general audience.
Scientific reports or general comments by people active in the field or a closely related field. In the early stages of scientific research, scientists will normally hedge their findings carefully – “as far as we know at this time,” “this appears to be the case.” When things are slippery, trust the expert, because they not only know what can go right and wrong in later findings, but they also know the range of research being done, so they have a basis for comparison. An immunologist knows the field of immunology and can therefore talk about the differences between the flu, measles, HIV, and COVID-19 without making false or inaccurate comparisons.
Click through to the original research. I find a lot of clickbait articles on science include links to the original paper. (If they don’t, that’s a problem.) Go read the abstract for yourself, and then decide if there is something there. In one case I recall, the actual paper was far more reasonable, which was fair given that it only had 14 research subjects.
Ask a friend more knowledgeable in science, or who can get you to someone who is. Someone with good scientific training might be able to read a paper in a related field, or at least know how to interpret the statistics and academic language involved.
How Real Scientists Speak (and Why)
Scientists use different phrasing, and sometimes the same words with different meanings. It’s worth clarifying this.
They distinguish between settled science, well-supported findings, and early exploration, and they work in the latter two, not the first.
There are concepts that experts in a field can safely take for granted, e.g., Evolution, relativity, anthropogenic global warming, because they are so well established as matters of fact that there is no point discussing them anymore. That's settled science. It doesn’t mean we can’t learn more - Einstein predicted the concept of "gravity waves," which are a logical extension of his other work but which were only first detected for real in 2016 - but the essential or core fact is accepted. The next tier, “well-supported findings,” have lots of research backing them, but we may not know as much as we would like to know yet - lots of good, solid data all pointing the same way, but we know there are some holes. If you read about the development of atomic theory in the 1930s, you can get a good sense of that - people were constantly coming up with new findings and new ways to break down the atom in a way that made enough sense to use, and they were ultimately able to make something that worked, which is a good test. Finally, early exploration is just that: trying something that hasn't been done, which means you cannot assume one study is enough to nail it down – and in fact it often provokes much more research to prove or deny the original study. Scientists don’t need to do research into the existence of gravity anymore, but they are definitely looking at gravity waves.
That means that scientists may be working in fields so far beyond everyday knowledge that they choke up trying to explain it – they’d have to explain centuries of work, and they just don’t know how. That's why they say things like “trust me, I could explain it, but it would take a while.” If you have to go through Galilean relativity, Newtonian physics, Einsteinian relativity, and quantum physics to get up to your question on string theory, it’s going to take a while. It’s not that you can’t learn – it’s that there is so deep a foundation that has to be understood first. Having said that, I do think scientists should work harder at finding and communicating answers, and we know it can be done: Isaac Asimov, Carl Sagan, and Stephen Jay Gould, to name only three, were actual scientists who could communicate recent findings in understandable ways.
Scientists use cautious scientific language. Many people get frustrated by scientists, who seem to be hedging everything. That's because they are: scientific method assumes you can always be wrong. What people do not always understand is that there are degrees of being wrong. In other words, we know we are right about big things like the existence of evolution or relativity - those will not be proved wrong. But that doesn’t mean that we might not learn more about how genes produce proteins or how gravity waves work, or be wrong about smaller mechanisms within the bigger fact. But the habit remains, especially on very complex issues where it’s hard to be precise, like global warming - it is a fact that it is happening because of humans, it is a fact with some variation on how fast it is happening - it is definitely years, not centuries, but how fast within that is unclear, and exact details of specific weather patterns are likely to be unclear for some time to come because they are hideously complicated. This is why good articles asking, “did global warming cause hurricanes?” always say something like “yes and no.” There is no doubt that increased oceanic warmth makes hurricane formation more likely, but does that mean more hurricanes, or the same number of hurricanes only bigger? And talking about this hurricane is kind of silly – this hurricane was helped along by global warming, but you can’t say it happened solely because of that.
A proper scientist will say something like, “the data seem to support the conclusion that so-and-so is happening.” This can be annoying when you want a straight answer, but it is a fairer way to talk about things that are happening right now. For example, we did not know how COVID-19 was transmitted at first, so the experts, using what they know of other viruses, suggested being careful about cleaning surfaces. Later, it was found that it traveled primarily by droplets in the breath, and it did not survive on surfaces for very long at all, so we could focus on masks rather than sanitizing every surface in sight. That doesn’t mean they are wrong about everything - we know a lot about how viruses work, and the incredible speed of development of new, effective vaccines proves it. It just means they are still learning. It is a fact that COVID-19 is contagious and deadly, but a year and a half ago we did not know enough to know all the ways it was contagious.
Scientists use “THEORY” differently. People usually use “theory” as synonymous with “idea” or “proposal” or as the opposite of a “proven fact.” That's not how scientists use it. A theory is a construct that has a vast amount of good data and research under it, and is considered a fact for all intents and purposes. A scientific theory not only explains virtually all the good research to date, but it can also make predictions about things that have not been tested. Relativity made specific predictions (e.g., gravity bends light), which were testable and found to be true. “The Theory of Evolution” is not something that is just an idea, it is the best and only explanation for what we know about genetics, speciation, fossils, mutation, biological mechanisms, etc., etc., etc. What that means is that we will simply not overturn evolution with any single discovery, or really at all. There's simply too much evidence that is explained by the fact of evolution. One frustration of scientists is when someone claims they have “disproved evolution” or some such with one finding. Nope. As mathematician Laplace (and Carl Sagan, David Hume, and others) said: “Extraordinary claims require extraordinary evidence.” One finding ain’t gonna do it. It is far more likely that the one thing is either an anomaly explainable by other means, or that it actually fits into the larger theory if you study it enough. For example, you may hear that Einstein “disproved” Newton. Not really – as one writer put it, Einsteinian theory swallowed Newton whole, by using it under most circumstances, and then explaining the exceptions that Newtonian theory could not.
Red Flags
Simplistic answers for complex issues. For example, one tactic of global warning deniers is to say smugly, “have they considered THE SUN?” The temptation by someone who is even moderately well informed is to say “no duh, Sherlock.” One key reason we know human-caused global warming is happening is precisely because the solar cycle is cooling right now. It would be idiotic for atmospheric physicists or climate scientists to ignore the single largest source of Earth’s heat.
Questioning experts without sources, or using ad hominem attacks. That's just saying “nuh-uh!” If someone has a challenge, then ask them to cite their sources. If they say, “do your research,” write them off at once. Remember: extraordinary claims require extraordinary evidence. If some amateur thinks they are more knowledgeable than world-renowned experts, they had better have real sources. And those sources had better be world-renowned experts in the field. On occasion I have challenged people claiming, for example, that “masks don’t work.” In one case I challenged a person to provide citations. He gave me a couple, and I was able to show that he had misinterpreted what the paper said, and in return provided six separate articles explaining that masks do indeed work - and why. He ignored these, failed to respond, and said, “masks don’t work” a week later. You can safely ignore those people - they're just trolls.
Using “common sense.” That's an appeal to emotion, not fact. Common sense said planes couldn't fly. Relativity is mind-bending at first - in part because it doesn’t all reflect daily life, but special situations we don’t experience (like traveling near the speed of light). That doesn’t mean it’s wrong - it just means you can’t detect it. I can’t see infrared light, but I know it exists.
No link to actual research paper, using phrases like “science says” or “new finding” without an author or citation.
NON-cautious language - AKA, clickbait. If you see something like "science says bananas can extend your life!" it is immediately untrustworthy. Indeed, exclamation marks appear to be a red flag here on their own!
I don’t know if anyone will read all this, or care, but if nothing else it makes me feel better, as a research-trained scientist myself, who has been in the business of translating his field into daily life for some time. I feel strongly that science should be accessible, and hopefully this will help even with fields that are too complex to learn right away.