Where is
the Skeptic Exercise Physiologist?
Tommy Boone, PhD, MPH, FASEP,
EPC
Professor and Chair
Department of Exercise Physiolgoy
College of St. Scholastica
Duluth, MN 55811
Skepticism is essentially non-existent
in exercise physiology! What a statement. Is it true that exercise
physiologists, who are researchers at heart, simply don’t debunk bad science?
Is it possible because either they don’t read the research or they simply
choose to look the other way? Better yet, how could the integrity
of research get lost or so poorly understood? Is it logical and right
that researchers should know better than to publish an article with obvious
flaws just to get it in print? Obviously, these are very important
questions for all professionals who strive to think critically and who
believe they are on top of their science.
Possibly, part of the problem is the unquestioned
belief in science. Surely no one would purposefully alter research
findings or make up data just to get tenure or to get an article published!
Well, unfortunately, the reality is that some researchers do just that.
They are so driven to get their point of view across that they are willing
to publish their views even when their work is premature or wrong.
What is also hard to understand is that, when the majority of the researchers
in a given field fail to recognize the findings as lacking, inappropriate,
or just plain wrong, the published work often goes unchallenged for years
if not decades. Interestingly, if someone were to disagree with the
published findings, without the majority to back the person, the person
may himself be on the outside looking in if not completed disregarded.
Hence, the noble tradition of thinking
like scientists has come up short in creating intellectual steps in critical
reflection. Too many scientists are too interested in publishing
their work, and they appear equally motivated to leave bad science to itself.
In the end, research can be and is frequently research for research sake.
In fact, I must confess at times to being guilty of similar thinking.
Most college professors don’t become professors without a significant list
of publications. As a result, it appears that many college teachers
don’t take the time away from their busy research efforts to teach students
about skeptical research or how to think. Instead, they teach students
what to think because it is easier and accepted. To find time outside
of class to address the weaknesses of published research isn’t a priority.
Instead of questioning what is written and why authors present their thinking
as they do, the student is left to believe the content as written.
The question of whether the content is a fair and reasonable representation
of the topic investigated isn’t addressed, yet an unknown percent of what
is published is absolute nonsense! It seems reasonable to argue,
therefore, that the peer-review process should screen out bad research,
bad thinking, and manuscripts designed to confirm the authors’ bias from
good research.
Without critical reflection, whatever is
published is considered good research and thus acceptable and/or proven
statements of fact. As previously stated, the student is not likely
to think the content could be wrong. Few, if any, students are prone
to disbelieve a published account of science! The idea that the content
in the article might be wrong or that the authors might have mis-represented
the conclusions is not questioned. In other words, there is a major
failure in teaching students to think for themselves. And, yet science
is so necessary and important that everyone seems to have forgotten that
it is created by colleagues who are subject to problems in thinking like
anyone else. If they are not ethical, then the science is unethical.
This is the crux of this article. The unquestioning faith in published
articles needs examination. Carl Sagan (1) said
it best, the need for critical thinking is “nothing less than our survival
– because baloney, bamboozles, bunk, careless thinking, flimflam and wishes
disguised as facts are not restricted to parlor magic and ambiguous advice
on matters of the heart.”
Traditionally, being healthy is so important
that students are told what not to do. Abstain from smoking, don’t
drink alcohol, avoid fatty foods, and get regular exercise are the obvious
statements of good intentions. Those who don’t change their lifestyle
are left feeling worried if not guilty. While it is logical that
moderation appears to be the key to most views about individual health
and health in general, seldom are students taught to keep an open mind
toward opposing views. This is an unfortunate outcome because students
are set up to see only one side of a controversial issue. The nature
of critical thinking demands the opportunity to examine all the facts to
solve problems and make decisions.
Taking Sides is a book that encourages
a better understanding of the subject of cause and effect (2).
It contains 38 articles arranged in 19 pro and con pairs. Each pair
addresses a controversial issue in health and society. In addition
to presenting the accepted thinking about health sensitive and complex
disease-related topics, students should be encouraged to study and understand
the points made by the opposition. Not only do students need clear
thinking to judge the statements of newspapers, books, teachers, doctors,
and other healthcare personnel, it is important that exercise physiologists
have the ability to think clearly, to know the difference between opinion
and facts, and to know when to take action to solve differences in thinking.
As professionals, exercise physiologists
must not become so attached to an idea that it becomes the only approach
to an issue (e.g., the role of cholesterol in causing heart attacks).
Ruchlis (3) said that, “People who believe strongly in
an idea often become so attached to it that they bend and twist any facts
that cast doubt on the idea. This ‘mindset’ or bias (prejudice that
inhibits judgment) is often subconscious and so deeply embedded that most
people are unaware of its influence on their reasoning.” The
point is that “doing what is assumed to be right” doesn’t always translate
as being factual, appropriate, or right.
While honest errors will always happen,
and while some errors happen for the wrong reasons, all college professors
should be driven to correct factual errors. After four years of college,
students should have a good understanding of reputable science and reliable
ways to think and argue about the accuracy of facts. Professors,
therefore, shouldn’t teach just one side of an issue, especially under
circumstances where that particular side confirms an embedded belief.
Students deserve more, society expects more, and all department chairs
should hold their faculty accountable for teaching their biased opinion
masquerading as “fact”. Just, as an example, if a professor stands
up in front of class and says that cholesterol causes coronary artery disease,
is it true? Must the department chairs accept the professor’s way
of thinking and lecturing?
The short answer to both questions is “no”.
Ruchlis (3) says, “It is not necessarily true that if
one event always follows another then the first event is the cause and
the second event is its effect.” The danger of an absolute and unyielding
mindset is that everyone starts to think exactly the same, which is a major
mistake. Professors, students, and the public sector should join
in the common belief that learning what to think should never take the
place of learning how to think. Learning what to think is only the
beginning of learning from reading books, journals, and so forth.
It seems that everyone knows what to think. That is, according to
Glenberg et al. (4), they have the illusion of knowing
whereby they think they understand but don’t and, perhaps, even worst,
they may never realize they have a problem thinking correctly and therefore
a need to learn how to think.
Having said this, it is time that the language
professors use to discuss causation should be updated, particularly with
regard to coronary artery disease. The truth is, aside from discussing
causation to prevent disease, the idea of knowing helps everyone feel more
in control and bring about desirable outcomes and avoid undesirable ones
(5). But, what if the idea of knowing (and hence
the prevention of an undesirable outcome) is based on an incorrect causal
inference? The consequences of unclear thinking and for not knowing
the real causes are potentially very troubling, if not dangerous.
“It ain’t so much the things we
don’t know that get us into trouble. It’s the things we know that just
ain’t so.” -- Artemus Ward
When lecturing on risk factors for coronary
artery disease, professors may understand that there are simply too many
unanswered questions for what causes a myocardial infarction (heart attack)
but students only know what is brought to their attention. Without
placing equal emphasis on a discussion of the “unknown causes” of heart
attack, students fail to grasp the language of causal inference.
Clearly, not every person who suffers a heart attack has elevated low-density
lipoprotein cholesterol. It is important therefore that students
are taught the distinction between necessary and sufficient causes.
An incorrect causal inference can have considerable emotional and mental
consequences.
Zechmeister and Johnson (5)
state that “a necessary cause is one without which the effect cannot occur.”
When speaking of high cholesterol being a necessary cause of coronary artery
disease (and thus a heart attack), a heart attack will not occur unless
cholesterol is elevated. If it is possible to have a heart attack
in the absence of high cholesterol, then high cholesterol is not a necessary
cause and, therefore, not the cause of a heart attack. Exercise physiology
professors should think some more about causality. It isn’t a failure
in thinking to admit that decades of research has resulted only in the
identification of different conditions associated with coronary artery
disease. Certainly the covariation of high cholesterol and incidence
of heart attacks needs continued study because, at the present time, the
strength of the correlation (i.e., the relationship between the two) is
not strong enough to predict who will have a heart attack. Unfortunately,
there are numerous plausible alternative causes including, but not limited
to, cigarette smoking, high blood pressure, genetics, inflammatory conditions
of the coronary arterial wall, and dozens upon dozens of other potentially
cause-and-effect relationships.
Exercise physiology professors are aware
of the pitfalls in publishing inappropriate conclusions in their research.
However, few of the professors appear to understand the pitfalls in drawing
conclusions about cause and effect. The most obvious pitfall occurs
when lecturing on risk factors for heart disease. It extends to drawing
conclusions that may lead the professor to seeing covariation that is not
there. Chapman (6) has referred to this error in
thinking as illusory correlation. The error stems from many factors;
the most important of which is someone (such as a colleague or a professor)
being told about the degree of relationship to expect (5).
Students, colleagues, and readers who are told what to think are predisposed
to see covariation that doesn’t exist.
“Learning without thought is labor
lost. Thought without learning is intellectual death.” -- Confucius
There is also the pitfall of “the before-after
argument” where professors fail to acknowledge coincidence and natural
causes as plausible explanations for an assumed relationship between the
variables. The problem stems from the observation that individuals
who have had heart attacks may have elevated cholesterol. In other
words, the research (or professor) argues that the presence of the high
cholesterol is responsible for the heart attack. For all anyone knows,
the heart attack may have resulted from one of a dozen different variables
or a complex combination of anyone of them with other known or unknown
variables. Interestingly, most exercise physiologists probably understand
this point, but it is not widely believed or taught.
What needs explanation is why so many exercise
physiologists persist in teaching questionable beliefs as absolute statements
of fact. Part of the reason is clear at the outset. Exercise
physiologists are not taught how to teach. Rather, it is assumed
that anyone with the doctorate degree can teach. Erroneous beliefs
like this plague the teaching profession. They are not all professionals
interesting in teaching! Instead of providing their students with
clear information that would enable them to “think” better, it is easier
(and possibly deliberate) to argue the status quo points of view.
This is a tragedy since it fails to support straight thinking that is required
for good teaching. It is likely the reader has read about similar
views elsewhere.
According to Gilovich (7),
“The most likely reason for the excessive influence of confirmatory information
is that it is easier to deal with cognitively.” This view in itself
is a sufficient reason to move forward in a better understanding of the
implications of confirming one’s bias. For example, once information
is mis-identified yet perceived as a real association, it is integrated
and explained in accordance with the exercise physiologist’s prior thinking,
theories, and beliefs. Unfortunately, since exercise physiologists
are taught very early during their academic programs that high cholesterol
causes heart attacks, their entrenched beliefs shape their thinking and
teaching in the face of evidence that states otherwise. As previously
stated, on closer inspection, it is clear that the adoption of information
as undisputed facts discredits the notion of straight thinking.
“I know it is true that high cholesterol
causes heart attacks.” “I know that regular exercise lowers blood
pressure.” “I am convinced that cardiac rehabilitation results in
regression of coronary artery disease.” What these statements have
in common is that students often cite them after hearing the statements
made by professors in support of their beliefs. “I am convinced that
the sports massage increases running economy.” I know somebody who
is trained in meditation, and who can voluntarily lower oxygen consumption
during steady state submaximal exercise.” These statements represent
the person’s belief, which is either based on good evidence or a conviction
that the beliefs are correct.
Although many of the beliefs are based
on good intentions, straight thinking and good teaching require more.
Believing strongly in a popular relationship between two variables does
not constitute the scientific method. As Gilovich noted in his book
(7), How We Know What Isn’t So, “People exhibit
a parallel tendency to focus on positive or confirming instances when they
gather, rather than simply evaluate information relevant to a given belief
or hypothesis.” The end result is that people, including students
and professors, do not use all the scientific information available to
them. The hazards of drawing conclusions solely from information
that confirms one’s beliefs can be seen in the passing of mis-information.
Thus, students who are given only one side of the relationship between
two variables aren’t likely to think clearly and in a factually correct
manner. This problem of hidden or absent data is a major discrimination
against the true effectiveness of college teaching.
Another, often-neglected point is the treatment
of information as though it is “the” way and the “only way” to think about
what is important research, the future of exercise physiology, and so forth.
Hence, to think differently is viewed as completely unjustified or potentially
closed-minded. How dare you think differently! On closer inspection,
though, the question of whether any idea has the right to not be critically
scrutinized is is often unnoticed and complicated by the assumption of
knowledge associated with advanced degrees or academic positions.
Obviously the issue is complex because it is appropriate to associate an
increased understanding of new information with advanced study. Yet,
it is entirely inappropriate behavior to not struggle with the notion of
what is right regardless of academic distinction and/or influence.
Exercise physiologists are justified in
allowing their beliefs and instincts to influence their skepticism of those
who have built their careers on the backs of possibilities with and without
serious merit. All this is to say that when a doctorate prepared
exercise physiologist has declared what is the appearance of the only desirable
research direction and, thus the only means to advance exercise physiology,
it is plausible that the word dysfunctional may be appropriately applied
without bias. An interesting example of the expectations of some
who may not even be exercise physiologists yet profess nonetheless to write
about exercise physiology is the idea that exercise physiologists are suppose
to harness new technologies including biochemical techniques, radioisotope
and imaging technologies that enable “gene-expression profiling to assess
gene-expression patterns in cells and…determine changes due to…physiological…interventions.”
(8)
The point here is not that research designed
to identify the genes associated with a disorder may correct the condition
or that such technology is bad. Rather, the question is whether the
responsibility of exercise physiologists and the profession of exercise
physiology are only important if it is dedicated to scientific and health-related
problems. It may be time within the field of exercise physiology
to carve out its own view of usefulness within the public sector.
For certain, the future of exercise physiology is not dependent upon “…laboratory
training that spans molecular biology through integrative systems physiology….”
(8)
The commonsense psychology is obvious.
The future of exercise physiology lies not in providing a voice to the
NIH and other funding agencies. It’s future exists with an understanding
of the infrastructure necessary to professionalize exercise physiology.
Research per se, however important, simply isn’t enough to ensure that
the emerging profession of exercise physiology (vs. the traditional thinking
of a scientific discipline) survives. It is time to emphasize professionalism
along with the emphasis on research because the false consensus of the
latter cannot establish a profession. Unfortunately, the collective
inexperience of exercise physiologists in understanding this point is partly
responsible for the present reality of undesirable financial conditions
and other employment issues. Why so many fail to appreciate this
point is a serious question that needs answering.
To be fair, all research is important.
But, the tendency to state that only one type of research is relevant is
questionable. Explaining this to PhDs isn’t easy and, in fact, is
contrary to the bloodletting that is part of doctorate training.
Simply stated, the aura of plausibility, that is, exercise physiologists
who believe a particular way because their mentors taught them to believe
a certain point of view, is important but it shouldn’t control one’s thinking.
Similarly, there is the issue of why aren’t more college professors teaching
students about the fallibility of marketed “scientific” products (i.e.,
the published manuscript). The emphasis on whatever is published
must be right is wrong for all kinds of reasons but, in particular, because
publications are in reality products marketed for specific publications.
The effectiveness of the author bears directly on promotion and tenure
and, thus the net effect is generally positive. Indeed, it is not
always positive though!
The continued problem of receiving filtered
thinking is compounded by the lack of having learned how to think about
alternative issues, possibilities, and strategies. At first glance,
a person may say “yes, but it is easily corrected”. The statement
certainly would not be endorsed by anyone who has taken the time to study
the problem. To start thinking about how to think, to start teaching
about how to think, and to start thinking therefore about the future of
clear thinking and, in particular, as it relates to exercise physiology,
the next step is to grasp what we cannot grasp which is embedded in the
following piece:
“My eyes already touch the sunny
hill, going far ahead of the road I have begun. So we are grasped
by what we cannot grasp; it has its inner light, even from a distance –
and changes us, even if we do not reach it, into something else, which,
hardly sensing it, we already are; a gesture waves us on, answering our
own wave.” -- Rainer Maria Rilke
It is important, then, to appreciate that
exercise physiology students must be taught how to uncover similarities
and differences between good and not so good research findings. Students
must be taught how to recognize contradictions and inconsistencies in the
authors’ sentences, paragraphs, and conclusions. The oversimplifications
of findings or the publishing of findings that are easily accepted and
the conclusions from published work that are not justified without having
evaluated the credibility of sources quoted do not constitute critical
thinking. Exercise physiologists, as teachers, therefore, must be
held accountable for increasing the students’ ability to think.
For example, do something in class that
will encourage the students to remember what is important. Tell a
joke that approximates the reality of the lecture content. Similarly,
alternating fast and slow speech may help break up the lecture into parts
that cause students to listen more closely and hence think more.
Don’t assume students automatically can engage in problem solving or that
they can reason correctly. Years of bad teaching or no teaching have
left many students hopelessly lost about how to think. Teach students
how to analyze problems and issues that require the development of specific
strategies to “read between the lines” and, thus to get at what the author(s)
didn’t say in the manuscript. Bear in mind however, getting students
to a “constructive discontent” point of view isn’t easy. In most
classrooms, it is a major hurdle to jump because students have difficulty
in questioning the authority of science. Students interpret published
articles as “the” information; the author(s) are “the” experts and so students
are not likely to question the content. Knowing how to probe into
whether the content is sound or whether the conclusions are correct isn’t
taught because, generally, professors do not challenge research findings
in front of their students. So, learning to think isn’t something
done out loud in class. Students are not usually afforded the opportunity
to hear the professor think out loud. Paul (9)
believes they should think out loud. Thinking aloud in front of students
encourages an understanding of how to think through problems. Stated
somewhat differently, but equally as effective is Miller’s point of view
(10):
“The teacher who questions, who
tries to find relationships, differences, and analogies, who shows facility
in simplifying, elaborating, substituting, varying, and combining; the
teacher who shows enthusiasm and appreciation for discovery; the teacher
who demonstrates imagination and curiosity and encourages these qualities
in others – this teacher by his own attitude and intensity produces conditions
in which students can be as creative as they are able. This teacher
awakens students’ capabilities.”
Straight thinking is helpful in recognizing
shared bias by trying to identify the main point and, hence the relevance
of all that surrounds an issue. Sadly, too few professors and most
students fail to understand this point and, thus the roots of poor thinking
lead directly to a lack of open-mindedness. Professors and their
students must not only be able, but also be encouraged, to consider the
strengths and weaknesses of opposing points of view. Students live
in a world where they need to construct meaning from ideas based on assumptions
differing from their own. In short, they have an undeniable right
to develop good thinking skills but, if the professor isn’t a skeptic him-
or herself, students are not likely to learn how to think since it requires
a new way of approaching health, fitness, and rehabilitation problems.
Professors have the responsibility to prepare
and assist students to learn what has been learned and to continue the
search for truth. Dispensing information by itself has little to
nothing to do with how to think. Professors understand this point,
but there isn’t an organizational analysis of the problem and, yet the
professor is exactly the person who should create an environment in which
genuine learning can occur. Why it isn’t reality is part of the continuous
problem of lack of emphasis on professionalism, in general, and the lack
of attention to integrity in thought and action, specifically. In
short, it is utmost important that all the evidence is analyzed, including
information that may demonstrate that the original evidence was flawed.
Professors must help their students to distinguish between the significant
and the trivial, to check authoritative sources for information, and to
keep in check the tendency to place unbounded faith in views expressed
by others.
Unfortunately, professors seldom teach
students about suppression of evidence and the failure to consider both
sides of an issue even when failure to do so is dishonest, intolerable,
and unethical (11). Scientific misconduct is more
than just mis-representing published data; it is also the underreporting
of scientific information during the teaching of a subject. Students
simply cannot make good judgments about the validity of an idea or the
results of a particular study without an adequate account of all the evidence.
Thus, to permit an informed judgment about the validity of published data,
it is important that “disappointing” or “uninteresting” results are given
equal analysis. It is surprising that so few exercise physiologists
have made an attempt to help their students make better decisions, particularly
with regard to the tendency to dichotomize research into those with positive
results and those with negative results. Students should understand
that research manuscripts ought to be interpreted on the basis of whether
they have been well executed, not whether they have positive results (12).
To help students think critically about
research findings, professors will have to “read more and consult a variety
of books and/or journals….they will no longer be able to teach the popular
(or biased) belief, however well-founded. Instead, teachers will
teach students how to think and to scrutinize old and/or contemporary ideas
for new insights” (13). The problem is that there
seems to be an assumption that professors are predisposed mentally and
psychologically to be critical thinkers. Shaking the foundation of
the professor’s education (or the lack of it) to think critically is a
challenge to the integrity of the doctorate programs of study. Yet,
there seems to be no end to the number of PhD exercise physiologists who
profess to be excellent teachers. When asked, “what-if “ questions
or the “it’s as if” statements, professors just don’t get it. Their
reply is, “But, stroke volume always plateaus at 50% of VO2 max” or “but,
CO2 rebreathing is problematic….” They can’t get beyond what they
were taught and, seemingly, are expected to teach, and yet they must learn
to shift perspectives on data and/or events to move on for the benefit
of the emerging profession.
This kind of behavior is common, but it
is not right. The lack of knowing how to teach is always a transgression
against the methods of teaching, never purposely against the body of knowledge.
Perpetrators always think they know how to teach and, therefore, do so
without troubling themselves with their own misconduct. The reason
is obvious enough. Most of us and, in particular, in my generation,
the business of how to teach and to conduct critical thinking lectures
didn’t exist. No one responded to the notion that the doctoral students
would be unprepared to teach. In truth, this seems exactly to have
been borne out. The reluctance to talk about it is a form of self-protection
and a process that becomes the standard. No one is interested in
becoming a whistleblower, and so the legitimacy of the academic programs
is not challenged.
Clearly, the credibility of the doctorate
programs are universally accepted; an idea that strikes at the heart of
the values of an education. While an excellent education is the basis
from which to understand honest errors and even deception, the lack of
teaching how to think undermines the entire set of values on which the
educational enterprise rests. Anyone who attends a college or university
is at risk of not knowing how to think if professors are not taught the
conduct, value, and mechanisms of straight thinking. Having said
this, it is equally noteworthy that one half of the top 50 research institutions
in the United States have had fraud investigations (14).
While fraud, alleged fraud, and deception in teaching are not new in science
or in exercise physiology, the point is that even unintentional mis-representation
of lecture content is unacceptable. As a result, it reasonable to
conclude that college professors of exercise physiology should adhere to
the following principles of fairness and responsible reporting of content
and research findings during course lectures. Professors should:
1. Be aware of the importance
of providing vigorous leadership in the pursuit of critical thinking.
2. Ensure that every lecture is presented
with absolute confidence that the content is presented with all issues,
that is, the extent to which is possible.
3. Treat alternative thinking with equal
respect to the contemporary view by colleagues.
4. Be sensitive to the students’ views
on issues that differ from the text and/or lecture notes.
5. Seek to always present course content
and the interpretation of research data with maximum objectivity.
Professional integrity in teaching is the
act of maintaining excellent and critically reflective teaching standards
despite the need to present disappointing data or controversial results.
To maximize the learning process, the presentation of “all” the data versus
“some” or “a biased version” of the data by the professor must be an integral
part of the professors’ job and the students’ educational program.
Similarly, authors who publish research articles need to present the same
disappointing data to maintain the right view of science when read by students,
colleagues, and others. Take, as an example, the work of Benson et
al. (15) who apparently believed that their subjects
could control the autonomic nervous system during exercise. Interestingly,
this is a common belief held by many exercise physiologists. They
seem to believe that by practicing the elicitation of the relaxation response
at rest it can then be elicited, perhaps, equally as easily during exercise
and therefore decrease oxygen consumption (VO2). The reduction in
VO2 is interpreted as an increase in running economy, given that the same
amount of work can be done with less oxygen (and hence less energy).
The problem is that there are only a few studies across several decades
of research in this area that have demonstrated a reduction in submaximal
steady state oxygen consumption (16).
Benson’s work was published in 1978 in
the Journal of Human Stress. The title of the manuscript is
“Decreased VO2 consumption during exercise with elicitation of the relaxation
response.” It is easy to imagine that athletes can learn to control
the physiologic response to exercise. The down side, however, is
that athletes and others may have been misled. The illusion of the
mind possessing more control over the physiology of exercise than actually
is the case is a problem. The fact that the title appears to market
the study and, of course, the fact that the authors are nationally known
scholars appears to help position the manuscript for publication.
Clearly, publishing is important and Harvard is no exception with its unambiguous
pressures to publish.
So, what is the problem? The title
is too leading. There is therefore the possibility that the title, stated,
in effect, as an alternative hypothesis, sets the readers’ acceptance for
an outcome that may not be correct. Without getting too involved
in a critical analysis of the paper, several factors are obvious problems:
First, the authors used a one-tailed paired t-test to compare the mean
values of the experimental period with those of the control periods.
Sprinthall (17) states, “The advantage is that we do
not have to obtain as high a t value to reject the null hypothesis as we
do when using the two-tail t table.” Although the calculation of
a one-tail t test is identical to that of the two-tail t test, the distinction
between the two t-tests is that the one-tail t-test can demonstrate statistical
significance when otherwise a significant difference would not exist.
In fact, after re-calculating the VO2 data presented in Table 1 of Benson
and colleagues’ study, using the more appropriate analysis of variance
with repeated measures test, there is no statistical difference between
the treatment condition and the two controls. In other words, the
subjects’ elicitation of the relaxation response during the fixed submaximal
exercise did not result in a decrease in VO2. As difficult as it
may be to point out, there is also the ethical question of whether researchers
suddenly switch to the one-tail t test just to get the rejection of the
null hypothesis.
Second, there is the possibility that certain
data were not presented to allow for an easier support of the basic premise
that the subjects’ were able to decrease VO2 and, therefore, increase exercise
economy. This point is obvious given that the authors report no change
in respiratory quotient (RQ), but failed to report whether the treatment
had an effect on the volume of carbon dioxide produced (VCO2). Given
that RQ and, in actuality, RER (i.e., respiratory exchange ratio) is determined
by dividing VCO2 by VO2, the authors had the VCO2 data for each of the
three conditions. Failure to report the data raises doubt that the
treatment had any significant effect on VCO2. From the authors’ view,
another non-significant result wouldn’t sit as well as the idea of one
significant change (i.e., VO2) in three variables (with RQ and heart rate
staying the same). That is, had the VCO2 values been reported with
no change and, therefore, the no change in the product of VCO2, that is,
expired ventilation (VE) and the fractional expired carbon dioxide (FECO2),
the idea of VO2 being significant would not have been as easily accepted
by the reader. While these comments are speculations, they provide
information about the manner in which the manuscript was marketed for publication,
which is not too dissimilar from other such scientific publications.
This is exactly the reason students need to learn how to read between the
lines to determine the degree to which they want to embrace the conclusions
of a particular research article.
Consider the data published by Abdulhadi
and colleagues (18) in the Archives of Physical Medicine
and Rehabilitation journal. The conclusion presented in the 1996
article appears incorrect. For example, the authors concluded “A
statistically significant finding in this study was the improvement in
oxygen cost when either a half-inch or a one-inch shoe-lift was added to
the contralateral foot of an immobilized extended knee.” The authors’
results are based on an overall average walking efficiency (VO2, ml/kg/m)
at a comfortable walking speed (CWS) for four different conditions: a)
normal walking with no knee immobilizer and no shoe-lift; b) knee immobilizer
but no shoe-lift; c) knee immobilizer and half-inch shoe-lift; and d) knee
immobilizer and one-inch shoe-lift. The statistical analysis demonstrated
that the oxygen cost for walking with the knee immobilized unilaterally
in full extension was significantly greater by an average of 20% compared
to normal walking (p = .002). This finding seems reasonable given
the increased effort to swing the immobilized, fully extended lower limb
forward in the swing phase of the subjects’ walking. But, then, the
authors state that oxygen cost was significantly less (11% vs. 20% above
that of normal walking) for the half-inch shoe-lift. The question,
here, is whether the authors are comparing the half-inch shoe-lift to normal
walking or to the knee immobilized condition without a shoe-lift?
Obviously, as the statistical results suggest, if the significant reduction
with the half-inch shoe-lift is compared to normal walking without the
knee immobilized, then the results do not support the conclusions.
Similarly, this is true for the comparison of the one-inch shoe-lift (where
a significant reduction in oxygen cost was reported to be 12% vs. 20% above
that of normal walking).
Without corroborative evidence from other
published articles, the findings in the two studies just described appear
questionable. This is straightforward logic. To write a research
study, the authors must be persuasive as well as correct to persuade the
readers to take it seriously. It is the readers’ responsibility to
judge whether the content in the published article makes sense. Questions
often used by readers to determine in what way or to what degree an article
makes sense may include some of the following:
1. What part of the article doesn’t
make sense? Is it the introduction, results, discussion, and/or conclusion?
2. What is the author saying or trying
hard not to say?
3. Are the references used in the introduction
good and/or appropriate for the purpose of the study?
4. Are additional references used in the
explanation of the findings?
5. Are there other reasons for the findings
the author has not discussed?
6. Have the authors offered an opportunity
to view the data from a different perspective or does the article appear
to be a marketed product?
7. What are the good and bad parts of
the article?
8. Is the article organized and, to what
extent, does its organization (or the lack of it) influence the readers’
support of (or failure to support) the authors’ conclusions?
9. How do the findings change the readers’
work and professional implications?
10. What are the consequences of not believing
the authors’ findings?
11. What kind of statistical analysis
was used in the study?
12. What are the consequences of having
used the wrong type of statistics?
13. What are the implications of too few
subjects in the study?
14. Have the authors achieve their purpose
using traditionally recognized principles of scientific research and writing?
15. Have the authors published similar
articles, and do the articles’ findings agree or disagree?
16. Are the authors’ data presented to
allow the readers the opportunity to analyze them?
17. Is the study’s design too complex
and unnecessary for illuminating the research question?
18. Are there good reasons to believe
that the findings are based on honest claims?
19. How could the readers evaluate the
findings in the study?
20. Have the authors distorted the results?
21. What are the implications for the
findings supporting the granting agency?
22. Are the opposing views adequately
addressed?
23. What is left unexplained in the study?
24. How has the study changed the readers’
thinking?
25. How can the reader reconcile what
has been concluded in the article?
These are just some of the questions adapted
from Paul’s (9) thinking about the logic of persuasive
writing. Clearly, research writing is a form of persuasive writing.
Authors are required (if they want anyone to take the article seriously)
to entertain the questions, ideas, relationships, and claims that are used
to support their work. Their interpretation of the data must be sound.
Students need to understand the steps used by the authors to generate the
findings and the implications. Serious flaws in thinking result from
misleading ideas presented in published articles. Students need skills
to understand the “scientific method” and the “reasons” why authors publish
articles as they do. Professors ought to devise scenarios within
the classroom for the cultivation of fair-mindedness and intellectual integrity.
To achieve this end, however, professors should teach their students how
to think independently. To become aware of this important directive,
professors will need to become more skeptical of pseudo critical thinking
that encourages flawed thinking.
References
1. Sagan, C. (1987).
The fine art of baloney. Parade. February. p. 12.
2. Daniel, E. L. (1998).
Taking sides: clashing views on controversial issues in health and society.
Guilford, Connecticut: Dushkin/McGraw-Hill.
3. Ruchlis, H. (1990).
Clear thinking: a practical introduction. Buffalo. New York: Prometheus
Books.
4. Glenberg, A. M., Wilkinson,
A. C., & Epstein, W. (1982). The illusion of knowing: failure
in the self-assessment of comprehension. Memory & Cognition. 10:597-602.
5. Zechmeister, E. B.
& Johnson, J. E. (1992). Critical thinking: a functional approach.
Pacific Grove, California: Brooks/Cole Publishing Company.
6. Chapman, L.J. (1967).
Illusory correlation in observational report. Journal of Verbal Learning
and Verbal Behavior. 6:151-155.
7. Gilovich, T. (1991).
How we know what isn’t so: the fallibility of human reason in everyday
life. New York, New York: The Free Press.
8. Baldwin, K.M. (2000).
Research in the exercise sciences: where do we go from here? Journal of
Applied Physiology. 88:332-336.
9. Paul, R. W. (1993).
Critical thinking: how to prepare students for a rapidly changing world.
Sana Rosa, CA: Foundation for Critical Thinking.
10. Miller, D. M. (1963).
Toward creative thinking. In Toward Excellence in College Teaching by Earl
V. Pullias et al., Dubuque, Iowa: Wm. C. Brown Company, pp. 61-69.
11. Lockhart, A. (1963).
Toward critical thinking. In Toward Excellence in College Teaching
by Earl V. Pullias et al., Dubuque, Iowa: Wm. C. Brown Company, pp. 49-60.
12. Chalmers, I. (1990).
Underreporting research is scientific misconduct. Journal of American Medical
Association. 263:1405-1408.
13. Boone, T. (1995).
What critical thinking may mean to the student and the teacher. College
Student Journal. 29:30-33
14. Holthaus, D. (1988).
Research fraud often goes undetected. Hospitals. 62:87.
15. Benson, H., Dryer,
T. & Hartley, H. (1978). Decreased VO2 consumption during exercise
with elicitation of the relaxation response. Journal of Human Stress. 4:
38-42.
16. Caird, S. J., McKenzie,
A.D., & Sleivert, G. G. (1999). Biofeedback and relaxation techniques
improve running economy in sub-elite long distance runners. Medicine &
Science in Sports & Exercise. 31:717-722.
17. Sprinthall, R. C.
(1990). Basic statistical analysis. Englewood Cliffs, New Jersey: Prentice
Hall.
18. Abdulhadi, H. M.,
Kerrigan, C., & LaRaia, P. J. (1996). Contralateral shoe-lift: effect
on oxygen cost of walking with an immobilized knee. Archives of Physical
Medicine and Rehabilitation. 77:670-672.
Acknowledgment:
I
am especially grateful to Dr. Larry Birnbaum, Chair of the Department of
Clinical Laboratory Science, for guidance and insight into the nature of
critical thinking and for his perceptive editorial comments which helped
to make this a better article.
Copyright
©1997-2001
American
Society of Exercise Physiologists
All
Rights Reserved.
ASEP
Table of Contents
Questions/comments
