The problem with n=1 and ‘evidence’

by admin on February 16, 2010

You often see barefoot runners responding to comments about the lack of evidence for barefoot running by doing one of two things:

  1. They point to evidence. However, when you look at it, it’s not evidence (they are really good at doing this intellectual dishonesty. For example, see how they fell for some of the research: Hook, line and sinker).
  2. They claim that they have all the evidence they need as they can run barefoot and they know some others that have successfully transitioned to barefoot. If they think this is ‘evidence’, then it is really further evidence of how they delude themselves. They need to go back to school to learn what the word ‘evidence’ means.

Within medical and health research, there is what is known as a hierarchy of evidence. At the top of the hierarchy are the meta-analyses that combine data from several prospective randomized controlled trials. At the very bottom of the hierarchy is the individual experiences. The n=1 (ie research involving one person individual experience) is not research. It is just one persons individual experiences. These experiences can be flawed or misinterpreted. Let’s look at examples and analogies:

I think everyone is clear on the evidence of the risk of harm of smoking during pregnancy, yet a number of women still smoke during pregnancy. Often when asked why they are doing it, they respond with something like, ‘My friend smoked during pregnancy and she had no problems’. That is pretty much the sort of argument that barefoot runners like to make. They have a friend that can run barefoot, therefore it must be OK, despite all the scientific evidence. Well, this kind of argument is not acceptable in the context of smoking and pregnancy, why is acceptable in the barefoot running community? I will concede that there is NO evidence that barefoot running is harmful, but there is NO evidence that it is beneficial either, despite the claim that the barefoot runners like to make – in this pregnancy and smoking analogy, the evidence for barefoot running is knowing some others that do it! That is NOT evidence.

Would you like to be operated on by a surgeon that has only done one of a particular type of procedure before? Would you be happier about it if the surgeon had a couple of other surgeons they knew who had also done the procedure? What kind of evidence is this? I assume you can see where this analogy is going…

Any number of medical and health treatments that people swear work have been shown not to work by good clinical trials. There are no good trials on if barefoot running is beneficial or not, yet the barefoot running community constantly talk about the evidence of the n=1. This kind of evidence is not acceptable elsewhere, so why is it acceptable to the barefoot running community?

How often do you see comments that go something like “I used to get all these injuries and now that I run barefoot, I don’t”. Lets dissect that kind of comment. How do you know that the injuries are less because of the barefoot running?:

  • most runners tend to go through phases of getting more injuries and then not getting injuries. It is just the way it happens. So, how do we know that a particular individual that transitions to barefoot running was not just due to enter a phase of less or no injuries. The reduction in injury could be due to this and NOT the barefoot running (but then again it may not be).
  • now that they are running barefoot, there is a good chance that they are now running on a softer surface. Maybe the lack of injuries is due to surface and nothing to do with the barefoot running (but then again it may not be).
  • maybe at the same time they transitioned to barefoot running, they also started doing more stretching exercises. The reduction in injury could be due to that and nothing to do with the barefoot running (but then again it may not be).
  • maybe they changed how many sugars that they have in a cup of coffee in the morning at the same time they started barefoot running.  Maybe that is the reason for the no injuires (but then again it incredibly unlikely to be the reason, but we can not rule it out).

Can you see where this is heading? It is often easy to blame the most obvious factor and not consider other possible intervening factors. It does not mean that the barefoot running was the reason or not the reason, it is just you cannot jump to that conclusion based on individual experiences (or are you happy for a surgeon to operate on you who has done something once and knows a few others that have also done it?).

Why are individual experiences so flawed? For those who want to explore this further, I suggest you read: Why do ineffective treatments seem helpful?. While this is written in the context of medical and health interventions, and about how an individual’s own clinical experiences can be flawed, the principle can be applied here to an individual’s experiences with what they perceive as the benefits to themselves of barefoot running. If you want to make claims about your experiences, that’s fine, but convince me first that you understand what this article is talking about. Otherwise stop making the silly claims about what you think is evidence.

{ 1 trackback }

What does the barefoot running community continually fall for this nonsense?
February 21, 2010 at 6:03 am

Comments on this entry are closed.

Previous post:

Next post: