JonStarbuck.co.uk

Data is not the plural of anecdote, or to put it another way an old Yiddish saying goes “for example is not proof”.

Understanding the difference between good science, bad science and anecdote is vital for intellectual self defence, sorting lies from truth, real from unreal.

Data is what scientists try to create, it is the results from their experiments. If the scientist has been doing their job properly then the data is of good quality, the experiment is well documented, the results are repeatable and the margin of error well understood. There is nothing secret or “commercially sensitive” about data, being able to review it and judge if the test was fair and the results believable is the essence of peer review and science, not being able to makes a nonsense of it.

Bad science is the crutch of, among many others, climate-change-deniers and people who peddle “alternative medicine”, and unfortunately it also exists… Very worth a read on the subject is Robert L. Park’s “Seven Warning Signs Of Bogus Science” where he lists his seven warning signs that some piece of new science or miraculous new scientific discovery is, in fact, baloney.

In his book, “Voodoo Science” he tells the tale of many such endeavours and how well meaning people, and even scientists, can quickly go from mere foolishness, through self deception and into outright fraud.

Anecdotes are what “alternative medicine” practitioners, charlatans, scammers and con-merchants the world over operate with. It is data which is badly collected in uncontrolled conditions, it is unverifiable and un-checkable, you simply have to take their word for it that the “data” is true or of good quality.

No matter how many anecdotes you collect it is not a substitute for real data.

In Robert L. Park’s warning sign #4 is as follows:

If modern science has learned anything in the past century, it is to distrust anecdotal evidence. Because anecdotes have a very strong emotional impact, they serve to keep superstitious beliefs alive in an age of science. The most important discovery of modern medicine is not vaccines or antibiotics, it is the randomized double-blind test, by means of which we know what works and what doesn’t. Contrary to the saying, “data” is not the plural of “anecdote.”

Other definitions of Anecdotal Evidence:

http://www.skepdic.com/testimon.html

Anecdotes are unreliable for various reasons. Stories are prone to contamination by beliefs, later experiences, feedback, selective attention to details, and so on. Most stories get distorted in the telling and the retelling. Events get exaggerated. Time sequences get confused. Details get muddled. Memories are imperfect and selective; they are often filled in after the fact. People misinterpret their experiences. Experiences are conditioned by biases, memories, and beliefs, so people’s perceptions might not be accurate. Most people aren’t expecting to be deceived, so they may not be aware of deceptions that others might engage in. Some people make up stories. Some stories are delusions. Sometimes events are inappropriately deemed psychic simply because they seem improbable when they might not be that improbable after all. In short, anecdotes are inherently problematic and are usually impossible to test for accuracy.

http://en.wikipedia.org/wiki/Anecdotal_evidence

The expression anecdotal evidence has two distinct meanings.

(1) Evidence in the form of an anecdote or hearsay is called anecdotal if there is doubt about its veracity; the evidence itself is considered untrustworthy.

(2) Evidence, which may itself be true and verifiable, used to deduce a conclusion which does not follow from it, usually by generalizing from an insufficient amount of evidence. For example “my grandfather smoked like a chimney and died healthy in a car crash at the age of 99″ does not disprove the proposition that “smoking markedly increases the probability of cancer and heart disease at a relatively early age”. In this case, the evidence may itself be true, but does not warrant the conclusion.

In both cases the conclusion is unreliable; it may not be untrue, but it doesn’t follow from the “evidence”.

http://www.wisegeek.com/what-is-anecdotal-evidence.htm

Anecdotal evidence has several definitions, which usually relate to how certain types of evidence cannot be used to logically conclude something. We see examples of this type of evidence all the time in commercials. A person tells us how their breath feels fresher after using a certain brand of toothpaste, or people testify to the clearing of their acne as a result of special products. Anecdotal evidence is often used in place of clinical or scientific evidence, and may completely ignore research or harder evidence that points to an opposite conclusion.