The C.R.A.P. Test
If you have read An Engineer’s Guide to Solving Problems you already are familiar with the Five Questions.
- What do you know?
- What don’t you know?
- What are the rules?
- How can you find out the stuff you don’t know?
- How do you know when you are ready to solve (or have already solved) the problem?
Today, I want to add something about the 4th question: How can you find out the stuff you don’t know? This question consumes many chapters in the book. Of course I discuss the benefits and dangers of looking for information on the web. Recently, I was introduced to the CRAP test by a really smart librarian.
In short, the CRAP test helps you assess the quality of any information you obtain, regardless of the source (online or not).
C is for currency. You have to understand whether the information is up-to-date (current) and how much that matters for your particular subject matter. For example, the American Civil War spanned the years 1861-1865. That fact can be verified multiple ways and is unlikely to change. But if instead you were trying to determine the appropriate treatment for a specific type of cancer, you might find answers changing within the past few months or even weeks. Likewise, the best local interface between two processor subsystems changes every few years. Currency of information is critical for some parts of electrical design, yet E = I*R has not changed in 100 years nor is it likely to change soon.
R is for reliability/relevance. You need to know where the information comes from and if it really applies to your situation. There are usually a bunch of secondary questions here:
- Is this a primary or secondary source?
- Are methods or references provided? (Do they show their work?)
- Who published the information? Are they reliable?
- Is the information peer-reviewed? (Don’t worry, not all good information is peer-reviewed, but if they show their work, you can do some peer-review of your own.)
- How specific is the information? Is there some bias in the information?
A is for authority. You need to know who authored this information. What is their reputation, expertise, or credentials? I personally find some writers to be quite authoritative, even outside of their formal education. But these authoritative writers show their work. They don’t hide behind smoke and mirrors. See, nothing hidden up their sleeves.
P is for Purpose or Point of View. You want to understand something about the motivation of the information source. Ask yourself, is this idea is too good to be true?
I guess I should comment here about my own purpose or point of view. Heck yes, I want to sell you a copy of my book. But I hope you find that what I put on these pages is reasonable, logical, verifiable, or otherwise passes what my wife calls the dumb-ass test.
I have listed several references below for the CRAP test. See, nothing up my sleeves.
Molly Beestrum and Kenneth Orenic, two librarians from Dominican University in River Forest Illinois, developed and first described the CRAP test. I put my post here through the CRAP test by contacting Ken Orenic, who confirmed that they had indeed authored the presentation (referenced below) which had shared this clever test with the rest of the world. He said that Ms. Beestrum had already been using this test when he first joined their library.
http://lisagoldresearch.wordpress.com/2010/09/20/the-crap-test-for-evaluating-sources/
http://www.workliteracy.com/pages/the-crap-test
http://loex2008collaborate.pbworks.com/f/Final_Wiki_Uses_In_Library_Instruction.ppt