Many of you will already have heard of the "Dunning and Kruger Effect", a piece of psychological research which has made its way into the popular consciousness. In summary it suggests that those who are more incompetent at a particular task are also more likely to overrate their competence, since their ignorance prevents them from realising just how bad they are.
Anyway, I finally got around to reading the article, "Unskilled and Unaware of it: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self Assessments", by Justin Kruger and David Dunning, from the Journal of Personality and Social Psychology, 1999, Vol 77, No 6. Much of it is not scintillating reading, being after all an academic research paper filled with statistical jargon. However, it is more comprehensible than many similar articles and shot through with flashes of psychologist humour.
The paper reports a series of four linked studies. All were carried out on undergraduate psychology students, coerced or bribed into being experimental subjects as they are the world over. Psychology is the study of first year psychology students. In each experiment, the students were asked to perform a test - one on assessing the quality of jokes, one on grammar, and two using tests of logical reasoning. In each case they were asked to rate their own performance in relation to their peers, and where it was possible to estimate their own score. The result - the students who scored the lowest were the ones who most overestimated their ability. Those who did best, on the other hand, tended to underestimate their scores.
In a follow-up, students who did either very well or very poorly were asked to come back a couple of weeks later and assess a sample of the work of their peers, being handed a set of papers of varying standard. After marking these papers as best they could, students were asked again how they thought they had gone compared to everyone else. The least competent showed no change - they still thought they had gone really well because, of course, they didn't know enough to make a proper assessment of the papers they had been asked to examine. The most competent, on the other hand, revised their assessment of their skills upwards - because they realised they had done a lot better than most of the tests they had marked.
The third activity involved training people, then asking them to reassess. Prior to rating some of their peers' performance on a logic test, some of the poorest performing students were given a short education module on logical processes. Unsurprisingly, after this brief presentation, they were much more realistic about their own performance, because they now understood a little more about the subject.
When you think about it, none of this should be a surprise. Some people don't know enough to be aware of their own ignorance, and pronounce confidently on subjects about which they know absolutely nothing. Often such people end up running whole countries. A little exposure to the discipline in question, whether it be logic or grammar or the relative funniness of jokes, makes us aware of the yawning gaps in our knowledge and we suddenly become more circumspect. Someone, however, has to be game to point out our ignorance. This is usually easier for people to do before you become Prime Minister - say, perhaps, when you're a first year psychology student who hasn't yet grasped the basics of grammar and logical thinking. What to they teach them in those schools?
The conclusion to Kruger and Dunning's article is a classic, worth reading to the end for.
Although we feel we have done a competent job in making a strong case for this analysis, studying it empirically, and drawing out relevant implications, our thesis leaves us with one haunting worry that we cannot vanquish. That worry is that this article may contain faulty logic, methodological errors, or poor communication. Let us assure our readers that to the extent this article is imperfect, it is not a sin we have committed knowingly.
Yes folks. Kruger and Dunning fear that they may have painted a fake. Join the club, boys.
Anyway, I finally got around to reading the article, "Unskilled and Unaware of it: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self Assessments", by Justin Kruger and David Dunning, from the Journal of Personality and Social Psychology, 1999, Vol 77, No 6. Much of it is not scintillating reading, being after all an academic research paper filled with statistical jargon. However, it is more comprehensible than many similar articles and shot through with flashes of psychologist humour.
The paper reports a series of four linked studies. All were carried out on undergraduate psychology students, coerced or bribed into being experimental subjects as they are the world over. Psychology is the study of first year psychology students. In each experiment, the students were asked to perform a test - one on assessing the quality of jokes, one on grammar, and two using tests of logical reasoning. In each case they were asked to rate their own performance in relation to their peers, and where it was possible to estimate their own score. The result - the students who scored the lowest were the ones who most overestimated their ability. Those who did best, on the other hand, tended to underestimate their scores.
In a follow-up, students who did either very well or very poorly were asked to come back a couple of weeks later and assess a sample of the work of their peers, being handed a set of papers of varying standard. After marking these papers as best they could, students were asked again how they thought they had gone compared to everyone else. The least competent showed no change - they still thought they had gone really well because, of course, they didn't know enough to make a proper assessment of the papers they had been asked to examine. The most competent, on the other hand, revised their assessment of their skills upwards - because they realised they had done a lot better than most of the tests they had marked.
The third activity involved training people, then asking them to reassess. Prior to rating some of their peers' performance on a logic test, some of the poorest performing students were given a short education module on logical processes. Unsurprisingly, after this brief presentation, they were much more realistic about their own performance, because they now understood a little more about the subject.
When you think about it, none of this should be a surprise. Some people don't know enough to be aware of their own ignorance, and pronounce confidently on subjects about which they know absolutely nothing. Often such people end up running whole countries. A little exposure to the discipline in question, whether it be logic or grammar or the relative funniness of jokes, makes us aware of the yawning gaps in our knowledge and we suddenly become more circumspect. Someone, however, has to be game to point out our ignorance. This is usually easier for people to do before you become Prime Minister - say, perhaps, when you're a first year psychology student who hasn't yet grasped the basics of grammar and logical thinking. What to they teach them in those schools?
The conclusion to Kruger and Dunning's article is a classic, worth reading to the end for.
Although we feel we have done a competent job in making a strong case for this analysis, studying it empirically, and drawing out relevant implications, our thesis leaves us with one haunting worry that we cannot vanquish. That worry is that this article may contain faulty logic, methodological errors, or poor communication. Let us assure our readers that to the extent this article is imperfect, it is not a sin we have committed knowingly.
Yes folks. Kruger and Dunning fear that they may have painted a fake. Join the club, boys.
Comments