The case for qualitative assessment of intelligence

A cropped photo of a white man holding a journal with a dinosaur motif on it.
Photo by Sticker Mule on Unsplash

While IQ tests work for a large portion of the population, they are less successful in properly gauging the intelligence of people whose disabilities interfere with the non-intellective aspects of modern intelligence testing; for example, the Wechsler intelligence scales (the WISC, WAIS, etc) place a premium on response speed, oral response and fine-motor skills. These requirements may make it difficult for a participant with cerebral palsy, an autism spectrum condition, ADHD or a similar disability to have their true level of intelligence identified correctly. There are also a number of subjective considerations in individual testing; if the participant connects poorly with the administrator, these social factors may exert an influence on the participant’s performance. Research already suggests that standard IQ tests may underestimate the intelligence of autistic people, especially non-speaking autistic people.

There’s nothing wrong with measuring these non-intellective cognitive tasks in order to identify what people’s strengths and weaknesses are, but they should be distinguished from tasks designed to identify a person’s learning and inferential abilities. The tasks on standard intelligence tests are relatively simple; paradoxically, some people who excel at integrating large amounts of discrete information may overthink some of the simpler questions that one encounters on tests, which may incur time penalties that subtract from their overall score. In Giftedness 101, Linda Silverman discusses this issue when assessing highly intelligent children on Wechsler tests that force the examiner to end a particular subtest after a participant fails the first set of simpler tasks. After modifying the criteria such that the discontinue criteria were no longer applied and the participants were allowed to complete all the items in the subtest, the participants were able to handle more complex problems that engaged them more so than the simpler questions did. The structure of these tests is ideal for identifying clinical cognitive issues or assessing members of the general population, but paradoxes like these may make them sub-optimal for identifying extreme intelligence if there are other indicators of it that appear throughout the testing or qualitative assessment. Again, if diagnosticians want to evaluate intelligence, then they should look for signs of reasoning and pattern recognition in ways that do not penalise them for having a disability or being methodical. If they want to identify difficulties with motor skills or processing time, then those assessments should be decoupled from assessments of the skills commonly referred to as general intelligence to avoid confusion.

The consequence is that some highly intelligent people, especially ones who have a concomitant disability, may not have their intellectual abilities correctly represented by a standard IQ test. They may end up with artificially deflated scores that disqualify them for gifted education, high-IQ societies and other spaces designed to enrich the education of highly intelligent people or connect them to people with similar learning styles. Using a qualitative model to supplement or replace standard intelligence tests will help detect people who would otherwise go unnoticed.

Some diagnosticians who are familiar with the traits of high intelligence do use qualitative assessment, but there is no systematic process by which professionals can look for these traits in routine neuropsychological examinations. For example, the late Annemarie Roeper developed a qualitative scale for professionals to look for signs of high intelligence, but they are not used in routine neuropsychological examinations. Most professionals may only have the vaguest idea of what intelligence looks like and may use the IQ score as an absolute determiner while ignoring a respondent’s case history, cultural background, behaviour and previous performance on intelligence tests. A qualitative assessment tool that correlates well with IQ testing, but that can identify highly intelligent people whose disabilities or backgrounds interfere with traditional testing, can help ensure that people who need more enriching educational options or other support related to their learning receive it. It may be helpful to look at a related practice used for people on the other extreme of the normal distribution. When diagnosticians determine whether a person has an intellectual disability, they don’t rely on an IQ score alone. Similarly, identifying the educational needs of students who are too intelligent to thrive in standard educational settings must also incorporate assessment that goes beyond a full-scale IQ, especially if this full-scale IQ is derived from a test that combines tests of intellectual ability with other cognitive skills.

I don’t think IQ tests are entirely useless, but the possibility of their producing false negatives in certain distinct populations should be a clarion call for researchers to develop a qualitative intelligence assessment that can replace or supplement traditional IQ testing.

Finally, I think Tony Kushner’s quotation from Angels in America is apropos:

…it should be the questions and shape of a life, its total complexity gathered, arranged, and considered, which matters in the end, not some stamp of salvation or damnation that disperses all the complexity into some unsatisfying little decision…

The context of the play may have been totally different, but the sentiment still fits: we cannot make decisions about people’s abilities based on a single score taken on a single day during a single point in someone’s life. Diagnosticians can, and should, use the ‘shape of people’s lives’ to assess someone’s intellectual capacity throughout the lifespan.

Further reading

Linda Silverman’s Giftedness 101 (2013) has an excellent chapter on the identification and assessment of highly intelligent children and adolescents; it's the main source I referred to when writing this article.

Henry Schlinger’s The Myth of Intelligence (2003; PDF article) argues for assessing intelligent behaviour through qualitative observations, rather than relying on IQ tests that purport to identify a singular and isolated quality called ‘intelligence’. I actually found the article after writing this, but the themes it discusses are close enough to my own that I decided to link to it.