How do modern cognitive assessments measure more than just intelligence?

My friend commented the other day that an accurate reading of someone’s intelligence would be taken from one photograph of them – them smiling (obviously), terrible lighting, one of them blinking in the picture and a big red mark on their forehead from having stayed up the night before studying for their test (trying to be as intelligent as possible). My friend was absolutely correct in her assessment that such a dated way of measuring a person’s intelligence is largely still in use today.

The number given to represent a person’s intelligence has an unusually long shelf life given that by modern standards of scientific measurement, the number given by an IQ test only gives a limited indication of a person’s cognitive abilities. To accept that one number can give an adequate representation of a person’s intelligence is to accept that one chapter of a person’s life can give an adequate sense of the whole book. Obviously, there are too many variables that can affect how a person does on an IQ test to be given as the sole criteria for judging a person’s intelligence. The results can be skewed by a variety of factors including test anxiety, cultural bias, and a host of other factors that cannot be fully accounted for by current test measuring cognitive ability.

What we've been getting wrong

First, Intelligence is many things. So one number cannot possibly hope to capture all the different aspects of the full spectrum of human potential. As a field we have been patching our models of Intelligence for decades. We have added new domains, modified scoring, and attempted many different solutions. However, the core problem with our current model is that it is fundamentally flawed. Thus, the best of patches are not good enough to cover for the basic shortcomings of the model. So instead of merely using lots of electrical tape to hold together a wire that is fundamentally fraying, it would be wiser to replace the wire entirely.

Human intelligence is not a single trait but rather a number of different cognitive functions. For example, a person can be very verbally oriented and have a slow processing speed. Another person can have poor working memory but excellent spatial reasoning. Thus, intelligence cannot be easily measured or quantified. Those who attempt to do so are oversimplifying the complex traits that make up human cognition.

So what are we measuring with all these new tests of intelligence?

The domains that actually matter

A large part of modern intelligence assessment consists of looking at several domains of cognitive function. Most contemporary tools, therefore, have been developed to test individual domains. Although the IQ scores from such assessments provide a rough outline of a person’s intelligence, there is a lot of individual variation within each of the assessed domains. For clinical, as well as for work and educational purposes, tools that assess several domains of cognitive function have been widely adopted and shown a lot of potential.

The main domains most contemporary tools assess include:

• Working memory: your ability to hold and manipulate information in real time. Think of it as your mental RAM. It predicts learning outcomes more reliably than IQ in many contexts.
• Processing speed: how quickly your brain takes in and responds to information. This one tends to decline with age and is often an early signal in neurodegenerative conditions.
• Reasoning ability: both fluid reasoning (solving novel problems without prior knowledge) and crystallized reasoning (applying learned knowledge). These can diverge dramatically between individuals.
• Attention and inhibition: the ability to focus and, critically, to suppress irrelevant information. Underrated. Enormously consequential for real-world performance.
• Verbal and visual-spatial processing: how different people encode and manipulate language versus spatial relationships.

Current practice of assessment using intelligence tests to obtain a single number which attempts to describe the intelligent individual.

Why the shift actually matters in practice

This would all be irrelevant if we were limiting our discussion to quaint philosophical debates taking place in obscure university departments far far away from our own struggles to get our arms around the nature of intelligent behavior and how to measure it. But no, the yardstick against which much of our decision-making about children, about adults with brain damage, about workers who don't seem to be performing as we had hoped, is a real, existing tool, the scores from which are frequently used to make very real decisions about the lives of real people.

For individuals who have suffered from brain injuries, and are trying to recover from their injuries, an IQ score is of little assistance. On the other hand, a report from a cognitive assessment that details an individual’s strengths and weaknesses can be extremely helpful to that individual, as well as to that individual’s family, friends, and rehabilitation team. The information that is gained from a cognitive assessment can highlight areas where an individual may need assistance with processing information received by the senses. For example, an individual who has suffered from a brain injury and was referred by their physician for a cognitive evaluation by a rehabilitation psychologist, in order to assist that individual with processing information by their senses, could find out that their verbal reasoning ability is within the normal range. However, they may find out that their processing speed is below average, and that they have difficulty with their working memory. This type of information is extremely helpful for a number of reasons.

A student who is not performing academically in school could be suffering from learning disabilities or have a processing speed that is significantly behind his or her reasoning ability. In either case, a different intervention could be required and a number of very different outcomes could be brought about by correctly identifying the problem. A good cognitive assessment can help a clinician, teacher or business owner to arrive at a number of possible interventions and to design a battery of tests that will help the individual in question to reach his or her full potential.

(As an aside, I used to dismiss the “IQ score is evil” debate as an esoteric argument amongst a few academic snobs and was blind to the real world implications of how intelligence was actually being measured. I was wrong. Stupidly, smugly wrong. The stakes are so much higher than I could have ever imagined).

A quick comparison worth keeping in mind

The honest caveat

No test of intelligence is ever going to be perfect. That is why the vast majority of intelligences tests are between 30 and 40 tasks long and administered over a period of 30-45 minutes. The best test of intelligence is one that provides a reasonably accurate measure of a child’s intelligence. Every test will fail to measure some aspect of a child’s intelligence. The key is whether the test fails to measure something important. There will be kids for whom their test results do not provide an accurate measure of their intelligence. There will be kids from cultures in which kids are not typically expected to complete tasks of the kind that are required by the tests, for whom their test results will not provide an accurate measure of their intelligence. But, for the vast majority of kids, a good test will provide a reasonably accurate measure of their intelligence.

This is not to say that a test that evaluates multiple domains and differentiates between them within those domains cannot exist. But until we create a test that can completely ‘test’ an individual’s cognitive potential over a longer period of time in different contexts (to mitigate for test anxiety, for example, as well as to capture different cultural practices), current tools are the best that we have. But that does not mean they are perfect, and they are definitely better than many of the tools that have been used before.

As mentioned earlier, shifting from measuring a single score to a more in-depth look at a person’s performance through a profile can help paint a true picture of a person’s strengths and weaknesses. This better enables individuals and organizations alike to work with and develop individuals in a respectful and constructive manner. Most assessments today measure the different cognitive functions that individuals use to solve problems and complete tasks using given information. And, importantly, a profile of a person’s test results can provide a more realistic indication of a given individual’s ability to perform in real world situations as opposed to simply being told they have a certain score.

That felt overdue. Still does.