The Future of Assessments

The future of assessments the missing half

I was sitting in my room, the world dark outside my window. Thoughts of watching a sci-fi show I like were interspersed with re-focusing on the task at hand, a project or idea for Founders School, when an idea struck me. We live today with a view of the future of education, infinite personalized generated lessons, but I have yet to see anything about the future of assessments.

Alpha and Founders School have adaptive computer software, generated lessons, AI at the forefront of their philosophy, all pieces of my favorite media, but don't have a core piece of this future. Measuring creative thinking. (From what I know and can find the closest is DOK4's and this isn't situational thinking)

In Ender's game, Red Rising, The Hunger Games, Star Trek, Ready Player One each has a scene where the paradigm shifts for the main character because they change the paradigm of a problem.

In the real world this is true too, it is Elon realizing that the cost of rocketships was not tied to the cost of materials to make rocket ships. It is Edwin Land reading every book on light in the NYPL to make instant photography really instant. It is Henry Ford realizing charcoal, a waste product from the sawmill for Ford cars, could be a product and business in itself!

So how is it tested? Ender, Darrow, Katniss

In Ender's game he is given a simulation tapped directly into his brain that is generated as he goes. His character is a little mouse, and he begins down the path in the game. As he walks along he comes across a giant ogre blocking his path. This guard growls out an ultimatum and offers two cups. "One is poison, one is not. Guess right and I take you to Fairyland."

Ender, knowing that he can rejoin and go again, so decides quickly and takes the 50/50. He jumps into the left cup and dies. Unemotional, he rejoins and jumps into the right cup only to die again.

Now he's frustrated, he tries again and again to choose the un-poisoned cup only to die.

Eventually he loads it up a last time, he approaches the giant, is given the ultimatum, and decides to take neither cup. He instead jumps up onto the cup and attacks the giant.

With a huge thud the giant falls to the ground, and the problem is passed.

In another story, Red Rising, a miner of the lowest social class is saved from dying and given a chance instead to join the rebellion. He agrees, and is given a final test that determines whether or not he is fit to join said rebellion. Dancer, the leader of these rebels offers him two cards, one with a scythe representing death, the other a sun offering him a place with the rebels. The miner Darrow guesses both cards are scythes so he eats the card he pulls to change the paradigm so it is his word against Dancer's.

In a crowd favorite movie, Hunger Games, there is no such two options test, just the skills assessment before the games. This skills demonstration is a chance to get a high-rating and with it sponsors and fans that may save her life in the arena. Fighting without sponsors is hugely negative to her chances of getting back to her family. She is ignored by the assessors so she shoots an arrow between them and into the decoration apple of the pig they're about to eat.

She was awarded the highest score that year of 11/12

Has this been made for schools already? no

If Founders School is the future of founders, then it only follows they would have the future of assessments to measure their performance especially on this paradigm shifting thinking. The kind of thinking that changes the world. With that realization I asked, has this been made for schools already?

The answer is no, but there are some one time tests typically used in labs.

The age of the captain 26 sheep, 10 goats

One is testing for kids' ability to identify 'un-solvable' problems and is referred to as the age of the captain study. In it you ask kids 'A captain has 26 sheep and 10 goats, how old is the captain?' within a set of other questions that are possible to answer with the information given. The test is whether or not kids can identify what's improbable to answer correctly without more information.

This identification and willingness to admit the low possibility of guessing right is the first step to changing the paradigm. Ender identified both obvious paths were traps, same as Darrow, only then could they ask "How can I change the paradigm?"

The nine dot problem since the 1930s

Since the 1930's there's even been a nine dot problem where the goal is connecting the dots without crossing the line you are connecting them with. The solution? Draw outside the box.

There are puzzles like making all 8 coins touch three others shown as a 2d plane, but then the answer is stacking them, but this is not measured or analyzed well.

PISA, 2012 buttons that do nothing

A more recent instance of this kind of assessment I found was in 2012, there was a test done by PISA which recorded students' actions as they tried to identify and learn an unfamiliar interface where some of the buttons didn't work. This problem solving test was meant to analyze a student's process of isolating variables and trying things outside of what's visually obvious i.e. clicking hidden buttons.

Matchstick arithmetic Ohlsson

There are also some scientists like Ohlsson who have been studying this kind of creative thinking using tests like the matchstick arithmetic test where people are asked to move one matchstick to balance an equation. While most people try changing the numbers, moving a matchstick to the operator like the minus sign into a plus sign is what gets the person success, that or just throwing the matchsticks away.

Each test is one time use the big issue

The big issue though, is that each test of these is one time use. When a student finds a way through, or talks to their friends after, the path is solved.

This is a truth created by the technology of past times, but does not need to be true anymore. Generative AI can create infinite novel situations to assess students on.

So I built one the MVP

So, I have built an MVP assessment demonstrating some of the tests that encourage mapping constraints, functions, and chunks to find novel solutions. I propose using these kinds of tests to encourage and assess kids ability to shift the paradigm because after all what shifting the paradigm is, is being limitless.

An advanced version could call APIs to create new situations for each student based on their knowledge graph, interest graph, and any other data inputted by their guides or assessors.