Imagine computers that can read your mind and brain implants that control computers by thought alone.
Professor Sook-Lei Liew, Associate Professor, Neural Plasticity and Neurorehabilitation Laboratory
University of Southern California
Everybody is fascinated about the idea of potentially being able to control things just using their mind.
This technology could revolutionise everything from astronauts in space to care for the severely paralysed.
He’ll save his life by turning himself into the world’s first cyborg.
The gap between the virtual and real worlds is closing but how long till headsets and screens disappear?
This is not the real world.
And The Matrix is no longer fiction.
This is the headquarters of Varjo, a startup company based in Finland. It might look like they’ve created a typical VR experience but it’s not. Varjo has created a headset which is changing our people interact with virtual and computer-generated worlds by changing how their brains engage with them.
Here in this example we have a Volvo car based on a CAD model that was just transformed into fully photorealistic accurate replica of the actual design model.
Varjo’s ground-breaking headset is built using advanced eye tracking technology. This means only the part of the image the eye is looking at needs to be processed and with no perceivable lag, it feels realistic.
Urho Konttori, Founder and Chief Technology Office
Varjo
We design optics and displays that make it possible to see in the same resolution as your own eyes see. This means that whatever we replicate looks exactly the same as it would look like in the real world.
Photorealism is one of the first steps towards convincing our brains that computer-generated simulations are real but achieving this will also require improvements to devices called Brain Computer Interfaces or BCIs. The Varjo headset uses hardware and software from a company called Open BCI.
Professor Sook-Lei Liew, Associate Professor, Neural Plasticity and Neurorehabilitation Laboratory
University of Southern California
BCI is a brain computer interface, so it’s essentially a way for the brain to directly interface with computers by reading out signals from the brain.
The term ‘brain computer interface’ was coined in the 1970s in California. The aim was to control external objects using signals from the brain but it’s only within the last twenty years that researchers have started to pull this off.
Turn right and fire.
Today, there are two kinds of BCIs. The first type are called non-invasive and can take the form of portable devices.
Professor Sook-Lei Liew, Associate Professor, Neural Plasticity and Neurorehabilitation Laboratory
University of Southern California
Using non-invasive ways of reading out brain activity, those signals are directly translated into ways of controlling computers, whether it’s a video game or it’s a robotic arm or any sort of machine.
To make computers better at reading brain signals, scientists need to get better at understanding the brain.
Professor Sook-Lei Liew, Associate Professor, Neural Plasticity and Neurorehabilitation Laboratory
University of Southern California
You can’t put a BCI on, think to yourself, “Do I want to eat,” you know, “fried chicken or a sandwich or a salad for lunch” and then think of it and then order it online. You don’t have clear brain regions for fried chicken versus salad in the brain. If we know what the brain activity is that we’re looking for in the brain region then we can use that signal to control the BCI.
Improving the understanding of the brain could lead to the type of BCI featured in the film ‘Ready Player One’. Set in 2045, it shows much of humanity escaping the real world by using headsets which fully immerse their brains in the oasis – a near perfect virtual reality simulation. Getting the design of non-invasive BCIs right is vital to fulfilling their potential. Some find devices like headsets impractical and bulky. But scientists are bringing designs down to size.
Dr Nataliya Kos’myna, Research Scientist, Fluid Interfaces
Massachusetts Institute of Technology, Media Lab
On top you can see one of the early prototypes of attention view glasses which have sensors on the back of the ears.
Dr Nataliya Kos’myna is a technologist at MIT who is working on a series of new designs in conjunction with NASA.
Dr Nataliya Kos’myna, Research Scientist, Fluid Interfaces
Massachusetts Institute of Technology, Media Lab
We have a project with NASA about optimising performance of astronauts.
The project is using this technology to read the brain activity of astronauts on missions to the International Space Station in order to monitor their health and performance.
Dr Nataliya Kos’myna, Research Scientist, Fluid Interfaces
Massachusetts Institute of Technology, Media Lab
So it’s a suite of different variable wireless sensing devices. It can pick up brain activity, eye movements and those can be put in the form factor of a pair of glasses.
We’re gonna try on a pair of AttentivU glasses. So we can pick up the brain activity using these electrodes.
The AttentivU glasses allow someone to answer questions by simply thinking of the answer. The computer is able to understand their answer by recording their brain waves.
Dr Nataliya Kos’myna, Research Scientist, Fluid Interfaces
Massachusetts Institute of Technology, Media Lab
So, what Kareem will need to do, using only his brain activity to give an answer to this question. So, do you like it here?
Kareem
Yes.
Dr Nataliya Kos’myna, Research Scientist, Fluid Interfaces
Massachusetts Institute of Technology, Media Lab
Okay. Did we get it right? I assume, yes.
It’s a technology that offers great promise on Earth as well as in space. For example, for people suffering from neurodegenerative disorders like ALS, which causes a loss in the ability to communicate.
Dr Nataliya Kos’myna, Research Scientist, Fluid Interfaces
Massachusetts Institute of Technology, Media Lab
What we actually are doing, we are providing them with this basic communication. The care giver can ask a question to the patient and the patient can respond yes, no.
These devices also have the potential to physically alter the brain and change how it works.
Professor Sook-Lei Liew, Associate Professor, Neural Plasticity and Neurorehabilitation Laboratory
University of Southern California
Try not to think about anything so, we ask people to think about a wall or a chair and just rest your head for ten seconds, okay.
Professor Sook-Lei Liew is a neuroscientist who has been working in this field for over ten years.
Professor Sook-Lei Liew, Associate Professor, Neural Plasticity and Neurorehabilitation Laboratory
University of Southern California
Over time, if you train or practice with a BCI a lot, you can actually see changes in both the brain activity patterns that people have and in the brain structures. Using a BCI can not only replace or serve as a compensatory role for replacing function but it can actually also help to rehabilitate or restore brain networks and pathways and it’s essentially like a, like a brain training sort of tool.
One day, the world may wake up to the reality of another kind of brain computer interface. The invasive kind. When The Matrix will no longer be just a Hollywood dream. Invasive BCIs promise the future merger of brains and computers through surgical implants. It’s early days for this technology and so far testing has mostly been done on animals rather than humans. Like this monkey, Pager.
Pager simply thinks about moving his hand up or down.
With electrodes surgically inserted into his brain, he is able to play the game, Pong, simply by thinking about moving his hand up or down and sending brain signals to the computer. The experiment was conducted by Elon Musk’s company, Neuralink. Neuralink says its ultimate aim is to improve the lives of humans with impaired abilities. However, its experiments have proved controversial.
Protestors are lining up calling for Neuralink’s animal testing labs to be shut down.
The company says it has euthanised eight monkeys but insists it has followed the law at all times.
Invasive BCIs can also pose ethical and health risks for humans.
Professor Sook-Lei Liew, Associate Professor, Neural Plasticity and Neurorehabilitation Laboratory
University of Southern California
So, they actually have to cut your skull open and then implant an electrode and then put the skull back on.
Despite this, many consider these implants to be the most promising path to removing the gap between the virtual and real worlds.
Professor Sook-Lei Liew, Associate Professor, Neural Plasticity and Neurorehabilitation Laboratory
University of Southern California
The most accurate tend to be the ones that are implanted in the brain. The upside is that you are reading directly from neural populations so, the cells in the brain that are actually firing.
Invasive BCIs have the potential to integrate human biology with technology in unprecedented ways and scientists are slowly moving in this direction. In 2018, scientist, Peter Scott-Morgan, offered his own body to his profession. He suffered from motor neurone disease and allowed surgeons to implant various chips into his body in the hope of extending his life.
Peter Scott-Morgan
Whatever happens, we will learn from it and we will do it better the next time.
The pioneering operations helped him live for another four years until his death in 2022.
The brain remains one of humanity’s greatest enigmas. Technology that melds the brain with computers opens up new possibilities, highlighting both how powerful the brain is and how little it is understood. Whilst scientists are a long way from understanding its secrets, there are grounds for both hope and concern.
Urho Konttori, Founder and Chief Technology Office
Varjo
It’s a very scary thought, when you start looking into, into what you could do when you are directly interfacing the brain. Yet at the same time, you can completely understand that this is the trajectory we are on.
Hello. I’m Tom Standage, Deputy Editor at The Economist. If you’d like to learn more about this topic, click on the link opposite and if you’d like to watch more of our Now & Next series, click on the other link. Thanks for watching and don’t forget to subscribe.