A year ago, I was in front of Carmen, a 90-year-old woman from Barcelona. I asked her, "Carmen, what is your earliest memory?" Now, for someone who has almost seen a century unfold, this can be a difficult question. I was half expecting her to say, "Son, my earliest memory is of you asking me this question."
(Laughter)
But the truth was that it was from 1941. She was six years old, and her mother would pay another family so they could enter their house and go up to their balcony. What was particular about that balcony was that it was facing La Modelo prison. During that time, during the Spanish dictatorship in Spain, it was a political prison and her father, a doctor for the antifascist front, was a prisoner there. So the only way they could see each other was from that balcony to the window of the prison. And that was her earliest memory of him. Between bars, through that street.
I asked her, "Carmen, would you like to have an image of that memory?" And she said, "Yes, of course. I would love to show to my family what I experienced, the things that I went through so they can remember where we all come from." And this is how we started this experiment, trying to transform a memory into an image, in this case using generative AI.
We started by creating a description of the memory, a prompt, to say so. And then Carmen generated tens of images. You could see her going through all of them. But it was not until she saw these two images that something clicked in her. She pointed at the image and said, "Yes, this is me and my mother on the balcony. I can see the haircut." It was as if she had found something long lost.
From there we did something else. We used that image as a starting point to generate a video, but this time, not only showing her and her mother, but also showing her father looking back at them from the other side of the street. When she saw it, there was a long silence. And later she told me that it had been a long time since she had seen an image of her father.
And as you can see, these are not factual recreations of the past. This feels a bit more like dreams, right? And in fact, they are. They are visualizations based [on] memories with their imperfections and undefinition. That's how we discovered that blurry, undefined images work much better than hyper-realistic ones when we are trying to reconstruct memories. That's why early generative AI models work much better than the ultimate state-of-the-art ones. It's not the factual accuracy that moves us but the emotional truth that we find embedded into some of these systems.
And we call this project Synthetic Memories. It's one of the several projects we are doing at Domestic Data Streamers, a team of researchers, designers and engineers exploring the impact that generative AI can have in society, going from media literacy to cultural heritage and of course, memories.
Now, memories are the architects of our identity, right? Memories remind us who we are. And visual memories are very important. We all have a picture of someone we love in our wallet, in our phones, in our homes. Visual memories shape our sense of self, and they can shape our sense of belonging to a specific place. They can teach us things from the past. And because of that, they can actually make us understand in a deeper level how we react to things.
And like organic memories, which are formed and stored in the human brain, synthetic memories are visual memories from a person's past which have been never documented or lost, and that we can now, using gen-AI, transform from a text or an oral description into an image.
Now, there are many reasons why we can lose visual memories and memories itself, but the most common one is the one that comes with aging and diseases like Alzheimer's or Parkinson's. I myself have lived through that in my family, and I guess many of you have had similar experiences. It's a terrible disease as you not only see the memories fade away, but also the character and the identity of someone that you love.
So two years ago, we started to create these meetups to meet people from very different disciplines, trying to figure out how synthetic memories could be used in different social setups. And then I met David, a social worker in a nursing home. He was taking care of a group of Alzheimer’s patients and he was using a therapy called reminiscence therapy. Now, reminiscence therapy is a kind of therapy that uses music and old photographs from a person's past to trigger these very visceral, emotional memories. Think of it as kind of lubricating an old rusty lock, making it much easier to open the doors of certain forgotten rooms that we have in our head. And it has been proven to, for certain people, help very much in the prevention of depression, of cognitive decay, and in some cases, even improve cognitive abilities.
So we designed a pilot experiment using synthetic memories within reminiscence therapy. And for over two months, 12 patients did both individual sessions and group therapy. And we could see a direct relation between the level of engagement and their cognitive abilities, pointing out that this could not only be a new way of doing therapy, but also it could be another way to detect early signs of cognitive decay.
Now, this image was generated with Nuria, a 96-year-old woman from a nursing home. And she was telling me about one of her favorite memories of her husband was during Sundays, he used to read the newspaper in the living room. And she will always play this old, jumpy gramophone with some music just to annoy him. When she saw the image, she was so happy her eyes [lit] up, but after a while, she looked at me and she said, "But Pau, we have a problem here. This man over here is not my husband." And I said, "How come?" And she said, half laughing, “Well, my husband was much more ugly.”
(Laughter)
[Proving] once again that love is not just blind, but also nearsighted.
(Laughter)
As you can see, this is far from being a cure of Alzheimer’s, but it could be a way of making the journey a bit less daunting. Another patient told me "It feels a bit like finding your glasses on your head after searching for them everywhere." It's small but very significant relief. And from a medical perspective, it could be a new way to temporarily enhance cognitive abilities, making it much easier for family members and caregivers in moments of distress or disconnection.
So the pilot experiment was promising enough to partner up with the AGE-WELL network here in Canada, together with researchers from the University of Toronto and the University of British Columbia. And during this summer, we will be exploring and designing an ethical framework to protect both family members and patients, as well as comparing this therapy to other therapeutic interventions and exploring how we can make it truly accessible for both families and institutions all over the world.
But as you can imagine, synthetic memories and this project can go beyond the research on dementia. There are many other reasons why we can lose visual memories. There is the loss that comes from sudden displacement due to war, political persecution, or natural disasters. Situations in which our photo albums, hard drives, phones, diaries can be lost, damaged or just left behind. Synthetic memories is a way to recover a part of this past heritage that have been lost and that right now is at risk of being forgotten. It's a new way of reconstructing a past that has been hidden to our eyes.
Now, soon we will open the first public office for visual memory reconstruction in the city of Barcelona, with the support of the city council. The idea is to make a city-scale experiment, a place where any citizen will be able to generate their own memories, to reconstruct a part of their past together in community, understanding which are the subjectivities of the past from different communities.
This image was sent to me not long ago by David, the social worker, and he did it with a dementia patient that for very long time had not spoken with his daughter. They were in kind of bad terms, and he told David that he was terrified about horses, but because his daughter loved them, he would bring her to ride every week during her childhood. And he asked David to generate this image of both of them riding in this kind of dreamy landscape. This was not a memory, this never happened. It was a remembrance. It was a way that he wanted to remember his daughter.
With his permission, David sent this image to his daughter, who, because of that, after almost ten years without speaking to each other, came back to Barcelona to speak back to him. Afterwards, she told David that when she saw the image, she realized that her father still had good memories of her.
An image can work as a witness to the world, we know that, but it can also change the way we see it. We have now tools that can help us understand a time that was not ours. Experiences from others that have never been shown in images and bridge realities that were hidden to our eyes. Montserrat Roig, a beloved Catalan writer, once wrote (speaking in Spanish). "If there is an act of love, that is to remember." And that is truly my hope. That we all get the opportunity to love and be loved in that way.
Thank you very much.
(Applause)
