Could AI help elderly people and refugees reconstruct unrecorded pasts?

Pau Aleikum Garcia was in Athens helping those arriving in the Greek capital after a dangerous sea voyage as a record 1.3 million people, mostly Syrians fleeing the civil war, sought asylum in 2015, at the height of the refugee crisis in Europe.

The then 25-year-old Spanish volunteer arranged housing for refugees in abandoned facilities like schools and libraries, and set up community kitchens, language classes and art activities.

Garcia recalls that it was “kind of a massive cascade of people.”

“My own memory of that time is oddly patchy”, he admits. Despite one particular circumstance that stood out.

In one of those schools in Athens ‘ Exarcheia neighbourhood, where refugees painted the external wall to illustrate their memories of their journeys, Garcia met a Syrian woman in her late 70s.

I’m not afraid to be a refugee, she said. I have lived all my life. He recalls her telling him, “I’m happy with what I’ve lived.” “I’m afraid that my grandkids will be refugees for all their life”.

She protested when he tried to persuade her that they would find a new home when he said, “No, no, I’m worried, because when my grandkids grow up, they ask themselves, “Where do I come from?” they won’t be able to answer that question”.

All but one of the family’s photo albums, according to the woman, were lost while the family was traveling to Greece.

Now, she said, all the memories of their lives in Syria existed only in her and her husband’s minds, unrecorded and unrecoverable for the next generation.

Connecting generations

After Garcia’s return to Barcelona and his involvement as the cofounder of the design studio Domestic Data Streamers (DDS), Garcia continued to tell the woman’s story.

Over the years, the studio has grown into a 30-person team of experts in varied disciplines such as psychology, architecture, cognitive science, journalism and design. The studio uses technology to “emotions and humanity” to visualize data, including museums, prisons, and churches, as well as the likes of the United Nations.

Then, in around 2019, with the rise of generative artificial intelligence – a model of machine learning that uses algorithms to create new content from data scraped from the internet – the team began to explore image-generating technology, following the release of ChatGPT.

Garcia and the Syrian grandmother were both impacted by the use of technology to create images based on memories and how it might benefit someone like her.

He believes that memories – captured through records like photographs – play an integral role in connecting generations.

The architects of who we are are are, according to memory. … It’s a big part of how social identities are built”, he says.

He also likes to mention Catalan author Montserrat Roig, who said that remembering something is the biggest act of love.

But in the past, people had fewer opportunities to document their lives than their mobile phone-wielding contemporaries, he says. Due to a lack of access, persecution, censorship, or marginalization, many experiences have been omitted or erased from collective memory.

So with this in mind, in 2022, Garcia and his team launched the Synthetic Memories project to use AI to generate photographic representations of memories that were lost, due to missing photos, for instance, or never recorded in the first place.

Garcia describes the development of the idea as “I don’t believe there was a eureka moment.” “I’ve always been intrigued by how documentaries reconstruct the past … our goal and approach were more focused on the subjective and personal side, trying to capture the emotional layers of memory”.

The chance to reclaim such memories is crucial for Garcia’s recovery of his past. “The fact that you have an image that tells this happened to me, this is my memory, and this is shown and other people can see it, is also a way to say to you, ‘ Yes, this happened’. It’s a way to say that you deserve more respect for the historical context you haven’t been depicted in.

forming memories

To create a synthetic memory, DDS uses open-source image-generating AI systems such as DALL-E 2 and Flux, while the team is developing its own tool.

A subject is first asked to recall their earliest memory by an interviewer. They explore various narratives as people recount their life stories before picking the one they think can be best encapsulated in an image.

The interviewer collaborates with a prompter, who is trained in the syntax used by AI to create visuals, who inputs specific words to create the image based on the details provided by the interviewee.

Nearly everything, such as hairstyles, clothing, and furniture, is recreated as accurately as possible. Figures are typically portrayed from behind or, if faces are shown, with some opacity.

This is intentional. Garcia says, “We want to be very clear that this is not real photography, but this is a synthetic memory.” This is partly because they want to ensure their generated images don’t add to the proliferation of fake photos on the internet.

The final images, which can last up to an hour, can appear undefined and dreamlike.

“As we know, memory is very, very, very fragile and full of imperfections”, Garcia explains. It’s a good illustration of how our memory functions because we wanted a model that could be brittle and fragile as well.

People who participated in the project reported feeling more connected to less-detailed images because of their suggestive nature, which allowed their imagination to fill in the blanks. The higher the resolution, the more someone focuses on the details, losing that emotional connection to the image, Airi Dordas, the project’s lead, explains.

The team’s grandparents were the first to try this technology out. The experience was moving, Garcia says, and one that grew into medical trials to determine whether synthetic memories can be used as an augmentation tool in reminiscence therapy for dementia sufferers.

The team then collaborated with Barcelona’s city council to document local memories before working with the Brazilian communities of Bolivia and Korea to tell their migration stories. The sessions were open to the public and held last summer at the Design Museum in Barcelona, generating more than 300 memories.

Some people wanted to survive traumatic experiences, like a woman who was abused by a relative who escaped jail and who wanted to share his memories with her family. Others recalled moments from their childhood, like 105-year-old Pepita, who recreated the day she saw a train for the first time. Couples exchanged experiences and memories.

There was always a moment, Ainoa Pubill Unzeta, who carried out interviews in Barcelona, says, “when people actually saw a picture that they would relate to, you could feel it … you can see it”. Some people just grinned, while others cried. For her, this was confirmation that the image was done well.

Carmen, who is now in her 90s, was one of the first memories Garcia and Garcia had of their pilot sessions. She remembers going up to a stranger’s balcony as a child, her mother having paid the owners to let them in, because it looked into the courtyard of the jail where her father, a doctor for the Republican front during the Spanish Civil War, was being held. The family could only see him from his cell phone window in this manner.

By incredible coincidence, Carmen’s son was employed in the same prison as a social worker decades later, but neither son nor mother knew that. Her son recognized the prison right away from his mother’s reconstruction when the entire family attended a display last year at the Public Office of Synthetic Memories. “It was a kind of closing the loop … it was beautiful”, Garcia says.

clandestine gatherings

The team was particularly interested in telling stories of civic activists who have played a key role in different social movements in the city over the last 50 years, including those concerning LGBTQ and workers ‘ rights. Although the initial focus was not on the dictatorship era, Dordas claims that it “naturally brought us to interact with people who, under the circumstances, were activists against the regime”.

One of them was 74-year-old Jose Carles Vallejo Calderon.

Vallejo was born in Barcelona in 1950 to Republican parents who fled the oppression under General Francisco Franco. He was a child during one of Europe’s longest dictatorships, one that lasted from 1939 to 1975. During the civil war of 1936-39, and following the defeat of the Republican forces by Franco’s Nationalists, enforced disappearances, forced labour, torture and extrajudicial killings claimed the lives of more than 100, 000 people.

Vallejo first tried to organize a democratic student union at school before working as a young worker at Barcelona’s SEAT car factory in opposition to the fascist regime.

He recalls an atmosphere of fear, with most people terrified of speaking out against the authoritarian government. He explains that fear stemmed from the terrible defeat of the Spanish Civil War, the numerous deaths that resulted from the war, as well as the harsh repression that persisted from the dictatorship’s end.

Informants were everywhere, and the circle of trusted individuals was small. This was living in the world of darkness, silence, fear, and repression, Vallejo writes.

“There were few of us – very few – who dared to move from silence to activism, which involved many risks”.

Vallejo spent a year in prison, including 20 days of tortured by Barcelona’s secret police, while trying to organize a labor union among SEAT employees in 1970. After another arrest in late 1971 and the prosecution demanding 20 years for what were then considered crimes of association, organisation and propaganda, Vallejo crossed the border with France in January 1972. Following Franco’s death in 1975, the first limited amnesty of 1976, which granted political prisoners pardons, led to his eventual political asylum in Italy, where he spent his time exiled before resuming his life there.

Today, Vallejo dedicates his time to human rights activism. The Catalan Association of Former Political Prisoners of Francoism, which was established in the dictatorship’s final years, is led by him.

Through Iridia, a human rights organization that collaborated with DDS to help visualize the memories of police abuse victims during the regime in a central Barcelona police station, he was given the idea of synthetic memories.

Vallejo was drawn to the project, curious about how the technology might be applied to capturing resistance activities too dangerous to record during Franco’s rule.

SEAT employees started hosting clandestine breakfasts in Vallvidrera’s forests in 1970. On Sunday mornings, disguised as hikers, they would make their way through the dense forests surrounding the Catalan capital to discuss the struggle against the dictatorship.

Vallejo recalls that “I believe I must have attended more than 10 or 15 of these forest gatherings.” Other times, they met in churches. These exist inadvertently.

Vallejo’s synthetic memory of these meetings is in black and white. The detail in the image is unclear, as if someone had wiped out the detail. But it is still possible to make out the scene: a crowd of people gathered in a forest. Some people sit, and others stand under the trees’ canopy.

Looking at the image, Vallejo says he felt transported to the clandestine assemblies in the Barcelona woods, where as many as 50 or 60 people would gather in a tense atmosphere.

He claims, “I truly found myself fully immersed in the image.”

“It was like entering a kind of time tunnel”, he adds.

During the ordeal of his arrests, imprisonment, and torture, Vallejo lost memories.

The process of creating the image provided “a feeling – not exactly of relief – but rather of reconciling memory with the past and perhaps also of filling that void created by selective amnesia, which results from complicated, traumatic, and above all, distant experiences”. He described the reconstruction as a “valuable experience” that assisted him in some of these events.

We are not reconstructing the past, they say.

Emphasising that memory is subjective, Garcia says, “One of the things that we are kind of drawing a very big red line about is historical reconstruction”.

The disadvantages of AI, which reinforce cultural and other biases in the data it sources, contribute to this.

David Leslie, director of ethics and responsible innovation research at the Alan Turing Institute, the United Kingdom centre for data science and AI, cautions that using data that was initially biased against marginalised groups could create revisionist histories or false memories for those communities. He asserts that “simply generating something from AI” can neither help to restore or correct historical narratives.

For DDS, “It is never about the bigger story. Garcia asserts that “we are not reconstructing the past.”

“When we talk about history, we talk about one truth that somehow we are committed to”, he elaborates. However, he points out that these memories are generated by the individual rather than the events that occurred, even though synthetic memories can reveal a portion of the human experience that history books cannot.

The team believes synthetic memories could not only help communities whose memories are at risk but also create dialogue between cultures and generations.

In areas where cultural heritage is at risk of being destroyed by natural disasters, like southern Brazil, which was last year hit by floods, they intend to set up “emergency” memory clinics. There are also hopes to make their finished tool freely available to nursing homes.

Garcia, however, wonders what role the project might play in a world where everything is “over-registrated” in the process. “I have 10 images of my father when he was a kid”, he says. When I was younger, I had more than 200. But my friend, of her daughter,]has] 25, 000, and she’s five years old”!

He speculates that the memory image issue will be resolved by finding the appropriate image to tell the story.

Yet in the present moment, Vallejo believes the project has a role to play in helping younger generations understand past injustices. According to him, forgetting is not useful for activists like him because memory is like “a weapon for the future.”

Instead of trying to numb the past, “I think it is more therapeutic – both collectively and individually – to remember rather than to forget”.

Source: Aljazeera