Richard Moss

7 November 2020

11 min read

Related
Links

The state of synthetic media

By Iskender Dirik

6 min read

How avatars change who we are

By Wagner James Au

10 min read

One day in the not-too-distant future, you could be the star of a blockbuster movie — or rather, a digital avatar of you could be. Your synthetic self could stand in for Tom Cruise as he battles to save the world, giving you a taste of fame and glory that the latest TikTok sensation could barely dream of. You could instead project yourself into a virtual classroom, learning and making friends with a dozen people spread around the world in a course tailor-made for that group. Or you could be watching a football game from the perspective of your favorite player on the field.

These are just a few examples of life in the hyper-personalized world of the future, in which synthetic media generates and transforms digital people and content on a mass scale. Once the stuff of science fiction, this technology is now being realized. And it’s poised to change everything about how we work, play, socialize, and learn.

Entering the synthetic reality

Our media today is largely driven by social media networks and a democratized distribution of content. But we’re quickly transitioning to a new, synthetic age thanks to artificial intelligence (AI). Other major forces include emerging modes of interaction like virtual and augmented reality, as well as technologies that aren’t seen by consumers like edge computing and 5G.

Once fully realized, this synthetic reality will enable AI systems or people using AI-powered tools to quickly generate and modify any and every kind of content at an extremely high quality, from text and images to video, voice, and music, and even the thinking and feeling of synthetic people or avatars.

“The first thing I think will happen is that the technology we interact with automatically adjusts to our schedules — and our activities — as everything becomes trackable,” says Pim de Witte, co-founder and CEO of game clip-sharing platform Medal.

Our homes, devices, software, vehicles, and AI assistants will learn our habits and preferences and find ways for us to spend more time with our friends. De Witte believes they’ll also have “a very clear split between play time and focus time and view time.”

This new synthetic reality enables “a higher degree of presence and a lower amount of multitasking,” adds Philip Rosedale, previously the founder of Second Life and now the CEO of High Fidelity.

Rosedale thinks a large part of this future will not be personal so much as communal. And our communal experiences won't be mediated by AI. Rather, they'll involve hanging out with various clusters of our wider social group. “Because ultimately I think people are the best experience in entertainment and community,” he says.

“Our homes, devices, software, vehicles, and AI assistants will learn our habits and preferences and find ways for us to spend more time with our friends.” — Pim de Witte, co-founder and CEO of Medal
Evolving meta-worlds

Visions of how we’ll spend our time in a synthetic media future typically posit a single all-encompassing virtual world or “metaversethat monopolizes our time and replaces our reality.


In novels like Snow Crash or movies like Ready Player One the metaverse is portrayed as an alternate reality we’ll live in, where our digital avatars will replace our true selves. But synthetic reality will more likely augment rather than replace the physical world, and there won’t be a single virtual world we connect into -- there will be many.

De Witte envisions a few different types of metaverse destination: “The first one is most comparable to how we currently play sports,” he says. That means skill-based and intense. Second will be the more generalized virtual worlds, filled with lots of different things to do, like an extension of the present-day experiences kids have hanging out in Minecraft, Roblox, and Fortnite. And the third type is like movie theaters, with immersive AI-augmented experiences that you can get lost in.

News and current affairs videos, meanwhile, will need to go through various authentication systems before you see them to guard against deepfakes and validate that they came from trusted sources. De Witte believes we'll regularly experience moments of our friends' lives, as seen through their eyes, via technology akin to today’s Instagram and SnapChat stories.

Rosedale thinks we’ll also have better-than-real-life communication through synthetic reality — even across diverse languages and cultures — by replicating and extending our facial expressions, voices, and other natural gestures, which will expand what we can do and how we interact online.

Present-day social networks may be obsolete in this future. De Witte and Rosedale both consider them a poor substitute for human interaction. By contrast, “meaningfully spending time in 3D environments is not a substitute; it’s actually a better replacement,” says de Witte.

Rosedale believes we need look no further than the impact of the COVID-19 pandemic to understand this, as we long for our pre-pandemic freedom to wander around and meet new people. ”We’ve suddenly been given a crash course in why the real world is interesting,” he says.

Thanks to new digital worlds, our future selves will have new ways to connect, work, and socialize remotely, often via drop-in-drop-out private or semi-private shared virtual spaces. These spaces will feel just as real as being in the same room together and may facilitate the type of everyday chance encounters we yearn for in the real world more easily. This is where Rosedale sees the greatest potential of the future to connect us, on a deeper level, with people all around the globe.

Take education, for example. Rosedale notes that virtual classrooms could allow any student to study under any teacher. This means you could get an on-campus, in-person experience of hyper-specific subject matter that may only appeal to you and a very small number of other people in the world.

“Will we become more human or not by using all these kinds of synthesized media? Will these media augment us? Or will they take things away from us?" — Sander Duivestein, author and analyst, Sogeti
We are synthetic media

Our future lives will still revolve around people and community, but they will be augmented by AI as they are digitized and interact in virtual worlds. “We, as human beings — we are becoming media ourselves,” says Sander Duivestein, an author and analyst on new technology for technology and engineering consulting firm Sogeti.


Humans already communicate with each other through screens. We exchange text messages, make videos, record audio messages, and photograph our lives to share online. Already, at the dawn of synthetic media, we are part of the media that we consume. Duivestein believes this revelation will reshape our entire world.

Duivestein describes four types of reality: “The objective reality is basically the trees, the rivers, the buildings around you. The subjective reality is about your feelings, your emotions, crying, laughing. The third one is the imaginary reality. It’s the things we come up with in our minds, the ideas we are creating. It’s money. It’s religion, gods. And the fourth reality we are now creating is this synthetic reality.”

He gives an example of taking a photo of yourself, adjusting it with filters, then asking a surgeon to make you look like your selfie — a behavior known today as “SnapChat dysmorphia” that increasing numbers of young people are already engaging in.

“Now we are using the synthetic reality to reshape our objective reality,” Duivestein says. “And that will become more and more important. New kinds of ideas, new kinds of religions will pop up in these synthetic realities and they will alter our objective reality. And take over.”

Hyper-personal is hyper-relevant

Our world today is drowning in notifications and noise. Between social media, news, podcasts, and video feeds — not to mention email, advertising, and everything else — it’s becoming increasingly difficult to identify what’s relevant to us as individuals and what can safely be ignored. But that problem fades away in a hyper-personal world because “a hyper-personal world is also a hyper-relevant one,” says Dave Morgan, CEO of Simulmedia.


Today, advertisers are incentivized to only care about their return on ad spend. It’s not unusual for 99.9 percent of their audience to be offended, while 0.1 percent clicks and completes the desired engagement. Morgan says this contorts the economics of advertising: “You start optimizing for ads that are only relevant to 0.1 percent, not the 99.9 percent that also have to receive them.”

Morgan thinks the equation will change as new data and analytics capabilities combine with smarter AI and other synthetic media tools to enable a more-anticipatory and less-annoying future.

This kind of hyper-personal data-driven anticipation of your needs could go very wrong very easily, but Morgan thinks that brands and AI bots and synthetic shop assistants inevitably “will learn how to keep out of the way when they should, and to be there at the right time.”

The empathy quotient
In a world where context is king, empathy becomes critical. If an AI algorithm or agent cannot distinguish between emotions like pain, sadness, anger, curiosity, and confusion, for example, or if it can’t tell if we’re on a mission or just casually browsing, then it will struggle to anticipate our needs with any reliability.

“You will have to have technologies with a lot of not just IQ, but EQ,” says Morgan. “Because we will have different sensibilities. And a brand knowing when to actually be available to someone will become maybe one of the biggest, most defining moments.”

That also means fostering trust with consumers on a more individual level than today, which Duivestein thinks in turn requires brands design a synthetic face to act as their direct link to consumers.

That leads to new, potentially fraught questions for companies: If your brand is a character, what does it look like? What is its gender? Or its skin color, voice, communication style, clothing, and rendering style? Is it photorealistic, or drawn like a cartoon?

These decisions will become paramount to a brand’s identity, as well as its ultimate success, and they’ll also tie-in to what Duivestein calls your “reality bubble.”

“You will have to have technologies with a lot of not just IQ, but EQ, because we will have different sensibilities." — Dave Morgan, CEO of Simulmedia.
You are in control

Today we live in algorithmic filter bubbles, Duivestein explains. That is, our media is selected by opaque data analytics to determine what news, opinions, events, and information bubbles up to the top of our feeds.


In the future, he believes we’ll live in reality bubbles controlled not by machine algorithms but by us as individuals. “What if you are a Nike fan, for example? Why would you allow Adidas in your bubble?”

Under such conditions, the consumer-brand relationship is flipped. “So you go from consumer relationship management into vendor relationship management,” he says, “where I am in control of which vendors can enter into my space.”

This shift in power dynamics also feeds back into the idea of a hyper-relevant, hyper-contextual, and hyper-personalized world. In an augmented reality (AR) world, technology is used to overlay contextual information onto the real world.

If we can work through the privacy issues of cameras on our faces, then it very well might do this on a mass scale. Rosedale points to an idea from The Sims creator Will Wright that sometimes better context means less information -- for instance, if you blur out all the signs at a major intersection except for the one you need.

Or imagine, as Duivestein does, a different travel experience: “If I want to go on a holiday to Melbourne, I can broadcast my own intention into this space,” he says.

This could entail listing some basic details about his traveling party, preferred ticket type, and a skeleton itinerary. “Then all kinds of vendors, they can make a plan or they can make an offer with a certain discount. And that gives me the possibility to look at the offers they make,” he adds.

On a day-to-day level, this might mean things like watching the same screen as the person next to you — but only one of you sees closed captions. Or sending your avatar to a planned engagement in lieu of your physical self, either as an AI agent or a remote projection of you, because you are unable -- or perhaps don’t want to -- leave your home.

Avoiding dystopia and making a new world

Whether or not we arrive at this future and how it takes shape will depend in large part on the business models it’s built upon. Social networks today are predicated on an economy of attention, while data collection and use policies are rarely transparent. Meanwhile, online advertising is unpleasant because it’s mostly based on direct-response conversion.


Duivestein, de Witte, Rosedale, and Morgan all agree that adopting these practices in a synthetic future will forge a path to dystopia. The key, then, is deciding what user experience outcomes and business models we strive for.

Morgan expects there’ll be a clear incentive to be wary of potential consumer backlash, with a nod to the fan dynamics that drive triple-A video games and major movie franchises like the 007 films and the Marvel Cinematic Universe. “And so if you think about solving problems on a fan basis — not just a target-customer basis — we’re more likely to create a better experience,” he says.

Rosedale also warns that the technology could lead us much further down the road of hyper-personalization than is safe. As such, we’d do well to remember the original promise of the Internet was, in his words, “to connect everybody to everybody without a whole bunch of companies being in the way.”

For Duivestein, meanwhile, building a synthetic reality will require a reset of capitalism to focus on profit with purpose — that is, to do good for society and the environment, because new generations will demand it.

But, he cautions, we must keep in sight our humanity. “I think one of the underlying questions is will we become more human or not by using all these kinds of synthesized media,” he says. “Will these media augment us? Or will they take things away from us?”

Either way, the future looks set to be hyper-contextual, hyper-relevant, and stacked with individually and communally tuned opportunities to play, work, socialize, and learn in a synthetic reality that’s as much about meaningful connections with other people as it is about the content that’s created within it.

Richard Moss HS
Richard Moss
Journalist, podcaster, & author of "Shareware Heroes: Independent Games at the Dawn of the Internet"

Richard Moss is an award-winning writer, journalist, and storyteller who explores the future — and the past — of innovation, video games, and technology. He is the author of The Secret History of Mac Gaming (Unbound) as well as the upcoming book Shareware Heroes: Independent Games at the Dawn of the Internet, and he creates the documentary-style games history podcast The Life & Times of Video Games.