Continuing my conversation with Mark Pesce, who argues for a ‘metaverse of the real’.
What follows is the second half of a recent conversation I had with futurist Mark Pesce on the danger and the potential of the metaverse. Mark’s book Augmented Reality came out a year ago, and he’s been closely following developments in the virtual reality field as well. Part one of our conversation was titled, “Will the Metaverse Be an ‘Omnidirectional Panopticon?” and ended with Mark suggesting that Meta and Mark Zuckerberg were trying to monopolize the future.
Q: So, given everything you’ve said, the question remains, can we head off the dystopia, the enhanced surveillance that the metaverse seems to portend, with the tools that we now have, and with the laws that we now have? Or are we still in great danger? You outline in your book the idea of a different approach to locative data that would not be owned by one company, right. I mean, it seems to me that’s still something we want.
A: Yeah, and I think that the reasons for it are becoming clearer the further down this path. Metaverse creators kind of fall into two classes of people: those who are firmly committed in open standards and then people who are happy to have entirely closed worlds now. So you can take a prebuilt product offered by a company like Meta or Apple or you can if you have enough technical chops to roll your own. Remember the barrier on this, in terms of technical chops, will continue to drop over time. I now judge science competitions where seventh graders are using TensorFlow. You have tools like Minecraft and Roblox, neither of which are open, but both of which provide a great deal of agency to their users. So there are many different models here that we can draw from. Whether people will come to the metaverse expecting agency ends up being partly the stories that we’re telling right now about what it could be.
And so that is a great big red warning sign to us that we need to be telling different stories around the value that you can have with your own agency in that space. I don’t think that’s an easy road. I have spent the last 10 years telling people to log off Facebook and they won’t, because everyone’s there. And all of this even though everyone now knows how bad Facebook is, they won’t do it.
In terms of data collection, you’re right, we need we need stronger laws. In Australia, you need stronger laws in America, they’re continuously revisiting the laws in the EU around this. But as I point to in the book, the user has to have complete agency about where that data goes, how it gets used, and so on. None of the devices that we have now are really designed around that.
Q: What we seem to have right now is largely a hands-off approach where we just have to hope that the marketplace offers a privacy-centered product, while everyone races to collect more data on us. I’ve yet to see a car manufacturer who decides that they’re going to be, for argument’s sake ,the Apple of cars, for example. That is where they view you the customer as the end user whose privacy they’re trying their best to protect and serve. I know Apple is not perfect either. But give me a better one.
A: You’re right, you’re right. It’s a thorny path.
Q: In the book, you talk a lot about Niantic and its success with Pokemon Go. Which captivated millions of people and showed that a game that is augmented reality could move people around in real life. Why haven’t there been more of these kinds of successes?
A: It definitely has the feeling in that sense of a hits-based business and that was clearly a hit. Wizards Unite made a fraction of what Pokemon Go made. Game design is notoriously fickle. And the sophomore album always stinks if the first album is a hit. Still, writing on space changed the behavior of tens of millions of people. And that’s the thing we need to lean into rather than, oh well, let’s just create another game. That’s what Niantic knows how to do so that’s what they’re doing. But it’s not leading.
Q: I saw a piece that you did after Meta made its big announcement where you talk about this idea of ‘a metaverse of the real’ that instead of only seeing the dark side of this future you offer I think a little bit of a hopeful vision for how we conceive of this technology working for people. Can say a bit more about that?
A: David Gelernter wrote a book called Mirror Worlds before the Unabomber blew him up, where he suggested that building a digital twin, a one-to-one representation of the world, might help us understand it better. To model something, you need to understand it. Once you actually have that model, you then also have something that allows you to see things in depth about the world that might not be visible to you either because you’re not at the right scale or timescale. It’s very hard for us to see things that are very slow because in a cosmological sense we have very short attention spans.
People will say, well we have Google Earth, right? It’s a photographic representation of the world. But there’s very little there. There is very little data. And given how much Google actually knows about the world, the amount of data that’s portrayed in Google Earth is just this tiny, tiny little bit. When you get to that depth, you start to get to focus on correlations on how this thing over here is affecting this thing over here or how this thing over here could be used to help this thing over there.
A: The first reaction I have, though, is to remember that power is not evenly distributed. And that maps have been for hundreds of years instruments of power. I also think of the work that my friend Emily Jacobi does with an organization she runs called Digital Democracy that has been working with indigenous tribes in the Amazon to fight resource extractors. One of the things they have done is taught them how to build their own drones to map their own communities. They also built their own mapping software which has its own intricate visual language showing what the indigenous people know about their own lands. Most importantly, none of these maps are digitally accessible. And that’s because of their understanding that when you make your own land more transparent, you just make it easier for other people to penetrate and conquer. The concept of a metaverse that works for all of us is hard to imagine. Everything we’ve seen about making things more transparent seems to work to the advantage of the haves rather than the have-nots.
A: You point to a fundamental tension. You don’t solve a dilemma, you endure it. The question then becomes the question of agency. So if you preserve that agency, but then also provide the open space for people to be able to collaborate and work around shared goals, you may preserve the best aspects of it and limit the worst.
Q: Last question. Given that the the technological challenge of VR/AR is surmountable, do you see any other actor that could then actually tackle the problem in a way that would work for all of us? For example, do you think that a group of say really forward-thinking municipalities, say, mega cities that recognize that they actually are troves of all kinds of important information and that it’s in their interest make location data, work for all their inhabitants work for the entire city? Not to be the playpen of a Sidewalk Labs say. That was an interesting experiment, where a Google subsidiary chummed up with the president of Canada to basically colonize some choice real estate in Toronto. It kind of blew up in their faces in part because one of my heroes, Bianca Wylie, the present day equivalent of Jane Jacobs, and a bunch of other great local activists just doggedly kept peeling back the onion and, and created enough public awareness to stop the project. Toronto is a relatively uncorrupt place compared to many. So I’m wondering if you can imagine a benevolent actor here?
A: Let’s step back a little bit and take a look. So augmented reality is a capital problem. The metaverse is a standards problem. And money doesn’t buy standards. Even Google doesn’t get the internet it wants. Facebook doesn’t get the internet it wants. They are forced into a position where they are negotiating. Remember 20 years ago, Microsoft was literally trying to encompass the entire web. And they were stopped, I think probably by the skin of our teeth. But we also learned you hold the lines on these things. So I think when you’re talking about the metaverse that those battles are going to be fought in terms of standards. And so we have to get people deeply interested in the kind of Metaverse they want. And to think about that rigorously in the sense of how does that become the standards and simple designs that would then flow into any products they get built.
Now come back out to this specific example of locative information. I do occasionally still get to code. I get to do it on weekends because I don’t have time for it. And over the course of a couple of weekends, I put together this little app. It’s just a little web thing. Right? What it does is uses Wikipedia as a database, and allows you to search Wikipedia by the things that are around you, and it shows where they are. We never think about using Wikipedia that way — we think about it as being text, right? My point is we actually already have some of these tools in place and we just don’t know that we have to use them. When you’re talking about populating a metaverse with data, whether that’s city data — all that data is fundamentally locative. Which means we could actually have a hackathon right now and say, okay kids, let’s see what we can do with this.
The funny thing is — as much as Mark Zuckerberg causes my blood pressure to rise 25 points just by thinking about him — he has got everyone thinking about the metaverse, and that’s not a bad thing, because there’s a lot of things we now need to ask. We also know the kind of baggage he’s carrying. And so people know to ask interesting, critical questions that maybe we wouldn’t have if it had been some other purveyor of this idea.