Institute For The Future

Designing an Ethical Operating System

How do you build something you won’t regret? It’s a question that deserves its fair share of thought, especially in Silicon Valley where countless tech companies compete to develop the next brilliant innovation that’ll change the world. With the rapid advancements in science and technology paired with a culture to compete and get ahead of so-and-so company or so-and-so country, we face the dangerous risk of building a technological innovation that’s not thoroughly thought out. Without pausing to envision the possible harmful consequences of technology, or anything else we design, we’ll likely find ourselves in Dr. Frankenstein’s shoes, having a monster whom we can no longer control. Dr. Jane McGonigal, Director of Game Research and Development at the Institute for the Future, says “Silicon Valley risks falling into a long period of ‘Post-Traumatic Innovation’ in which our imagination is limited to solving the problems of the past instead of preventing the problems of the future.” It was with this risk in mind that the Institute for the Future (IFTF) and the Omidyar Network Tech and Society Solutions Lab partnered to create the Ethical Operating System Toolkit as a lodestar to guide anyone building or consuming new tech. The toolkit includes a checklist of eight risk zones to help identify possible pathways to harm, fourteen scenarios, or stories, of tech gone wrong, and seven strategies you can employ to make your product future-ready.

On Tuesday, April 2nd, Institute for the Future and Stanford University’s d.school co-hosted a 2.5 hours-“pop-out” course at the IFTF office, in downtown Palo Alto, to introduce the Ethical Operating System to 27 Stanford students and alumni from various departments, including Management Science and Engineering, Economics, and, of course, Design, amongst others. This particular partnership between IFTF and the d.school was born as an attempt to fuse together futures thinking with design thinking. As d.school’s Designer in Residence Lisa Kay Solomon states, futures-centered design “asks that we become more observant and attuned to highly important and uncertain trends that will meaningfully interact and change the broader context….It empowers us to make more thoughtful choices by suggesting that the future isn’t pre-determined, but is a wider range of possible futures that we can and should explore and shape.” By placing designers in the mindset of thinking long-term, say 10 years out, they are forced to consider the larger forces within which they create. In other words, a futures-savvy designer will think outside the box of its myopic problem and constraints and, therefore, design a solution that’ll endure and weather the oncoming forces of the unknown.

As students began to arrive at IFTF for the pop-out, they were encouraged to look at a long table of “artifacts from the future”, which are tangible objects from a possible future that provide a moment of intrigue, allowing the viewer to connect with and feel the future. The course started with an introduction from Kay Solomon, one half of the teaching team, who emphasized design’s role in shaping not just products but also experiences, systems, and technologies, and, ultimately the implications that might follow. Solomon shared the design abilities that many d.school school classes focus on, including  1) navigating ambiguity, 2) learning from others, 3) experimenting rapidly, 4) synthesizing information, 5) building and crafting intentionally, 6) communicating deliberately, 7) moving between concrete and abstract, and 8) designing your design space. After McGonigal, the second half of the teaching team, followed with an introduction to futures thinking, she dove into the heart of the class, the Ethical OS Toolkit. McGonigal gave an overview, and signals (specific and compelling examples), of all eight “risk zones”, which are 1) Truth, Disinformation, and Propaganda; 2) Addiction and the Dopamine Economy; 3) Economic and Asset Inequalities; 4) Machine Ethics and Algorithmic Biases; 5) Surveillance State; 6) Data Control and Monetization; 7) Implicit Trust and User Understanding; and 8) Hateful and Criminal Actors. Participants were given time to go through the full detailed checklist while thinking about a specific technology, ideally one each of them are designing themselves. Then, using a Google form, the participants entered what kind of tech they ran through the checklist and voted live on which risk zone they consider most urgent (see images below). The top two winners were Truth, Disinformation and Propaganda, and Data Monetization and Control.

The final exercise of the evening was when futures and design thinking really came together. Kay Solomon presented four out of the fourteen scenarios from the Toolkit, after which everyone formed small groups and used what they learned to create an advertisement for a new technology designed to fit one of the presented scenarios. They created ads for technologies which included a personal virtual bodyguard who protected the user from online hatred, a chatbot that mimicked one’s most revered celebrity, and a service that sent the user’s loved ones a recorded message of the user emotionally appealing them to vote. Participants left the pop-out more conscious of their responsibility in making better decisions today which will impact how scary or not the future will look like.

IFTF Executive Director Marina Gorbis says “Once something is built, you can’t take it back.” As tech developments interject their way into the numerous aspects of our lives -- food, medicine and biology, city infrastructure, transportation, communications, etc. -- the harmful consequences around bad tech can be colossal, so it’s increasingly essential that everyone stop to play out possible scenarios, even the ones that seem ridiculous because, as McGonigal cautions, it’s the futures that sound the most ridiculous that are the ones that blindside us. The Ethical OS’s goal is to fight against the idea that we can’t imagine future consequences. Futures thinking isn’t meant for us to predict the future but to be prepared for it. “By embracing a posture of long-term thinking, new processes that make futures concrete and accessible, and a wider set of practices that collaboratively question, imagine, and communicate new possibilities,” Kay Solomon says, “we can catalyze a new movement of futures-centered designers to shape a better tomorrow for generations to come.”

McGonigal wants everyone to know that Ethical OS is free to use and is a public endeavor, so all users are encouraged to contribute their insights and foresight back into the public domain. If you use the Toolkit, go you! If you want to send us your feedback, even better! So, go forth, design your tech with ridiculous imagination and supported by the EOS Toolkit, and be Futures Designing Superstars!