How Technology Spreads Bias: And How You Can Stop It
Transcript
Thank you everyone for coming to this workshop on a rainy evening. It's great to have you here.
My name is CJ; I am nonbinary, and my pronouns are they/them. I'm a researcher, game developer, and artist. I've been working in UX design for over 10 years and at companies such as Apple, frog design, Goodpatch, and most recently, a little startup in Tokyo called yamaneco.
I've been giving talks publicly since 2016, talking in Finland, Russia, the U.S. and Japan.
I do research; last year in April my co-dev and I released a game called Terranova about fangirls on the early 00's internet, and I also blow glass. When I first got started professionally I felt pressure to focus on one thing to make myself more marketable, but nowadays I like being a messier person.
I like to tell stories. And this workshop is about the stories we have, and the stories we have yet to see.
A common thread throughout my work is that I love technology as much as I love people. I love people a lot.
And genuinely, when you love something and you're invested in it, you want it to succeed. You believe in its potential to do great things. When I was a baby queer in my conservative suburban hometown in Texas, there was no one I felt I could open up to at school. But online, my horizons expanded. The internet saved my life. It helped me come out.
If any of y'all have been in a long-term relationship, you know; love isn't about pretending things are always okay. It's about doing the hard work; about having hard conversations when you need to.
I love this quote from Steve Jobs. "Stay hungry. Stay foolish." When I hear this used, it's in relation to appetite for innovative technology or testing the limits of our skills. But I've noticed in tech that suddenly we have no appetite when we start talking about bias and equity. And I am putting this quote to remind you to stay hungry throughout this workshop. Stay curious, stay open. Always be pushing the edges of what you know.
My promise to you: by the end of this session, you will understand how bias in technology affects others, reflect on your own biases and how they may affect your work, and discuss real-life scenarios where you will talk to others about bias and how it influences technology.
Alright, so what's bias? Simply put, bias is a preference for an idea that does not give a chance to an equal idea. What does this mean? If I have a bias towards Idea A, and I have two ideas, A and B, then I will lean towards Idea A and perhaps ignore or discredit Idea B. I have a preference for Idea A.
We do not often think of technology as having a preference. Rather, we see people as having preferences and technology as having data.
But who makes the technology? Who feeds the technology data?
People.
The same can be said for systems, too; systems are very complex, and they are made up of processes. Many people say these processes are neutral, but these processes were made by people. And people have biases. Therefore, any technology, any system made by people or using data collected by people has bias.
The good news? We made technology and systems, which means we can dismantle them, too.
So what's an example of bias in technology? Let me take an example from my own personal life. For context, I do not feel comfortable with the label she or her, wife, and recently, daughter. I've had varying levels of distaste for these labels before I came out as nonbinary, but the feeling has gotten even stronger now. I dress normally like you see me; masculine, but my voice is feminine.
So my two friends, who are both female, and I, are taking a trip to Hakone. I'm booking the tickets for us and the website requires I pick my gender; Male or Female.
At first, I thought that they were going to check my ID, and I didn't want a fuss, so I picked what was on my ID. I have to stress to you—I wanted the least amount of fuss possible. I did not want to think about this. I wanted to have a fun time with my friends.
This is what the form looked like.
When we got on the bus, I realized that without my consent, the bus had automatically assigned us to the "Woman Only" priority seats in the front. I was dressed in a t-shirt and a baseball cap, and looked like a man.
It was frustrating and embarrassing for me to be sitting here. I asked for another seat, but the bus was full. So I rode the hour and a half bus ride in a seat that said, “Woman Only.”
I just wanted to have a pleasant bus ride with my friends, not sit in a pink seat that said, "woman."
Another example from Algorithms of Oppression by Safiya Noble: Kandis, a black hair hairdresser, is often listed last on Yelp (a local business listing website) when customers search “hair care” or “black hair care.”
Why? Because Yelp's search algorithm saw "hair care" as something primarily white people do, and even specifically the words "black hairdresser" was interpreted as "hair dressers who specialize in hair dye (to dye the hair black)."
Hell, even when I was looking up the words "black hairdresser" on Unsplash the first entry was one photo of a black woman doing hair and the rest were white hairdressers on black or dark backgrounds.
Even if customers are black and want to find black hair care, they will not find Kandis' salon. They will find hair salons for white people instead.
In both Kandis' and my situation, this is what we felt. “I felt invisible. Like I shouldn't be here.”
Those are examples of microaggressions. A microaggression is a small daily insult and indignity againstmarginalized people.
The effect is a constant reminder that the person is “less than human” by belonging to this marginalized group.
Did the people coding the website intend to embarrass me? Did the people at Yelp intend for Kandis' hair salon to not show up? No. But that doesn't change the fact this was the outcome.
Microaggressions are usually unintentional, small and can easily be explained away, and done by many different people and hard to address individually without coming off as hypersensitive.
The best way it's been explained to me is this: imagine that you're walking down the street and someone bumps into you. No big deal. But if you walk down the same street every day for two months and every day someone bumps into you, you start to wonder..."is it just me?"
You look around. No one else you see is getting bumped into. In fact, when you talk about it, people tell you what a friendly street it is. So why is it only you?
You start to expect the bumps. You start to hate walking down that street. Your shoulder is sore. And one day, one person bumps into you and you say, "Hey! Watch it!"
And they say, "Geez, I'm sorry. Don't be so sensitive."
But see, it's not about that one incident. If it were, it wouldn't be a big deal. It's about that incident and the sixty that came before it, every day. It's about the way you used to enjoy walking down the street and now you fear it.
Some examples include a black person hearing, “Is that your real hair? Can I touch it?” or a queer person and their partner hearing, "“Ok, but which one of you is the top?”
There's a meaning, even if unintentional, behind that statement. And the question I like to ask myself when thinking about the hidden meaning is, "if this person was white, or if this person was straight, would I ask the same question?" In the case of asking about black hair, the hidden meaning here is, I want to touch you to satisfy my curiosity (but I wouldn't do the same with a non-black person). The hidden meaning behind asking a queer person about their sex life is, I can ask you sexual questions to satisfy my curiosity (but I wouldn’t do the same with a non-LGBTQ+ coworker).
It's embarrassing to talk about, but I have said those literal words to a black friend of mine in college. I feel sorry I did. When I first reflected on my actions, the first thing I thought was, "Well, I didn't think..."
Yeah. And that's exactly the point. I didn't think. And in addition to that, I treated someone like their personal space didn't matter as much as my white friends.
We are natually curious as humans. We are natually hungry. But we have to use that curiosity to empower others, not discriminate and cause them harm. Everyone is deserving of privacy, bodily autonomy and personal space.
I'd like to propose we call these small moments in tech what they are: tech microaggressions.
At its least destructive, as in my case, tech microaggressions make marginalized people feel like they don't belong. It was an uncomfortable bus ride, but not unbearable.
At its most destructive, tech microaggressions lock out marginalized people from accessing basic services and income. Kandis' hair salon lost out on potential customers that wanted to find her because Yelp's search algorithm was biased against the term "black hair."
So you might be asking yourself, what can I do to help?
Well, let's start by looking inward. Let's talk about privilege.
Privilege is a system of advantages or rights that people have because of their sex, race, ability, etc. Keep the word "system" in mind, here.
When I first encountered this term, I felt defensive. “I can't have privilege," I said to myself. "I've had my own struggles as a nonbinary and queer person.”
Yeah, and my experience is valid. But my struggle and someone else's struggle can exist at the same time. Talking about privilege is about acknowledging a system of advantages. It's not about saying an someone is bad. It's not about competition. It's not about individuals at all. It's about talking about the system and how we fit into it.
So, for our first exercise, I'd like you to fill this diagram out to your level of comfort. It's called The Power You Have. It is a diagram of some, not all, axes of privilege and marginalization. Use each of the lines as a spectrum and map out your unique set of privileges and marginalizations. Where are you on this diagram?
Next, find a partner or two (2-3 people groups) and discuss: What did you notice? What does it feel like to be visualized in this way?
Make sure to share your views equitably. Here are some tips.
Reflecting on that exercise, you can probably see where you can empower other people, and lift them up. And where you may have bias, where you may not know what another person who is different than you's life is like.
Now, let's take our pairs and find another pair (4-5 people groups). Let's talk about some real-life scenarios and what we could do in the real world.
Scenario 1: You are working on a product team as a junior developer. You get a task from a senior to “design a signup page for the product.” In her design, you notice that the form has a required question about gender; it asks the user to choose either Male or Female. You are not transgender or nonbinary, but something about the question feels wrong. The designer shrugs and says, “the marketing team asked for this button.”
What could you say?
What did you think? There is no one right answer here; only the one that you feel was the best move from your intuition and with the amount of power you had.
Keep in mind when giving feedback in real work situations to state the facts. Ask hard questions like, "why did you do or say that?" or "Can you help me understand..." Take some time to listen and see if you can elevate others. If you're a senior, openly stand up for junior developer's positions if you feel they are right.
And remember that some of these conversations sometimes can be unproductive at first. We all dream of being the hero; blazing in with our righteousness and saving the day. But it doesn't work like that. People reflect in their own time and move in their own time. Sometimes people will push back on you at first, and reflect later. You may not see the growth later, or at all.
When someone gives you feedback that you are being biased in the workplace, take a minute. Pause. If you feel threatened or uncomfortable, ask yourself, compassionately, why. Thank them for their feedback and research more, later. And if you're sorry, apologize.
I think apologizing when you're not sorry; when you feel forced to do so, does more harm than help. If you are sorry, offering a genuine apology with specifics is the best way to repair damage and move forward intentionally.
Next, we're going to discuss another scenario. Find a new group of 4-5 people. This time, you will be on the defensive.
Scenario 2: You are working on a project as a senior designer. You are responsible for overseeing a new chatbot feature. It helps users connect from your website to the right support agent. Your company is a well-known and trusted PC company. When you present your designs, your manager seems pleased. However, your coworker, who is a researcher looks uncomfortable.
Afterwards, she talks to you in private.
"I'm worried about your design," she says. "Many of our support agents get very personal questions like, 'how do I come out to my parents?' and 'I'm depressed, what do I do?' What will the chatbot do if a customer asks this?"
You feel defensive. You answer: “That's not my problem to think about that. We're a PC company, not a counselor.” Your coworker is silent for a while and then leaves.
What could you do next?
Scenario 3: You are working for a big-name social media company as a content moderator. You've received an email to expect “a lot of LGBTQ+ political debate” during June (Pride Month). Normally, you would delete comments like, “transgenderism is a disease that pollutes our children's minds” or, “gay women need a strong man to keep them in line” but the email asks that you “consider being more forgiving in your moderation” to “encourage engagement on our platform.”
This means you would not delete comments that have misinformation about LGBTQ+ people. Your say to your manager that you are uncomfortable with this, but he says that “There's no harm in free speech. Everyone knows those things are false.”
What could you do?
You've talked about what you might do in these scenarios; now, let's talk about action. What will you do after you get out of this workshop?
First, here are some resources that I used when working on this talk:
- So you want to talk about race by Ijeoma Oluo
- Algorithms of Oppression by Safiya Umioa Noble
- Technically Wrong by Sara Wachter-Boettcher
Here's some resource groups in Japan continuing conversations around race and LGBTQ+ issues specifically in Tokyo:
So how will you use these? I want to invite you to make a promise to yourself. On the next worksheet, you will find a template. We've passed out two so that one you can take home, and the other you can put on the board there for other workshop participants to see.
Some possible actions might include:
- ...start reading one of the resources mentioned in this talk to educate myself.
- ...bring these scenarios to my coworkers and discuss with them.
- ...do the privilege/marginalization exercise with my coworkers.
- ...speak up the next time I notice tech microaggressions.
- ...reflect and journal on how I can use my privilege to empower other people.
As a final reflection: take a look at the workshop participant's promises and reflect.
And remember everyone: “Stay hungry. Stay foolish. Never let go of your appetite to go after new ideas, new experiences, and new adventures.”
I hope everyone continues to be hungry. To make mistakes and to apologize. And to move forward.
Thank you.
Wolf illustration on the first and last slides is by artist @hookieduke.
Slides and Worksheets
PDF version of the slides are here.
During the workshop, we used multiple worksheets and scenarios to facilitate discussion. They are available to free to download here. Please use these in your organization to facilitate your own discussions, or contact me if you'd like to hire me for your organization.