and HCI

Technically Wrong

Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech

By Sara Wachter-Boettcher

This book does a great job of digging into tech products with a social analysis lens. Wachter-Boettcher is a web consultant who helps tech organizations with product strategy and content strategy. Her extensive experience of identifying bias within products and analysing potential sources for biases through guiding questions really shines through in the book, and has helped me think a bit deeper about the digital products around me.

For example, I saw the “Your Year in Review” cards on Facebook at the end of every year and didn’t really think much of them besides perhaps recalling some fond memories from the photos Facebook pulled from my timeline and plopped onto the card. However, Wachter-Boettcher highlights the case of Eric Meyer, whose daughter died of cancer earlier in the year. Eric included a photo of her in a grievance post he wrote and posted, and to his horror, Facebook framed that photo with partying figures and balloons in his Year In Review card. It’s likely that the algorithm for the Year In Review card takes the pictures from posts others interacted with the most with and displays them on the card. This works well for people who had a good year or only post positive content, but because the design team focused only on positive experiences, it hadn’t thought enough about what would happen for everyone else - for people whose years were marred by grief, illness, heartbreak, or disaster.

The book broadly claims that tech companies build products that are not intentionally biased, and the issue stems from homogeneity of the tech investors and employees. I have no doubt that this is true, and can recommend the book because it does a great job illustrating how this issue permeates to various products and its direct effects on end users. I also enjoyed Wachter-Boettcher’s point about edge cases: she claims that engineers and designers build for the majority and not edge cases, which makes sense because that’s where most of the money is. She argues that the harm caused by inconsideration for a small group of users can cause significant damage, perhaps more the monetary benefits, and proposes “edge cases” to be thought of as “stress cases” instead. The subtle difference allows designers to realize the strength of their work and where it breaks down instead of identifying cases that are of lower priority or outside the bounds of concern.

I do, however, wish that Wachter-Boettcher addressed other tech scenes outside of California. The book is heavily focused on tech companies and culture based in the west (Facebook, Apple, Twitter, various startups in Silicon Valley). Sure, the tech scene may not the healthiest in the Valley, but what about other places? Also, I get the feeling that the overall book is guided by articles and opinions in tech media – many of which are focused on the big tech companies. The next step would be to go out and interview some users to test the validity of some of the claims. There are specific cases mentioned in the book, but I their content are all quite narrow and similar in that they all unroot an unpleasant experience with a product. Overall, I think some of the claims could’ve been made stronger with some more diverse user research and/or more scholarly content such as ethnographic research.

As a side note, reading this book while building Rehack was a surprisingly nice complement: it helped me solidify some of ideas and goals I wanted to incorporate into the event.