Joy Buolamwini, the renowned AI researcher and activist, appears on the Zoom screen from home in Boston, wearing her signature thick-rimmed glasses.
As an MIT grad, she seems genuinely curious about seeing old covers of MIT Technology Review that hang in our London office. An edition of the magazine from 1961 asks: “Will your son get into college?”
I can tell Buolamwini finds the duvet amusing. She takes an image of it. Times have modified rather a lot since 1961. In her latest memoir, Buolamwini shares her life story. In some ways she embodies how far tech has come since then, and the way much further it still must go.
Buolamwini is best known for a pioneering paper she co-wrote with AI researcher Timnit Gebru in 2017, called “Gender Shades,” which exposed how business facial recognition systems often failed to acknowledge the faces of Black and brown people, especially Black women. Her research and advocacy led firms similar to Google, IBM, and Microsoft to enhance their software so it could be less biased and back away from selling their technology to law enforcement.
Now, Buolamwini has a brand new goal in sight. She is asking for a radical rethink of how AI systems are built. Buolamwini tells MIT Technology Review that, amid the present AI hype cycle, she sees a really real risk of letting technology firms pen the foundations that apply to them—repeating the very mistake, she argues, that has previously allowed biased and oppressive technology to thrive.
“What concerns me is we’re giving so many firms a free pass, or we’re applauding the innovation while turning our head [away from the harms],” Buolamwini says.
A specific concern, says Buolamwini, is the premise upon which we’re constructing today’s sparkliest AI toys, so-called foundation models. Technologists envision these multifunctional models serving as a springboard for a lot of other AI applications, from chatbots to automated movie-making. They’re built by scraping masses of knowledge from the web, inevitably including copyrighted content and private information. Many AI firms are actually being sued by artists, music firms, and writers, who claim their mental property was taken without consent.
The present modus operandi of today’s AI firms is unethical—a type of “data colonialism,” Buolamwini says, with a “full disregard for consent.”
“What’s on the market for the taking, if there aren’t laws—it’s just pillaged,” she says. As an creator, Buolamwini says, she fully expects her book, her poems, her voice, and her op-eds—even her PhD dissertation—to be scraped into AI models.
“Should I find that any of my work has been utilized in these systems, I will certainly speak up. That’s what we do,” she says.
Buolamwini says an actual innovation price boasting about can be models that firms could show have a positive climate impact and legitimately sourced data, for instance.
“I see a chance to learn from so many mistakes of the past in relation to oppressive systems powering advanced technologies. My hope is we go in a distinct direction,” she says.
I ask what aspect of AI systems she would audit today if she could repeat the success of “Gender Shades.” Without missing a beat, Buolamwini says that as an alternative of 1 single audit, she would love AI to have an audit culture, where systems get rigorously tested before they’re deployed in the true world. She’d wish to see AI developers ask essential questions on any system they give you, similar to: How does it do what it does, and will we even be using it?
“Straight away we’re in guinea pig mode, where the systems haven’t been fully tested … That’s irresponsible, especially when these systems are getting used in ways in which impact people’s lived experiences and life opportunities,” she says.
An accidental activist
Buolamwini has been on the forefront of raising awareness around AI harms for the higher a part of a decade. But her path to becoming a outstanding researcher and activist was removed from typical.
Her memoir gives us an inside look into how she evolved from a young computer science student who didn’t need to take care of “consequences and humans,” as she puts it, into someone who couldn’t look away when confronted with deeply flawed technology that she realized was not created for people like her.
In a striking moment within the book, she describes discovering that common facial recognition software would recognize a white Halloween mask but not her own face. Although she was unsettled, Buolamwini’s initial response was that another person, perhaps a more seasoned AI researcher, would handle the issue. However the more examples she saw of algorithmic bias, the harder it became to not do something herself.
While still a PhD researcher, propelled by the findings of her research on facial recognition, Buolamwini founded the Algorithmic Justice League (AJL), a nonprofit organization fighting algorithmic biases and injustice, and have become a outstanding voice warning in regards to the harm posed by biased AI systems. In 2019, she gave a pair of expert testimonies to Congress on facial recognition technology and its effect on civil rights and liberties.
While Buolamwini’s story is in some ways inspirational, it’s also a warning. The pressure of raising awareness about AI harms and speaking up against a few of the world’s strongest firms took a toll. In a very sad and touching a part of the book, Buolamwini describes checking herself into an emergency room the night before an exam, exhausted from the stress of back-to-back congressional testimonies on top of intensive travel and advocacy.
After she’d done years of intense work establishing the AJL and advising governments about AI while working on her PhD, her “burnout mode” was so overwhelming that in 2021, she wrote a letter to her professors, saying she might drop out. She asked them to make a case for the actual advantage of an MIT PhD—which can be her fourth university degree.
She took a month off from writing her dissertation to rest, and took up skateboarding. “Perhaps it was not too late to turn into knowledgeable skateboarder and possibly even represent Ghana at the following Olympics,” Buolamwini writes, tongue in cheek.
Ultimately, she decided to complete her doctorate, graduating in 2022. She resolved to achieve this, she writes, after realizing that for her as a lady of color, unlike white colleagues, dropping out of a level program can be seen as a failure.
Big risk, big reward
Buolamwini also describes an episode wherein she went up against a tech “Goliath,” Amazon. Her PhD research about auditing facial recognition systems elicited public attacks from senior executives at the corporate, which was on the time—in 2019—competing with Microsoft for a $10 billion contract to offer AI services to the Pentagon. After research by Buolamwini and Inioluwa Deborah Raji, one other AI researcher, showed that Amazon’s facial recognition technology was biased, an Amazon vice chairman, Matt Wood, claimed that her paper and press coverage about it were “misleading” and drew “false conclusions.”
Amazon was the one technology company that took a combative approach to Buolamwini’s research, she says. She frightened she’d put Raji, who on the time was still an undergraduate, at risk: “Prospective computer science departments might perceive her as an excessive amount of of a risk. Future employers within the tech industry could blacklist her,” Buolamwini writes.
“I used to be concerned that if other researchers saw what Amazon was doing to us and no academics stood up in defense, other researchers would perceive the skilled risk of this type of research as being too high,” Buolamwini adds.
Many individuals got here to fight in her corner. The American Civil Liberties Union of Massachusetts and the Georgetown Law Center on Privacy and Technology supported her publicly on Twitter (now X). Within the research community, Margaret Mitchell and Gebru, who on the time were leading Google’s AI ethics work, organized an open letter in support of Buolamwini’s work. Seventy-five researchers, including Turing Prize winner Yoshua Bengio, signed it.
Disappointingly, she says, a lot of the public support she received was from outside MIT. In her book, she writes that she felt she had been “largely abandoned by nearly all of MIT leadership at that moment.”
“I assumed the Media Lab, which showcased my work to draw students and news coverage, would defend me. It was disappointing to feel like I needed to plead for cover after which receive little or no,” Buolamwini writes. (We now have contacted MIT Media Lab for comment on this allegation, and can update this piece in the event that they reply.)
Her advice to young people in tech who need to speak up in regards to the problems they see? “Don’t jump within the deep end not knowing that it’s deep,” Buolamwini says. She wants more people to have the option to call it out after they see injustices.
But at the identical time, she advises them to surround themselves with individuals who can offer support and protection. “Your voice matters,” she says.