Date: November 1 2021
Summary: How algorithms and mathematics can be used to construct active harm across populations
Keywords: #archive #book #weapon #math #destruction
O'neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. Broadway Books.
"Sometimes the job of a data scientist is to know when you don't know enough."
According to O'neil, profits have often come about to provide a stand-in as a value judgement on what is right or true. To that end, we are often vetted by algorithmically based on who we are (e.g. our backgrounds, race, etc.) and not on how we actually perform or fare.
Model construction happens from:
not just from data but from the choices we make about which data to pay attention to—and which to leave out.
Those choices are not just about logistics, profits, and efficiency. They are fundamentally moral. We have to learn to interrogate our data collection process, not just our algorithms.
When considering credit scores, money or wealth is no longer just about a means of survival. Wrapped up in these scores is also a value judgement on someone's self-worth.
Algorithms prioritize two factors: efficiency and profit. They do not care about justice or the well-being of the people underlying the data they are analyzing.
So the first step is to get a grip on our techno-utopia, that unbounded and unwarranted hope in what algorithms and technology can accomplish. Before asking them to do better, we have to admit they can’t do everything
audits have to be carefully designed and tested by human beings, and afterward automated
Zelko, Jacob. Weapons of Math Destruction. https://jacobzelko.com/11012021133502-weapons-math-destruction. November 1 2021.