Does the Algorithm have the Last Word?

In my post on the performativity of data (August 2021), I looked at some of the ways in which data and information can make something true. In this post, I want to go further. What if an an algorithm can make something final.

I've just read a very interesting paper by the Canadian sociologist Arthur Frank, which traces the curious history of a character called Devushkin - from a story by Gogol via another story by Dostoevsky into some literary analysis by Bakhtin.

In Dostoevsky's version, Devushkin complained that Gogol's account of him was complete and final, leaving him no room for change or development, hopelessly determined and finished off, as if he were already quite dead.

For Bakhtin, all that is unethical begins and ends when one human being claims to determine all that another is and can be; when one person claims that the other has not, cannot, and will not change, that she or he will die just as she or he always has been. Frank

But that's pretty much what many algorithms do. Machine learning algorithms extrapolate from historical data, captured and coded in ways that reinforce the past, while more traditionally programmed algorithms simply betray the opinions and assumptions of their developers. For example, we see recruitment algorithms that select men with a certain profile, while rejecting women with equal or superior qualifications. Because that's what's happened in the past, and the algorithm has no way of escaping from this.

 

I'm going to keep saying this, I think: just as we have a right to download the data any corp holds on us, we should also have the right to reset any view an algorithm has of us.

We used to be able to redefine and evolve ourselves. Algorithmic views of us hold us back.

— Russell Garner (@rgarner) March 4, 2022

The inbuilt bias of algorithms has been widely studied. See for example Safiya Noble and Cathy O'Neil.

David Beer makes two points in relation to the performativity of algorithms. Firstly through their material interventions.

Algorithms might be understood to create truths around things like riskiness, taste, choice, lifestyle, health and so on. The search for truth becomes then conflated with the perfect algorithmic design – which is to say the search for an algorithm that is seen to make the perfect material intervention.

And secondly through what he calls discursive interventions.

The notion of the algorithm is part of a wider vocabulary, a vocabulary that we might see deployed to promote a certain rationality, a rationality based upon the virtues of calculation, competition, efficiency, objectivity and the need to be strategic. As such, the notion of the algorithm can be powerful in shaping decisions, influencing behaviour and ushering in certain approaches and ideals.

As Massimo Airoldi argues, both of these fall under what Bourdieu calls Habitus - which means an inbuilt bias towards the status quo. And once the algorithm has decided your fate, what chance do you have of breaking free?

---

Massimo Airoldi, Machine Habitus: Towards a sociology of algorithms (Polity Press, 2022)

David Beer, The social power of algorithms (Information, Communication & Society, 20:1, 2017) 1-13, DOI: 10.1080/1369118X.2016.1216147

Safiya Noble, Algorithms of Oppression (New York University Press, 2018)

Cathy O'Neil, Weapons of Math Destruction (2016)

Arthur W. Frank, What Is Dialogical Research, and Why Should We Do It? (Qual Health Res 2005; 15; 964) DOI: 10.1177/1049732305279078

Carissa Véliz, If AI Is Predicting Your Future, Are You Still Free? (Wired, 27 December 2021)

Related posts: Could we switch the algorithms off? (July 2017), Algorithms and Governmentality (July 2019), Algorithmic Bias (March 2021), On the performativity of data (August 2021)