That's what I was thinking when I read a similar article on inserting fairness into search algorithms. Fair to whom?

This paper has been a struggle to write. It’s a strange paper in that the main technical contribution is mainly conceptual: establishing what we think are the right basic primitives that can be used to express (mathematically) concepts like fairness, nondiscrimination, and structural bias.

When you start adding algorithms into real life data, someone is playing "God" with the data. Who gets to determine what is fair and why?