“Ethics owners operate inside a fraught dynamic: on the one hand attempting to resolve critical external normative claims about the core logics of the tech industry; on the other hand doing so while fully embedded within those logics. Attempts to institutionalize ethics within entities structured by core logics of corporate power point towards a series of structural, conceptual, and procedural pitfalls that may ultimately stymie these efforts.” (451)
- The thin line that algorithm writers & ethicists have to walk.
“While the engineer wants to see more bottom-up engagement with ethics from her fellow engineers, the management consulting leader sees the future of ethics in the tech industry as the implementation of processes and protocols to eliminate organizational risk across a broad range of domains, from reputational risk to liability, as well as a hedge against being seen as inattentive to these risks (Power 1997). At the same time, they share the view that the practice of ethics would require financial or organizational sacrifice. These two ethics stakeholders come from different organizational vantage points, but they both also consistently refer to a set of cultural logics that are widely viewed as normative, if not desirable, in the tech industry.” (460)
“there may be no “silver bullet” for ethical problems, the search for technological solutions (Ball-Rokeach et al. 2004) that use “the super powers given to you by tech,” as one lead researcher put it, is strongly foregrounded in interviews and public discourse. Stated desires for toolkits and checklists posit that a technological solution is possible, even if it is yet to be fully developed. In some accounts, this gives the impression that ethics problems arise from imperfect technical solutions, which implies the inverse: ethical products will be better products. As a senior management consultant explained, “you’re not going to have an issue with the AI if your data is all good and you’ve curated that appropriately … checked for fairness … mitigated against negative bias … and carried metadata along so that you can revisit decisions. … And the problem exists when you don’t do those things.”” (464)
“As tech companies start to envision procedures to operationalize ethics, they do so according to their own internal logics. As outlined by the senior leader in management consulting, one common way of approaching ethics is through legal or business risk accounting. Such an approach tasks senior leaders with assessing the risk to the corporation in terms of legal, financial, or reputational risk (Orlitzky and Benjamin 2001). Risk accounting describes risk and determines its acceptable levels, defining the situation in ways that let leaders manage potential consequences within a range of acceptable costs to the firm. Another approach is more deeply rooted in engineering and security practices.” (467)
“Collectively, with an eye towards avoiding risk, companies tend to steer their decisionmaking to respond to, and ideally avoid, public calamities. They are far less likely to share or learn from others’ successful actions. In this process, they create a form of “blinkered isomorphism.” Like a pony wearing blinkers, tech companies reproduce the same ethical blind spots across the entire industry through a process rooted in what DiMaggio and Powell (1983) call “institutional isomorphism.” This concept explains how companies and sectors converge on similar structures. In the context of tech companies, this can become “innovation through imitation” (Caplan and boyd 2018).” (469)
“While it is too early to know if the nascent efforts to effect change in tech companies will actually produce more ethical organizations, we are concerned that the organizational, cultural, and structural conditions inhibit such lofty goals. If ethics is simply absorbed within the logics of market fundamentalism, meritocracy, and technological solutionism, it is unlikely that the tech sector will be able to offer a meaningful response to the desire for a more just and values-driven tech ecosystem. This is a fate we would prefer to avoid. The question at hand for both critical data scholars and ethics owners inside these companies is how to find a route to an improved mode of “doing ethics” that institutionalizes a more open, just, and critical everyday practice.” (472)
- We might not be able to change all at once, but can change little by little.
Metcalf, Jacob. Moss, Emanuel. Boyd, Danah. “Owning Ethics: Corporate Logics, Silicon Valley, and the Institutionalization of Ethics.” An International Quarterly, Volume 86, Number 2, Summer 2019, pp. 449-476