Sound Research WIKINDX
Sound Research WIKINDX |
|
 |
|
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. 2021 On the dangers of stochastic parrots: Can language models be too big? 🦜. Paper presented at Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency.
Added by: Mark Grimshaw-Aagaard 05/05/2026, 04:48 Last edited by: alexb44 29/09/2026, 07:51 |
|
| Resource type: Proceedings Article Peer reviewed Published DOI: 10.1145/3442188.3445922 ID no. (ISBN etc.): 9781450383097 BibTeX citation key: Bender2021 Email resource to friend View all bibliographic details  |
Language: en: English Categories: General Keywords: Artificial creativity, Artificial Intelligence Creators: Bender, Gebru, McMillan-Major, Shmitchell Collection: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency |
Views: 18/106
|
| Abstract |
|
The past 3 years of work in NLP have been characterized by the development and deployment of ever larger language models, especially for English. BERT, its variants, GPT-2/3, and others, most recently Switch-C, have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pretrained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We provide recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models.
Added by: Mark Grimshaw-Aagaard Last edited by: Mark Grimshaw-Aagaard |
| Quotes |
Hegemonic view of the world gets strengthened
"Thus at each step, from initial participation in Internet fora, to continued presence there, to the collection and finally the filtering of training data, current practice privileges the hegemonic viewpoint. In accepting large amounts of web text as ‘representative’ of ‘all’ of humanity we risk perpetuating dominant viewpoints, increasing power imbalances, and further reifying inequality."
Added by: alexb44
29/09/2026, 07:43
|
|
p. 612
Environmental racism
"When we perform risk/benefit analyses of language technology, we must keep in mind how the risks and benefits are distributed, because they do not accrue to the same people. On the one hand, it is well documented in the literature on environmental racism that the negative effects of climate change are reaching and impacting the world’s most marginalized communities first. Is it fair or just to ask, for example, that the residents of the Maldives (likely to be underwater by 2100) or the 800,000 people in Sudan affected by drastic floods pay the environmental price of training and deploying ever larger English LMs, when similar large-scale models aren’t being produced for Dhivehi or Sudanese Arabic? And, while some language technology is genuinely designed to benefit marginalized communities, most language technology is built to serve the needs of those who already have the most privilege in society. Consider, for example, who is likely to both have the financial resources to purchase a Google Home, Amazon Alexa or an Apple device with Siri installed and comfortably speak a variety of a language which they are prepared to handle. Furthermore, when large LMs encode and reinforce hegemonic biases, the harms that follow are most likely to fall on marginalized populations who, even in rich nations, are most likely to experience environmental racism."
Added by: alexb44
29/09/2026, 07:40
|
|
p. 613
The internet is far from representative
"The Internet is a large and diverse virtual space, and accordingly, it is easy to imagine that very large datasets, such as Common Crawl (“petabytes of data collected over 8 years of web crawling”,a filtered version of which is included in the GPT-3 training data) must therefore be broadly representative of the ways in which different people view the world. However, on closer examination, we find that there are several factors which narrow Internet participation, the discussions which will be included via the crawling methodology, and finally the texts likely to be contained after the crawled data are filtered. In all cases, the voices of people most likely to hew to a hegemonic viewpoint are also more likely to be retained. In the case of US and UK English, this means that white supremacist and misogynistic, ageist, etc. views are overrepresented in the training data, not only exceeding their prevalence in the general population but also setting up models trained on these datasets to further amplify biases and harms. Starting with who is contributing to these Internet text collections, we see that Internet access itself is not evenly distributed, resulting in Internet data overrepresenting younger users and those from developed countries. However, it’s not just the Internet as a whole that is in question, but rather specific subsamples of it. For instance, GPT-2’s training data is sourced by scraping outbound links from Reddit, and Pew Internet Research’s 2016 survey reveals 67% of Reddit users in the United States are men, and 64% between ages 18 and 29. Similarly, recent surveys of Wikipedians find that only 8.8–15% are women or girls"
Added by: alexb44
29/09/2026, 07:41
|
|
p. 614
Static data - always changing world
"Developing and shifting frames stand to be learned in incomplete ways or lost in the big-ness of data used to train large LMs — particularly if the training data isn’t continually updated. Given the compute costs alone of training large LMs, it likely isn’t feasible for even large corporations to fully retrain them frequently enough to keep up with the kind of language change discussed here. Perhaps fine-tuning approaches could be used to retrain LMs, but here again, what would be required is thoughtful curation practices to find appropriate data to capture reframings and techniques for evaluating whether such fine-tuning appropriately captures the ways in which new framings contest hegemonic representations."
Added by: alexb44
29/09/2026, 07:45
Comments:
This problem is only exacerbated as models grow bigger - they become increasingly more expensive to train and therefore will struggle more and more to deal with these issues.
I suppose the silver lining is that those who can actually afford to use LLMs in the future will all be sufficiently mecanised to the point that they fully align with the model biases, and there might be nobody else left who is actually discriminated against /s Added by: alexb44 29/09/2026, 07:45 |
|
p. 615
They feed it beauty, ugly and cruel and expect only beauty.
"Birhane and Prabhu note, echoing Ruha Benjamin [15], “Feeding AI systems on the world’s beauty, ugliness, and cruelty, but expecting it to reflect only the beauty is a fantasy.” [p.1541]"
Added by: alexb44
29/09/2026, 07:46
|
|
p. 616
Coherence in the Eye of the Beholder
"We say seemingly coherent because coherence is in fact in the eye of the beholder. Our human understanding of coherence derives from our ability to recognize interlocutors’ beliefs [30, 31] and intentions [23, 33] within context [32]. That is, human language use takes place between individuals who share common ground and are mutually aware of that sharing (and its extent), who have communicative intents which they use language to convey, and who model each others’ mental states as they communicate. As such, human communication relies on the interpretation of implicit meaning conveyed between individuals. The fact that human-human communication is a jointly constructed activity [29, 128] is most clearly true in co-situated spoken or signed communication, but we use the same facilities for producing language that is intended for audiences not co-present with us (readers, listeners, watchers at a distance in time or space) and in interpreting such language when we encounter it. It must follow that even when we don’t know the person who generated the language we are interpreting, we build a partial model of who they are and what common ground we think they share with us, and use this in interpreting their words."
Added by: alexb44
29/09/2026, 07:51
Comments:
Useful sources in relation to why we think LLM output is coherent:
[23] - Susan E Brennan and Herbert H Clark. 1996. Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology: Learning, Memory, and Cognition 22, 6 (1996), 1482 [29] - Herbert H. Clark. 1996. Using Language. Cambridge University Press, Cambridge. [30] - Herbert H. Clark and Adrian Bangerter. 2004. Changing ideas about reference. In Experimental Pragmatics. Springer, 25–49. [31] - Herbert H. Clark and Meredyth A Krych. 2004. Speaking while monitoring addressees for understanding. Journal of Memory and Language 50, 1 (2004),62–81 [32] - Herbert H. Clark, Robert Schreuder, and Samuel Buttrick. 1983. Common ground at the understanding of demonstrative reference. Journal of Verbal Learning and Verbal Behavior 22, 2 (1983), 245 – 258. https://doi.org/10.1016/S0022- 5371(83)90189-5 [33] - Herbert H. Clark and Deanna Wilkes-Gibbs. 1986. Referring as a collaborative process. Cognition 22, 1 (1986), 1 – 39. https://doi.org/10.1016/0010-0277(86) 90010-7 [128] - Greg J. Stephens, Lauren J. Silbert, and Uri Hasson. 2010. Speaker–listener neural coupling underlies successful communication. Proceedings of the National Academy of Sciences 107, 32 (2010), 14425–14430. https://doi.org/10.1073/pnas. 1008662107 arXiv:https://www.pnas.org/content/107/32/14425.full.pdf Added by: alexb44 29/09/2026, 07:51 |