Sound Research WIKINDX

"When we perform risk/benefit analyses of language technology, we must keep in mind how the risks and benefits are distributed, because they do not accrue to the same people. On the one hand, it is well documented in the literature on environmental racism that the negative effects of climate change are reaching and impacting the world’s most marginalized communities first. Is it fair or just to ask, for example, that the residents of the Maldives (likely to be underwater by 2100) or the 800,000 people in Sudan affected by drastic floods pay the environmental price of training and deploying ever larger English LMs, when similar large-scale models aren’t being produced for Dhivehi or Sudanese Arabic? And, while some language technology is genuinely designed to benefit marginalized communities, most language technology is built to serve the needs of those who already have the most privilege in society. Consider, for example, who is likely to both have the financial resources to purchase a Google Home, Amazon Alexa or an Apple device with Siri installed and comfortably speak a variety of a language which they are prepared to handle. Furthermore, when large LMs encode and reinforce hegemonic biases, the harms that follow are most likely to fall on marginalized populations who, even in rich nations, are most likely to experience environmental racism."

"The Internet is a large and diverse virtual space, and accordingly, it is easy to imagine that very large datasets, such as Common Crawl (“petabytes of data collected over 8 years of web crawling”,a filtered version of which is included in the GPT-3 training data) must therefore be broadly representative of the ways in which different people view the world. However, on closer examination, we find that there are several factors which narrow Internet participation, the discussions which will be included via the crawling methodology, and finally the texts likely to be contained after the crawled data are filtered. In all cases, the voices of people most likely to hew to a hegemonic viewpoint are also more likely to be retained. In the case of US and UK English, this means that white supremacist and misogynistic, ageist, etc. views are overrepresented in the training data, not only exceeding their prevalence in the general population but also setting up models trained on these datasets to further amplify biases and harms. Starting with who is contributing to these Internet text collections, we see that Internet access itself is not evenly distributed, resulting in Internet data overrepresenting younger users and those from developed countries. However, it’s not just the Internet as a whole that is in question, but rather specific subsamples of it. For instance, GPT-2’s training data is sourced by scraping outbound links from Reddit, and Pew Internet Research’s 2016 survey reveals 67% of Reddit users in the United States are men, and 64% between ages 18 and 29. Similarly, recent surveys of Wikipedians find that only 8.8–15% are women or girls"

"Thus at each step, from initial participation in Internet fora, to continued presence there, to the collection and finally the filtering of training data, current practice privileges the hegemonic viewpoint. In accepting large amounts of web text as ‘representative’ of ‘all’ of humanity we risk perpetuating dominant viewpoints, increasing power imbalances, and further reifying inequality."

"Developing and shifting frames stand to be learned in incomplete ways or lost in the big-ness of data used to train large LMs — particularly if the training data isn’t continually updated. Given the compute costs alone of training large LMs, it likely isn’t feasible for even large corporations to fully retrain them frequently enough to keep up with the kind of language change discussed here. Perhaps fine-tuning approaches could be used to retrain LMs, but here again, what would be required is thoughtful curation practices to find appropriate data to capture reframings and techniques for evaluating whether such fine-tuning appropriately captures the ways in which new framings contest hegemonic representations."

"Birhane and Prabhu note, echoing Ruha Benjamin [15], “Feeding AI systems on the world’s beauty, ugliness, and cruelty, but expecting it to reflect only the beauty is a fantasy.” [p.1541]"

"We say seemingly coherent because coherence is in fact in the eye of the beholder. Our human understanding of coherence derives from our ability to recognize interlocutors’ beliefs [30, 31] and intentions [23, 33] within context [32]. That is, human language use takes place between individuals who share common ground and are mutually aware of that sharing (and its extent), who have communicative intents which they use language to convey, and who model each others’ mental states as they communicate. As such, human communication relies on the interpretation of implicit meaning conveyed between individuals. The fact that human-human communication is a jointly constructed activity [29, 128] is most clearly true in co-situated spoken or signed communication, but we use the same facilities for producing language that is intended for audiences not co-present with us (readers, listeners, watchers at a distance in time or space) and in interpreting such language when we encounter it. It must follow that even when we don’t know the person who generated the language we are interpreting, we build a partial model of who they are and what common ground we think they share with us, and use this in interpreting their words."

"But to reason in this way is precisely not to submit to the disruption promoted by Californian ‘digital business’. For there is such a submission, not yet to come, as the poisonous fantasies distilled by a culture and a business of fear would have it, but rather right now, as resignation in the face of the diktat of ‘radical innovation’ in the service of the ‘ecosystem’ centred on Californian ‘digital business’."

"What is new is the systematic exploitation and physical reticulation of interindividual and transindividual relations – serving what is referred to today as the ‘data economy’, itself based on data-intensive computing, or ‘big data’, which has been presented as the ‘end of theory’. This amounts to the full realization of barbarism in Adorno and Horkheimer’s sense, but they could surely never have imagined how far this would extend onto the noetic plane."

"The automatic power of reticulated disintegration extends across the face of the earth through a process that has recently become known as disruption. Digital reticulation penetrates, invades, parasitizes and ultimately destroys social relations at lightning speed, and, in so doing, neutralizes and annihilates them from within, by outstripping, overtaking and engulfing them. Systemically exploiting the network effect, this automatic nihilism sterilizes and destroys local culture and social life like a neutron bomb: what it dis-integrates, it exploits, not only local equipment, infrastructure and heritage, abstracted from their socio-political regions and enlisted into the business models of the Big Four, but also psychosocial energies – both of individuals and of groups – which, however, are thereby depleted."

Continues...

The automatic protentions define our will and desires we have no chance to fight back
"Disruption moves quicker than any will, whether individual or collective, from consumers to ‘leaders’, whether political or economic.28 Just as it overtakes individuals via digital doubles or profiles on the basis of which it satisfies ‘desires’ they have most likely never expressed – but which are in reality herd-like substitutes depriving individuals of their own existence by always preceding their will, at the same time emptying them of meaning, while feeding the business models of the data economy – so too disruption outstrips and overtakes social organizations, but the latter recognize this only after the fact: always too late."

"In the contemporary epoch of the absence of epoch, the role of digital tertiary retentions in the intergenerational (non)relationship, and in the (non)formation of collective retentions and protentions, is both perfectly obvious and totally escapes comprehension – because there is no longer any adjustment between the new technical system and the social systems. Far from adjusting the social systems by reshaping them to suit a ‘new epoch’, the technical system short-circuits them and, ultimately, destroys them."

"[...] the shift from hand-tools to factory machines is a process of grammatization, and so is genetic engineering—cells and organs become replicated and revised like a kind of alphabet. In every case, a continuous flux (e.g., speech, the body, the genome) becomes broken down into a system of discrete elements (e.g., alphabetic characters, mechanical systems, recombinant DNA sequences). And, in every case, the latter’s emergence always disrupts, transforms, and reconfigures the former"

"Simply put, permanent innovation names a historical condition in which technical invention outpaces cultural adaptation (Stiegler, 1998, p. 15). This is not to suggest that technology and culture are divisible; for Stiegler, all social practices and institutions have a technological basis. Precisely because of this interdependence, the adoption of a new cultural technology precipitates the obsolescence, to greater and lesser degrees, of traditions that were developed in conjunction with the affordances of a becoming-obsolete technology. While this sort of technocultural evolution is evident throughout history, technical invention became infused with entirely unprecedented levels of resources and investments starting with the industrial revolution, during which economic incentives abounded to transform scientific knowledge from an academic pursuit into a lucrative industrial research and development enterprise (Stiegler, 1998, p. 40). As such, industrial technical invention has come to outpace conceptual innovation in other social systems such as law, government, and education"

Research turned ito a lucrative industrial research. This is still true with AI, perhaps more than ever, and it (mostly) hides under the guide of doing science that will help humanity as a whole. Importantly, regulation of AI as it is developing shows this very well - it cannot keep up at all.

"The computer revolution continues to intensify this rift as industrial activities turn toward the production of information technologies that archive, manage, and structure individual and collective memory [my emphasis]. The rapid evolution of global networks increasingly disorients and scrambles the traditions, conventions, and practices that have defined national institutions over previous centuries. Stiegler (1998) described this widespread sense of disorientation:

[W]e are experiencing the deep opacity of contemporary technics; we do not understand what is being played out in technics, nor what is being transformed therein, even though we unceasingly have to make decisions regarding technics. . [I]n day to day technical reality, we cannot spontaneously distinguish the long-term processes of transformation from spectacular but fleeting technical innovations. (p. 21)"

This is arguably even more dire with LLMs - their impact on our collective memory (touched on in a musing below) is inevitably going to be profound, and I think pernicious.

"Focusing on grammatization can help digital rhetoricians “distinguish the long-term processes of transformation from spectacular but fleeting technical innovations” (Stiegler, 1998, p. 21), and, most importantly, help us to identify robust categories of production and analysis integral to a variety of contemporary rhetorical situations. Electracy and remediation, I will argue, have only circled around this latter task."

Goes on to show how remediation functions primarily as a genealogy of a given technology/medium, and therefore fail to account for what is actually new about new media.

"Stiegler postulated that to study emerging media and writing technologies was to attend to the constitutive forces that condition the possibilities of human becoming, which were always shifting from epoch to epoch, especially in permanent innovation societies. Accordingly, his realignment of techno-human relations put writing at the epicenter of contemporary and projected cultural and ontological transformations. Humans, as a species, were not born into the world already equipped with mature cognitive capacities; these capacities developed over time in a transductive relationship with Neolithic technics, and they are still developing today hand in glove through our collective play with contemporary technics. Informed by philosopher Gilbert Simondon, Stiegler routinely defined technics as organized inorganic matter.” The term refers both to the history of fabricated objects (e.g., flint, hammers, pencils, computers) and to the domain of techne: the techniques and practices involved in making (something with) technology. Technics are more than merely a part of the environment humans inhabit; technics constitute—not determine—our experience on every possible level, from retention to anticipation, and from cultural history to genetics."

The "organised inorganic matter" comes directly from Simondon - computers, and LLMs by extension - are the same as hammers in this sense.

Similar to postphenomenology of Ihde and Verbeek

"Stiegler (2010a) asserted that “the static oppositions of Western metaphysics must be replaced by dynamic compositions: one must think in terms, not of hierarchies or totalizing systems, but of processes.” (p. 69). Beyond the dialogic sense of co-evolution, Stiegler circumvented the persistent opposition between humanity and technology that proponents of technogenesis still employ.3 Instead, critical attention starts by studying the constitutive processes that underwrite so-called humans and technologies—before considering them as separate, already-constituted entities. These constitutive processes, in Stiegler’s lexicon, are called processes of grammatization."

Again very similar to postphenomenology, which states the exact same thing-tech form humans form tech. Seems also quite like Simondon's concept of individuation, but in slightly different terms. 

This is one issue I take with the extended mind thesis - Clark especially believes we are natural born cyborgs, but fails to account for what might be lost when we replace certain cognitive functions with technics - the technologies only appropriate, but if they assume the position of a cognitive function, then our cognitive function matches that of the technology regardless of whether it is the same or not.

...This I suppose is the essence of simulacrum?

"Stiegler, even more than Derrida, professed to see life in the world—human becoming, historical change, social organization—as the evolution and play of gramme (the Greek word for “written mark”). What atoms are to physicists, gramme are to Stiegler."

"Gradually, the introduction of alphabetic writing furnished the Greeks with a new form of tertiary retention (or technical memory), and hence a new medium for educational practices. The students of early manuscript cultures (an elite group, to be sure) could inscribe the word into relative permanence, and do it without emotional identification, regular recitation, or a harmonious and mimetic style (the essential technics of oral poetry). Sharing the burden of memory with alphabetic writing materials, they were able to move from acts of recitation to exercises in abstraction (Havelock, 1963). More generally, grammatization “is the production and discretization of structures;” the formalization of a new gramme accrues through metonymic invention, whereby a part becomes detached from the whole of a continuous flux (Stiegler, 2011, p. 173)."

"Stiegler stretched grammatological theory even further, adapting it into a basis for understanding all manner of technical systems beyond those traditionally regarded as writing or communication technologies. In our age of ubicomp, this move is incredibly pertinent. Increasingly, more everyday objects are becoming smart objects, everyday practices are becoming digitally supplemented, and technics in every domain are becoming mnemotechnics (i.e., systems capable of capturing, storing, and sharing information). From an expanded notion of writing, Stiegler prompted us to recognize the expansive array of contemporary artif(acts) that now operate as gramme."

I think there's a case to be made that in many cases it doesn't just supplement, but supplant. Meta glasses being able to 'find' hidden keys means we don't even need to keep track of our keys (if anybody has keys nowadays).. What does that mean for instance for physical organisation? Who cares if our glasses are always there to be our eyes?

"This suggestive dimension of Stiegler’s (2012a) thought stems from his belief in a general interplay of gramme and gestures: “To separate the question of language, alphabet, photograph, cinematograph, audio recording, and so on from all the questions of gestures is a very big error.” Words, images, sounds: these are not the only gramme. According to Stiegler, some of the most important processes of grammatization—historically and especially today—involve the becoming-gramme of territories, bodies, and even cells. In addition to writing by virtue of bodily gestures, our seemingly non-discursive bodily gestures—by virtue of machines—can produce writing of sorts, to the extent that such gestures are been broken down from the flow of muscular continuity, detached from our bodies and iterable in their absence. For instance, the gesture or action of hammering a nail can be performed by a discreet and replicatable machinic configuration, just as the letters of the alphabet can be arranged to record a speaker’s every word. In grammatizing the laborer’s gesture, industrialization precipitates proletarianization, or “the loss of know-how” on the part of laborers (Stiegler, 2010a, p. 71). "

One important aspect here (ignoring the pride the workman takes in his physical labour, which is also important), is that assessing the quality of knowledge work (writing) is very difficult because it is inherently ambiguous. It's quality lies also in what was learned in the process (similar to the workman, but with differences). In this sense, the hammer goes in regardless of whether it is done by the workman or the machine, but with knowledge work, though the outcomes might functionally produce a similar bit of text (big if), then they are still not the same.

... continues

"For Stiegler, industrialization thus constitutes a pivotal stage of grammatization, which becomes intensified by digitization as industrial resources and investments center on the programming of behavioral models engendered through the production and commoditization of audiovisual media. Whereas prior grammatization processes dealt primarily in the “sphere of language, of logos” throughout much of writing’s history, industrialization “came to invest [in] the sphere of bodies,” and the age of mass media turned mechanization toward “apparatus-dependent reproducibilities of the visible and the audible” (Stiegler, 2010a, p. 70). "

RLHF workers for AI companies are literally filming everything to be able to automate every single human function (also the very point of meta glasses and the goal of that horrendous company). Ironically for text-training, some of these workers were fired for using AI to do their job - AI verifying AI leads to model collapse. There might also be another link here. Though prolitarianisation focuses on the labourer's know-how, Albert Borgmann's device paradigm says that something else is lost (for the consumer) when we let machines take over tasks.

The question is more so one of automation - what do we lose when we automate certain tasks? Not merely in the actual quality of the work, which in some cases would be actual gains, but in terms of things like skills, quality of life, expertise and deep knowledge of institutions and projects...

"Today, runners wearing a Nike+ sensor generate a digital data visualization as they run; each stride produces an indexical trace recorded and stored on an online database. They run, and their running generates computing, writing, or data in real-time. In addition to more traditional mnemotechnics, grammatization now engulfs all manner of machines, which variously enable and delimit a general writing of particular gestures and embodied activities. Defining all writing technologies as pharmakon, Stiegler (2011) warned that hyperindustrial investment in digital machines was contributing to a general proletarianization of the consumer’s existence to an even more pervasive extent than the industrial investment of factory machines effected a proletarianization of the worker’s labor."

Very much in line with Borgmann's device paradigm - something is lost in the consumer's existence.

"Ultimately, Stiegler’s grammatological analysis of (hyper)industrial activities suggests an altogether different account of technical evolution and techno-human relations than the familiar notion of augmentation, which models all technologies as prosthetic extension of innate human capabilities. Conceived as processes of grammatization, the organization of inorganic matter that technologies are results not from “an extension of man” but an appropriation, a writing, defined by its disjunction from the continuity of a flux or movement, human or otherwise. Once disjointed and inscribed autonomously as inorganic matter, the movement becomes a gramme and, therefore, unfolds by the logic of the supplement and the nuclear traits of (arche-)writing. Theories rooted in augmentation cannot account for the autonomous, iterability of technics and their originary, constitutive force within the development of cultural practices, social organization, and even the human body itself."

This to me is a core issue of LLMs. They essentially outsource (downright supplant) thinking, but they're not thinking, so a person heavily reliant on LLMs do their thinking in the exact same manner as an LLM. For instance, in the original extended mind thesis paper by Chalmers and Clark, they show that keeping a notebook works directly as supplanting memory for a patient who can't remember. But the function of this person's memory is then also 'reduced' (arguably - some would argue augmented in some instances) to whatever the technology offers.

Typing speeds up recording of thoughts, but by doing so limit the time for thinking, making it suitable for recording ideas in a flow to later be organised and reflected upon, but offers no advantages, and definitely some disadvantages, for doing work that requires a deep, slow understanding - for instance note taking, writing a paper... (digital storage, collaboration practices, etc. in this case are different from the writing practice itself)...

Above - "a writing" refers then to the reduction of the physical world to symbols? It changes the very fabric of society and humanity and therefore, as stated, cultural practice, social organisation, even the human body.

"Supplements, such as writing technologies, are originary to human becoming; the exteriority of technics exceeds and conditions the interior life of the mind, “the already-there inhabits invention itself” (Stiegler, 2009, p. 42). "

"Amid permanent innovation, one can approach changes in techne from the standpoint of Stiegler’s episteme (i.e., grammatization), in order to identify which processes of change actually disrupt established gramme and, in so doing, contribute the emergence of new gramme. Such becomes the criteria by which to evaluate and describe the (lack of) significance that any given permutation bears for the history and future of writing and rhetoric. If an apparent technological innovation does not markedly disrupt the history of supplementarity, or extend its logic to new domains, then it will not likely hold significance for digital rhetoric moving forward"

"Grammatization, however, works rigorously on both sides of this dichotomy, oscillating between precedent and novelty. Media evolution is conceived as a matter of metonymic invention, whereby some part becomes detached from a continuous whole. This part, now discrete and iterable apart from the whole, acts as a gramme, which is to say it embodies the technics of writing, manifesting the logic of supplementarity within new spheres of practice. "

Another description of grammatisation as media evolution. 

"By contrast, grammatological analogies, which juxtapose the history and technics of writing with the present state of digital media, refer to older media with the aim of pinpointing:

1) disruptive or unprecedented aspects of emerging technologies, and

2) urgent tasks for theorists and practitioners concerned withthe cultural development of such technologies.

Comparing different processes of grammatization becomes a generative ground for conceptual invention, not a means of legitimating a myth of origin (like with remediation)

"YouTube—or, more accurately, the process of grammatization we can ascertain from its emergence—shows that electronic and digital media have not hopelessly handed over culture to the culture industry. In short-circuiting processes of collective individuation, the consumerist model of the cultural industry is the threat to democracy—not audiovisual production in and of itself, which is ultimately a pharmakon (Stiegler, 2010b)"

As I understand this - - - blame not the actual medium, but the conditions around it, because the medium is a pharmakon, meaning it can be both remedy and poison. This seems more true than ever - stealth adds through AI influencers, and earlier regular influencers, would have little incentive (beyond trolling, I suppose) unless it could generate income for the person doing it.

"Stiegler (2010b) claimed that “in its first stages of development, writing doesn’t produce any reflexivity” (p. 44), as he pointed out that cuneiform was performed exclusively by an elite class of scribes whose primary tasks focused on accounting and administrative recordkeeping."

This is also what postpheonmenology posits, namely that tech cannot be seen outside of how it is actually used. It's capabilities reveal themselves over time to reflect the needs and values of those who use them, and that would include the conditions surrounding it - as such, no tech is inherently evil.This is a sort of psychic and collective individuation.

...but perhaps a tech (like LLMs) have no actual use-case in which they are good? 

"Dog–human games are frequently claimed to have effects upon the relationship between the players. Consistently allowing a dog to win games is alleged to result in the dog perceiving itself as stronger than its owner, leading to subsequent conflict and behavioural problems McBride, 1995. This idea is based on extrapolation of intraspecific behaviours, particularly those of the wolf. Winning possession of toys is described as simulating the winning of the battle for the best meat at the end of a pack hunt  Appleby, 1997, p. 182., which can have consequences for the social hierarchy. This idea assumes that play is a contest and the goal is to possess the toy. Although this was the case during dog–dog play, we saw no evidence for it during dog–human play. Since dogs react differently to human and dog play partners, we see no reason to assume that the consequences of dog–human games are the same as dog–dog games. Decreased competitiveness may mean that the outcome of dog–human games is less likely to affect the players’ relationship than has been suggested by some authors."

Appleby, D., 1997. Ain’t Misbehavin: A Good Behaviour Guide for Family Dogs. Broadcast Books, Bristol.

"Without subscribing to a technological determinist position – for him there is no value in trying to oppose cultural or technological factors to each other in searching for some original condition of human–technical becoming – he will propose that the constitutive condition of human cultural development is one that is always already dis-adjusted, moving toward a metastabilization that is always conditional. It is a core claim of Stiegler’s work that the “who” and the “what” must not be thought in oppositional terms (Stiegler 1998: 177–8). Human and technics compose together a dynamic of mutual becoming."

"For Stiegler, Heidegger fails to follow his own analysis to the realization that technical objects are indispensable to the human experience of time. If as Heidegger argues Dasein is time, as temporality, that is, as remembering and anticipating from an always “enchained” present moment, then this is on the basis of its technically conditioned encounter with the world. The tool, the technique – and in this view both language and material objects are co-implied, composed in human being – are always memories whose deployment is always in anticipation of an outcome [my emphasis]. The knapped flint or the hammer preserves the memory of the gestures which produced it. As a medium or milieu of human existence, temporality is constituted out of this anticipatory projection from a past recollected on the basis of technical facticity. The development of tools and techniques specifically dedicated to support memory, which Stiegler calls “mnemotechnics,” from imaging to storytelling to writing to the digital database, are a specialization of the memorial aspect of every tool."