
Zusammenfassungen

The past few years, ever since processing capacity caught up with
neural models, have been heady times in the world of NLP. Neural
approaches in general, and large, Transformer LMs in particular,
have rapidly overtaken the leaderboards on a wide variety of benchmarks
and once again the adage “there’s no data like more data”
seems to be true. It may seem like progress in the field, in fact, depends
on the creation of ever larger language models (and research
into how to deploy them to various ends).
In this paper, we have invited readers to take a step back and
ask: Are ever larger LMs inevitable or necessary? What costs are
associated with this research direction and what should we consider
before pursuing it? Do the field of NLP or the public that it serves
in fact need larger LMs? If so, how can we pursue this research
direction while mitigating its associated risks? If not, what do we
need instead?
Von Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell im Text On the Dangers of Stochastic Parrots (2021) The past 3 years of work in NLP have been characterized by the development and deployment of ever larger language models, especially for English. BERT, its variants, GPT-2/3, and others, most recently Switch-C, have pushed the boundaries of the possible both through architectural innovations and through sheer size. Using these pretrained models and the methodology of fine-tuning them for specific tasks, researchers have extended the state of the art on a wide array of tasks as measured by leaderboards on specific benchmarks for English. In this paper, we take a step back and ask: How big is too big? What are the possible risks associated with this technology and what paths are available for mitigating those risks? We provide recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values, and encouraging research directions beyond ever larger language models.
Von Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, Shmargaret Shmitchell im Text On the Dangers of Stochastic Parrots (2021)
Dieser wissenschaftliche Zeitschriftenartikel erwähnt ...
![]() Personen KB IB clear | Safiya Umoja Noble, Claude Shannon, Warren Weaver | |||||||||||||||||||||||||||
![]() Aussagen KB IB clear | Machine learning benötigt Daten Machine Learning kann bestehende Vorurteile/Ungerechtigkeiten verstärken/weitertragen Textgeneratoren erleichtern das Generieren von Bullshit Textgeneratoren erleichtern das Generieren von Fake-News massiv | |||||||||||||||||||||||||||
![]() Begriffe KB IB clear | ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() | |||||||||||||||||||||||||||
![]() Bücher |
|
Dieser wissenschaftliche Zeitschriftenartikel erwähnt vermutlich nicht ... 
![]() Nicht erwähnte Begriffe | ChatGPT, facebook, Negative Rückkoppelung, Positive Rückkoppelung / Teufelskreis |
Tagcloud
Zitationsgraph
3 Erwähnungen 
- Re-examining AI, automation and datafication in education (Ben Williamson, Felicitas Macgilchrist, John Potter)
- Digital Warriors (Roberta Fischli) (2022)
- 4. Rassistische Software - sie zieht Google dafür zur Verantwortung (2022)
- 4. Rassistische Software - sie zieht Google dafür zur Verantwortung (2022)
- Do not feed the Google - Republik-Serie (2023)
- 4. Wenn ethische Werte nur ein Feigenblatt sind (Daniel Ryser, Ramona Sprenger) (2023)
- 4. Wenn ethische Werte nur ein Feigenblatt sind (Daniel Ryser, Ramona Sprenger) (2023)
Anderswo finden
Volltext dieses Dokuments
![]() | ![]() ![]() ![]() ![]() ![]() |
Anderswo suchen 
Beat und dieser wissenschaftliche Zeitschriftenartikel
Beat hat Dieser wissenschaftliche Zeitschriftenartikel erst in den letzten 6 Monaten in Biblionetz aufgenommen. Beat besitzt kein physisches, aber ein digitales Exemplar. Eine digitale Version ist auf dem Internet verfügbar (s.o.). Es gibt bisher nur wenige Objekte im Biblionetz, die dieses Werk zitieren.