The simple reason LLMs are not scientific models (and what the alternative is for linguistics).
Joe Collins
April 2024 

Response to Piantadosi (2023). There is a an explicit and mathematical reason why Large Language Models (LLMs) are not scientific theories. They belong to a class of Universal Function Approximators, which can approximate any mathematical function by summing over many generic functions, and whose representations are therefore arbitrary. They are closely related to approximation methods such as (generalised) Fourier series and Taylor expansions. This has consequences for how much we can learn from the practical limitations of LLMs and their behaviour. Finally, it is argued if linguistics is to tackle problems of complexity and emergence, it should take its cues from similarly "Galilean" fields such as statistical physics, rather than machine learning.

Format:	[ pdf ]
Reference:	lingbuzz/008026 (please use that when you cite this article)
Published in:
keywords:	piantadosi 2023, large language models, llms, chatgpt, transformers, ai, generative linguistics, chomsky, physics, syntax, phonology, semantics, morphology, machine learning, deep learning, connectionism
previous versions:	v1 [April 2024]
Downloaded:	397 times

 

[ edit this article | back to article list ]