First, a note to the reader: This blog post is longer than usual, as I decided to address multiple connected issues in the same post, without being too restrictive on length. With modern browsing habits and the amount of available online media, I suspect this post will be quickly passed over in favor of more interesting reading material. Before you immediately close this tab, I invite you to scroll down and read the conclusion, which hopefully can give you some food for thought along the way. If, however, you manage to read the whole thing, I applaud your impressive attention span.

A common criticism of the use of large language models (LLMs) is that it can deprive us of cognitive skills. The typical argument is that outsourcing certain tasks can easily cause some kind of mental atrophy. To what extent this is true is an ongoing discussion among neuroscientists, psychologists and others, but to me, the understanding that with certain skills you have to "use it or lose it" seems intuitively and empirically sound.

The more relevant question is whether certain kinds of use are better or worse than others, and if so, which? In the blog post The lump of cognition fallacy, Andy Masley discusses this in detail. His entry point to the problem is to challenge the idea that "there is a fixed amount of thinking to do", and how it leads people to the conclusion that "outsourcing thinking" to chatbots will make us lazy, less intelligent, or in other ways be negative for our cognitive abilities. He compares this to the misconception that there is only a finite amount of work that needs to be done in an economy, which often is referred to as "the lump of labour fallacy". His viewpoint is that "thinking often leads to more things to think about", and therefore we shouldn't worry about letting machines do the thinking for us — we will simply be able to think about other things instead.

Reading Masley's blog post prompted me to write down my own thoughts on the matter, as it has been churning in my mind for a long time. I realized that it could be constructive to use his blog post as a reference and starting point, because it contains arguments that are often brought up in this discussion. I will use some examples from Masley's post to show how I think differently about this, but I'll extend the scope beyond the claimed fallacy that there is a limited amount of thinking to be done. I have done my best to write this text in a way that does not require reading Masley's post first. My aim is not to refute all of his arguments, but to explain why the issue is much more complicated than "thinking often leads to more things to think about". Overall, the point of this post is to highlight some critical issues with "outsourcing thinking".

When should we avoid using generative language models?

Is it possible to define categories of activities where the use of LLMs (typically in the form of chatbots) is more harmful than helpful? Masley lists certain cases where, in his view, it is obviously detrimental to outsource thinking. To fully describe my own perspective, I'll take the liberty to quote the items on his list. He writes it's "bad to outsource your cognition when it:"

• Builds complex tacit knowledge you'll need for navigating the world in the future.
• Is an expression of care and presence for someone else.
• Is a valuable experience on its own.
• Is deceptive to fake.
• Is focused in a problem that is deathly important to get right, and where you don't totally trust who you're outsourcing it to.

I was surprised to discover that we are to a large extent in agreement on this list, despite having fundamentally different views otherwise. The disagreement lies, I believe, in the amount of activities that fall within the categories outlined above, particularly three of them.

Personal communication and writing

Let's start with the point "Is deceptive to fake". Masley uses the example of:

If someone’s messaging you on a dating app, they want to know what you’re actually like.

Very true, but in my view, it's not only in such intimate or private situations where it is deceptive to fake what you are like. Personal communication in general is an area where it matters how we express ourselves, both for ourselves and those we talk or write to. When we communicate with each other, there are certain expectations framing the whole exchange. Letting our words and phrases be transformed by a machine is a breach of those expectations. The words we choose and how we formulate our sentences carry a lot of meaning, and direct communication will suffer if we let language models pollute this type of interaction. Direct communication is not only about the information being exchanged, it's also about the relationship between the communicators, formed by who we are and how we express ourselves.

I think this is not only relevant for communication between two humans, but also for text with a personal sender conveyed to a human audience in general. To a certain extent, the same principles apply. There has been a debate in the Norwegian media lately regarding the undisclosed use of LLMs in public writing, with allegations and opinions flying around. I'm very happy to see this discussion reaching broad daylight, because we need to clarify our expectations to communication, now that chatbots are being so widely used. While I clearly think that it is beneficial to keep human-to-human communication free from an intermediate step of machine transformation, not everyone shares that view. If, going forward, our written communication will for the most part be co-authored with AI models, we need to be aware of it, and shift our expectations accordingly. Some have started disclosing when they have used AI in their writing, which I think is a good step towards better understanding of our use of LLMs. Knowing whether a text is written or "co-authored" by an LLM has an important effect on how a receiver views it; pretending otherwise is simply false.

Many see LLMs as a great boon for helping people express their opinions more clearly, particularly for people not using their native language or those who have learning disabilities. As long as the meaning originates from a person, LLMs can help express that meaning in correct and effective language. I have two main objections against this. The first one is about what happens to the text: In most cases it's impossible to separate the meaning from the expression of it. That is in essence what language is — the words are the meaning. Changing the phrasing changes the message. The second one is about what happens to us: We rob ourselves of the opportunity to grow and learn, without training wheels. LLMs can certainly help people improve the text, but the thinking process — developing the ideas — will be severely amputated when leaving the phrasing up to an AI model. They quickly become a replacement instead of help, depriving us the opportunity of discovering our own voice and who we can be and become when we stand on our own two feet.

With great care, one may be able to use a chatbot without being affected by these two drawbacks, but the problem is that with LLMs, there is an exceptionally thin line between getting help with spelling or grammar, and having the model essentially write for you, thereby glossing over your own voice. This is unavoidable with the current design of chatbots and LLM-powered tools; the step from old-school autocorrect to a generative language model is far too big. If we really envision LLMs as a tool for helping people become better at writing, we need to have a much more carefully considered interface than the chatbots we have today.

At the same time, I realize many are far more utilitarian. They just want to get the job done, finish their work, file that report, get that complaint through, answer that email, in the most efficient way possible, and then get on with their day. Getting help from an LLM to express oneself in a second language also seems useful, without considering how much or little one learns from it (I would be more positive to LLMs for translation if it wasn't for the fact that current state-of-the-art LLMs are simply very bad at producing Norwegian text. I can only hope the state is better for other non-English languages, or that it will improve over time). Additionally, LLMs seem to be efficient for people who are fighting with bureaucracy, such is filing complaints and dealing with insurance companies. In this case the advantage seems greater. We must, however, remember that the "weapon" exists on both sides of the table. What will happen to bureaucratic processes when all parties involved are armed with word generators?

It is not without reservation that I express these opinions, because it may come across as I want to deny people something that looks like a powerful tool. The point is that I think this tool will make you weaker, not stronger. LLMs don't really seem to empower people. Some of the effect I currently see is the number of applications to various calls (internships, research proposals, job openings) multiplying, but the quality dropping. Students are asking chatbots for help with solving collaborative tasks, not realizing that everyone is asking the same chatbot, robbing us of the diversity of ideas that could have formed if they took a minute to think for themselves.

The chatbots may have lowered the threshold for participation, but the competition's ground rules hasn't changed. To get better at writing, you need to write. The same goes for thinking. Applying for a job means showing who you are, not who the LLM thinks you are, or should be. Participating in the public debate is having to work out how to express opinions in clear language. Am I really participating if I'm not finding my own words?

It is important to note that not all text is affected in the same way. The category of writing that I like to call "functional text", which are things like computer code and pure conveyance of information (e.g., recipes, information signs, documentation), is not exposed to the same issues. But text that has a personal author addressing a human audience, has particular role expectations and rests on a particular trust. An erosion of that trust will be a loss for humanity.

A pragmatic attitude would be to just let the inflation of text ensue, and take stock after the dust has settled. What will be left of language afterwards? My conservative viewpoint stems from believing that what we will lose is of greater worth than what we gain. While LLMs can prove useful in the short term, using them is treating a symptom instead of the problem. It is a crutch, although some may truly be in need of that crutch. My only advice would be to make sure you actually need it before you lean on it.

Valuable experiences

Using LLMs is not only about writing. Masley mentions that it's bad to outsource activities that are "a valuable experience on its own". I couldn't agree more, but I suspect that he will disagree when I say that I think this category encompasses a lot of what we already do in life. Major LLM providers love to show how their chatbots can be used to plan vacations, organize parties, and create personal messages to friends and family. I seldom feel more disconnected from the technological society than when I watch these advertisements.

To me, this highlights a problem that goes to the core of what it means to be human. Modern life brings with it a great deal of activities that can feel like chores, but at the same time it seems like we are hell-bent on treating everything as a chore as well. Humans are surprisingly good at finding discontentment in nearly anything, maybe because of an expectation in modern society that we should be able to do anything we want, anytime we want it — or perhaps more importantly, that we should be able to avoid doing things we don't feel like doing. Our inability to see opportunities and fulfillment in life as it is, leads to the inevitable conclusion that life is never enough, and we would always rather be doing something else.

In theory, I agree that automating some things can free up time for other things that are potentially more meaningful and rewarding, but we have already reached a stage where even planning our vacation is a chore that apparently a lot of people would like to avoid doing. I hope that AI's alleged ability to automate "nearly anything" helps us realize what is worth spending time and effort on, and rediscover the value of intentional living.

Building knowledge

The third point I would like to address is that we shouldn't use chatbots when it "builds complex tacit knowledge you'll need for navigating the world in the future", according to Masley. Again, I agree completely, and again, I think that this point encompasses a great deal of daily life. Building knowledge happens not only when you sit down to learn something new, but also when you do repetitive work.

This misconception is not new for chatbots, but has been present since we started carrying smartphones in our pockets. With internet at hand at all times, there's apparently no need to remember information anymore. Instead of using our brains for storing knowledge, we can access information online when we need it, and spend more time learning how to actually use the information and think critically. The point we are missing here, is that acquiring and memorizing knowledge is a huge part of learning to use the knowledge. It is naive to think that we can simply separate the storage unit from the processing unit, like if we were a computer.

I learned this lesson while being a piano student. I was trying to understand jazz, and figure out how good improvisers could learn to come up with new phrases so easily on the spot. How does one practice improvisation? Is it possible to exercise the ability to come up with something new that immediately sound good? I ended up playing similar riffs almost every time I tried. After a while I got convinced that good jazz players must be born with some inherent creativity, some inner musical inspiration that hummed melodies inside their heads for them to play.

One of my tutors taught me the real trick: Good improvisation comes not from just practicing improvisation. You need to play existing songs and tunes, many of them, over and over, learn them by heart, get the chord progressions and motifs under your skin. This practice builds your intuition for what sounds good, and your improvisation can spring from that. Bits and pieces of old melodies are combined into new music. In that sense, we are more like a machine learning model than a computer, but do not make the mistake of thinking that is actually what we are.

There is a need for clarification here: I'm not saying that nothing should be automated by LLMs. But I think many are severely underestimating the knowledge we are building from boring tasks, and we are in danger of losing that knowledge when the pressure for increased efficiency makes us turn to the chatbots.

The extended mind

As a sidenote, I would like to contest the idea of the extended mind, as explained by Masley:

[M]uch of our cognition isn’t limited to our skull and brain, it also happens in our physical environment, so a lot of what we define as our minds could also be said to exist in the physical objects around us.

It seems kind of arbitrary whether it’s happening in the neurons in your brain or in the circuits in your phone.

This statement is simply absurd, even when read in context. The fact that something happens in your brain rather than on a computer makes all the difference in the world. Humans are something more than information processors. Yes, we process information, but it is extremely reductionist to treat ourselves as objects where certain processes can be outsourced to external devices without consequences. Does it really matter if I remember my friend's birthday, when I can have a chatbot send them an automated congratulation? Yes, it matters because in the first case you are consciously remembering and thinking about your friend, consolidating your side of the relationship.

The quoted statement above is followed up with:

It’s true that you could lose your phone and therefore lose the stored knowledge, but you could also have a part of your brain cut out.

Losing your phone and losing a part of your brain are two tremendously different things, both in terms of likelihood and consequences. Not only does the statement above significantly underestimate the processes that happens in our brain, but to even liken having a part of your brain cut out to losing your phone reveals that the premiss of the argument is severely detached from reality.

The design of our built environments is also brought up to show how it's beneficial to minimize the amount of thinking we do:

[M]ost of our physical environments have been designed specifically to minimize the amount of thinking we have to do to achieve our daily goals.

Try to imagine how much additional thinking you would need to do if things were designed differently.

This doesn't hold up to scrutiny. Yes, if our environment suddenly changed, it would require extra mental effort of us to navigate. For a time. But, then we would have gotten familiar with that alternative design, and adapted ourselves. The only case where we would have had to do additional thinking is if the design of our physical environments changed all the time.

What we think about does matter

Regarding the "lump of cognition fallacy", I fully agree that we need not worry about "draining a finite pool" of thinking, leaving "less thinking" — whatever that means — for humans. There is, however, another fallacy at play here, which is that "it does not matter what we think about, as long as we think about something". It is easy to be convinced that if a computer can do the simple, boring tasks for me, I can deal with more complex, exciting stuff myself. But we must be aware that certain mental tasks are important for us to do, even though a machine technically could do them for us.

To illustrate: If I outsource all my boring project administration tasks to a chatbot, it can leave more time for my main task: research. But it will also rob me of the opportunity to feel ownership to the project and build a basis for taking high-level decisions in the project. In a hypothetical situation where a chatbot performs all administrative tasks perfectly on my behalf, I will still have lost something, which may again have impact on the project. I'm not saying that no tasks should be automated at all, but we must be aware that we always lose something when automating a process.

Comparing with the "lump of labour" fallacy again: While it may be true that outsourcing physical work to machines will simply create new types of work to do, it doesn't mean that the new work is useful, fulfilling, or beneficial for individuals and society. The same goes for thinking. We must acknowledge that all kinds of thinking have an effect on us, even the boring and tedious kinds. Removing the need for some cognitive tasks can have just as much influence, positive or negative, as taking up new types of cognitive tasks.

Conclusion

We have a major challenge ahead of us in figuring out what chatbots are suitable for in the long term. Personal communication may change forever (that is to say, maybe it won't stay personal anymore), education systems will require radical adaptations, and we need to reflect more carefully about which experiences in life actually matter. What is truly exciting about this new type of technology, is that it forces us to face questions about our humanity and values. Many formerly theoretical questions of philosophy are becoming relevant for our daily lives.

A fundamental point I'm trying to bring forth is that how we choose to use chatbots is not only about efficiency and cognitive consequences; it's about how we want our lives and society to be. I have tried to argue that there are good reasons for protecting certain human activities against the automation of machines. This is in part based on my values, and does not rely on research into whether or not our efficiency at work or cognitive abilities are affected by it. I cannot tell other people what they should do, but I challenge everyone to consider what values they want to build our communities on, and let that weigh in alongside what the research studies tell us.