I recently worked on something very complex I don't think I would have been able to tackle as quickly without AI; a hierarchical graph layout algorithm based on the Sugiyama framework, using Brandes-Köpf for node positioning. I had no prior experience with it (and I went in clearly underestimating how complex it was), and AI was a tremendous help in getting a basic understanding of the algorithm, its many steps and sub-algorithms, the subtle interactions and unspoken assumptions in it. But letting it write the actual code was a mistake. That's what kept me from understanding the intricacies, from truly engaging with the problem, which led me to keep relying on the AI to fix issues, but at that point the AI clearly also had no real idea what it was doing, and just made things worse.

So instead of letting the AI see the real code, I switched from the Copilot IDE plugin to the standalone Copilot 365 app, where it could explain the principles behind every step, and I would debug and fix the code and develop actual understanding of what was going on. And I finally got back into that coding flow again.

So don't let the AI take over your actual job, but use it as an interactive encyclopedia. That works much better for this kind of complex problem.

Writing code has just typically been how I've needed to solve those problems.

That has increasingly shifted to "just" reviewing code and focusing on the architecture and domain models.

I get to spend more time on my actual job.

Solve enough problems relying on AI writing the code as a black box, and over time your grasp of coding will worsen, and you wont be undestanding what the AI should be doing or what it is doing wrong - not even at the architectural level, except in broad strokes.

One ends like the clueless manager type who hasn't touched a computer in 30 years. At which point there will be little reason for the actual job owners to retain their services.

Computer programming on the whole relying on the canned experience of the AI data set, producing more AI churn as ratio of the available training code over time, and plateuing both itself and AI, with the dubious future of reaching Singularity its only hope out of this.

But I generally agree with your point.

Computing is always abstractions. We moved from plugging to assembly, then to c, then we had languages that managed memory for you -- how on earth can you understand what the compiler should be doing or what it is doing if you don't deal with explicit pointers on a day by day basis.

We bring in libraries when we need code. We don't run our own database, we use something else, and we just do "apt-get install mysql", but then we moved onto "docker run" or perhaps we invoke it with "aws cli". Who knows what teraform actually does when we declare we want a resource.

I was thinking the other day how abstractions like AWS or Docker are similar to LLM. With AWS you just click a couple of buttons and you have a data store, you don't know how to build a database from scratch, you don't need one. Of course "to build a database from scratch you must first create the universe".

Some people still hand-craft assembly code to great benefit, but that vast majority don't need to to solve problems, and they can't.

This musing was in the context of what do we do if/when aws data centres are not available. Our staff are generally incapable of working in a non-aws environment. Something that we have deliberately cultivated for years. AWS outputs are one option, or perhaps we should run a non-aws stack that we fully own and control.

Is relying on LLMs fundamentally any different than relying on AWS, or apt, or java. Is is different from outsourcing? You concentrate on your core competency, which is understanding the problem and delivering a solution, not managing memory or running databases. This comes with risk -- all outsourcing does, and if outsourcing to a single supplier you don't and can't understand is acceptable risk, then is relying on LLMs not?

When you use LLMs to write all your code you will lose (or never learn) the details. Your decision making will not be as good.

It is about the fact that LLM users struggle to recall their own work.

That is to say: they are not only losing the "inputs" to the generative work, they are losing the outputs too

Because they didn't understand the architecture or the domain models otherwise.

Perhaps in your case you do have strong hands-on experience with the domain models, which may indeed have shifted you job requirements to supervising those implementing the actual models.

I do wonder, however, how much of your actual job also entails ensuring that whoever is doing the implementation is also growing in their understanding of the domain models. Are you developing the people under you? Is that part of your job?

If it is an AI that is reporting to you, how are you doing this? Are you writing "skills" files? How are you verifying that it is following them? How are you verifying that it understands them the same way that you intended it to?

Funny story-- I asked a LLM to review a call transcript to see if the caller was an existing customer. The LLM said True. It was only when I looked closer that I saw that the LLM mean "True-- the caller is an existing customer of one of our competitors". Not at all what I meant.

> Because they didn't understand the architecture or the domain models otherwise.

My point is that requiring or expecting an in-depth understanding of all the algorithms you rely on is not a productive use of developer time, because outside narrow niches it is not what we're being paid for.

It is also not something the vast majority of us do now, or have done for several decades. I started with assembler, but most developers have never-ever worked less than a couple of abstractions up, often more, and leaned heavily on heaps of code they do not understand because it is not necessary.

Sometimes it is. But for the vast majority of us pretending it is necessary all the time or even much of the time is a folly.

> I do wonder, however, how much of your actual job also entails ensuring that whoever is doing the implementation is also growing in their understanding of the domain models. Are you developing the people under you? Is that part of your job?

Growing the people under me involves teaching them to solve problems, and already long before AI that typically involved teaching developers to stop obsessing over details with low ROI for the work they were actually doing in favour of understanding and solving the problems of the business. Often that meant making them draw a line between what actually served the needs they were paid to solve rather than the ones that were personally fun to them (I've been guilty of diving into complex low-level problems I find fun rather than what solves the highest ROI problems too - ask me about my compilers, my editor, my terminal - I'm excellent at yak shaving, but I work hard to keep that away from my work)

> If it is an AI that is reporting to you, how are you doing this? Are you writing "skills" files? How are you verifying that it is following them? How are you verifying that it understands them the same way that you intended it to?

For AI use: Tests. Tests. More tests. And, yes, skills and agents. Not primarily even to verify that it understands the specs, but to create harnesses to run them in agent loops without having to babysit them every step of the way. If you use AI and spend your time babysitting them, you've become a glorified assistant to the machine.

And nobody is talking about verifying if the AI bubble sort is correct or not - but recognizing that if the AI is implementing it’s own bubble sort, you’re waaaay out in left field.

Especially if it’s doing it inline somewhere.

The underlying issue with AI slop, is that it’s harder to recognize unless you look closely, and then you realize the whole thing is bullshit.

Why are you creating BS tests?

> And nobody is talking about verifying if the AI bubble sort is correct or not - but recognizing that if the AI is implementing it’s own bubble sort, you’re waaaay out in left field.

Verifying time and space complexity is part of what your tests should cover.

But this is also a funny example - I'm willing to bet the average AI model today can write a far better sort than the vast majority of software developers, and is far more capable of analyzing time and space complexity than the average developer.

In fact, I just did a quick test with Claude, and asked for a simple sort that took into account time and space complexity, and "of course" it knows that it's well established that pure quicksort is suboptimal for a general-purpose sort, and gave me a simple hybrid sort based on insertion sort for small arrays, heapsort fallback to stop pathological recursion, and a decently optimized quicksort - this won't beat e.g. timsort on typical data, but it's a good tradeoff between "simple" (quicksort can be written in 2-20 lines of code or so depending on language and how much performance you're willing to sacrifice for simplicity) and addressing the time/space complexity constraints. It's also close to a variant that incidentally was covered in an article in DDJ ca. 30 years ago because most developers didn't know how to, and were still writing stupidly bad sorts manually instead of relying on an optimized library. Fewer developers knows how to write good sorts today. And that's not bad - it's a result of not needing to think at that level of abstraction most of the time any more.

And this is also a great illustration of the problem: Even great developers often have big blind spots, where AI will draw onresults they aren't even aware of. Truly great developers will be aware of their blind spots and know when to research, but most developers are not great.

I encountered something like this recently. I had to replace an exact data comparison operation (using a simple memcmp) with a function that would compare data and allow differences within a specified tolerance. The AI generated beautiful code using chunking and all kinds of bit twiddling that I don't understand.

But what it couldn't know was that most of the time the two data ranges would match exactly, thus taking the slowest path through the comparison by comparing every chunk in the two ranges. I had to stick a memcmp early in the function to exit early for the most common case, because it only occurred to me during profiling that most of the time the data doesn't change. There was no way I could have figured this out early enough to put it in a spec for an AI.

Sure. But then that belongs in a test case that 1) documents the assumptions, 2) demonstrates if a specialised solution actually improves on the naive implementation, and 3) will catch regressions if/when those assumptions no longer holds.

In my experience in that specific field is that odds are the human are likely making incorrect assumptions, very occasionally are not, and having a proper test harness to benchmark this is essential to validate the assumptions whether or not the human or an AI does the implementation (and not least in case the characteristics of the data end up changing over time)

This is an odd statement to me. You act like the AI can only write the application once and can never look at any other data to improve the application again.

>only occurred to me during profiling

At least to me this seems like something that is at far more risk of being automated then general application design in the first place.

Have the AI design the app. Pass it off to CI/CD testing and compile it. Send to a profiling step. AI profile analysis. Hot point identification. Return to AI to reiterate. Repeat.

This function is a small part of a larger application with research components that are not AI-solvable at the moment. Of course a standalone function could have been optimised with AI profiling, but that's not the context here.

Only if you don't constrain the tests. If you use agents adversarially in generating test cases, tests and review of results, you can get robust and tight test cases.

Unless you're in research, most of what we do in our day jobs is boilerplate. Using these tools is not yet foolproof, but with some experience and experimentation you can get excellent results.

And with all the stacking LLMs against each other, that just sounds like more work than just… writing the damn tests.

I meant this more in the sense of there is nothing new under the sun, and that LLMs have been trained on essentially everything that's available online "under the sun". Sure, there are new SaaS ideas every so often, but the software to produce the idea is rarely that novel (in that you can squint and figure out roughly how it works without thinking too hard), and is in that sense boilerplate.

Your time would be better spent, in a permanent code base, trying to get that LLM to understand something than it would be trying to understand the thing yourself. It might be the case that you need to understand the thing more thoroughly yourself so you can explain it to the LLM, and it might be the case that you need to write some code so that you can understand it and explain it, but eventually the LLM needs to get it based on the code comments and examples and tests.

Yes, and there's often a benefit to having a human have an understanding of the concrete details of the system when you're trying to solve problems.

> That has increasingly shifted to "just" reviewing code

It takes longer to read code than to write code if you're trying to get the same level of understanding. You're gaining time by building up an understanding deficit. That works for a while, but at some point you have to go burn the time to understand it.

It's important that when you solve problems by writing code, you go through all the use cases of your solution. In my experience, just reading the code given by someone else (either a human or machine) is not enough and you end up evaluating perhaps the main use cases and the style. Most of the times you will find gaps while writing the code yourself.

Further elaborating from my experience.

1. I think we're in the early stages, where agents are useful because we still know enough to coach well - knowledge inertia.

2. I routinely make the mistake of allowing too much autonomy, and will have to spend time cleaning up poor design choices that were either inserted by the agent, or were forced upon it because I had lost lock on the implementation details (usually both in a causal loop!)

I just have a policy of moving slowly and carefully now through the critical code, vs letting the agent steer. They have overindexed on passing tests and "clean code", producing things that cause subtle errors time and time again in a large codebase.

> burn the time to understand it.

It seems to me to be self-evident that writing produces better understanding than reading. In fact, when I would try to understand a difficult codebase, it often meant that probing+rewriting produced a better understanding than reading, even if those changes were never kept.

This is true whether an AI wrote the code or a co-worker, except the AI is always on hand to answer detailed questions about the code, do detailed analysis, and run extensive tests to validate assumptions.

It is very rarely productive any more to dig into low level code manually.

What data are you basing this assumption on?

If I had turned an AI loose against the original codebase, I think it would have just churned away copying the existing patterns and debugging any runtime errors that result. I don't think an AI would have ever voluntarily told me "this form library is costing time and effort, we should replace it with such and such instead"

How we work changes and the extra complexity buys us productivity. The vast majority of software will be AI generated, tools will exist to continuously test/refine it, and hand written code will be for artists, hobbyists, and an ever shrinking set of hard problems where a human still wins.

This to me looks like an analogy that would support what GP is saying. With modern farming practices you get problems like increased topsoil loss and decreased nutritional value of produce. It also leads to a loss of knowledge for those that practice those techniques of least resistance in short term.

This is not me saying big farming bad or something like that, just that your analogy, to me, seems perfectly in sync with what the GP is saying.

Never use your phrase to say something is impossible. I mean there are driverless Waymo's on the street in my area so your statement is already partially incorrect.

Just about a week ago I launched a 100% AI generated project that shortcircuits a bunch of manual tasks. What before took 3+ weeks of manual work to produce, now takes us 1-2 days to verify instead. It generates revenue. It solved the problem of taking a workflow that was barely profitable and cutting costs by more than 90%. Half the remaining time is ongoing process optimization - we hope to fully automate away the reaming 1-2 days.

This was a problem that wasn't even tractable without AI, and there's no "explosion of AI generated code".

I fully agree that some places will drown in a deluge of AI generated code of poor quality, but that is an operator fault. In fact, one of my current clients retained me specifically to clean up after someone who dove head first into "AI first" without an understanding of proper guardrails.

People often say this when giving examples, but what specifically made the problem intractable?

Sometimes before beginning work on a problem, I dramatically overestimate how hard it will be (or underestimate how capable I am of solving it.)

You're like 836453th person to say this. It's not untrue, but many of us will take writing over reviewing any day. Reviewing is like the worst part of the job.

E.g. "show me why <this assumption that is necessary for the code I'm currently staring at> holds" makes it far more pleasant to do reviews. AI code review tooling works well to reduce that burden. Even more so when you have that AI cod review tooling running as part of your agent loop first before you even look at a delivery.

"prove X" is another one - if it can't find a test case that already proves X and resorts to writing code to prove X, you probably need more tests, and now you have one,.

The people framing this as "cognitive debt" are measuring the wrong thing. You're not losing the ability to think - you're shifting what you think about. That's not a bug, it's the whole point.

But I think the cognitive debt framing is useful: reading and approving code is not the same as building the mental model you get from writing, probing, and breaking things yourself. So the win (more time on problem solving) only holds if you're still intentionally doing enough of the concrete work to stay anchored in the system.

That said, if you're someone like me, I don't always need to fully master everything, but I do need to stay close enough to reality that I'm not shipping guesses.

[0] https://alisor.substack.com/p/i-never-really-wrote-code-now-...

It's "true" in a sense. It helps. But it is also largely irrelevant for most of us, in that most of us are writing code you can learn to read and write in a tiny proportion of the time we spend in working life. The notion that you need to keep spending more than a tiny fraction of your time writing code in order to understand enough to be able to solve business problem will seem increasingly quaint.

Completely disagree. Reading books doesn't make you an author. Reading books AND writing books makes you an author.

Most of us aren't paid to be authors in your analogy.

(Which is good, because outside of your analogy, most authors are paid peanuts, and most of those of us who do write do so because we enjoy it, not as a job)

But even if our jobs were to be authors, while I learned some things about writing books from writing the novels I have written and published, I learned far more from being a voracious reader for decades.

I probably needed both, and I'm sure I'd improve as a writer past what I could from just reading by writing more, I think your analogy if anything is a perfect fit for my point that we don't need to spend more than a tiny proportion our time writing to be competent at it (I won't claim great).

Many of us will probably keep doing it for fun, but it will be increasingly hard to justify "manual coding" at work.

Air quotes and more and more general words. The perfect mercenary’s tools.

The buck stops somewhere for most of us. We have jobs, we are compelled to do them. But we care about how it is done. We care whether doing it in a certain will give us short term advantages but hinder us in the long term. We care if the process feels good or bad. We care if it feels like we are in control of the process or if we are just swimming in a turbulent sea. We care about how predictable the tools we use. Whether we can guess that something takes a month and not be off by weeks.

We might say that we are the perfect pragmatists (mercenaries); that we only care about the most general description of what-is-to-be-done that is acceptable to the audience, like solving business problems, or solving technical problems, or in the end—as the pragmatist sheds all meaning from his burdensome vessel—just solving problems. But most of us got into some trade, or hobby, or profession, because we did concrete things that we concretely liked. And switching from keyboards to voice dictation might not change that. But seemingly upending the whole process might.

It might. Or it may not. Certainly could go in more than one direction. But to people who are not perfect mercenaries or business hedonists[1] these are actual problems or concerns. Not nonsense to be dismissed with some “actual job” quip, which itself is devoid of meaning.

[1] https://news.ycombinator.com/item?id=46692039

I do not want to be a supervisor of AI agents. I do not want to engineer prompts, I want to engineer software.

In practice, this isn't bearing out at all though both among my peers and with peers in other tech companies. Just making a blanket statement like this adds nothing to the conversation.

The only way I'll run out of credits is if my company isn't liquid any longer in which case I have bigger problems.

And there are plenty of LLM providers, almost only a few w/SOTA models but even for SOTA models there is no reason to be dependent on one.

if you're a consultant/contractor that's bid a fixed amount for a job: you're incentivised to slop out as much as possible to hit the complete the contract as quickly as possible

and then if you do a particularly bad job then you'll be probably kept on to fix up the problems

vs. an permanent employee that is incentivised to do the job well, sign it off and move onto the next task

Most of my work is on projects I have a long term vested interest in.

I care far more about maximally leveraging LLMs for the projects I have a vested interest in - if my clients don't want to, that's their business.

Most of my LLM usage directly affects my personal finances in terms of the ROI my non-consulting projects generate - I have far more incentives to do the job well than a permanent employee whose work does not have an immediate effect on their income.

https://hokstadconsulting.com/blog

Very impressive daily output! Keep up the good work.

Some people learn from rote memorization, some people learn through hands on experience. Some people have "ADHD brains". Some people are on the spectrum. If you visit Wikipedia and check out Learning Styles, there's like eight different suggested models, and even those are criticized extensively.

It seems a sort of parochial universalism has coalesced, but people should keep in mind we don't all learn the same.

ETA: I'd also like to say learning from LLMs are vastly similar, and some ways more useful, than finding blogs on a subject. A lot of time, say for Linux, you'll find instructions that even if you perform them to a tee, something goes pear shaped, because of tiny environment variables or a single package update changes things. Even Photoshop tutorials are not free of this madness. I'm used to mostly correct but just this side of incorrect instructions. LLMs are no different in a lot of ways. At least with them I can tailor my experience to just what I'm trying to do and spend time correcting that versus loading up a YT video trying to understand why X doesn't work. But I can understand if people don't get the same value as I do.

Trade offs around "room to do more of other things" are an interesting and recurring theme of these conversations. Like two opposites of a spectrum. On one end the ideal process oriented artisan taking the long way to mastery, on the other end the trailblazer moving fast and discovering entirely new things.

Comparing to the encyclopedia example: I'm already seeing my own skillset in researching online has atrophied and become less relevant. Both because the searching isn't as helpful and because my muscle memory for reaching for the chat window is shifting.

If you outsource a skill consistently, you will be engaging less with that skill. Depending on the skill, this may be acceptable, or a desirable tradeoff.

For example, using a very fast LLM to interactively make small edits to a program (a few lines at a time), outsources the work of typing, remembering stdlib names and parameter order, etc.

This way of working is more akin to power armor, where you are still continuously directing it, just with each of your intentions manifesting more rapidly (and perhaps with less precision, though it seems perfectly manageable if you keep the edit size small enough).

Whereas "just go build me this thing" and then you make a coffee is qualitatively very different, at that point you're more like a manager than a programmer.

I have a whole half-written blog post about how LLMs are the cars of the mind. Massive externalities, has to be forced on people, leads to cognitive/health issues instead of improving cognition and health.

[0] https://thereader.mitpress.mit.edu/when-cities-treated-cars-... [1] https://usa.streetsblog.org/2020/03/19/study-most-car-owners...

More like mobility scooter for disabled. Literally Wall-E in the making.

I do, which is exactly why I found the presumption that not spending your time doing the coding is equivalent to a disability both gross and arrogant.

> Where are they supposed to learn good design when slop takes over?

You're not learning good architecture and systems design from code. You learn good architecture and systems design from doing architecture and systems design. It's a very different discipline.

While knowing how to code can be helpful, and can even be important in narrow niches, it is a very minor part of understanding good architecture.

And, yes, I stand by the claim the coding is by far the simplest part, on the basis of having done both for longer than most developers have been doing either.

Doubling down on your ignorance, bold strategy.

Yes, that is my experience. I have done some C# projects recently, a language I am not familiar with. I used the interactive encylopedia method, "wrote" a decent amount of code myself, but several thousand lines of production code later, I don't I know C# any better than when I started.

OTOH, it seems that LLMs are very good at compiling pseudocode into C#. And I have always been good at reading code, even in unfamiliar languages, so it all works pretty well.

I think I have always worked in pseudocode inside my head. So with LLMs, I don't need to know any programming languages!

Claude Code seems to be a much better paradigm. For novel implementations I write code manually while asking it questions. For things that I'm prototyping I babysit it closely and constantly catch it doing things that I don't want it to do. I ask it questions about why it built things certain ways and 80% of the time it doesn't have a good answer and redoes it the way that I want. This takes a great deal of cognitive engagement.

Rule nombre [sic] uno: Never anthropomorphize the LLM. It's a giant pattern-matching machine. A useful one, but still just a machine. Do not let it think for you because it can't.

If I understand a problem and AI is just helping me write or refactor code, that’s all good. If I don’t understand a problem and I’m using AI to help me investigate the codebase or help me debug, that’s okay too. But if I ever just let the AI do its thing without understanding what it’s doing and then I just accept the results, that’s where things go wrong.

But if we’re serious about avoiding the trap of AI letting us write working code we don’t understand, then AI can be very useful. Unfortunately the trap is very alluring.

A lot of vibe coding falls into the trap. You can get away with it for small stuff, but not for serious work.

It's similar to other abstractions in this way, but on a larger scale due to LLM having so many potential applications. And of course, due to the non-determinism.

We already see the damage of a lack of understanding when we have to work with old codebases. These behemoths can become very difficult to work in over time as the people who wrote it leave, and new people don’t have the same understanding to make good effective changes. This slows down progress tremendously.

Fundamentally, code changes you make without understanding them immediately become legacy code. You really don’t want too much of that to pile up.

Outsourcing learning and thinking is a double edged sword that only comes back to bite you later. It's tempting: you might already know a codebase well and you set agents loose on it. You know enough to evaluate the output well. This is the experience that has impressed a few vocal OSS authors like antirez for example.

Similarly, you see success stories with folks making something greenfield. Since you've delegated decision making to the LLM and gotten a decent looking result it seems like you never needed to know the details at all.

The trap is that your knowledge of why you've built what you've built the way it is atrophies very quickly. Then suddenly you become fully dependent on AI to make any further headway. And you're piling slop on top of slop.

Funny - that's the hard part for me. I have yet to figure out what to use it for, since it seems to take longer than any other method of performing my tasks. Especially with regards to verifying for correctness, which in most cases seems to take as long or longer than just having done it myself, knowing I did it correctly.

If I can explain briefly what our issue is: we've got a really complex graph, and need to show it in a way that makes it easy to understand. That by itself might be a lost cause already, but we need it fixed. The problem is that our graph has cycles, and dagre is designed for DAGs; directed acyclic graphs. Fortunately it has a step that removes cycles, but it does that fairly randomly, and that can sometimes dramatically change the shape of the graph by creating unintentional start or end nodes.

I had a way to fix that, but even with that, it's still really hard to understand the graph. We need to cut it up into parts, group nodes together based on shared properties, and that's not something dagre does at all. I'm currently looking into cola with its constraints. But I'll take another look at elk.

But while I was able to understand it enough to steer the conversation, I was utterly unable to make any meaningful change to the code or grasp what it was doing. Unfortunately, unlike in the case you described, chatting with the LLM didn’t cut it as the domain is challenging enough. I’m on a rabbit hunt now for days, picking up the math foundations and writing the code at a slower pace albeit one I can keep up with.

And to be honest it’s incredibly fun. Applied math with a smart, dedicated tutor and the ability to immediately see results and build your intuition is miles ahead of my memories back in uni.

I think you not asking questions about the code is the problem (in so far it still is a problem). But it certainly has gotten easy not to.

Seriously though, I appreciated it because my curiosity got the better of me and I went down a quick rabbit hole in Sugiyama, comparative graph algorithms, and learning about the node positioning as a particular dimension of graph theory. Sure nothing ground breaking, but it added a shallow amount to my broad knowledge base of theory that continues to prove useful in our business (often knowing what you don't know is the best initiative for learning). So yeah man, lets keep name dropping pretentious technical details because thats half the reason I surf this site.

And yes, I did use ChatGPT to familiarize myself with these concepts briefly.

Everything to them is a social media post for likes.

I have explored all kinds of graph layouts in various network science context via LLMs and guess what? I don't know anything much about graph theory beyond G = (V,E). I am not really interested either. I am interested in what I can do with and learn from G. Everything on the right of the equals sign Gemini is already beyond my ability. I am just not that smart.

The standard narrative on this board seems to be something akin to having to master all volumes of Knuth before you can even think to write a React CRUD app. Ironic since I imagine so many learned programming by just programming.

I know I don't think as hard when using an LLM. Maybe that is a problem for people with 25 more IQ points than me. If I had 25 more IQ points maybe I could figure out stuff without the LLM. That was not the hand I was dealt though.

I get the feeling there is immense intellectual hubris on this forum that when something like this comes up, it is a dog whistle for these delusional Erdos in their own mind people to come out of the wood work to tell you how LLMs can't help you with graph theory.

If that wasn't the case there would be vastly more interesting discussion on this forum instead of ad nauseam discussion on how bad LLMs are.

I learn new things everyday from Gemini and basically nothing reading this forum.

Where I'm skeptical of this study:

- 54 participants, only 18 in the critical 4th session

- 4 months is barely enough time to adapt to a fundamentally new tool

- "Reduced brain connectivity" is framed as bad - but couldn't efficient resource allocation also be a feature, not a bug?

- Essay writing is one specific task; extrapolating to "cognition in general" seems like a stretch

Where the study might have a point:

Previous tools outsourced partial processes - calculators do arithmetic, Google stores facts. LLMs can potentially take over the entire cognitive process from thinking to formulating. That's qualitatively different.

So am I ideologically inclined to dismiss this? Maybe. But I also think the honest answer is: we don't know yet. The historical pattern suggests cognitive abilities shift rather than disappear. Whether this shift is net positive or negative - ask me again in 20 years.

[Edit]: Formatting

They were arguably right. Pre literate peole could memorise vast texts (Homer's work, Australian Aboriginal songlines). Pre Gutenberg, memorising reasonably large texts was common. See, e.g. the book Memory Craft.

We're becoming increasingly like the Wall E people, too lazy and stupid to do anything without our machines doing it for us, as we offload increasing amounts onto them.

And it's not even that machines are always better, they only have to be barely competent. People will risk their life in a horribly janky self driving car if it means they can swipe on social media instead of watching the road - acceptance doesn't mean it's good.

We have about 30 years of the internet being widely adopted, which I think is roughly similar to AI in many ways (both give you access to data very quickly). Economists suggest we are in many ways no more productive now than when Homer Simpson could buy a house and raise a family on a single income - https://en.wikipedia.org/wiki/Productivity_paradox

Yes, it's too early to be sure, but the internet, Google and Wikipedia arguably haven't made the world any better (overall).

It seems more likely that there were only a handful of people who could. There still are a handful of people who can, and they are probably even better than in the olden times [1] (for example because there are simply more people now than back then.)

[1] https://oberlinreview.org/35413/news/35413/ (random first link from Google)

I think they were right that something was lost in each transition.

But something much bigger was also gained, and I think each of those inventions were easily worth the cost.

But I'm also aware that one cost of the printing press was a century of very bloody wars across Europe.

As for the productivity paradox, this discounts the reality that we wouldn't even be able to scale the institutions we're scaling without the tech. Whether that scaling is a good thing is debatable.

They are, but you go on to assume that they will adapt in a good way.

Bodies are adaptive too. That didn't work out well for a lot of people when their environment changed to be sedentary.

You think it's likely that we offload cognitive difficulty and complexity to machines, and our brains don't get worse at difficult, complex problems?

It literally does. If your brain shuts down the moment you can't access your LLM overlord then you're objectively worse.

Launching a search engine and searching may spew incorrectness but it made you make judgement, think. You could have two different opinions one underneath each other; you saw both sides of the coin.

We are no longer critical thinking. We are taking information at face value, marking it as correct and not questioning is it afterwards.

The ability to evaluate critically and rationally is what's decaying. Who opens an physical encyclopedia nowadays? That itself requires resources, effort and time. Add in life complexity; that doesn't help us in evaluating and rejecting consumption of false information. The Wall-E view isn't wrong.

Please provide evidence that masses of people ever were critically thinking across general fields they were not involved in.

Everyone seems to take for face value there was a golden age of critical thinking done by the masses is at some time in the indeterminate past, but regardless of when you ask this question, the answer is always "in the past".

I surmise your thesis is incorrect and supplant this one instead.

The average person can only apply critical thinking on a very limited amount of information, and typically on topics they deal with that have a quick feedback loop of consequences.

Deep critical thinkers across vast topics are rare, and have always been rare. There are likely far more of them than ever now, but this falls into the next point

Information and complexity are exploding, the amount of data required to navigate the world we now live in is far larger than just a few generations ago. Couple this with the amount of information being presented to individuals and you run into actual physics constraints on the amount of information the human brain can distil into a useful model.

By (monetary) necessity people have become deep specialists in limited topics, analogies and paradigms don't necessarily work across different topics. For example, understanding code very well has very little bearing on if I grok the reality of practiced political sociology, and my idea of what is critical thinking around it is very likely to have a very large prediction mismatch to what actually happens.

You have a point with trusting AI, but I'm starting to see people around me realising that LLMs tend to be overconfident even when wrong and verifying the source instead of just trusting. That's the way I use something like perplexity, I use it as an improved search engines and then tend to visit the sources it lists.

I can't stress this enough, Homer Simpson is a fictional character from a cartoon. I would not use him in an argument about economics any more than I would use the Roadrunner to argue for road safety.

A 2025 Homer is only plausible if he had some kind of supplemental income (like a military pension or a trust fund), if Marge had a job, if the house was in a depressed region, or he was a higher level supervisor. We can use the Simpsons as limited evidence of contemporary economic conditions in the same way that we could use the depictions of the characters in the Canterbury Tales for the same purpose.

https://youtu.be/axHoy0hnQy8?t=29

This claim of "a single man could feed a whole family on one factory job" is misleading and untrue. It's usually the 1950s that people claim this was true and they wish we could go back to the 1950s. It's easy to show that that the 1950s were no picnic (https://archive.is/oH1Vx).

It's always some time in the past that the nation was great. They pick 1950s, you pick the 1990s. What you don't understand is that people are usually longing for a time when they weren't alive or when they were children. They want to go back to living the stress free life of a happy childhood, when your parents shielded you from all the vagaries of life.

You cite cartoons, they cite memes. If you ask them how a meme could possibly be used as evidence, they say much the same as you - anecdotes about their grandparents.

I would also trust 100 fictional cartoon characters before I would trust anything said in a pirated article written by Noah Smith about anything. If Noah Smith said that grass was green I would assume that it's blue.

In addition to these base skills, I also have specialized skills adapted to the modern world, that is my job. Combined with the internet and modern technology I can get to a level of proficiency that no one could get to in the ancient times. And the best part: I am not some kind of genius, just a regular guy with a job.

And I still have time to swipe on social media. I don't know what kind of brainless activities the ancient Greeks did, but they certainly had the equivalent of swiping on social media.

The general idea is that the more we offload to machines, the more we can allocate our time to other tasks, to me, that's progress, that some of these tasks are not the most enlightening doesn't mean we did better before.

And I don't know what economist mean by "productivity", but we can certainly can buy more stuff than before, it means that productivity must have increased somewhere (with some ups and downs). It may not appear in GDP calculations, but to me, it is the result that counts.

I don't count home ownership, because you don't produce land. In fact, that land is so expensive is a sign of high global productivity. Since land is one of the few things that we need and can't produce, the more we can produce the other things we need, the higher the value of land is, proportionally.

People will risk their and others' lives in a horribly janky car if it means they can swipe on social media instead of watching the road - acceptance doesn't mean it's good.

FTFY

Literacy, books, saving your knowledge somewhere else removes the burden of remembering everything in your head. But they don't come into effect into any of those processes. So it's an immensely bad metaphor. A more apt one is the GPS, that only leaves you with practice.

That's where LLMs come in, and obliterate every single one of those pillars on any mental skill. You never have to learn a thing deeply, because it's doing the knowing for you. You never have to practice, because the LLM does all the writing for you. And of course, when it's wrong, you're not wrong. So nothing you learn.

There are ways to exploit LLMs to make your brain grow, instead of shrink. You could make them into personalized teachers, catering to each student at their own rhythm. Make them give you problems, instead of ready-made solutions. Only employ them for tasks you already know how to make perfectly. Don't depend on them.

But this isn't the future OpenAI or Anthropic are gonna gift us. Not today, and not in a hundred years, because it's always gonna be more profitable to run a sycophant.

If we want LLMs to be the "better" instead of the "worse", we'll have to fight for it.

Yes, I wrote this comment under someone else's comment before, but it seems to apply to yours even better.

That said:

TV very much is the idiot box. Not necessarily because of the TV itself but rather whats being viewed. An actual engaging and interesting show/movie is good, but last time I checked, it was mostly filled with low quality trash and constant news bombardment.

Calculators do do arithmetic and if you ask me to do the kind of calculations I had to do in high school by hand today I wouldnt be able to. Simple calculations I do in my head but my ability to do more complex ones diminished. Thats down to me not doing them as often yes, but also because for complex ones I simply whip out my phone.

I got scared by how awfully my juniour (middle? 5-11) school mathematics had slipped when helping my 9 year old boy with his homework yesterday.

I literally couldn't remember how to carry the 1 when doing subtractions of 3 digit numbers! Felt literally idiotic having to ask an LLM for help. :(

What I have asked my children to do very often is back-of-the-envelope multiplications and other computations. That really helped them to get a sense of the magnitude of things.

Just expose them to everyday math so they aren't one of those people who think math has no practical uses. My father isn't great with math, but would raise questions like how wide a river was (solvable from one side with trig, using 30 degree angles for easy math). Napkin math makes things much more fun than strict classroom math with one right answer

Techniques of an "intuitive" character often lack or have formal underpinnings that are hard to understand, which means they do not to the same extent implicitly teach analytical methods that might later be a requirement for formal deduction.

Also aside, in the method I was taught in school (and I assume you and GP from terminology), "carrying" is what you do with addition (an extra 1 can be carried to the next column), "borrowing" is for subtraction (take a 1 away from the next column if needed).

That said, these kinds of studies are important, because they reveal that some cognitive changes are evidently happening. Like you said, it's up to us to determine if they're positive or negative, but as is probably obvious to many, it's difficult to argue for the status quo.

If it's a negative change, teachers have to go back to paper-and-pen essay writing, which I was personally never good at. Or they need to figure out stable ways to prevent students from using LLMs, if they are to learn anything about writing.

If it's a positive change, i.e., we now have more time to do "better" things (or do things better), then teachers need to figure out substitutes. Suddenly, a common way of testing is now outdated and irrelevant, but there's no clear thing to do instead. So, what do they do?

Here’s the key difference for me: AI does not currently replace full expertise. In contrast, there is not a “higher level of storage” that books can’t handle and only a human memory can.

I need a senior to handle AI with assurances. I get seniors by having juniors execute supervised lower risk, more mechanical tasks for years. In a world where AI does that, I get no seniors.

Shift to what? This? https://steve-yegge.medium.com/welcome-to-gas-town-4f25ee16d...

a) serious, but we live on different planets

b) serious with the idea, tongue-in-check in the style and using a lot of self-irony

c) an ironic piece with some real idea

d) he is mocking AI maximalists

Steve Yegge's a famous developer, this is not a joke :) You could say he is an AI maximalist, from your options I'd go with (b) serious with the idea, tongue-in-check in the style and using a lot of self-irony.

It is exaggerated, but this is how he sees things ending up eventually. This is real software.

If things do end up in glorified kanban boards, what does it mean for us? That we can work less and use the spare time reading and doing yoga, or that we'll work the same hours with our attention even more fragmented and with no control over the outputs of these things (=> stress).

I'd really wish that people who think this is good for us and are pushing for this future do a bit better than:

1. More AI 2. ??? 3. Profit

[1] https://www.nytimes.com/2025/01/15/books/review/open-socrate...

TV is the uber idiot box, the overlord of the army of portable smart idiot boxes.

Yes, but also the extra wrinkle that this whole thing is moving so fast that 4 months old is borderline obsolete. Same into the future, any study starting now based on the state of the art on 22/01/2026 will involve models and potentially workflows already obsolete by 22/05/2026.

We probably can't ever adapt fully when the entire landscape is changing like that.

> Previous tools outsourced partial processes - calculators do arithmetic, Google stores facts. LLMs can potentially take over the entire cognitive process from thinking to formulating. That's qualitatively different.

Yes, but also consider that this is true of any team: All managers hire people to outsource some entire cognitive process, letting themselves focus on their own personal comparative advantage.

The book "The Last Man Who Knew Everything" is about Thomas Young, who died in 1829; since then, the sum of recorded knowledge has broadened too much for any single person to learn it all, so we need specialists, including specialists in managing other specialists.

AI is a complement to our own minds with both sides of this: Unlike us, AI can "learn it all", just not very well compared to humans. If any of us had a sci-fi/fantasy time loop/pause that let us survive long enough to read the entire internet, we'd be much more competent than any of these models, but we don't, and the AI runs on hardware which allows it to.

For the moment, it's still useful to have management skills (and to know about and use Popperian falsification rather than verification) so that we can discover and compensate for the weaknesses of the AI.

He may have been right... Maybe our minds work in a different way now.

I think a better framing would be "abusing (using it too much or for everything) any new tool/medium can lead to negative effects". It is hard to clearly define what is abuse, so further research is required, but I think it is a healthy approach to accept there are downsides in certain cases (that applies for everything probably).

To be fair, I think this one is true. There's a lot of great stuff you can watch on TV, but I'd argue that TV is why many boomers are stuck in an echo chamber of their own beliefs (because CNN or fox news or whatever opinion-masquerading-as-journalism channel is always on in the background). This has of course been exacerbated by social media, but I can't think of many productive uses of TV other than sesame Street and other kids shows.

Writing did eliminate the need for memorization. How many people could quote a poem today? When oral history was predominant, it was necessary in each tribe for someone to learn the stories. We have much less of that today. Writing preserves accuracy much more (up to conquerors burning down libraries, whereas it would have taken genocide before), but to hear a person stand up and quote Desiderata from memory is a touching experience to the human condition.

Scribes took over that act of memorization. Copying something lends itself to memorization. If you have ever volunteered extensively for project Gutenberg you can also witness a similar experience: reading for typos solidifies the story into your mind in a way that casual writing doesn't. In losing scribes we lost prioritization of texts and this class of person with intimate knowledge of important historical works. With the addition of copyright we have even lost some texts. We gained the higher availability of works and lower marginal costs. The lower marginal costs led to...

Pulp fiction. I think very few people (but I would be disappointed if it was no one) would argue that Dan Brown's da Vinci Code is on the same level as War and Peace. From here magazines were created, even cheaper paper, rags some would call them (or use that to refer to tabloids). Of course this also enabled newspapers to flourish. People started to read things for entertainment, text lost its solemnity. The importance of written word diminished on average as the words being printed became more banal.

TV and the internet led to the destruction of printed news, and so on. This is already a wall of text so I won't continue, but you can see how it goes:

Technology is a double edged sword, we may gain something but we also can and did lose some things. Whether it was progress or not is generally a normative question that often a majority agrees with in one sense or another but there are generational differences in those norms.

In the same way that overuse of a calculator leads to atrophy of arithmetic skills, overuse of a car leads to atrophy of walking muscles, why wouldn't overuse of a tool to write essays for you lead to atrophy of your ability to write an essay? The real reason to doubt the study is because its conclusion seems so obvious that it may be too easy for some to believe and hide poor statistical power or p-hacking.

I also find exhausting the Socrates reference that's ALWAYS brought up in these discussions. It is not the same. Losing the collective ability to recite a 10000 words poem by heart because of books it's not the same thing as stopping to think because an AI is doing the thinking for you.

We keep adding automation layers on top of the previous ones. The end goal would be _thinking_ of something and have it materialized in computer and physical form. That would be the extreme. Would people keep comparing it to Socrates?

Were they? It seems that often the fears came true, even Socrates’

It hugely enhanced synthetic and contextual memory, which was a huge development.

AI has the potential to do something similar for cognition. It's not very good at it yet, but externalised cognition has the potential to be transformative in ways we can't imagine - in the same way Socrates couldn't imagine Hacker News.

Of course we identify with cognition in a way we didn't do with rote memory. But we should possibly identify more with synthetic and creative cognition - in the sense of exploring interesting problem spaces of all kinds - than with "I need code to..."

So we’ve externalised memory, we’ve externalised arithmetic. Personally the idea of externalising thinking seems to be the last one? It’s not clear what’s left inside us of being a human once that one is gone

Wouldnt the endgame of externalized cognition be that humans essentially become cogs in the machine?

In fact, I can feel my memory is easily worse now than from before ChatGPT's release, because we are doing less hard cognitive work. The less we use our brain's the dumber we get, and we are definitely using our brains less now.

Perhaps he could. If there’s an argument to be made against writing, social media (including HN) is a valid one.

Still is.

What do you mean? All of them were 100% right. Novels are brain softening, TV is an idiot box, and writing destroys memory. AI will destroy the minds of people who use it much.

Critical thinking, forming ideas, writing, etc, those are too stuff that can atrophy if not used.

For example, a lot of people can't locate themselves without a GPS today.

To be frank I see it really similar to our muscles: don't want to lose it? Use it. Whether that is learning a language, playing an instrument or the task llms perform.

In my opinion, they've almost always been right.

In the past two decades, we've seen the less-tech-savvy middle managers who devalued anything done on computer. They seemed to believe that doing graphic design or digital painting was just pressing a few buttons on the keyboard and the computer would do the job for you. These people were constantly mocked among online communities.

In programmers' world, you have seen people who said "how hard it could be? It's just adding a new button/changing the font/whatever..."

And strangely, in the end those tech muggles were the insightful ones.

Earlier, I had to only keep my phone away and not open Instagram while studying. Now, even thinking can be partially offloaded to an automated system.

this actually does include a crazy amount of long form latex expositions on a bunch of projects im having a blast iterating on. i must be experiencing what its almost like not having adhd

It certainly hasn't inhibited learning either. The most recent example is shaders. I started by having it just generate entire shaders based on descriptions, without really understanding the pipeline fully, and asking how to apply them in Unity. I've been generally familiar with Unity for over a decade but never really touched materials or shaders. The generated shaders were shockingly good and did what I asked, but over time I wanted to really fine tune some of the behavior and wound up with multiple passes, compute shaders, and a bunch of other cool stuff - and understanding it all on a deeper level as a result.

If you aren't, then sure you'll be a PM with a lackluster team of engineers.

LLMs can engineer small well defined functions / scripts rather well in my experience. Of course it helps to be able to understand what it outputs and prod it to engineer it just the way you want it. Still faster than me writing it from scratch, most of the time. And even if it's the same time as me doing it from scratch it feels easier so I can do more without getting tired.

It just feels like AI upskills everyone a little.

Meanwhile they're pulling the same or greater comp, working half the hours, and "drinking beers with important people" is an accepted part of their job. The status hierarchy you're describing where they suck isn't real. It's a useful fiction that keeps you grinding while they harvested your output.

Everyone becoming a PM is a good thing precisely because PMs don't work as hard. Wouldn't a job be more pleasant if you could meet expectations by lunch? Imagine how psychologically freeing that would be. Dreadful future my ass.

The study shows that the brain is not getting used. We will get stupid in the same way that people with office jobs get unhealthy if they don't deliberately exercise.

I haven't been diagnosed with ADHD or anything but i also haven't been tested for it. It's something I have considered but I think it's pretty underdiagnosed in Spain.

That must be how normal people feel.

One of my favorite things is that I no longer feel like I need to keep up with "framework of the year"

I came up over a decade ago, places I worked were heavy on Java and Spring. Frontends were Jquery back then. Since then I've moved around positions quite a bit, many different frameworks, but typically service side rendered MVC types and these days I work as an SRE. The last 5 years I've fiddled with frontend frameworks and SPAs but never really got into it. I just don't have it in me to learn ANOTHER framework.

I had quite a few projects, all using older patterns/frameworks/paradigms. Unfortunately these older paradigms don't lend themselves to "serverless" architecture. So when I want to actually run and deploy something I've gotta deploy it to a server (or ecs task). That shit starts to cost a bit of money, so I've never been able to keep projects running very long... typically because the next idea comes up and I start working on that and decide to spend money on the new things.

I've been working at a cloud native shop the last 7 years now. Damn, you can run shit CHEAP in AWS if you know what you're doing. I know what I'm doing for parts of that, using dynamodb instead of rds, lambdas instead of servers. But I could never get far enough with modern frontend frameworks to actually migrate my apps to these patterns.

Well, now it's easy.

"Hey Claude, look at this repo here, I want to move it to AWS lambdas + apigw + cloudfront. Break the frontend out into a SPA using vue3. I've copied some other apps and patterns {here} so go view those for how to do it"

And that's just the start.

I never thought I'd get into game development but it's opened that up to me as well (though, since I'm not an artist professionally I have issues getting generative AI to make assets, so I'm stuck plodding along in aseprite and photoshop make shit graphics lol). I've got one simple game like 80% done and ideas for the next one.

I never got too far down mobile development either. But one of the apps I made it could be super useful to have a mobile app. Describe the ux/ui/user flow, tell it where to find the api endpoints, and wham bam, android app developed.

Does it make perfect code one shot? Sometimes, but not often, I'll have to nudge it along. Does it make good architectural decisions? Not often on its own, again, I'l nudge it, or even better, I'll spin up another agent to do code reviews and feed the reviews back into the agent building out the app. Keep doing that loop until I feel like the code review agent is really reaching or being too nitpicky.

And holy shit, I've been able to work on multiple things at the same time this way. Like completely different domains, just have different agents running and doing work.

btw, I have a couple of questions just out of curiosity: What tools do you use besides Claude? Do you have a local or preferred setup? and do you know of any communities where discussion about LLM/general AI tool use is the focus, amongst programmers/ML engineers? Been trying to be more informed as to what tools are out there and more up to date on this field that is progressing very quickly.

1: https://www.catharsisinsight.com 2: https://ashleyjuavinett.com

How about some more info on what their main conclusions are?

The hosts condemn the study’s "bafflingly weak" logic and ableist rhetoric, and advise skepticism toward "science communicators" who might profit from selling hardware or supplements related to their findings: one of the paper's lead authors, Nataliya Kosmyna, is associated with the MIT Media Lab and the development of AttentivU, a pair of glasses designed to monitor brain activity and engagement. By framing LLM use as creating a "cognitive debt," the researchers create a market for their own solution: hardware that monitors and alerts the user when they are "under-engaged". The AttentivU system can provide haptic or audio feedback when attention drops, essentially acting as the "scaffold" for the very cognitive deficits the paper warns against. The research is part of the "Fluid Interfaces" group at MIT, which frequently develops Brain-Computer Interface (BCI) systems like "Brain Switch" and "AVP-EEG". This context supports the hosts' suspicion that the paper’s "cognitive debt" theory may be designed to justify a need for these monitoring tools.

In my own (classic) engineering work, AI has become so phenomenally powerful that I can only imagine that if I was still in college, I'd be mostly checked out during those boring lectures/bad teacher classes, and then learning on my own with the textbook and LLMs by night. Which begs the question, what do we need the professor for?

I'd be interested to see stats on "office hours" visitation time over the last 4 years (although admittedly its the best tool for gaining a professor's favor, AI doesn't grant that)

The media is extremely pro-AI (and a quick look at their ownership structure gives you a hint as to why). You seem to be projecting your own biases here, no?

And how would those LLMs learn? How would you learn to ask the right questions that further scientific research?

> To summarize, the delta-band differences suggest that unassisted writing engages more widespread, slow integrative brain processes, whereas assisted writing involves a more narrow or externally anchored engagement, requiring less delta-mediated integration.

There is no intellectual judgement regarding this difference, though the authors do supply citations from related work that they claim may be of interest to those wanting "to know if the offloading of cognitive tasks changes my own brain and my own cognition". If your brain changes, it might change for the worse at least as far as you experience it. Is this ableism, to examine your own cognitive well-being and make your own assessment? If you don't like how you're thinking about something, are you casting aspersion on yourself and shaming your own judgement? Ableist discourse is, unsurprisingly, a stupid language game for cognitively impaired dummies. It's a pathetic attempt to redefine basic notions of capability and impairment, of functioning and dysfunction as inherently evil concepts, and then to work backward from that premise to find fault with the research results. Every single person experiences moments or lifetime's of psychological and mental difficulty. Admitting this and adapting to it or remediating harmful effects has nothing to do with calling stupid people stupid or ableism. It's just a means of providing tools and frameworks for "cognitive wellness", but even just the implication of "wellness" being distinct from "illness" makes the disturbed and confused unwell.