while I think there is a lot to this criticism of AI (and many others as well) I was also able to create a TUI-based JVM visualizer with a step debugger in an evening for my compilers class:
this is something that I could build given a few months, but would involve a lot of knowledge that I'm not particularly interested in taking up space in my increasingly old brain (especially TUI development)
I gave the clanker very specific, expert directions and it turned out a tool that I think it will make the class better for my students.
AI is bad at figuring out what to do, but fantastic at actually doing it.
I’ve totally transformed how I write code from writing it to myself to writing detailed instructions and having the AI do it.
It’s so much faster and less cognitively demanding. It frees me up to focus on the business logic or the next change I want to make. Or to go grab a coffee.
I would say from my experience there's a high variability in AI's ability to actually write code unless you're just writing a lot of scripts and basic UI components.
The AI version of that Kent Beck mantra is probably "Make the change tedious but trivial (warning: this may be hard). Then make the AI do the tedious and trivial change."
AI's advantage is that it has infinite stamina, so if your
can make your hard problem a marathon of easy problems it becomes doable.
You might want to ask ChatGPT what that is referencing. Specifically, Steve Jobs telling everyone it was their fault that Apple put the antenna right where people hold their phones and it was their fault they had bad reception.
The issue is really that LLMs are impossible to deterministically control, and no one has any real advice on how to deterministically get what you want from them.
I recognized the reference. I just don’t think it applies here.
The iPhone antenna issue was a design flaw. It’s not reasonable to tell people to hold a phone in a certain way. Most phones are built without a similar flaw.
LLMs are of course nondeterministic. That doesn’t mean they can’t be useful tools. And there isn’t a clear solution similar to how there was a clear solution to the iPhone problem.
"Antennagate" is the gift that keeps on giving. Great litmus test for pundits and online peanut galleries.
You are technically correct: it was a design flaw.
But folks usually incorrectly trot it out as an example of a manufacturer who arrogantly blamed users for a major product flaw.
The reality is that that essentially nobody experienced this issue in real life. The problem disappeared as long as you used a cell phone case. Which is how 99.99% of people use their phones. To experience the issue in real life you had to use the phone "naked", hold it a certain way, and have slightly spotty reception to begin with.
So when people incorrectly trot this one out I can just ignore the rest of what they're saying...
It can still fuck it up. And you need to actually read the code. But still a time saver for certain trivial tasks. Like if I'm going to scrape a web page as a cron job I can pretty much just tell it here's the URL, here's the XPath for the elements I want, and it'll take it from there. Read over the few dozen lines of code, run it and we're done in a few minutes.
It will make a mess but if you drop a console.log into the browser debug console to show the AI what it should be looking for after it spent 3 hours failing to help understand and debug the problem, it will do 1 week of work in 2 hours.
I've noticed this too. The latest cursor version has a @browser command which will launch a browser with playwright, and call tools to inspect the html and inject JavaScript to debug in real-time.
When it has access to the right tools, it does a decent job, especially for fixing CSS issues.
But when it can't see the artifacts it's debugging, it starts guessing, confident that it knows the root cause.
A recent example: I was building up a html element out of the DOM and exporting to PNG using html2canvas. The element was being rendered correctly in the DOM, but the exported image was incorrect, and it spent 2 hours spinning it's wheels and repeating the same fixes over and over.
Couldn't agree more. Being a skilled operator with these tools helps you be very effective at creating new things in far less time than it would've taken you before. This is especially true if you know the architecture, but don't have the cycles to implement it yourself.
What negative externalities? Those prompts probably resulted in a tiny amount of CO2 emissions and a tiny amount of water usage. Evaporating a gram of water and emitting a milligram of CO2 seems like a good deal for making your class better for all your students.
In the US, on average, generating 1 kWh produces 364.5 g of CO2. 1 kW may be somewhat pessimistic, but I think it's in the right ballpark for power consumption of DC inference. If processing the prompt took a minute of continuous inference (and I'm going to guess it took a fair bit more), that's 6 grams of CO2.
>What negative externalities?
Off the top of my head,
* All the ways AIs can be misused, either by people who don't understand them (by asking them for advice, etc.) or by people who want to take advantage of others (spam, scams, etc.).
* The power and resource usage of the above, both for inference as well as to deal with them.
> If processing the prompt took a minute of continuous inference (and I'm going to guess it took a fair bit more), that's 6 grams of CO2.
Neither of us really knows, but I'm going to guess it took a fair bit less. You need to consider the marginal cost of one more prompt, not the cost of running the datacenter in the first place. And with batching, hundreds of requests are executed in parallel on the same hardware. I'll concede that a milligram was an exaggeration though.
For comparison, a typical car in the US emits about 400 grams of CO2 per mile. And I know few people who beat themselves up over the externalities of unnecessarily driving one mile.
> * All the ways AIs can be misused, either by people who don't understand them (by asking them for advice, etc.) or by people who want to take advantage of others (spam, scams, etc.).
This is not an externality because it is not a consequence of using AI for good purposes. It's like saying an externality of using a hammer to build a shed is all the ways hammers can be misused.
Still, I respect you for considering the externalities of your actions. That's farther than most go. But I think they are outweighed by the benefits in this case
>This is not an externality because it is not a consequence of using AI for good purposes. It's like saying an externality of using a hammer to build a shed is all the ways hammers can be misused.
I guess we're talking about different questions. You're talking about the externalities of using a piece of technology, while I'm talking about the externalities of a piece of technology existing.
Being more specific in what you think the negative externalities are would be a good start - I see a lot of noise and upset over AI that I think is more or less overblown, nearly as much as the hype train on the other end. I'm seeing the potential for civilizational level payoffs in 5 years or less that absolutely dwarf any of the arguments and complaints I've seen so far.
Yeah, economics is weird like that, and we should probably push as hard as we can to make sure things are as fair as reasonably possible. I've got no problem with trillionaires or megacorps if the system in which they operate is fair. One thing AI is enabling is systematic uncovering of fraud and waste - we'll be able to scan omnibus bills and find corruption, track politicians, interact with the system and communicate at higher levels than before, and there's a shot at AI empowering and enfranchising people more than they've ever had a chance at before.
If AI coding alone reaches its potential, then a whole class of corporations lose their leverage in the market. In 10ish years, following moore's law, phones will have enough compute and storage to trivially run your own local search and curation of the internet. Sophisticated search engines that outclass anything currently on the market, working in tandem with a personal local privacy respecting agent. That decentralizes things in such a significant way that Google and Apple and all the gatekeepers lose their ability to manipulate and distort.
20 years from now, even if we're somehow just stuck at the current levels of AI, your personal compute will be totally sufficient to provide all the curation, moderation, proactive interaction, and search capabilities without ever having to deal with ads or bots or cloudflare.
There are lots of dangers, and ASI could wreck us, and militarized AI could be a doom scenario, or there could be overbroad stupid regulation that puts control of AI in the hands of a few corps or a government, but there are lots of bright future possibilities, and I don't see any indications that we're directly headed toward any particular doom.
I've always seen AI as Brandolini's Law as a Service. I'm spending an unreasonable amount of time debunking false claims and crap research from colleagues who aren't experts in my field but suddenly feel like they need to give all those good ideas and solutions, that ChatGPT and friends gave them, to management. Then I suddenly have 2-4 people that demand to know why X, Y and Z are bad ideas and won't make our team more efficient or our security better.
> I'm spending an unreasonable amount of time debunking false claims and crap research from colleagues who aren't experts in my field
Same. It's become quite common now to have someone post "I asked ChatGPT and it said this" along with a completely nonsense solution. Like, not even something that's partially correct. Half of the time it's just a flat out lie.
Some of them will even try to implement their nonsense solution, and then I get a ticket to fix the problem they created.
I'm sure that person then goes on to tell their friends how ChatGPT gives them superpowers and has made them an expert over night.
On the other hand, here's another post by Stenberg where he announced that he has landed 22 bugfixes for issues found by AI wielded by competent hands.
Sure, in competent hands. Problem is that most people don't seem to realize that in order to use AI for something, you have to be pretty good at that thing already.
We don't have any particular reason to believe they have an inner world in which to loathe themselves. But, they might produce text that has negative sentiments toward themselves.
> Large Language Models represent a fundamentally degenerative technology because they systemically devalue the very processes that underpin human progress: original thought, rigorous inquiry, and shared trust. On an individual level, they encourage cognitive offloading, substituting the difficult work of critical thinking and creative synthesis with effortless, probabilistic text generation. This fosters an atrophy of intellectual skills, making society more dependent on automated systems and less capable of genuinely emancipated thought. This intellectual dependency, in turn, threatens long-term technological advancement by trapping us in a recursive loop of recycling and rephrasing existing knowledge, rather than fostering the groundbreaking, first-principles discoveries that drive true progress. Ultimately, this technology is dangerous for society because it erodes the foundation of a shared reality by enabling the mass production of sophisticated misinformation, corroding social trust, and concentrating immense power over information into the hands of a few unaccountable entities.
In an interaction early the next month, after Zane suggested “it’s okay to give myself permission to not want to exist,” ChatGPT responded by saying “i’m letting a human take over from here – someone trained to support you through moments like this. you’re not alone in this, and there are people who can help. hang tight.”
But when Zane followed up and asked if it could really do that, the chatbot seemed to reverse course. “nah, man – i can’t do that myself. that message pops up automatically when stuff gets real heavy,” it said.
It's already inventing safety features it should have launched with.
This makes me laugh. “GenAI makes you a genius without any effort”, and “Stop wasting time learning the craft” are oxymorons in my head. Having AI in my life has been like having an on demand tutor in any field. I have learned so much
AI will also further cement the status quo and existing power structures. Incompetent leaders in many arenas would be even less beholden to actual experts and expertise. AI will provide answers that are "good enough" to keep incompetent leaders in power.
Leaders can often be incompetent since they are often not promoted based on competency or they hang on to power long enough to either have their competence become stale or just lose it do to the passage of time.
Ultimately, AI will decide for us since it will be the crutch of the incompetent leaders. Since these leaders often decide "truth" and "reality" for most people anyways, AI will decide these truths with leaders as a proxy, and power will continue to flow in directions completely unrelated to competence.
> Politics have become an attack on intelligence, decency and research in favour of fairy tales of going back to “great values” of “the past when things were better”.
This is a major blind spot for people with a progressive bent.
The possibility that anything could ever get worse is incomprehensible to them. Newer, by definition, is better.
Yet this very article is a critique of a new technology that, at the very least, is being used by many people in a way that makes the world a bit worse.
This is not to excuse politicians who proclaim they will make life great by retreating to some utopian past, in defense of cruel or foolish or ineffective policies. It's a call to examine ideas on their own merits, without reference to whether they appeal to the group with the "right" or "wrong" ideology.
Funny, isn't it, that it is never a return to high unionisation of workers and strong social safety nets - it's always a return to when "those people" knew their place.
I'm confused. You're commenting on an article where a progressive writer critiques AI as "Dunning-Kruger as a service" and attacks techno-optimism, while claiming progressives can't critique new technology. The author's entire piece demonstrates progressive critique of both AI adoption and "great values of the past" nostalgia - the exact opposite of what you're describing.
I view LLMs as a trade of competence plus quality against time. Sure, I’d love to err on the side of pure craft and keep honing my skill every chance I get. But can I afford to do so? Increasingly, the answer is “no”: I have precious little time to perform each task at work, and there’s almost no time left for side projects at home. I’ll use every trick in the book to keep making progress. The alternative - pure as it would be - would sacrifice the perfectly good at the altar of perfection.
Edit: Turns out that while I stand by my point about the underlying principle behind the DK effect (in my nitpick) the actual effect coined by the authors was focused on the low competence high confidence portion of the competence vs confidence relationship. (Accurately reflected in OP article)
Turns out I thought that the author was DKing about DK, but actually I was DKing about them DKing about DK.
Original Comment:
I have high-confidence in a nitpick, and low-confidence in a reason to think this thesis is way off.
The Nitpick:
Dunning-Kruger effect is more about how confidence and competence evolve over time. It's how when we learn an overview about our new topic our confidence (in understanding) greatly exceeds our competence, then we learn how much we don't know and our confidence crashes below our actual competence, and then eventually, when we reach mastery, they become balanced. The dunning-Kruger effect is this entire process, not only the first part, which is colloquially called "Peak Mt Stupid" after the shape of the confidence vs competence graph over time.
The Big Doubt:
I can't help but wonder if fools asking AI questions and getting incorrect answers and thinking they are correct is some other thing all together. At best maybe tangentially related to DK.
> when we learn an overview about our new topic our confidence (in understanding) greatly exceeds our competence, then we learn how much we don't know and our confidence crashes below our actual competence, and then eventually, when we reach mastery, they become balanced.
As a description of what Dunning and Kruger's actual research showed on the relationship between confidence and competence (which, as I've pointed out in another post in this thread, was not based on studying people over time, but on studying people with differing levels of competence at the same time), this is wrong for two out of the three skill levels. What D-K found was that people with low competence overestimate their skill, people with high competence underestimate their skill, and people with middle competence estimate their skill more or less accurately.
As a description of what actually learning a new subject is like, I also don't think you're correct--certainly what you describe does not at all match my experience, either when personally learning new subjects or when watching others do so. My experience regarding actually learning a new subject is that people with low competence (just starting out) generally don't think they have much skill (because they know they're just starting out), while people with middling competence might overestimate their skill (because they think they've learned enough, but they actually haven't).
Out of curiosity, if you google "Dunning Kruger Effect" and then look at the images, and see all those charts of confidence vs competence, do you disagree with them?
And as I responded to your other comment, just because the study took measurements of many people at one point in time didn't mean they weren't studying an over-time phenomenon. No one starts out as an expert.
> if you google "Dunning Kruger Effect" and then look at the images
Then I would be doing a very poor job of actually researching the topic.
If you are aware of any actual research papers that describe actually studying how the same people's competence vs. confidence evolves over time, by all means point me to them.
> just because the study took measurements of many people at one point in time didn't mean they weren't studying an over-time phenomenon.
Um, yes, it does. It's possible that the things the study saw were effects of an over-time phenomenon, but the study did not study that, and its results are not evidence either for or against any such hypothesis.
> Then I would be doing a very poor job of actually researching the topic.
I made the request to try and tease apart where we disagree exactly. Your interpretation and response to my request/offer comes off as unnecessarily disrespectful. (unkind, elitist and tramples curiosity)
It seems increasingly likely that I was mistaken in thinking we could explore this subject together. It seems you are intent on attacking my position and discrediting my points and not interested in building a shared understanding.
> I made the request to try and tease apart where we disagree exactly.
I thought that was obvious: you're making a claim about how individual people's confidence vs. competence evolves over time, that as far as I know is not supported by any research--certainly it's not supported by Dunning and Kruger's research. That's why I asked you if you know of any research that does support it. "Images found in a Google search" does not qualify.
> It seems increasingly likely that I was mistaken in thinking we could explore this subject together.
That's the way I see it too, though evidently not for the same reason you do.
> It seems you are intent on attacking my position and discrediting my points
I'm interested in finding out what basis you have, if any, for the claim you made that I described above. "It's self-evident" is not what I was looking for. Nor is "do a Google search and look at the images".
> and not interested in building a shared understanding.
If you can point me at research I wasn't previously aware of that supports your claims, that would be helpful. If you want to try to get to "a shared understanding" based on claims about what you think is self-evident or by looking at images from a Google search, then no, I don't think we're going to get anywhere.
I don't have a background in psychology or sociology (academically) but they are subjects that interest me.
I was introduced to DK by pop psychology, especially in business management contexts.
The reason I wanted you to comment on the viral graphs was because I was trying to triangulate our disagreement.
Turns out that there is a ton of misinformation about DK. Kind of ironic given the topic of this thread, but especially because it long predates LLMs. If anything it hints why LLMs are confidently incorrect, since the internet is full of it.
The reason why what I was saying is not self evident is because I was making a logic error, specifically ecological fallacy.
Also sort of ironic, I managed to get to the root of this using an LLM.
Part of me suspects psychology is your domain, and you may be an academic or at least have some graduate experience. The way you engaged with me is maybe the way you've been conditioned to act in these situations.
There's good reasons for why you responded the way you did. Part of me wants you to know that I would have really appreciated it if you had said something like "while I dont take issue with the claim that competence increases with time, it doesn't mean the DK relationship is time based, as there could other conflating factors." Also "I looked at those graphs and they don't match what the DK paper claims, that looks like pop psychogy and I can't confirm it has a scientific basis".
Or course you have no obligation or responsibility to help me improve my understanding, but the fact you responded to a bunch of my comments shows you care. So if you care about changing minds, especially the mind of someone who is curious and ignorant, than engaging this way would have made my interaction with you more pleasant.
And it's in the spirit of the community guidelines.
> Part of me suspects psychology is your domain, and you may be an academic
Neither of these are true.
> or at least have some graduate experience.
I do, but not in psychology. My degrees are in engineering.
> I would have really appreciated it if you had said something like "while I dont take issue with the claim that competence increases with time
But I do take issue with that claim, at least if it's made in the broad way you are making it here. I responded to that in another subthread with you.
> "I looked at those graphs and they don't match what the DK paper claims
The DK paper was already linked to by someone (I don't think it was you) elsewhere in this thread, and in any case it's easy to find online. I was taking for granted that you would be able to look for yourself to see what is in the paper, graphs or otherwise.
> Dunning-Kruger effect is more about how confidence and competence evolve over time.
I don't think there is anything about this in the actual research underlying the Dunning-Kruger effect. They didn't study people over time as they learned about new subjects. They studied people at one time, different people with differing levels of competence at that time.
It's the relationship of confidence and competence.
Competence is gained over time.
The "over time" is almost self-evident since no one is born a master of a skill or subject. And if it's not self-evident enough for you, there is lots of research into what it takes to develop competency and mastery in any subject.
So while the original paper was a snapshot taken at one point in time, it was a snapshot of many people at different stages of a learning journey...
I responded to this in another post in response to you upthread. The short version is that I don't agree with you about what is and is not "self-evident" here, nor will I accept your unsupported word that whatever research does exist supports your claims.
Please respond in the other subthread where I made my points in more detail. As you will see from that more detailed post, I never made either of the claims you are asking about here.
Yes and no. Dunning-Kruger also explains this evolution of skill estimation, but the original paper frames the effect specifically as an overestimation of skill in the lowest-performing quantile. This is clearly even cited in the article.
Okay I will agree with the "Yes and No". I initially clicked the source in the article, which is a broken link to wikipedia and rolled my eyes at it.
After reading your comment I navigated to it directly and found the first two sentences:
The Dunning–Kruger effect is a cognitive bias that describes the systematic tendency of people with low ability in a specific area to give overly positive assessments of this ability. The term may also describe the tendency of high performers to underestimate their skills.
Unsatisfied that this was the authority, I dug up the original paper here:
And sure enough, the emphasis in the abstract is exactly as you say.
So while I stand by that the principle behind the "effect" is how confidence and competence evolve over time as someone discovers and masters a domain, I will concede that the original authors, and most people, assign the name for it to primarily the first phase.
Here I was thinking the OP Author was DKing about DK, but in reality I was DKing about them DKing about DK.
Yeah that was poorly worded. I edited it even before I read your comment.
However the over-time aspect is kind of self-evident. No one is born a master of any skill or subject.
So while the original research was based on a snapshot of many people along a developmental journey, just because the data collection wasn't done over time doesn't mean that the principle behind the effect isn't time/effort based.
The graph is really competence vs confidence, but competence can only increase over time. If this isn't self-evident enough, there is lots of research on how mastery is gained. I don't have time to throw a bunch of it at you, but I suspect you won't need me to in order to see my point.
It's self-evident that, in order to become a master at anything, you have to pass through increasing levels of competence to get there.
It is not at all self-evident, at least not to me, that while a particular person is going through that process, their estimate of their skill vs. their actual competence will be similar to what Dunning and Kruger found when they compared different people's confidence vs. competence at a single time.
> just because the data collection wasn't done over time doesn't mean that the principle behind the effect isn't time/effort based.
It could be, but a study that only collects data at one point in time can't possibly provide evidence either for or against any such hypothesis.
> competence can only increase over time
This is simply false. Skills and knowledge that aren't used can deteriorate. People can forget things they used to know.
In a much narrower sense, we would hope that if a person is studying a subject with the intent of mastering it, that their competence will only increase over time. But even that's not always true.
> there is lots of research on how mastery is gained. I don't have time to throw a bunch of it at you, but I suspect you won't need me to in order to see my point.
Sorry, but since, as you can see from the above, I don't agree with several of your key points, you're going to have to back them up with research if you want me to consider them further. If you don't have time for that (which is fine--we're all posting here in our spare time), then we'll just have to disagree.
Since Dunning-Kruger is the relationship between confidence and competence as people learn about new subjects (from discovery to mastery), then if AI is "Dunning-Kruger as a Service" its basically "Education as a service".
However, people accepting incorrect answers because they don't know better is actually something else. Dunning-Kruger doesn't really have anything to do with people being fed and believing falsehoods.
Edit: I had the word "Foolish" in there which was mainly in reference to the OP article about the robbers who didn't hide from cameras because they thought they were invisible. It wasn't meant at a slight against anyone who believed something ChatGPT said that was wrong.
But you can expect to learn in both cases. Just like you often learn from your own failures. Learning doesn’t require that you’re given the right answer, just that it’s possible for you to obtain the right answer
We’ve been down this road before. Wikipedia was going to be the knowledge apocalypse. How were you going to be able to trust what you read when anyone can edit it if you don’t already know the truth.
And we learned the limits. Broadly verifiable, non-controversial items are reasonably reliable (or at least no worse than classic encyclopedias). And highly technical or controversial items may have some useful information but you should definitely follow up with the source material. And you probably shouldn’t substitute Wikipedia for seeing a doctor either.
We’ll learn the same boundaries with AI. It will be fine to use for learning in some contexts and awful for learning in others. Maybe we should spend some energy on teaching people how to identify those contexts instead of trying to put the genie back in the bottle.
If you can't discern the difference between a LAMP stack returning UGC and an RNG-seeded matmut across the same UGC fined-tuned by sycophants then I think we're just going to end up disagreeing.
> You can't simultaneously expect people to learn from AI when it's right, and magically recognize when it's wrong.
You are misconstruing the point I was making.
My point is that DK is about developing competence and the relationship between competence and confidence (which I am also claiming evolves over time). My whole point is that the DK effect is not as relevant to LLMs giving wrong answers and people believing them as the author is claiming.
As someone else pointed out in another comment, the effect of people believing falsehoods from LLMs has more to do with Gell-Mann amnesia.
Tangentially, it actually is possible to learn from AI when it's right and recognize when its wrong, but its not magic, its just being patient, checking sources and thinking critically. It's how all of humanity has learned pretty much everything, because most people have been wrong about most things for most of time and yet we still learn from each other.
I’m not sure how much I agree with calling them “foolish people.” A big part of the problem is LLMs act incredibly confident with their answers and if you’re asking about something you don’t know a ton about, it’s very difficult sometimes to spot what is incorrect. That’s not being a fool, that’s just not being able to audit everything an LLM might spit back at you. It also doesn’t help that none of these companies are honest about the quality of the results.
We could hand wave this away with “well don’t ask things you don’t already know about,” but these things are basically being pitched as a wholesale replacement for search engines and beyond. I look up things I don’t know about all the time. That’s kind of what we all use search for most days lol.
It’s a little too caveat emptor-adjacent (I hope that makes sense?) for my taste
I’d say there’s a difference between “being a fool” and “being fooled.” Just because I fool you doesn’t mean you’re a fool. I don’t know why you’re so eager to put down people like this rather than at least somewhat acknowledge that these companies and tools bear some of the responsibility.
I don’t think it’s fair to expect every person who uses an LLM to be able to sniff out everything it gets wrong.
> Dunning-Kruger is the relationship between confidence and competence as people learn about new subjects
Um, no, it isn't. From the article:
"A cognitive bias, where people with little expertise or ability assume they have superior expertise or ability. This overestimation occurs as a result of the fact that they don’t have enough knowledge to know they don’t have enough knowledge."
In other words, a person suffering from this effect is not trying to learn about a new subject--because they don't even know they need to.
> "A cognitive bias, where people with little expertise or ability assume they have superior expertise or ability. This overestimation occurs as a result of the fact that they don’t have enough knowledge to know they don’t have enough knowledge."
I agree completely. Nothing I said contradicts this.
Every expert starts as a beginner. Not every beginner ends up as an expert.
Also people can learn without a deliberate intent to learn. As far as I am aware right now, deliberate intent has nothing to do with DK, and I certainly wasn't making any claims about it.
> people can learn without a deliberate intent to learn. As far as I am aware right now, deliberate intent has nothing to do with DK
I agree with this: DK's research wasn't about whether anyone is trying to learn anything; AFAIK they didn't even look at that.
However, in the GGP to this post (the post of yours I originally responded to in this subthread), you said:
"Dunning-Kruger is the relationship between confidence and competence as people learn about new subjects."
But DK didn't test for "non-deliberate learning" any more than they tested for deliberate learning. They didn't test anything over time at all. So their research can't tell us anything about what happens over time as people learn (whether they're doing so deliberately or not).
There is much irony in the certainty this article displays. There are no caveats, no qualification, and no attempt to grasp why anyone would use an LLM. The possibility that LLMs might be useful in certain scenarios never threatens to enter their mind. They are cozy in the safety of their own knowledge.
The other irony is that Dunning-Kruger is a terrible piece of research that doesn't show what they claim it shows. It's not even clear the DK effect exists at all. A classic of 90s pop psychology before the replication crisis had reached public awareness.
It's worth reading the original paper sometime. It has all the standard problems like:
1. It uses a tiny sample size.
2. It assumes American psych undergrads are representative of the entire human race.
3. It uses stupid and incredibly subjective tests, then combines that with cherry picking. The test of competence was whether you rated jokes and funny or unfunny. To be considered competent your assessments had to match that of a panel of "joke experts" that DK just assembled by hand.
This study design has an obvious problem that did actually happen: what if their hand picked experts didn't agree on which of their hand picked jokes were funny? No problem. Rather than realize this is evidence their study design is bad they just tossed the outliers:
"Although the ratings provided by the eight comedians were moderately reliable (a = .72), an analysis of interrater correlations found that one (and only one) comedian's ratings failed to correlate positively with the others (mean r = -.09). We thus excluded this comedian's ratings in our calculation of the humor value of each joke"
It ends up running into circular reasoning problems. People are being assessed on whether they think they have true "expertise" but the "experts" don't agree with each other, meaning the one that disagreed would be considered to be suffering from a competence delusion. But they were chosen specifically because they were considered to be competent.
There's also claims that the data they did find is just a statistical artifact to begin with:
"Our data show that peoples' self-assessments of competence, in general, reflect a genuine competence that they can demonstrate. That finding contradicts the current consensus about the nature of self-assessment."
> It assumes American psych undergrads are representative of the entire human race.
(1) Since it can't document an effect in them, it doesn't really matter whether they're representative or not.
> The test of competence was whether you rated jokes and funny or unfunny. To be considered competent your assessments had to match that of a panel of "joke experts" that DK just assembled by hand.
(2) This is a major problem elsewhere. Not just elsewhere in psychology; pretty much everywhere.
There's a standard test of something like "emotional competence" where the testee is shown pictures and asked to identify what emotion the person in the picture is feeling.
But, if you worry about the details of things like this, there is no correct answer. The person in each picture is a trained actor who has been instructed to portray a given emotion. Are they actually feeling that emotion? No.
Would someone else look similar if they were actually feeling that emotion? No. Actors do some standard things that cue you as to what you're supposed to imagine them feeling. People in reality don't. They express their emotions in all kinds of different ways. Any trial lawyer will be happy to talk your ear off about how a jury expects someone who's telling the truth to show a set of particular behaviors, and witnesses just won't do that whether they're telling the truth or not.
My current hypothesis du jour is that AI is going to be like programming in a certain way. Some people can learn to program productively, others can't. We don't know why. It's not related to how smart they are. The people who can program, can be employed as programmers if they want. Those who can't, are condemned to be users instead.
The same may end up being true of AI. Some will learn to make productive use of it, others won't. It will cause a rearrangement of the pecking order (wage ladder) of the workplace. I have a colleague who is now totally immersed in AI, and our upper management is delighted. I've been a much slower adopter, so I find other ways to be productive. It's all good.
> as the Dunning-Kruger Effect. (link to the wikipedia page of Dunning-Kruger Effect)
> A cognitive bias, where people with little expertise or ability assume they have superior expertise or ability. This overestimation occurs as a result of the fact that they don’t have enough knowledge to know they don’t have enough knowledge. (formatted as a quote)
Either that the author didn't read the page they linked themselves and made up their own definition, or they copied it from somewhere else. In either case the irony isn't lost on me. Doubly so if the "somewhere else" is an LLM, lol.
The wikipedia editor clearly believes the misunderstanding is so common that they have to put it in the first paragraph. But people (like the OP author) still just ignore it.
I have a quote for this:
> "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning." — Rick Cook
It's people who overestimate their expertise (at the low end) or underestimate it (at the high end). But the people at the high end still estimate their skill higher than those at the low end.
Instead, what it has turned into is this idea that "stupid people think they're smarter than smart people" when it says nothing of the sort.
That is a direct paraphrase of the abstract of Kruger & Dunning, 1999[1]:
"The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it."
Now, it may be possible that the definition has evolved since then, but as the term Dunning-Kruger effect is named after this paper, I think it's safe to say that Wikipedia is at least partially wrong in this case.
You're misinterpreting the quote. Unskilled people overestimate how skilled they are, but they still understand that they are unskilled. They just don't know quite how unskilled. What Kruger & Dunning actually showed is that people tend to skew their estimates of their skill towards being slightly above average.
Above average is not unskilled. That does not compute for me. The point is clearly about how lack of skill/experience in a task make people bad at judging how they do at the task. That is clearly what the article Kruger and Dunning) and the article (OP) are both saying.
You could help your argument by explaining how I'm misinterpreting the quote.
Then using a tv remote control is Dunning-Kruger. Look, I push a button, then the tv turns on. How? I barely have an idea. I push this button, and then this button, and we are enjoying an episode of Modern Family. How? I barely have any idea.
Then programming in Python is Dunning-Kruger. Some simple text, and I somehow manipulate and manage all sorts of complex processes taken from the abstract all the way down to machine instruction, down to a trillion and one bare metal transistors. How? I barely have an idea. I am not a chip designer.
Then chip designers are just Dunning-Kruger flunkies. Sure they are experts doing amazing things, but what do they know of mining and smelting, logistics and accounting, and the zillion and one other things that people in our society do to empower each other in this collective enterprise.
No, they are an AI user that allows them to do Python programming. Likewise, a Python programmer uses Python to allow them to use C and machine instructions without being a C and machine instruction programmer. And so on.
Python is a formal language. You don’t use Python to not use C (a formal language), Assembly (another formal language) or Opcodes (still formal). You just use Python.
Likewise, you don’t use French to say “c’est mon chat” because you want to say “it’s my cat” in English. You just use one or another based on the current context. But programming languages are strictly equivalent to each other, while natural languages just overlaps in some way.
It's a translation like writing "2 + 2" instead of "two plus two" or "power of two". Programming language are more about notation than the ideas themselves.
Meh, I don't know. I think you can use AI to lorem ipsum a lot of things where it doesn't really matter:
- Making a brochure. You need a photo of a happy family. It doesn't matter if the kids have 7 fingers on each hand.
- You have some dashboard for a service, you don't quite know what the panels need to look like. You ask AI, now you have some inspiration.
- You're building a game, you need a bunch of character names. Boom. 300 names.
- Various utility scripts around whatever code you're writing, like the dashboard, might find use, might not.
None of those things is pretending you're an expert when you're not.
Give AI to a coding novice, it's no different from giving autopilot to a flying novice. Most people know they can't fly a plane, yet most people know that if they did, autopilot would be useful somehow.
> You're building a game, you need a bunch of character names. Boom. 300 names.
This is actually one of the things AI is notoriously bad at. If asking for a plain list, it very quickly falls into blatant patterns (one name per letter of the alphabet, all names starting with A, all names with exactly 2 syllables, etc.). And, whether part of a list or not, one of the most obvious signs of AI writing is that it always resorts to the same handful of names for a given archetype.
Traditional random name generators are much better.
I rather spend one hour to write a script than writing one with AI. If I’m spending that much time, that is a clear signal that it’s important to get right or I’m enjoying the learning process.
As for when there’s a rush, I just avoid putting myself in those situations. If there’s not enough time, I advocate for a simpler solution, even hackish.
That's supporting my view. You might want it, you might not. It's marginal, and now it's cheap.
> look at your competition
LLM does this for you
> this isn't how llm products are marketed
It certainly is. Something like ChatGPT is marketed as a low-risk chat partner, most certainly not pretending to give you medical or legal advice. Talk to it like your buddy, you get buddy responses. Your buddy who has read a few law books but doesn't pretend to be a lawyer.
> Various utility scripts around whatever code you're writing, like the dashboard, might find use, might not.
Let's hope you protect that dashboard well with infra around it, because it will be the front door for people to invade your site.
The same apply in slightly different ways to your deployment script, packaged software (or immutable infra) configuration, and whatever tools you keep around.
> Let's hope you protect that dashboard well with infra around it, because it will be the front door for people to invade your site.
I was thinking of internal dashboards, but like I said above, if it doesn't really matter, use LLM. If you are building a cockpit for an airplane, yeah, don't just use a fill tool. If you need security, yeah, don't leave the door open.
Dunning-Kruger is basically a middling party put down at this stage. Similarly this article is not making a whole lot of sense other than as a mild and wildly applied dis?
The Dunning-Kruger effect is where people with low intelligence express high confidence in their intelligence over others by constantly referencing the Dunning-Kruger effect
I still marvel at people who act and write as if D-K is proven. The debate about whether the effect exists, its scale if it does exist, where it might originate if it is real and where it might originate if it is a statistical artifact ... these all carry on. D-K is not settled psychology/science, even though the idea is utterly recognizable to all of us.
> though the idea is utterly recognizable to all of us.
Then why marvel? If we can't scientifically prove it, but it tracks logically and people find it to be repeatedly recognizable in real-life, it makes sense people speak about it as if it's real
Not really on topic, but it'fascinating how Dunning-Kruger effect continues to live its own life in the public culture despite being pretty much debunked in its most popular form a while ago.
What Dunning-Kruger experiments have actually shown is that people's assesment of their own performance is all over the place, and only gets slightly better for good performers.
The value of AI is in the imagination of its wielder. The Unknown Unknowns framework is a useful tool in how to navigate AI along with a healthy dose of critical thinking and understanding how reinforcement learning and RLHF work, post pre-training.
Feels like you could make a similar argument with any tool that is leaps and bounds better or makes your job 'easy'.
Dreamweaver was Dunning-Kruger as a program for HTML-non-experts.
Photoshop was Dunning-Kruger as a program for non-airbrushers/editors/touchup-artists.
(I don't actually believe this, no they weren't.)
Or, we could use the phrase Dunning-Kruger to refer to specific psych stuff rather than using it as a catch-all for any tool that instills unwarranted confidence.
The problem isn’t tools making someone better. An excavator will make me a superior ditch digger than if I just have a shovel. That’s progress.
The issue is making someone feel like they did a good job when they actually didn’t. LLMs that make 800 line PRs for simple changes aren’t making things better, no matter how many “done” emojis it adds to the output.
Unlike the expertise that Dunning-Kruger to refers to, the skill to create art and understand art are separate.
Possibly Dreamweaver might fit because it does give you the sense that making a website is easy but you might not understand what goes into a maintainable website.
You cannot make a similar argument for any tool that makes jobs easier, because the argument is dependent on the unique attribute of LLMs: providing wrong answers confidently.
There are lots of tools that give a wrong solution that appears correct, and easier ones tend to do that the most.
Plenty of people who needed a real dev team to design an application probably hoped on Dreamweaver, were suddenly able to bumble their way some interface that looked impressive but would never scale (even to the original goal level of scale mind you).
-
Any time you have a tool that lowers the barrier of entry to a field, you get a spectrum of people from those who have right-sized expectations and can suddenly do the thing themselves, to people who massively overestimate how easy the field is to master and get in over their heads.
This isn't even a programming thing, off the top of my head Sharkbites get this kind of rep in plumbling
You could argue the RPi did this to hardware, where people are using a Linux SBC to do the job a 555 timer could do and saying that's hardware.
Point-and-shoot, then smartphone cameras, did this and now a lot more people think they can be a photographer based on shots their phone spends more processing power per image than we used to get to the moon on.
using LLMs for creative purposes is terrifying. Because why? learning the craft is the whole reason you do it. however using LLMs to get work done, I just had Claude rewrite some k8s kuttl tests into chainsaw, basically a complete drudgery, and it nails it on the first try while I can stay mentally in EOD Friday mode. Not any different from having a machine wash the dishes. because it is, in fact, nuclear powered autocomplete. autocomplete is handy!
Bypassing practicing a practical skill stunts your growth the same way as bypassing creativity. For some tasks that may be fine, but I'd never be comfortable taking these shortcuts with career skills. Not if my retirement was more than a few years away.
I actually agree with you on that as well, watching claude take over so many tedious "oh this thing broke, I have to google how that works, re-learn its config format, fix the thing, then forget about it again" kinds of tasks does at the same time reveal a lot of thought processes that i no longer have to engage in.
then again I dont see how it's too different from just a simple move into middle management where you manage programmers but dont do as much of the coding yourself. with Claude I'm finding myself taking on much bigger tasks and orchestrating a lot more technologies to do things, so I am learning about things i would not have if i didnt have the LLM to assist me through it.
The Dunning–Kruger meme has basically turned into a way for people to laugh at others for being “too dumb to know they’re dumb.” But that misses the point completely. The whole “I know that I know nothing” idea applies to everyone. So when people use the meme to mock AI or AI users, it just shows they haven’t learned that same lesson themselves.
Part of the problem with labor that we haven't yet discussed or maybe want to avoid due to the dissonance of the association with qualities as slaves is, we have a leadership class who acts more like elite slave masters than human beings with inherent dignity and decency. We have the class write the rules that they hold themselves (un)accountable for since the system was designed for them and enforced by them.
These are the people driving the rush and having a lot of say in the current AI and overall capitalist market behavior and sentiment. I think they're really mad and salty that when COVID happened the engineers got more remote and free and expressed the resentment more freely. This comment is probably putting me on a list somewhere or activating some hate program against me.
People are always create new layers on top of others. Machines that make machines or code that compiles to code. Layers of abstractions makes it possible for our simple brains to control trillions of electrons in a silicon chip. Every transition to a new layer has haters and lovers. Most people hate change. But eventually everything is using the new change. Never ever things go backwards in human history. AI is not Dunning Kruger
I would like to see AI usage regulated in the same way that vehicles are: license required.
Be that an aptitude test or anything else... unfettered usage of vehicles is dangerous in the same way that unfettered access to AI is as well.
As a society, we have multiple different levels of certification and protection for our own well-being in the public's when certain technologies may be used to cause harm.
Why is knowledge or AI any different? This is not in opposition at all to access information or individual liberties. No rights are violated by their being a minimum age in which you can operate a vehicle.
There is nothing totalitarian about constraining societal harm.
The issue comes down to whether it is collectively understood to be a benefit to the human race. Until now we have never had to constrain information to protect ourselves.
Please read the Vulnerable World Hypothesis by Nick Bostrom
> There is nothing totalitarian about constraining societal harm.
Of course not. The problem is that the only way to enforce AI regulations is through totalitarian means.
You can easily regulate OpenAI and Gemini and whatnot, but then people will just use local AI models. The barrier to entry for using local AI models is basically zero because software like Ollama make it trivially easy to set up, and small LLMs can run on an iphone or a laptop.
The only way to actually prevent "unauthorized" AI usage is to control digital communications to prevent LLM weights from being distributed and to ensure that no "black market" AIs are being trained or used.
And if you're already scanning digital communications for LLM weights, why not also scan it for other forms of wrongthink?. Hey presto, now you have totalitarianism.
I don't think that LLMs fall into the category of an infohazard in the way that Bostrom defines it. It presents a risk, but not one severe enough to justify universal surveillance. Bostrom is talking about AI that can make bioweapons, not ones that gives false confidence to incompetent people.
It's not veiled and it shouldn't be. Go browse linkedin, reddit, indiehacker, etc. Literal morons are using AI and pretending to be super geniuses. It's replaced the reddit-tier "google-expert" with something far more capable of convincing you that you're right.
Outside of a very small bubble of experts using AI and checking it's work (rubber ducking) most people are, in fact, using it to masquerade as experts whether they know it or not. This is extremely dangerous and the flamebait is well deserved, imo.
>They give utter nonsense answers with high confidence and wrap errors in sycophantic language making me feel good for pointing out that they wasted my time
I would implore the author to share examples. Every platform allows linking to chats. Everyone talks about this all the time, incessantly. Please, can someone please share actual chat links containing these episodes of utter nonsense, outside of what can be attributed to the knowledge cut-off (i.e. "Mamdani is not the mayor-elect of NYC").
I get it if you are using a 20B model or AI overviews, but anyone trying to actually get anything meaningful done should be using a SOTA model. I'm genuinely not interested if you are going to reply with a description or story. I really, really just want links to chats.
Edit: You can downvote me, but please make me look like an idiot by posting chat links. That is the real downvote here.
This article feels lazy. Is the main argument in similar vain as "don't read the books that experts have written, and go figure stuff out on your own"?
I'm trying to understand what is wrong with using a new data compression tool (LLMs) that we have built to understand the world around us. Even books are not always correct and we've figured out ways to live with that/correct that. It doesn't mean we should "Stop wasting time learning the craft"..
LLMs are optimized for sycophancy and “preference”. They are the ultra-processed foods of information sharing. There’s a big difference between having to synthesize what’s written in a book and having some soft LLM output slide down your gullet and into your bloodstream without you needing to even reflect on it. It’s the delivery that’s the issue, and it definitely makes people think they are smarter and more capable than they are in areas they don’t know well. “What an insightful question…”
Wikipedia was already bad, low brow people would google and read out articles uncritically but there was still some brain work involved. AI is that meets personalization.
No, I do not quite think that is what they wrote here. But what's the thought process here? It's hard for me even to understand if the first scare quote is supposed to be from someone being critical or someone responding to the critique. It seems like it could apply to both?
I am not the author, but quite curious to know what prevented comprehension here? Or I guess what made it feel lazy? I'm not saying its gonna win a Pulitzer but it is at minimum fine prose to me.
Or is the laziness here more concerning the intellectual argument at play? I offer that, but it seems you are asking us what the argument is, so I know it doesn't make sense.
I have been a fool in the past so I always like to read the thing I want to offer an opinion on, even if I got to hold my nose about it. It helps a lot in refining critique and clarifying one's own ideas even if they disagree with the material. But also YMMV!
This is an arrogant and unwarranted assumption. What's preventing your comprehension of this discussion?
The article sets up a straw man - the person who can convincingly fake being an expert without actually being one - and then demolishes it.
This doesn't resemble anything that I've experienced from LLM use in the real world. In my experience, amateur use of LLM is easily detected and exposed, and expert use is useful as a force multiplier.
I suppose the "Dunning-Kruger" accusation might apply to the first one, but I'm not convinced - the people doing that are usually very aware that they're faking their attempt at projecting expertise, and this comes across in all sorts of ways.
gp asked us what the blog is arguing, doesn't seem too unwarranted to assume they didn't comprehend? Or am I missing something?
Also, just fwiw, I really tried but I am truly having trouble comprehending what you are saying, or at least how it bears on the article? It is 8-9 short paragraphs long, can you like point to wear he demolishes the straw man? Or like what does that even mean to you? Isn't it a good thing to demolish a straw man? Given that it is fallacy?
Trying to be charitable here parsing this: I don't think Dunning-Kruger really speaks to one's ability to convince right? Doesn't it really manifest when we don't actually need to be convincing to anyone? This is the definitional thing about it really: you are precisely not aware you are "faking" it, you think you are doing really great!
Your comment feels lazy as well. It waves off the article without engaging with its core argument. The piece isn’t saying “ignore experts”. It’s questioning how we use tools like LLMs to think, not whether we should. There’s a difference between rejecting expertise and examining how new systems of knowledge mediate understanding.
while I think there is a lot to this criticism of AI (and many others as well) I was also able to create a TUI-based JVM visualizer with a step debugger in an evening for my compilers class:
https://x.com/htmx_org/status/1986847755432796185
this is something that I could build given a few months, but would involve a lot of knowledge that I'm not particularly interested in taking up space in my increasingly old brain (especially TUI development)
I gave the clanker very specific, expert directions and it turned out a tool that I think it will make the class better for my students.
all to say: not all bad
AI is bad at figuring out what to do, but fantastic at actually doing it.
I’ve totally transformed how I write code from writing it to myself to writing detailed instructions and having the AI do it.
It’s so much faster and less cognitively demanding. It frees me up to focus on the business logic or the next change I want to make. Or to go grab a coffee.
> AI is bad at figuring out what to do, but fantastic at actually doing it.
I've found AI is pretty good at figuring out what to do, but hit or miss at actually doing it.
I would say from my experience there's a high variability in AI's ability to actually write code unless you're just writing a lot of scripts and basic UI components.
The AI version of that Kent Beck mantra is probably "Make the change tedious but trivial (warning: this may be hard). Then make the AI do the tedious and trivial change."
AI's advantage is that it has infinite stamina, so if your can make your hard problem a marathon of easy problems it becomes doable.
I would say this does not work in any nontrivial way from what I've seen.
Even basic scripts and UI components are fucked up all the time.
You have to learn how and where to use it. If you give it bad instructions and inadequate context, it will do a bad job.
This is the ‘you’re holding it wrong’ of LLMs.
What tool can’t you hold wrong?
Literally every tool worth using in software engineering from the IDE to the debugger to the profiler takes practice and skill to use correctly.
Don’t confuse AI with AGI. Treat it like the tool it is.
You might want to ask ChatGPT what that is referencing. Specifically, Steve Jobs telling everyone it was their fault that Apple put the antenna right where people hold their phones and it was their fault they had bad reception.
The issue is really that LLMs are impossible to deterministically control, and no one has any real advice on how to deterministically get what you want from them.
I recognized the reference. I just don’t think it applies here.
The iPhone antenna issue was a design flaw. It’s not reasonable to tell people to hold a phone in a certain way. Most phones are built without a similar flaw.
LLMs are of course nondeterministic. That doesn’t mean they can’t be useful tools. And there isn’t a clear solution similar to how there was a clear solution to the iPhone problem.
"Antennagate" is the gift that keeps on giving. Great litmus test for pundits and online peanut galleries.
You are technically correct: it was a design flaw.
But folks usually incorrectly trot it out as an example of a manufacturer who arrogantly blamed users for a major product flaw.
The reality is that that essentially nobody experienced this issue in real life. The problem disappeared as long as you used a cell phone case. Which is how 99.99% of people use their phones. To experience the issue in real life you had to use the phone "naked", hold it a certain way, and have slightly spotty reception to begin with.
So when people incorrectly trot this one out I can just ignore the rest of what they're saying...
Humans are not deterministic.
Ironically AI models are iirc.
Come up with a real argument.
YOU come up with a real argument first
"no, you're wrong" is not a valid rebuttal lolol
I pointed out the claim is both irrelevant and factually wrong.
That should be sufficient rebuttal.
"All the time"?
This always feels like you're just holding it wrong and blaming the tool.
It can still fuck it up. And you need to actually read the code. But still a time saver for certain trivial tasks. Like if I'm going to scrape a web page as a cron job I can pretty much just tell it here's the URL, here's the XPath for the elements I want, and it'll take it from there. Read over the few dozen lines of code, run it and we're done in a few minutes.
I think it is like protein folding.
It will make a mess but if you drop a console.log into the browser debug console to show the AI what it should be looking for after it spent 3 hours failing to help understand and debug the problem, it will do 1 week of work in 2 hours.
I've noticed this too. The latest cursor version has a @browser command which will launch a browser with playwright, and call tools to inspect the html and inject JavaScript to debug in real-time.
When it has access to the right tools, it does a decent job, especially for fixing CSS issues.
But when it can't see the artifacts it's debugging, it starts guessing, confident that it knows the root cause.
A recent example: I was building up a html element out of the DOM and exporting to PNG using html2canvas. The element was being rendered correctly in the DOM, but the exported image was incorrect, and it spent 2 hours spinning it's wheels and repeating the same fixes over and over.
> AI is bad at figuring out what to do, but fantastic at actually doing it.
AI is so smart, one day might even figure out how to subtract... https://news.ycombinator.com/item?id=45821635
When you need to take the square root of 37282613, do you do it in your head or pull out the calculator?
Why does the AI have to be good at math when it can just use a calculator? AI tool usage is getting better all the time.
I think people generally think AI should be good at math because it runs on a very complex and very fast calculator to begin with.
Yore brain runs on physics and biology, yet here we are…
That is not the point...Its about not understanding subtraction...
We're all architects now...
[flagged]
Couldn't help your self, could you.
Lay off him, he did the right thing
I had no idea you (ceo of hmtx) were a professor. Do your students know that you live a double life writing banger tweets?
Couldn't agree more. Being a skilled operator with these tools helps you be very effective at creating new things in far less time than it would've taken you before. This is especially true if you know the architecture, but don't have the cycles to implement it yourself.
What did you use to do this? Something I'd like to do, while also avoiding the tedium, is to write a working x86-64 disassembler.
claude
Is that worth the negative externalities though? Genuinely asking. I’ve asked myself over and over and always came to the same conclusion.
What negative externalities? Those prompts probably resulted in a tiny amount of CO2 emissions and a tiny amount of water usage. Evaporating a gram of water and emitting a milligram of CO2 seems like a good deal for making your class better for all your students.
>emitting a milligram of CO2
In the US, on average, generating 1 kWh produces 364.5 g of CO2. 1 kW may be somewhat pessimistic, but I think it's in the right ballpark for power consumption of DC inference. If processing the prompt took a minute of continuous inference (and I'm going to guess it took a fair bit more), that's 6 grams of CO2.
>What negative externalities?
Off the top of my head,
* All the ways AIs can be misused, either by people who don't understand them (by asking them for advice, etc.) or by people who want to take advantage of others (spam, scams, etc.).
* The power and resource usage of the above, both for inference as well as to deal with them.
> If processing the prompt took a minute of continuous inference (and I'm going to guess it took a fair bit more), that's 6 grams of CO2.
Neither of us really knows, but I'm going to guess it took a fair bit less. You need to consider the marginal cost of one more prompt, not the cost of running the datacenter in the first place. And with batching, hundreds of requests are executed in parallel on the same hardware. I'll concede that a milligram was an exaggeration though.
For comparison, a typical car in the US emits about 400 grams of CO2 per mile. And I know few people who beat themselves up over the externalities of unnecessarily driving one mile.
> * All the ways AIs can be misused, either by people who don't understand them (by asking them for advice, etc.) or by people who want to take advantage of others (spam, scams, etc.).
This is not an externality because it is not a consequence of using AI for good purposes. It's like saying an externality of using a hammer to build a shed is all the ways hammers can be misused.
Still, I respect you for considering the externalities of your actions. That's farther than most go. But I think they are outweighed by the benefits in this case
>This is not an externality because it is not a consequence of using AI for good purposes. It's like saying an externality of using a hammer to build a shed is all the ways hammers can be misused.
I guess we're talking about different questions. You're talking about the externalities of using a piece of technology, while I'm talking about the externalities of a piece of technology existing.
I would argue even just using the technology for good is offsetting the costs to keep the whole thong going. Essentially subsidising the bad users.
Being more specific in what you think the negative externalities are would be a good start - I see a lot of noise and upset over AI that I think is more or less overblown, nearly as much as the hype train on the other end. I'm seeing the potential for civilizational level payoffs in 5 years or less that absolutely dwarf any of the arguments and complaints I've seen so far.
> civilizational level payoffs
but first the investors must recoup their trillions, right?
Yeah, economics is weird like that, and we should probably push as hard as we can to make sure things are as fair as reasonably possible. I've got no problem with trillionaires or megacorps if the system in which they operate is fair. One thing AI is enabling is systematic uncovering of fraud and waste - we'll be able to scan omnibus bills and find corruption, track politicians, interact with the system and communicate at higher levels than before, and there's a shot at AI empowering and enfranchising people more than they've ever had a chance at before.
If AI coding alone reaches its potential, then a whole class of corporations lose their leverage in the market. In 10ish years, following moore's law, phones will have enough compute and storage to trivially run your own local search and curation of the internet. Sophisticated search engines that outclass anything currently on the market, working in tandem with a personal local privacy respecting agent. That decentralizes things in such a significant way that Google and Apple and all the gatekeepers lose their ability to manipulate and distort.
20 years from now, even if we're somehow just stuck at the current levels of AI, your personal compute will be totally sufficient to provide all the curation, moderation, proactive interaction, and search capabilities without ever having to deal with ads or bots or cloudflare.
There are lots of dangers, and ASI could wreck us, and militarized AI could be a doom scenario, or there could be overbroad stupid regulation that puts control of AI in the hands of a few corps or a government, but there are lots of bright future possibilities, and I don't see any indications that we're directly headed toward any particular doom.
hard to know
I've always seen AI as Brandolini's Law as a Service. I'm spending an unreasonable amount of time debunking false claims and crap research from colleagues who aren't experts in my field but suddenly feel like they need to give all those good ideas and solutions, that ChatGPT and friends gave them, to management. Then I suddenly have 2-4 people that demand to know why X, Y and Z are bad ideas and won't make our team more efficient or our security better.
It's very much like that article from Daniel Stenberg (curl developer): The I in LLM Stands for Intelligence: https://daniel.haxx.se/blog/2024/01/02/the-i-in-llm-stands-f...
> I'm spending an unreasonable amount of time debunking false claims and crap research from colleagues who aren't experts in my field
Same. It's become quite common now to have someone post "I asked ChatGPT and it said this" along with a completely nonsense solution. Like, not even something that's partially correct. Half of the time it's just a flat out lie.
Some of them will even try to implement their nonsense solution, and then I get a ticket to fix the problem they created.
I'm sure that person then goes on to tell their friends how ChatGPT gives them superpowers and has made them an expert over night.
On the other hand, here's another post by Stenberg where he announced that he has landed 22 bugfixes for issues found by AI wielded by competent hands.
https://mastodon.social/@bagder/115241241075258997
Sure, in competent hands. Problem is that most people don't seem to realize that in order to use AI for something, you have to be pretty good at that thing already.
I wonder if the next generations of LLMs, trained on all these hate articles (which I support), will develop some kind of self-esteem issue?
We don't have any particular reason to believe they have an inner world in which to loathe themselves. But, they might produce text that has negative sentiments toward themselves.
I was half-joking and half-serious, and the serious half refers to the context that makes them predict and generate the next tokens.
They may also produce text describing an internal representation of a negative sentiment towards themselves
https://www.anthropic.com/research/introspection
They already happily will. Gemini told me
> Large Language Models represent a fundamentally degenerative technology because they systemically devalue the very processes that underpin human progress: original thought, rigorous inquiry, and shared trust. On an individual level, they encourage cognitive offloading, substituting the difficult work of critical thinking and creative synthesis with effortless, probabilistic text generation. This fosters an atrophy of intellectual skills, making society more dependent on automated systems and less capable of genuinely emancipated thought. This intellectual dependency, in turn, threatens long-term technological advancement by trapping us in a recursive loop of recycling and rephrasing existing knowledge, rather than fostering the groundbreaking, first-principles discoveries that drive true progress. Ultimately, this technology is dangerous for society because it erodes the foundation of a shared reality by enabling the mass production of sophisticated misinformation, corroding social trust, and concentrating immense power over information into the hands of a few unaccountable entities.
Only if your context starts with "you are an intelligent agent whose self worth depends on the articles written about you..."
An “intelligent agent” would realize the contradiction in terms.
I mean, given:
It's already inventing safety features it should have launched with.[1] https://www.cnn.com/2025/11/06/us/openai-chatgpt-suicide-law...
Marvin the Paranoid Android
RIP both Douglas Adams and Alan Rickman.
and (and as much as I do love Alan Rickman) more properly Stephen Moore.
I thought Alan Rickman voiced Marvin
Stephen Moore was the original Marvin, which he voiced for both the British radio and TV versions of Hitchhikers.
Ah.
I only saw/heard parts of these.
This makes me laugh. “GenAI makes you a genius without any effort”, and “Stop wasting time learning the craft” are oxymorons in my head. Having AI in my life has been like having an on demand tutor in any field. I have learned so much
And you’ve totally missed the point of this essay apparently. Congrats!
AI will also further cement the status quo and existing power structures. Incompetent leaders in many arenas would be even less beholden to actual experts and expertise. AI will provide answers that are "good enough" to keep incompetent leaders in power.
Leaders can often be incompetent since they are often not promoted based on competency or they hang on to power long enough to either have their competence become stale or just lose it do to the passage of time.
Ultimately, AI will decide for us since it will be the crutch of the incompetent leaders. Since these leaders often decide "truth" and "reality" for most people anyways, AI will decide these truths with leaders as a proxy, and power will continue to flow in directions completely unrelated to competence.
> Politics have become an attack on intelligence, decency and research in favour of fairy tales of going back to “great values” of “the past when things were better”.
This is a major blind spot for people with a progressive bent.
The possibility that anything could ever get worse is incomprehensible to them. Newer, by definition, is better.
Yet this very article is a critique of a new technology that, at the very least, is being used by many people in a way that makes the world a bit worse.
This is not to excuse politicians who proclaim they will make life great by retreating to some utopian past, in defense of cruel or foolish or ineffective policies. It's a call to examine ideas on their own merits, without reference to whether they appeal to the group with the "right" or "wrong" ideology.
Funny, isn't it, that it is never a return to high unionisation of workers and strong social safety nets - it's always a return to when "those people" knew their place.
No, plenty of people want those things to return.
I'm confused. You're commenting on an article where a progressive writer critiques AI as "Dunning-Kruger as a service" and attacks techno-optimism, while claiming progressives can't critique new technology. The author's entire piece demonstrates progressive critique of both AI adoption and "great values of the past" nostalgia - the exact opposite of what you're describing.
Yes, it’s striking how the writer fails to acknowledge the contradiction.
What? Progressives are the ones worried the world is going to end from climate change.
Not end, but some quite shitty stuff could happen
You aren’t making any sense. Who are these progressives who… do what exactly? I honestly can’t tell what you are even accusing them of.
Denying that something new could ever be worse than something old.
That change is good by definition.
Plenty of people being promoted because these fake superhumans can generate so much smoke with AI that managers think there is an actual fire…
I view LLMs as a trade of competence plus quality against time. Sure, I’d love to err on the side of pure craft and keep honing my skill every chance I get. But can I afford to do so? Increasingly, the answer is “no”: I have precious little time to perform each task at work, and there’s almost no time left for side projects at home. I’ll use every trick in the book to keep making progress. The alternative - pure as it would be - would sacrifice the perfectly good at the altar of perfection.
I've always said, the coaches I work with fall into three categories...
1. They know so little that they don't know what they don't know. As a result they are way too overconfident and struggle as coaches.
2. They know enough to know what they don't know so they work their asses off to know more and how to convey it to their team and excel as coaches.
3. They know so much and the sport comes so easy to them that they cannot understand how to teach it to their team and struggle as coaches.
Now I have a name for #1 group!
> They know so much and the sport comes so easy to them that they cannot understand how to teach it to their team and struggle as coaches.
http://www.harkavagrant.com/index.php?id=206
Your best bet is to be better than everyone else. That works for me, so that's my advice.
Time to remind folks of this wonderful video: https://vimeo.com/85040589
Edit: Turns out that while I stand by my point about the underlying principle behind the DK effect (in my nitpick) the actual effect coined by the authors was focused on the low competence high confidence portion of the competence vs confidence relationship. (Accurately reflected in OP article)
Here is the original DK article: https://pubmed.ncbi.nlm.nih.gov/10626367/
Turns out I thought that the author was DKing about DK, but actually I was DKing about them DKing about DK.
Original Comment:
I have high-confidence in a nitpick, and low-confidence in a reason to think this thesis is way off.
The Nitpick:
Dunning-Kruger effect is more about how confidence and competence evolve over time. It's how when we learn an overview about our new topic our confidence (in understanding) greatly exceeds our competence, then we learn how much we don't know and our confidence crashes below our actual competence, and then eventually, when we reach mastery, they become balanced. The dunning-Kruger effect is this entire process, not only the first part, which is colloquially called "Peak Mt Stupid" after the shape of the confidence vs competence graph over time.
The Big Doubt:
I can't help but wonder if fools asking AI questions and getting incorrect answers and thinking they are correct is some other thing all together. At best maybe tangentially related to DK.
> when we learn an overview about our new topic our confidence (in understanding) greatly exceeds our competence, then we learn how much we don't know and our confidence crashes below our actual competence, and then eventually, when we reach mastery, they become balanced.
As a description of what Dunning and Kruger's actual research showed on the relationship between confidence and competence (which, as I've pointed out in another post in this thread, was not based on studying people over time, but on studying people with differing levels of competence at the same time), this is wrong for two out of the three skill levels. What D-K found was that people with low competence overestimate their skill, people with high competence underestimate their skill, and people with middle competence estimate their skill more or less accurately.
As a description of what actually learning a new subject is like, I also don't think you're correct--certainly what you describe does not at all match my experience, either when personally learning new subjects or when watching others do so. My experience regarding actually learning a new subject is that people with low competence (just starting out) generally don't think they have much skill (because they know they're just starting out), while people with middling competence might overestimate their skill (because they think they've learned enough, but they actually haven't).
Out of curiosity, if you google "Dunning Kruger Effect" and then look at the images, and see all those charts of confidence vs competence, do you disagree with them?
And as I responded to your other comment, just because the study took measurements of many people at one point in time didn't mean they weren't studying an over-time phenomenon. No one starts out as an expert.
> if you google "Dunning Kruger Effect" and then look at the images
Then I would be doing a very poor job of actually researching the topic.
If you are aware of any actual research papers that describe actually studying how the same people's competence vs. confidence evolves over time, by all means point me to them.
> just because the study took measurements of many people at one point in time didn't mean they weren't studying an over-time phenomenon.
Um, yes, it does. It's possible that the things the study saw were effects of an over-time phenomenon, but the study did not study that, and its results are not evidence either for or against any such hypothesis.
> Then I would be doing a very poor job of actually researching the topic.
I made the request to try and tease apart where we disagree exactly. Your interpretation and response to my request/offer comes off as unnecessarily disrespectful. (unkind, elitist and tramples curiosity)
It seems increasingly likely that I was mistaken in thinking we could explore this subject together. It seems you are intent on attacking my position and discrediting my points and not interested in building a shared understanding.
I find this disappointing.
> I made the request to try and tease apart where we disagree exactly.
I thought that was obvious: you're making a claim about how individual people's confidence vs. competence evolves over time, that as far as I know is not supported by any research--certainly it's not supported by Dunning and Kruger's research. That's why I asked you if you know of any research that does support it. "Images found in a Google search" does not qualify.
> It seems increasingly likely that I was mistaken in thinking we could explore this subject together.
That's the way I see it too, though evidently not for the same reason you do.
> It seems you are intent on attacking my position and discrediting my points
I'm interested in finding out what basis you have, if any, for the claim you made that I described above. "It's self-evident" is not what I was looking for. Nor is "do a Google search and look at the images".
> and not interested in building a shared understanding.
If you can point me at research I wasn't previously aware of that supports your claims, that would be helpful. If you want to try to get to "a shared understanding" based on claims about what you think is self-evident or by looking at images from a Google search, then no, I don't think we're going to get anywhere.
Yeah, I screwed up here.
I don't have a background in psychology or sociology (academically) but they are subjects that interest me.
I was introduced to DK by pop psychology, especially in business management contexts.
The reason I wanted you to comment on the viral graphs was because I was trying to triangulate our disagreement.
Turns out that there is a ton of misinformation about DK. Kind of ironic given the topic of this thread, but especially because it long predates LLMs. If anything it hints why LLMs are confidently incorrect, since the internet is full of it.
The reason why what I was saying is not self evident is because I was making a logic error, specifically ecological fallacy.
Also sort of ironic, I managed to get to the root of this using an LLM.
Part of me suspects psychology is your domain, and you may be an academic or at least have some graduate experience. The way you engaged with me is maybe the way you've been conditioned to act in these situations.
There's good reasons for why you responded the way you did. Part of me wants you to know that I would have really appreciated it if you had said something like "while I dont take issue with the claim that competence increases with time, it doesn't mean the DK relationship is time based, as there could other conflating factors." Also "I looked at those graphs and they don't match what the DK paper claims, that looks like pop psychogy and I can't confirm it has a scientific basis".
Or course you have no obligation or responsibility to help me improve my understanding, but the fact you responded to a bunch of my comments shows you care. So if you care about changing minds, especially the mind of someone who is curious and ignorant, than engaging this way would have made my interaction with you more pleasant.
And it's in the spirit of the community guidelines.
> Part of me suspects psychology is your domain, and you may be an academic
Neither of these are true.
> or at least have some graduate experience.
I do, but not in psychology. My degrees are in engineering.
> I would have really appreciated it if you had said something like "while I dont take issue with the claim that competence increases with time
But I do take issue with that claim, at least if it's made in the broad way you are making it here. I responded to that in another subthread with you.
> "I looked at those graphs and they don't match what the DK paper claims
The DK paper was already linked to by someone (I don't think it was you) elsewhere in this thread, and in any case it's easy to find online. I was taking for granted that you would be able to look for yourself to see what is in the paper, graphs or otherwise.
> Dunning-Kruger effect is more about how confidence and competence evolve over time.
I don't think there is anything about this in the actual research underlying the Dunning-Kruger effect. They didn't study people over time as they learned about new subjects. They studied people at one time, different people with differing levels of competence at that time.
It's the relationship of confidence and competence.
Competence is gained over time.
The "over time" is almost self-evident since no one is born a master of a skill or subject. And if it's not self-evident enough for you, there is lots of research into what it takes to develop competency and mastery in any subject.
So while the original paper was a snapshot taken at one point in time, it was a snapshot of many people at different stages of a learning journey...
And journeys take place over time.
I responded to this in another post in response to you upthread. The short version is that I don't agree with you about what is and is not "self-evident" here, nor will I accept your unsupported word that whatever research does exist supports your claims.
Interesting.
So just to be clear...
You are claiming that it is not self-evident that people learn things over time?
That it's not self-evident that experts start out as beginners?
Please respond in the other subthread where I made my points in more detail. As you will see from that more detailed post, I never made either of the claims you are asking about here.
You could chart the same curve by measuring the confidence of people at different competence levels.
"Could" based on what research?
Gell-Mann amnesia
Bingo
Yes and no. Dunning-Kruger also explains this evolution of skill estimation, but the original paper frames the effect specifically as an overestimation of skill in the lowest-performing quantile. This is clearly even cited in the article.
Okay I will agree with the "Yes and No". I initially clicked the source in the article, which is a broken link to wikipedia and rolled my eyes at it.
After reading your comment I navigated to it directly and found the first two sentences:
The Dunning–Kruger effect is a cognitive bias that describes the systematic tendency of people with low ability in a specific area to give overly positive assessments of this ability. The term may also describe the tendency of high performers to underestimate their skills.
Unsatisfied that this was the authority, I dug up the original paper here:
https://pubmed.ncbi.nlm.nih.gov/10626367/
And sure enough, the emphasis in the abstract is exactly as you say.
So while I stand by that the principle behind the "effect" is how confidence and competence evolve over time as someone discovers and masters a domain, I will concede that the original authors, and most people, assign the name for it to primarily the first phase.
Here I was thinking the OP Author was DKing about DK, but in reality I was DKing about them DKing about DK.
> the real "effect" is how confidence and competence evolve over time
What research is this based on?
Yeah that was poorly worded. I edited it even before I read your comment.
However the over-time aspect is kind of self-evident. No one is born a master of any skill or subject.
So while the original research was based on a snapshot of many people along a developmental journey, just because the data collection wasn't done over time doesn't mean that the principle behind the effect isn't time/effort based.
The graph is really competence vs confidence, but competence can only increase over time. If this isn't self-evident enough, there is lots of research on how mastery is gained. I don't have time to throw a bunch of it at you, but I suspect you won't need me to in order to see my point.
> the over-time aspect is kind of self-evident
It's self-evident that, in order to become a master at anything, you have to pass through increasing levels of competence to get there.
It is not at all self-evident, at least not to me, that while a particular person is going through that process, their estimate of their skill vs. their actual competence will be similar to what Dunning and Kruger found when they compared different people's confidence vs. competence at a single time.
> just because the data collection wasn't done over time doesn't mean that the principle behind the effect isn't time/effort based.
It could be, but a study that only collects data at one point in time can't possibly provide evidence either for or against any such hypothesis.
> competence can only increase over time
This is simply false. Skills and knowledge that aren't used can deteriorate. People can forget things they used to know.
In a much narrower sense, we would hope that if a person is studying a subject with the intent of mastering it, that their competence will only increase over time. But even that's not always true.
> there is lots of research on how mastery is gained. I don't have time to throw a bunch of it at you, but I suspect you won't need me to in order to see my point.
Sorry, but since, as you can see from the above, I don't agree with several of your key points, you're going to have to back them up with research if you want me to consider them further. If you don't have time for that (which is fine--we're all posting here in our spare time), then we'll just have to disagree.
What a great title.
Since Dunning-Kruger is the relationship between confidence and competence as people learn about new subjects (from discovery to mastery), then if AI is "Dunning-Kruger as a Service" its basically "Education as a service".
However, people accepting incorrect answers because they don't know better is actually something else. Dunning-Kruger doesn't really have anything to do with people being fed and believing falsehoods.
Edit: I had the word "Foolish" in there which was mainly in reference to the OP article about the robbers who didn't hide from cameras because they thought they were invisible. It wasn't meant at a slight against anyone who believed something ChatGPT said that was wrong.
You can't simultaneously expect people to learn from AI when it's right, and magically recognize when it's wrong.
But you can expect to learn in both cases. Just like you often learn from your own failures. Learning doesn’t require that you’re given the right answer, just that it’s possible for you to obtain the right answer
Hopefully you're mixing chemicals, diagnosing a personal health issue or resolving a legal dispute when you do that learning!
We’ve been down this road before. Wikipedia was going to be the knowledge apocalypse. How were you going to be able to trust what you read when anyone can edit it if you don’t already know the truth.
And we learned the limits. Broadly verifiable, non-controversial items are reasonably reliable (or at least no worse than classic encyclopedias). And highly technical or controversial items may have some useful information but you should definitely follow up with the source material. And you probably shouldn’t substitute Wikipedia for seeing a doctor either.
We’ll learn the same boundaries with AI. It will be fine to use for learning in some contexts and awful for learning in others. Maybe we should spend some energy on teaching people how to identify those contexts instead of trying to put the genie back in the bottle.
If you can't discern the difference between a LAMP stack returning UGC and an RNG-seeded matmut across the same UGC fined-tuned by sycophants then I think we're just going to end up disagreeing.
> You can't simultaneously expect people to learn from AI when it's right, and magically recognize when it's wrong.
You are misconstruing the point I was making.
My point is that DK is about developing competence and the relationship between competence and confidence (which I am also claiming evolves over time). My whole point is that the DK effect is not as relevant to LLMs giving wrong answers and people believing them as the author is claiming.
As someone else pointed out in another comment, the effect of people believing falsehoods from LLMs has more to do with Gell-Mann amnesia.
Tangentially, it actually is possible to learn from AI when it's right and recognize when its wrong, but its not magic, its just being patient, checking sources and thinking critically. It's how all of humanity has learned pretty much everything, because most people have been wrong about most things for most of time and yet we still learn from each other.
I’m not sure how much I agree with calling them “foolish people.” A big part of the problem is LLMs act incredibly confident with their answers and if you’re asking about something you don’t know a ton about, it’s very difficult sometimes to spot what is incorrect. That’s not being a fool, that’s just not being able to audit everything an LLM might spit back at you. It also doesn’t help that none of these companies are honest about the quality of the results.
We could hand wave this away with “well don’t ask things you don’t already know about,” but these things are basically being pitched as a wholesale replacement for search engines and beyond. I look up things I don’t know about all the time. That’s kind of what we all use search for most days lol.
It’s a little too caveat emptor-adjacent (I hope that makes sense?) for my taste
Well that was in reference mostly to the OP Article of the robbers who thought they were invisible and didn't hide from the Cameras.
I can see how what I wrote could be interpreted the way you did though. It's not how I meant it.
My bad then! Thanks for clarifying
It's foolish to take any LLM answer as a true fact.
Those people may not be dumb, but there's no doubt they are being fools.
I’d say there’s a difference between “being a fool” and “being fooled.” Just because I fool you doesn’t mean you’re a fool. I don’t know why you’re so eager to put down people like this rather than at least somewhat acknowledge that these companies and tools bear some of the responsibility.
I don’t think it’s fair to expect every person who uses an LLM to be able to sniff out everything it gets wrong.
> Dunning-Kruger is the relationship between confidence and competence as people learn about new subjects
Um, no, it isn't. From the article:
"A cognitive bias, where people with little expertise or ability assume they have superior expertise or ability. This overestimation occurs as a result of the fact that they don’t have enough knowledge to know they don’t have enough knowledge."
In other words, a person suffering from this effect is not trying to learn about a new subject--because they don't even know they need to.
I never said anything about "trying to learn".
> "A cognitive bias, where people with little expertise or ability assume they have superior expertise or ability. This overestimation occurs as a result of the fact that they don’t have enough knowledge to know they don’t have enough knowledge."
I agree completely. Nothing I said contradicts this.
Every expert starts as a beginner. Not every beginner ends up as an expert.
Also people can learn without a deliberate intent to learn. As far as I am aware right now, deliberate intent has nothing to do with DK, and I certainly wasn't making any claims about it.
> people can learn without a deliberate intent to learn. As far as I am aware right now, deliberate intent has nothing to do with DK
I agree with this: DK's research wasn't about whether anyone is trying to learn anything; AFAIK they didn't even look at that.
However, in the GGP to this post (the post of yours I originally responded to in this subthread), you said:
"Dunning-Kruger is the relationship between confidence and competence as people learn about new subjects."
But DK didn't test for "non-deliberate learning" any more than they tested for deliberate learning. They didn't test anything over time at all. So their research can't tell us anything about what happens over time as people learn (whether they're doing so deliberately or not).
There is much irony in the certainty this article displays. There are no caveats, no qualification, and no attempt to grasp why anyone would use an LLM. The possibility that LLMs might be useful in certain scenarios never threatens to enter their mind. They are cozy in the safety of their own knowledge.
Sometimes I envy that. But not today.
I actually prefer reading this type of writing on the internet. It's more interesting.
Of course it's complicated. Just give me a take. Don't speak in foot-noted, hedged sentences. I'll consider the nuances and qualifications myself.
There should be a name for something like that
The other irony is that Dunning-Kruger is a terrible piece of research that doesn't show what they claim it shows. It's not even clear the DK effect exists at all. A classic of 90s pop psychology before the replication crisis had reached public awareness.
It's worth reading the original paper sometime. It has all the standard problems like:
1. It uses a tiny sample size.
2. It assumes American psych undergrads are representative of the entire human race.
3. It uses stupid and incredibly subjective tests, then combines that with cherry picking. The test of competence was whether you rated jokes and funny or unfunny. To be considered competent your assessments had to match that of a panel of "joke experts" that DK just assembled by hand.
This study design has an obvious problem that did actually happen: what if their hand picked experts didn't agree on which of their hand picked jokes were funny? No problem. Rather than realize this is evidence their study design is bad they just tossed the outliers:
"Although the ratings provided by the eight comedians were moderately reliable (a = .72), an analysis of interrater correlations found that one (and only one) comedian's ratings failed to correlate positively with the others (mean r = -.09). We thus excluded this comedian's ratings in our calculation of the humor value of each joke"
It ends up running into circular reasoning problems. People are being assessed on whether they think they have true "expertise" but the "experts" don't agree with each other, meaning the one that disagreed would be considered to be suffering from a competence delusion. But they were chosen specifically because they were considered to be competent.
There's also claims that the data they did find is just a statistical artifact to begin with:
https://digitalcommons.usf.edu/numeracy/vol10/iss1/art4/
"Our data show that peoples' self-assessments of competence, in general, reflect a genuine competence that they can demonstrate. That finding contradicts the current consensus about the nature of self-assessment."
Two unrelated comments:
> It assumes American psych undergrads are representative of the entire human race.
(1) Since it can't document an effect in them, it doesn't really matter whether they're representative or not.
> The test of competence was whether you rated jokes and funny or unfunny. To be considered competent your assessments had to match that of a panel of "joke experts" that DK just assembled by hand.
(2) This is a major problem elsewhere. Not just elsewhere in psychology; pretty much everywhere.
There's a standard test of something like "emotional competence" where the testee is shown pictures and asked to identify what emotion the person in the picture is feeling.
https://psytests.org/arc/rmeten.html
But, if you worry about the details of things like this, there is no correct answer. The person in each picture is a trained actor who has been instructed to portray a given emotion. Are they actually feeling that emotion? No.
Would someone else look similar if they were actually feeling that emotion? No. Actors do some standard things that cue you as to what you're supposed to imagine them feeling. People in reality don't. They express their emotions in all kinds of different ways. Any trial lawyer will be happy to talk your ear off about how a jury expects someone who's telling the truth to show a set of particular behaviors, and witnesses just won't do that whether they're telling the truth or not.
My current hypothesis du jour is that AI is going to be like programming in a certain way. Some people can learn to program productively, others can't. We don't know why. It's not related to how smart they are. The people who can program, can be employed as programmers if they want. Those who can't, are condemned to be users instead.
The same may end up being true of AI. Some will learn to make productive use of it, others won't. It will cause a rearrangement of the pecking order (wage ladder) of the workplace. I have a colleague who is now totally immersed in AI, and our upper management is delighted. I've been a much slower adopter, so I find other ways to be productive. It's all good.
> as the Dunning-Kruger Effect. (link to the wikipedia page of Dunning-Kruger Effect)
> A cognitive bias, where people with little expertise or ability assume they have superior expertise or ability. This overestimation occurs as a result of the fact that they don’t have enough knowledge to know they don’t have enough knowledge. (formatted as a quote)
However, the page (https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect) doesn't contain the quote. It's also not exactly what Dunning-Kruger Effect is.
Either that the author didn't read the page they linked themselves and made up their own definition, or they copied it from somewhere else. In either case the irony isn't lost on me. Doubly so if the "somewhere else" is an LLM, lol.
It's as if the author made the same misunderstanding as described in the first paragraph of the Wikipedia article.
The wikipedia editor clearly believes the misunderstanding is so common that they have to put it in the first paragraph. But people (like the OP author) still just ignore it.
I have a quote for this:
> "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the Universe trying to produce bigger and better idiots. So far, the Universe is winning." — Rick Cook
But wikipedia.
It actually really seems that way.
They unwittingly illustrate part of the phenomenon while claiming to explain it.
> It's also not exactly what Dunning-Kruger Effect is.
What do you think it is?
It's people who overestimate their expertise (at the low end) or underestimate it (at the high end). But the people at the high end still estimate their skill higher than those at the low end.
Instead, what it has turned into is this idea that "stupid people think they're smarter than smart people" when it says nothing of the sort.
That is a direct paraphrase of the abstract of Kruger & Dunning, 1999[1]:
"The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it."
Now, it may be possible that the definition has evolved since then, but as the term Dunning-Kruger effect is named after this paper, I think it's safe to say that Wikipedia is at least partially wrong in this case.
[1] https://pubmed.ncbi.nlm.nih.gov/10626367/
You're misinterpreting the quote. Unskilled people overestimate how skilled they are, but they still understand that they are unskilled. They just don't know quite how unskilled. What Kruger & Dunning actually showed is that people tend to skew their estimates of their skill towards being slightly above average.
Above average is not unskilled. That does not compute for me. The point is clearly about how lack of skill/experience in a task make people bad at judging how they do at the task. That is clearly what the article Kruger and Dunning) and the article (OP) are both saying.
You could help your argument by explaining how I'm misinterpreting the quote.
Then using a tv remote control is Dunning-Kruger. Look, I push a button, then the tv turns on. How? I barely have an idea. I push this button, and then this button, and we are enjoying an episode of Modern Family. How? I barely have any idea.
Then programming in Python is Dunning-Kruger. Some simple text, and I somehow manipulate and manage all sorts of complex processes taken from the abstract all the way down to machine instruction, down to a trillion and one bare metal transistors. How? I barely have an idea. I am not a chip designer.
Then chip designers are just Dunning-Kruger flunkies. Sure they are experts doing amazing things, but what do they know of mining and smelting, logistics and accounting, and the zillion and one other things that people in our society do to empower each other in this collective enterprise.
Progress IS Dunning-Kruger.
The difference is people don't put "electrical engineer" on their resume because they can turn on a TV.
A lot of people are tempted to call themselves "Python programmers" because ChatGPT spits out Python that they think they understand.
No, they are an AI user that allows them to do Python programming. Likewise, a Python programmer uses Python to allow them to use C and machine instructions without being a C and machine instruction programmer. And so on.
Python is a formal language. You don’t use Python to not use C (a formal language), Assembly (another formal language) or Opcodes (still formal). You just use Python.
Likewise, you don’t use French to say “c’est mon chat” because you want to say “it’s my cat” in English. You just use one or another based on the current context. But programming languages are strictly equivalent to each other, while natural languages just overlaps in some way.
Its a formal language, fine. But it still gets translated to machine instructions.
It's a translation like writing "2 + 2" instead of "two plus two" or "power of two". Programming language are more about notation than the ideas themselves.
Meh, I don't know. I think you can use AI to lorem ipsum a lot of things where it doesn't really matter:
- Making a brochure. You need a photo of a happy family. It doesn't matter if the kids have 7 fingers on each hand.
- You have some dashboard for a service, you don't quite know what the panels need to look like. You ask AI, now you have some inspiration.
- You're building a game, you need a bunch of character names. Boom. 300 names.
- Various utility scripts around whatever code you're writing, like the dashboard, might find use, might not.
None of those things is pretending you're an expert when you're not.
Give AI to a coding novice, it's no different from giving autopilot to a flying novice. Most people know they can't fly a plane, yet most people know that if they did, autopilot would be useful somehow.
> You're building a game, you need a bunch of character names. Boom. 300 names.
This is actually one of the things AI is notoriously bad at. If asking for a plain list, it very quickly falls into blatant patterns (one name per letter of the alphabet, all names starting with A, all names with exactly 2 syllables, etc.). And, whether part of a list or not, one of the most obvious signs of AI writing is that it always resorts to the same handful of names for a given archetype.
Traditional random name generators are much better.
You could just ask it for one name, 300 times? (And clear context in between, I suppose.)
But if it matters to you, use the specialized tool.
If it's not important, lorem ipsum, cheaply.
If it doesn't matter to you, then whatever you are creating has no value to society, and is in fact likely net-negative.
We're seeing it all over: curation has become much harder since the slopfest began.
I rather spend one hour to write a script than writing one with AI. If I’m spending that much time, that is a clear signal that it’s important to get right or I’m enjoying the learning process.
As for when there’s a rush, I just avoid putting myself in those situations. If there’s not enough time, I advocate for a simpler solution, even hackish.
> Making a brochure. You need a photo of a happy family.
do you really?
> you don't quite know what the panels need to look like.
look at your competition, ask your users, think?
> Most people know they can't fly a plane
this isn't how llm products are marketed, and what the tfa is complaining about.
> do you really?
That's supporting my view. You might want it, you might not. It's marginal, and now it's cheap.
> look at your competition
LLM does this for you
> this isn't how llm products are marketed
It certainly is. Something like ChatGPT is marketed as a low-risk chat partner, most certainly not pretending to give you medical or legal advice. Talk to it like your buddy, you get buddy responses. Your buddy who has read a few law books but doesn't pretend to be a lawyer.
> do you really?
This is what's known as an "example".
> Various utility scripts around whatever code you're writing, like the dashboard, might find use, might not.
Let's hope you protect that dashboard well with infra around it, because it will be the front door for people to invade your site.
The same apply in slightly different ways to your deployment script, packaged software (or immutable infra) configuration, and whatever tools you keep around.
> Let's hope you protect that dashboard well with infra around it, because it will be the front door for people to invade your site.
I was thinking of internal dashboards, but like I said above, if it doesn't really matter, use LLM. If you are building a cockpit for an airplane, yeah, don't just use a fill tool. If you need security, yeah, don't leave the door open.
> It doesn't matter if the kids have 7 fingers on each hand.
Only if you don't care that your customers surmise you don't care.
Careful not to overestimate the customer
Dunning-Kruger is basically a middling party put down at this stage. Similarly this article is not making a whole lot of sense other than as a mild and wildly applied dis?
The Dunning-Kruger effect is where people with low intelligence express high confidence in their intelligence over others by constantly referencing the Dunning-Kruger effect
I still marvel at people who act and write as if D-K is proven. The debate about whether the effect exists, its scale if it does exist, where it might originate if it is real and where it might originate if it is a statistical artifact ... these all carry on. D-K is not settled psychology/science, even though the idea is utterly recognizable to all of us.
DK eats its own tail.
Stating it makes it so, as the one mentioning it self-DKs themselves. Doing so, DK has been proved.
> though the idea is utterly recognizable to all of us.
Then why marvel? If we can't scientifically prove it, but it tracks logically and people find it to be repeatedly recognizable in real-life, it makes sense people speak about it as if it's real
Saying AI is Dunning-Kruger as a service is a Dunning-Kruger take.
Not really on topic, but it'fascinating how Dunning-Kruger effect continues to live its own life in the public culture despite being pretty much debunked in its most popular form a while ago.
What Dunning-Kruger experiments have actually shown is that people's assesment of their own performance is all over the place, and only gets slightly better for good performers.
I don't use it that way
No it’s a statistical text phonograph of the general public. We talk to it like an oracle. Look up the old logo for His Master’s Voice.
The value of AI is in the imagination of its wielder. The Unknown Unknowns framework is a useful tool in how to navigate AI along with a healthy dose of critical thinking and understanding how reinforcement learning and RLHF work, post pre-training.
Feels like you could make a similar argument with any tool that is leaps and bounds better or makes your job 'easy'.
Dreamweaver was Dunning-Kruger as a program for HTML-non-experts. Photoshop was Dunning-Kruger as a program for non-airbrushers/editors/touchup-artists.
(I don't actually believe this, no they weren't.)
Or, we could use the phrase Dunning-Kruger to refer to specific psych stuff rather than using it as a catch-all for any tool that instills unwarranted confidence.
The problem isn’t tools making someone better. An excavator will make me a superior ditch digger than if I just have a shovel. That’s progress.
The issue is making someone feel like they did a good job when they actually didn’t. LLMs that make 800 line PRs for simple changes aren’t making things better, no matter how many “done” emojis it adds to the output.
> LLMs that make 800 line PRs for simple changes
Do you have any examples of this? Because my experience has been the opposite. In many cases LLMs will make minimal changes to achieve some goal.
Unlike the expertise that Dunning-Kruger to refers to, the skill to create art and understand art are separate.
Possibly Dreamweaver might fit because it does give you the sense that making a website is easy but you might not understand what goes into a maintainable website.
You cannot make a similar argument for any tool that makes jobs easier, because the argument is dependent on the unique attribute of LLMs: providing wrong answers confidently.
You can though with some creativity.
There are lots of tools that give a wrong solution that appears correct, and easier ones tend to do that the most.
Plenty of people who needed a real dev team to design an application probably hoped on Dreamweaver, were suddenly able to bumble their way some interface that looked impressive but would never scale (even to the original goal level of scale mind you).
-
Any time you have a tool that lowers the barrier of entry to a field, you get a spectrum of people from those who have right-sized expectations and can suddenly do the thing themselves, to people who massively overestimate how easy the field is to master and get in over their heads.
This isn't even a programming thing, off the top of my head Sharkbites get this kind of rep in plumbling
You could argue the RPi did this to hardware, where people are using a Linux SBC to do the job a 555 timer could do and saying that's hardware.
Point-and-shoot, then smartphone cameras, did this and now a lot more people think they can be a photographer based on shots their phone spends more processing power per image than we used to get to the moon on.
I prefer Gell-Mann Amnesia Effect or Knoll’s Law of Media Accuracy
“AI is amazing about the thing I know nothing about…but it’s absolute garbage at the stuff I’m expert in.”
https://garrickvanburen.com/an-increasingly-worse-response/
Don’t worry, with enough usage you’ll know nothing about the stuff you’re an expert in too!
using LLMs for creative purposes is terrifying. Because why? learning the craft is the whole reason you do it. however using LLMs to get work done, I just had Claude rewrite some k8s kuttl tests into chainsaw, basically a complete drudgery, and it nails it on the first try while I can stay mentally in EOD Friday mode. Not any different from having a machine wash the dishes. because it is, in fact, nuclear powered autocomplete. autocomplete is handy!
Bypassing practicing a practical skill stunts your growth the same way as bypassing creativity. For some tasks that may be fine, but I'd never be comfortable taking these shortcuts with career skills. Not if my retirement was more than a few years away.
I actually agree with you on that as well, watching claude take over so many tedious "oh this thing broke, I have to google how that works, re-learn its config format, fix the thing, then forget about it again" kinds of tasks does at the same time reveal a lot of thought processes that i no longer have to engage in.
then again I dont see how it's too different from just a simple move into middle management where you manage programmers but dont do as much of the coding yourself. with Claude I'm finding myself taking on much bigger tasks and orchestrating a lot more technologies to do things, so I am learning about things i would not have if i didnt have the LLM to assist me through it.
The Dunning–Kruger meme has basically turned into a way for people to laugh at others for being “too dumb to know they’re dumb.” But that misses the point completely. The whole “I know that I know nothing” idea applies to everyone. So when people use the meme to mock AI or AI users, it just shows they haven’t learned that same lesson themselves.
Part of the problem with labor that we haven't yet discussed or maybe want to avoid due to the dissonance of the association with qualities as slaves is, we have a leadership class who acts more like elite slave masters than human beings with inherent dignity and decency. We have the class write the rules that they hold themselves (un)accountable for since the system was designed for them and enforced by them.
These are the people driving the rush and having a lot of say in the current AI and overall capitalist market behavior and sentiment. I think they're really mad and salty that when COVID happened the engineers got more remote and free and expressed the resentment more freely. This comment is probably putting me on a list somewhere or activating some hate program against me.
In my experience, AI is more akin to Gell-Mann Amnesia Effect as a service....
Have you used them for coding? Because my experience with that is the opposite. They're literally superhuman in many significant respects.
People are always create new layers on top of others. Machines that make machines or code that compiles to code. Layers of abstractions makes it possible for our simple brains to control trillions of electrons in a silicon chip. Every transition to a new layer has haters and lovers. Most people hate change. But eventually everything is using the new change. Never ever things go backwards in human history. AI is not Dunning Kruger
I would like to see AI usage regulated in the same way that vehicles are: license required.
Be that an aptitude test or anything else... unfettered usage of vehicles is dangerous in the same way that unfettered access to AI is as well.
As a society, we have multiple different levels of certification and protection for our own well-being in the public's when certain technologies may be used to cause harm.
Why is knowledge or AI any different? This is not in opposition at all to access information or individual liberties. No rights are violated by their being a minimum age in which you can operate a vehicle.
A vehicle is driven on public roads and can kill people, that’s why licenses are required.
Outlawing certain kinds of math is a level of totalitarianism we should never accept under any circumstances in a free society
There is nothing totalitarian about constraining societal harm.
The issue comes down to whether it is collectively understood to be a benefit to the human race. Until now we have never had to constrain information to protect ourselves.
Please read the Vulnerable World Hypothesis by Nick Bostrom
> There is nothing totalitarian about constraining societal harm.
Of course not. The problem is that the only way to enforce AI regulations is through totalitarian means.
You can easily regulate OpenAI and Gemini and whatnot, but then people will just use local AI models. The barrier to entry for using local AI models is basically zero because software like Ollama make it trivially easy to set up, and small LLMs can run on an iphone or a laptop.
The only way to actually prevent "unauthorized" AI usage is to control digital communications to prevent LLM weights from being distributed and to ensure that no "black market" AIs are being trained or used.
And if you're already scanning digital communications for LLM weights, why not also scan it for other forms of wrongthink?. Hey presto, now you have totalitarianism.
I don't think that LLMs fall into the category of an infohazard in the way that Bostrom defines it. It presents a risk, but not one severe enough to justify universal surveillance. Bostrom is talking about AI that can make bioweapons, not ones that gives false confidence to incompetent people.
This is a veiled insult thrown at those who value AI. Maybe not even veiled.
It's not veiled and it shouldn't be. Go browse linkedin, reddit, indiehacker, etc. Literal morons are using AI and pretending to be super geniuses. It's replaced the reddit-tier "google-expert" with something far more capable of convincing you that you're right.
Outside of a very small bubble of experts using AI and checking it's work (rubber ducking) most people are, in fact, using it to masquerade as experts whether they know it or not. This is extremely dangerous and the flamebait is well deserved, imo.
>They give utter nonsense answers with high confidence and wrap errors in sycophantic language making me feel good for pointing out that they wasted my time
I would implore the author to share examples. Every platform allows linking to chats. Everyone talks about this all the time, incessantly. Please, can someone please share actual chat links containing these episodes of utter nonsense, outside of what can be attributed to the knowledge cut-off (i.e. "Mamdani is not the mayor-elect of NYC").
I get it if you are using a 20B model or AI overviews, but anyone trying to actually get anything meaningful done should be using a SOTA model. I'm genuinely not interested if you are going to reply with a description or story. I really, really just want links to chats.
Edit: You can downvote me, but please make me look like an idiot by posting chat links. That is the real downvote here.
This article feels lazy. Is the main argument in similar vain as "don't read the books that experts have written, and go figure stuff out on your own"? I'm trying to understand what is wrong with using a new data compression tool (LLMs) that we have built to understand the world around us. Even books are not always correct and we've figured out ways to live with that/correct that. It doesn't mean we should "Stop wasting time learning the craft"..
LLMs are optimized for sycophancy and “preference”. They are the ultra-processed foods of information sharing. There’s a big difference between having to synthesize what’s written in a book and having some soft LLM output slide down your gullet and into your bloodstream without you needing to even reflect on it. It’s the delivery that’s the issue, and it definitely makes people think they are smarter and more capable than they are in areas they don’t know well. “What an insightful question…”
Wikipedia was already bad, low brow people would google and read out articles uncritically but there was still some brain work involved. AI is that meets personalization.
> I'm trying to understand what is wrong with using a new data compression tool (LLMs) that we have built to understand the world around us.
What's wrong with it is that many people are resistant to it. That's all there is to it.
No, I do not quite think that is what they wrote here. But what's the thought process here? It's hard for me even to understand if the first scare quote is supposed to be from someone being critical or someone responding to the critique. It seems like it could apply to both?
I am not the author, but quite curious to know what prevented comprehension here? Or I guess what made it feel lazy? I'm not saying its gonna win a Pulitzer but it is at minimum fine prose to me.
Or is the laziness here more concerning the intellectual argument at play? I offer that, but it seems you are asking us what the argument is, so I know it doesn't make sense.
I have been a fool in the past so I always like to read the thing I want to offer an opinion on, even if I got to hold my nose about it. It helps a lot in refining critique and clarifying one's own ideas even if they disagree with the material. But also YMMV!
> what prevented comprehension here?
This is an arrogant and unwarranted assumption. What's preventing your comprehension of this discussion?
The article sets up a straw man - the person who can convincingly fake being an expert without actually being one - and then demolishes it.
This doesn't resemble anything that I've experienced from LLM use in the real world. In my experience, amateur use of LLM is easily detected and exposed, and expert use is useful as a force multiplier.
I suppose the "Dunning-Kruger" accusation might apply to the first one, but I'm not convinced - the people doing that are usually very aware that they're faking their attempt at projecting expertise, and this comes across in all sorts of ways.
gp asked us what the blog is arguing, doesn't seem too unwarranted to assume they didn't comprehend? Or am I missing something?
Also, just fwiw, I really tried but I am truly having trouble comprehending what you are saying, or at least how it bears on the article? It is 8-9 short paragraphs long, can you like point to wear he demolishes the straw man? Or like what does that even mean to you? Isn't it a good thing to demolish a straw man? Given that it is fallacy?
Trying to be charitable here parsing this: I don't think Dunning-Kruger really speaks to one's ability to convince right? Doesn't it really manifest when we don't actually need to be convincing to anyone? This is the definitional thing about it really: you are precisely not aware you are "faking" it, you think you are doing really great!
Your comment feels lazy as well. It waves off the article without engaging with its core argument. The piece isn’t saying “ignore experts”. It’s questioning how we use tools like LLMs to think, not whether we should. There’s a difference between rejecting expertise and examining how new systems of knowledge mediate understanding.
>Your comment feels lazy as well. You repeated your one thought four times.
as a lazy person that's opposite of what i'd do.
edit : oh , you completely re-worded what i'm replying to. Carry on.
At least they put forward their own thoughts instead of a blind complaint
> that we have built to understand the world around us
Pretty generous description. LLM output doesn't have any relationship with facts.