It also does an amazing job of taking snapshots of posts. I use it all the time to grab posts from HackerNews so I can mark them up.
And it automagically syncs everywhere across devices and lets me store way too many files on the web like the ADD packrat I am, without breaking a sweat.
In short, this software just works, and it works well.
So when somebody behind Zotero talks about how to develop software, I listen.
And it's a fun post with some history. You should save this post to Zotero, and then read it.
Unrelated, but I love when people describe software with the phrase "just works".
It suggests a certain... simplicity that I chase in all of my developments. The amount of EFFORT to get software from "works but..." is interesting. You can get something that works 99% of the time, but that never quite fits the "just works" level of quality that we love to see.
Really, there's an immediate feeling of joy I get when I interact with software that "just works" knowing how much effort it takes to get something to that point.
Like an application to book a haircut at a salon or log into the WiFi at a hotel that does something "on rails" and should work without any learning or training is one thing. A complex application for professional creative work is something else entirely.
Then there are the things that are just weird, like life got me looking at mobile games lately and I was really shocked to see how many of them have a horrific onboarding experience. The best games in the "clothes collecting" genre, for instance, like Love Nikki have a gentle onboarding process that teach you the game mechanics and get you into the story and maybe get you hooked. When I was developing consumer-facing social and entertainment apps (before Facebook!) I had my own belief that "easy onboarding is good for users and good for profits" and worked with founders who got it and pushed me to outdo myself. Most games in that genre have horrific UI and don't try to explain the mechanics because they could care less if users understand the currencies used in the system because they only care for the worst "whales" who click on any red dot and fold green at the slightest difficulty.
These days when it comes to "slow software", I think five years is an eternity from a platform perspective so if you start something you may need to rebuild it. Like I find it hard to justify developing on anything other than the web platform (like "write once and it runs on platforms you never thought of like VR headsets", "for christ's sake did you realize every Windows software vendor had at least one InstallShield engineer back in the 1990s and IT was pulling their hair out updating all their desktops?", "it took three weeks for the App Store to reject your update? what did you think would happen?") but the app I am working on right now depends on React components that haven't been updated since 2021 and the bill is coming due.
Oh man, the state of dress up games is a personal pet peeve of mine, poor onboarding tutorials is just one problem. The art direction is often simply atrocious, proper game loops barely exist (let alone creative ones), the microtransactions are the default business model. I honestly don't know what causes it, perhaps the games are just not that profitable so they don't attract highly skilled or at least conscientious game developers? I came to accept that they are just made by people who either dislike/don't care about games in general or dislike dress up games specifically. The best dress up games are normally made by Nintendo who have a strong game design and gave development culture in general.
P.s. As a side note, I started making small dress up games that I draw myself to scratch my own itch. I am not a programmer so they are absolutely horrendous and I only show them to friends. Making videogames is not as easy as making paper clothes for paper dolls like I did as a kid, but it is so fun and I feel 30 years younger somehow. :D
Yeah, I think those games have more potential than most people think. I love voting mechanics (spent a lot of time on ranking systems) and some have good stories ranging from "the heartwarming story of Nikki exploring a continent, making friends and proving her mad skills" to "a vicious fight for survival in an ancient court"
What's a good voting system? I remember checking out Dress to Impress (not a mobile but an incredibly popular Roblox game), and it had a somewhat unpleasant, imo, voting system -- players vote for each other's outfits and the score is a sum of the scores your co-players' gave you which incentivizies players to give everyone the lowest score. The game tries to "punish" this behavior by detecting "unfair" voters and lowering the weight of their votes, but I feel like you could tweak the original incentives instead by changing the mechanic. Having to rank outfits instead of giving each outfit a score could work I think? But there are probably other problems I'm missing. I dnn't play too many mutliplayer games.
(On the other hand, while I believe thinking about "gaming" the ranking takes away from the fun of the game, Dress to Impress is massively popular, so maybe this frustration is what makes it popular. IDK if kids play it because there are no competitors in the genre or because some competitive toxicity is addictive.)
Well I'm interested in voting theory from the viewpoint of political science and also from the viewpoint of developing ranking systems. It's actually a pretty depressing topic because you can easily come to the conclusion that the public doesn't have an interest at all.
In short, pairwise rankings are real and comparable between people whereas different people might use a scale differently. It's a difficult problem to deal with "i like the bagel shop a little bit more than the rice bowl restaurant but the party member who has celiac's opinion matters more than mine" See
now if you have N options you don't have to have everyone evaluate N(N-1) pairs because preferences are correlated and should be transitive. There are many algorithms, like Elo and the methods in that paper that can be used to run tournaments and compute scores. The game Covet uses pairwise ranking and I think gets good scores.
To get back to social decision theory, it's probably not good for players to be in a team for purposes of winning and losing. Like I think sharing clothes with your friends is a really fun mechanic (might make you want to really buy or work for something!) but if people can help or hurt their friends you get into the whole can of worms of coalitions in N-person games.
I just quickly tried Covet and I can imagine how the act of comparing and picking one of the two options can feel fun. I can see myself just voting on outfits in the game without creating my own when I need some really simple "mindless" fun. Not a productive effect but also not addictive enough AFAIK to be a big problem.
What I dislike about Zotero is that it's a pain in the ass to self host. In my department I could spin up a local Zotero server for our PhD students and give them basically unlimited sync, but it's really finnicky
What difficulties are you having? All you need is WebDAV. I use a nextcloud server for mine and my partner's zotero storage, and its entirely trivial to set up.
I have many issues with Zotero (which has been dumbing down at the expense of power users for some time) – but easy private sync is definitely not one of them.
Looking at my own Zotero account, I can confirm that all metadata storage is free and not counted (I am using none of my 300Mb storage in my account according to zotero.org, despite syncing metadata for several thousand bibliography items). The only storage Zotero seems to count is attachment storage, and that's all synced via my WebDAV.
I thought the discussion was about self-hosting data, such that the provided free storage limits by Zotero remained sufficient? And for that use case, self-hosted WebDAV storage for file attachments is absolutely fine.
(The poster I was replying to was suggesting that their departmental users would go over their storage limits for metadata alone, even if they used WebDAV for attachments -- but that seems to be incorrect from my own experience, and everything I can find online.)
It is not supported natively (you can use a webdav server but it doesn't sync everything) and the third party options don't have the level of stability I'd need
I was looking for software that would let me bookmark recipes (from web pages) and let me scrawl my notes on them. "Needs more of this or that ingredient" "Use less salt". This might be it.
It's baffling actually that Firefox doesn't let you annotate bookmarks, it seems such an obvious feature.
> You should save this post to Zotero, and then read it.
I might be dense - but I added this story as a URL to my library on Zotero on Android - and I don't see any way to read it "on" Zotero? I can apparently create a citation, however.
I could dig into their documentation but I'd rather ask an enthusiast. Is it still possible to get your own data out in a nice and easy format? I was once all-in on a note-taking app that slowly got worse and worse and it was a hassle to escape, so now I always consider it before starting.
If Zotero ever enshittifies or just disappears would I still be able to access my data and transfer it to something else?
It's also all open source and local-first. You can also add your own self-hosted WebDAV server for syncing across clients. Also, it's developed by a nonprofit organization.
Not the person you're replying it to. I had that happen with Microsoft's OneNote. I really liked it and I had no problems parsing the XML files it made so I could pipe data into other systems but then it went 100% cloud storage.
It was a classic example of "Microsoft made something surprisingly good, made people think it was crap because they stuffed five OneNote icons on your taskbar and three up your nose, and then they wrecked it", must have come from the same mind that thought buying Activision was a good idea.
Yes, but in addition to that, it is owned by a non-profit who hasn’t made a single controversial update in 20 years, so it’s got a pretty decent track record.
I have Zotero set up with some plugins that create a .bib file I create notes for in Emacs. I enjoy this setup because it allows me the convenience of saving files quickly using Zotero, but saves all the data it extracts to an open format with an auto export. If Zotero were to disappear tomorrow, I’d still have all my notes in org files and my citations in their bib file.
> But that slow formation led to software that was durable rather than ephemeral, with a strong foundation that could be built upon.
People say that agentic development is great because you can churn out so much so fast. But that doesn't mean that any of it will be truly good and reliable.
The things that are truly insightful and solid end up being used exponentially more, which makes the linear cost of extra development time (asymptotically) insignificant in the cost/benefit equation.
The emphasis on quantity while neglecting quality calls to mind a passage from EWD1175[0]:
> My second warning remark is that I shall refuse to discuss the academic enterprise in financial terms. The first reason is that the habit of trying to understand, explain, or justify in financial terms is unhealthy: it creates the ethics of the best-seller society in which saleability is confused with quality. The other day we had to discuss the professional quality of one of our colleagues, in whose favour it was then mentioned that one of his Ph.D.s had earned lots and lots of money in the computer business, and few people seemed to notice how ridiculous a recommendation this was. We also know that the financial success of a product can be totally independent of its quality (as everyone who remembers for instance the commercially successful IBM360 should know). The second reason for my refusal is that the value of money is a very fuzzy notion, so fuzzy in fact, that efforts to understand in financial terms always lead to greater confusion. [Remember this, for it is quite likely that this afternoon will give you the opportunity to observe the phenomenon. Note that money need not be mentioned explicitly for the nonsense to emerge, a reference to "the taxpayer" can do the job. The role of "the taxpayer" then invariably leads to the conclusion that of State Universities at least the undergraduate curriculum has to be second- or third-rate.] The final reason for my refusal is that the habit appeals to the quantitative mind and I come from a culture in which the primarily quantitative mind does not evoke admiration. [A major reason that we considered Roman Catholics to belong to a lower class was precisely their quantitative bent: they always counted, number of faithful, number of days in purgatory, you name it.....]
Everyone I know who used the 360 has a lot of nostalgia for it.
It was well thought through. I mean, the 360 architecture is still with us because it avoided the traps that killed the PDP-11, VAX and the 68k -- as much as I loved the PDP-11. It's true when it came out that nobody had any idea what a general-purpose operating system looked like and it took a decade for them to productize VM so that you could run as many operating systems as you needed simultaneously.
The trade-off between quality and speed of development has always been a tenet of software engineering. Agentic coding changes the equation a lot, but the equation is still there.
> The trade-off between quality and speed of development has always been a tenet of software engineering. Agentic coding changes the equation a lot, but the equation is still there.
The quality and speed tradeoff doesn't matter if other incentives aren't aligned. In today's product-driven world, where usage stats are compiled in real-time, features are added and then pop-ups, nudges, etc are added to software to get product usage up. These features may or may not add quality, relevance, etc. But the "success" is measured in usage, forced or not. This is essentially Microsoft today (especially with copilot), but even after every major apple OS update, I get "what's new" pop ups for every Apple app (notes, reminders, mail, etc) on every platform (iPad/iPhone/Mac) that I own. I opened the email app to check my email, don't get in my way!
It may be so but the article does not claim that the strong foundation is the code, rather it seems to be product design, and design of other products, at that (the two predecessors, webapp and desktop app). No reason why you couldn't study existing products now and tell your agent to build something based on that.
This feels like a sleight of hand to me. The hard part of evolving Scribe and Web Scrapbook was discovering that a browser extension manipulating a local SQLite database was _the only_ architecture that could reconcile local offline persistence with live DOM scraping across arbitrary catalogs of academic data.
An agent can synthesize existing solutions but (because I see this failure mode at work constantly) it can't synthesize an architecture to resolve the sorts of tensions that the person prompting it doesn't yet understand (not that that is stopping anyone). You can't prompt it to build something if the operational primitives required to solve the problem haven't been mapped.
"Build a tool based on Scribe and Web Scrapbook" in 2003 would've made a fragile PHP wrapper because that's what the existing landscape looked like.
Yes, exactly. And this is why I don't think that the LLMs can make significant process beyond what humans have done and published.
"But the math proofs," people will say. A lot of those seem to be spam-solving things with a huge swath of existing lemmas, and a some of these are being debunked and retracted.
Just today I was quizzing ChatGPT about a basic grammar question for a language that has huge training data but for which the grammar was not well documented. It kept giving me confidently wrong answers until I drilled and drilled it and then finally it found/gave back an explanation that perfectly fit a pattern given in one particular grammar, citing that as a source. It doesn't appear to have been able to figure out the inner structure on it's own. It appears only able to pattern match and put things together from what humans have already discovered and written.
Right, there's a difference between statistical interpolation and semantic induction. The whole point is that LLMs can't reason from first principles to drive missing rules. It keeps confidently feeding you approximations until it collides with some source that already mapped it.
Interesting take, given that the whole reason LLMs are interesting is that they're the first system we have that can work in semantic space. Statistical interpolation, that we've solved long ago.
I think that's equivocating on the word semantic. Word embeddings map concepts like "king - man + woman = queen" or cluster synonyms together in high-dimensional vector space and the ML literature very loosely calls this "semantic space." But a high-dimensional topology of token co-occurrences isn't semantics in the sense of computation or formal semantics. It's still "just" measuring distributional similarity. Some vector that represents "thread deadlock" lives near tokens like "mutex" and "race condition" and "starvation," but the model itself has no concept of concurrency and contention.
Claiming we "solved" statistical interpolation long ago just means curve-fitting and basic regressions on structured data. Transformers are a truly impressive achievement, scaling all of this to unstructured high-dimensional text topologies, but it's fundamentally the same math operations on statistical proximity.
Like how do we explain hallucinations here? Tokens that are hallucinated are semantically "close" in that vector space but they're completely false in reality. If LLMs operated in a true semantic space they wouldn't hallucinate CLI flags that don't exist.
> Some vector that represents "thread deadlock" lives near tokens like "mutex" and "race condition" and "starvation," but the model itself has no concept of concurrency and contention.
I propose that concepts of "concurrency" and "contention" are themselves vector in latent space. All concepts are. Recall that we're talking about a 10^4 - 10^5 dimensional space. You can fit in pretty much any conceivable association as some direction in there.
And try to zoom in on any concept you know. If you do, it should quickly become apparent that there's never any concept you can give a closed definition for. We can only define concepts, and we can only learn them, through generalizing from examples. Which is conceptually (pun not intended) regression - finding a vector along which examples live.
As somebody responsible for the acquisition of users and growth of a company in terms of customer and revenue, this is a VERY salient point:
> “… we could not have accelerated Zotero’s conception, because we did not know exactly what we wanted, and so could not have written coherent prompts for an LLM.”
A surprising proportion of software products, maybe even businesses today, are solutions in search of a problem.
Sometimes that’s okay, but only sometimes. And being a solution in search of a problem requires you to get everything /else/ pretty much perfect if you want to succeed.
The fact Zotero paid attention to what people wanted, and gave it to them, and were market oriented, is demonstrably a big part of their success.
It is MUCH easier to make something people want, than to make them want something you made.
Indeed, the solutions in search of a problem are teeming in the day and age of LLMs! I think the opposite framing is quite interesting aswell "solutions in search of a problem" -> "problem in search of a solution" as in solution are not always self evident and what might work for one might not work for someone else, striking the right balance takes time. And to me this article highlight this, they are academics that are well aware of problem at hand, but still, they can't easily formulate a solution to it.
Agreed, but it is also much easier to get something made in the first place. LLMs enable rapid prototyping and rapid shifting to solve real problems. My applications are morphing from mediocre to highly useful problem solving machines much more rapidly now.
That’s a really good point! But I am always surprised by how often people will avoid putting their prototypes out there and let the market shape their product.
Rapid prototyping is an amazing opportunity afforded to us by AI. But some people use that potential to spend even longer on a more developed prototype that they are too attached to to get feedback on!
> But I am always surprised by how often people will avoid putting their prototypes out there and let the market shape their product
I see the opposite. I see thousands and thousands of quickly produced apps that have been put out there, but have 0 users that could shape it. Not so much because of their quality, but rather distribution in a very crowded and shouty marketplace.
> Today, software can be created more or less instantly with AI, for a large user base or for yourself, for any purpose or for no serious purpose at all.
I don’t think this is true. I’m working on medium to large features in companies that have around 1K engineers. These features typically involve 50% of work within your domain plus 50% of work in dependant domains. You cannot get anything doing without previous alignment with such domains. There are discussions, tradeoffs, design docs, approval committees, etc. AI can (and does) help in every step, but it’s not a magic wand that can solve the whole thing with a well crafted prompt.
Ithe hands of inexperienced people, AI slows down things (e.g., ai-generated robotic and lengthy slack messages that go nowhere, PRs that implement what a jira ticket says… but jira tickets without alignment are worthless, etc)
good things take time because they grow from something like seed. as that seed grows, it figures out its local and global context. a curious and patient caretaker of this seed will spend a lot of time looking at it, understanding it, trying to figure out the right way to give the small plant a steady foundation. with care and attention, it could grow into a tree and attract all sorts of other insects, animals, and all sorts of life.
speed kills quality. it’s literally impossible to make anything good fast.
we know this, and it still applies to software. while we may be able to make things faster, they will never become good (or great) without an incredible amount of care, patience, and joy from its maker.
there are no shortcuts to quality. it will always take a lot of time to make anything good.
I am not sure it IS impossible to make anything good fast. Look at how many incredible songs have been made in a single afternoon, or fantastic ideas for a simple product came to somebody in an instant.
Your comment made me think two things:
• Sometimes constraints make things better, and ‘speed’ can occasionally make you prioritise the things that actually matter so you deliver stuff that counts
• Sometimes, thinking longer about something doesn’t get you closer to the correct answer. You can rearrange and refactor and rewrite and redesign, but you won’t always get something objectively better than what you originally came up with. It’s still your thoughts, your brain and your ways of working that shape the output (and they haven’t changed).
Investing more time to make something better should be a conscious decision. Perhaps it’s one people decide against for the wrong reasons.
But sometimes things need something other than time and effort spent to make them better.
Keep up the good work. Your contrarian comments on this thread are saving me from wasting time being the contrarian myself.
Is Zotero good software? Yeah, it seems pretty good. I myself don’t use it, despite trying, because it didn’t solve my problems. It is free, has always been free, and its users are primarily academics who do not face market economics for their work output.
Speed is not the enemy. A good engineer knows that time is a precious resource, just like any other, and should be valued appropriately. And engineers are not academics.
Most good things are made fast, because once you slow down below certain degree, things won't get made at all.
Over the two decades of writing software professionally, I learned to appreciate the perspective of software companies: most software is crap, and without pressure to ship, smart programmers will forever keep polishing the turd, way beyond the point the software stops being relevant or useful.
The best, high quality, slow-developed software isn't the best because programmers knew when to stop. It's because the scope and timeline were bounded up front.
I agree with this deeply. I’d even go as far to say I don’t think it’s just software this is applicable too, it’s perhaps any body of collaborative, thoughtful and developed work
> Look at how many incredible songs have been made in a single afternoon
Ideas for songs. Good ideas for songs can come in an instant. And in the hands of an experienced songwriter, you can take that idea to a polished product very quickly. But that's because they spend years developing taste.
> fantastic ideas for a simple product came to somebody in an instant
What's the saying around here? Ideas are cheap. It's execution that matters? And the execution part often comes about through trial and error.
> But sometimes things need something other than time and effort spent to make them better.
> If AI had existed in the early aughts, we could not have accelerated Zotero’s conception, because we did not know exactly what we wanted, and so could not have written coherent prompts for an LLM. Instead, it took a great deal of time and collaboration to develop a clear vision for what Zotero should be.
AI doesn't preclude this. In fact, it can help accelerate parts of it.
He's describing the typical big project lifecycle:
- Examine the landscape
- User research (how they use existing software, what their frustrations are, etc)
- Brainstorming
- Early ideas and prototypes
- Refinement, user feedback
- Solidify the vision and high level process design
- Choose technologies
- Design & architecture
- Plan out phases
- Build phases, then test them with users
LLMs are great at research, and great at prototypes. Once you have your design, they're good at coding as well. They're also good at distilling user feedback.
The problem is that in the future, LLMs will change society so much that we won't even want to go after these things any more because the world will live and die on cheap prototypes and trends.
It's like all the food being replaced by MacDonalds food that is served from a replicator (from Star Trek). Sure, we can survive on it, sort of, but if all we ever know is this food and we're too busy to cook (or no one knows how to manufacture a stove) then we'll never know anything else. Maybe someone will learn how to tweak the recipe a little but our minds will be conditioned to context-switch so much that we'll never even think of learning to cook great meals.
We've already gone through that many times with COBOL, Excel, Visual Basic, PHP, Javascript, etc.
Whenever a new technology makes what used to be hard easy, all the non-experts pile in and produce shit for awhile, until they're finally forced to admit that software engineering is hard (or die trying).
The same will happen with LLMs in a couple of years.
Star Trek isn't the best example, because many characters on the show like to whine about replicator food having bland taste, and treating cooking as important skill to have (though I'd argue it's for social reason).
I can't speak for everyone, but the software I've built via agentic coding still takes a long time to build, refine and iron out all the bugs. I still puzzle out how to solve issues that the agents can't deal with. I still find it challenging and enjoyable.
There seems to be this pervasive false dichotomy about software currently: that it's either lovingly hand-tooled and crafted and perfect; or that it's a vibe-coded one-shot sloppy-slop-slop-fest of spaghetti code.
There is a middle ground, where rigorous software project management and careful architectural guidance and oversight continues, but agentic coding takes away much of the coding drudgery. It takes time, and it's not even remotely one-shot, but it's still way faster than coding everything by hand. From what I can tell, this quiet approach is where most agentic coding effort is going currently -- it's not flashy, it's not sloppy, and mostly you won't even notice.
I love great software and agree that great software is evolved - not built. The whole point of building v0.1 is to figure out what's wrong and should be fixed in v0.2
At the same time, there are broadly three motions in the loop:
- thinking/discussing (what should it do)
- building (Make it do that)
- using (Seeing whether that is actually what it SHOULD do)
And then of course you repeat until you run out of time, money, patience, volition, etc. For some software there is a terminal state - it truly does exactly what it should. For most you're always reaching for it.
LLMs definitely accelerate 2 and potentially can help accelerate 1 and 3. As such cycle time can be reduced. It still may take 100 turns to get what you want, but I'd be surprised if the clock time for the 100 turns would be unchanged using AI.
> LLMs definitely accelerate 2 and potentially can help accelerate 1 and 3.
People that do think of using LLMs to accelerate 2 usually don’t think enough about 1. And from what I’ve seen, they quickly get tired of 3.
Most quality software I’ve seen usually starts from a small subset for the cycle and then incrementally add to it. LLM projects usually rush the building part and do too big of a job and it’s become cumbersome to design (sunken cost) and evaluate (too many variables). If you want to build shelters, you start with a small hut, not with a cathedral.
> Most quality software I’ve seen usually starts from a small subset for the cycle and then incrementally add to it. LLM projects usually rush the building part and do too big of a job and it’s become cumbersome to design (sunken cost) and evaluate (too many variables). If you want to build shelters, you start with a small hut, not with a cathedral.
What on earth makes you think that you can't adopt this exact approach with agentic coding??
This is like saying that you shouldn't drive a car around suburban streets, because the maximum speed it can do is in excess of 250 kph. You absolutely can (and arguably should) take agentic coding slow and steady, and build software the way you always did. Like using a car to get to the shops at 50 kph, you'll still get there much faster than if you walk.
> What on earth makes you think that you can't adopt this exact approach with agentic coding??
It can be done, but it’s not where the hype and the practice is. It’s all about number of commits, number of PR, churns in LoC, which has no bearing on software quality and usefulness.
Something like caddy[0] is just 2700+ commits over 7 years. That’s like 32 commits a month in average. Even if you apply an exponential decay (going from greenfield to mature project), the latter years would have been way peaceful. I know AI can help in some cases, but is it such a pressing need. That would been like taking a car to go somewhere less than 10m away.
Most (mature) open source projects progress a few changes at a time. Most of the time is about ensuring that nothing breaks due to the change.
I completely agree. And yes, I think the current game-ification of repositories (more stars! more PRs!) is pretty counter-productive.
I suspect that projects that run hot and fast with LLMs will be very much like driving your car at 250 kph to the shops. You might get away with it briefly, but you'll ultimately crash and burn. As with all things -- the obsession will fade soon enough, and agentic coding will become just another tool, where those who use it well will be valued. At least, so I hope. We will see, I guess.
How many cathedral does the world needs? And even if you wanted to build a new one, it’s no longer the middle age. No one is inventing UNIX all over again. Lots of hurdles have been solved. The difficulty of software is not technical or implementation related, it’s mostly about defining the problem.
Durability is mostly boring decisions compounded: open formats, exportable data, and no dependency on a vendor mood. Speed wins users, but those choices are what keep them.
The reason great artists spend so much time developing their skills is that the result no one knew they wanted comes from skill and the new possibilities emerging from that.
So as a software developer now might be the best time to throw system design on it's head and develop without a direction but make sure everything is as good as one can forge it.
Surprising functionality and stuff not found elsewhere then more or less simply emerges from that.
Granted - that has a certain freedom and no pressure to make money as a prerequisite but so do the 5 years noted here.
This seems like a highly romanticized retelling of the origin story. LLMs would have more likely made that 5 years into 5 months. LLMs are excellent at accelerating work where you don't know exactly what you are after.
They will also take you down a road you didn't want to go down at a breakneck speed too. And you don't realize you didn't want to go down that road. Sure once you figure out you didn't want that it is quick to go back and start again, but how many times do you need to do that until you get where you wanted to be? Are you really saving time in the end?
I’m developing a software solution that integrates w/ Zotero (sortcite.com). Their API is very easy to work with. Some of their competitors (Endnote) don’t even have an API.
A very very strong point. I have personally detailed entire projects because adding a feature seemed easy with AI. It is very difficult to vibe code and not add a bunch of useless crap features.
Really? I find LLMs quite bad at deleting code. If you ask them to add a feature, then later take it out again, the codebase still grows. It always grows. Every time I've tried it, llms have failed to simplify code via refactoring. Even fable is incredibly bad at this for some reason.
LLMs are excellent at making prototypes though. And prototypes can be an excellent way to stop yourself from implementing the wrong features in the first place.
A small footnote regarding the intersection of this fascinating article and current(ish) events.
> That winter, with Roy as the principal investigator and Josh and me as co-directors, we applied for a grant from the Institute of Museum and Library Services.
> The Institute of Museum and Library Services (IMLS) is an independent agency of the United States federal government established in 1996. It is the main source of federal support for libraries and museums within the United States, having the mission to "advance, support, and empower America's museums, libraries, and related organizations through grantmaking, research, and policy development"
You can guess the next part, I’m sure:
> On March 14, 2025, President Trump issued an executive order virtually eliminating IMLS that directed that "the non-statutory components and functions ... shall be eliminated to the maximum extent consistent with applicable law, and such entities shall reduce the performance of their statutory functions and associated personnel to the minimum presence and function required by law", along with minimizing several other agencies. The entire 70 person staff was put on leave on March 31, 2025.
> On May 1, 2025, a lawsuit brought by the American Library Association and the American Federation of State, County and Municipal Employees resulted in the U.S. District Court for the District of Columbia granting a limited temporary restraining order to block any further actions to dissolve IMLS.
This post smells a lot like academia. I say that as someone who has spent quite a bit of time there myself, so I understand where the author is coming from, but “ivory tower” is still an apt metaphor.
The difference between an academic and an engineer is that the engineer understands that time spent in research and design does not magically justify itself. Quality and time-to-market are both important and to be an engineer is to figure out how to deliver along the optimal frontier.
I guess we'll only be able to verify this 10-15 years after coding agents arrived. Personally, I think there will always be a market for apps created with care and great attention to detail.
There is a large market for fine art. For everything else, there are printers and copiers. It's very difficult to maintain a career of painting and selling works for $100k. It's much easier to maintain a different career that happens to include printing things with little effort using an ordinary office printer.
That's where software development is going: of course you can still code by hand, but it will be increasingly difficult to get paid to do it. People will have a somewhat different career that happens to include coding things with little effort using an ordinary coding agent. Companies won't be able to afford to pay for expensive coders anymore (because otherwise their competition, who is getting along fine with a small software development budget, will take all their customers.)
For folks who are wondering:
This post is about Zotero. https://www.zotero.org/
If you are not an academic, you might not know Zotero.
It is such a pleasure to use. Every app should be like this.
I read everything in it, including books I'm going through right now from https://teachyourselfcs.com/
It also does an amazing job of taking snapshots of posts. I use it all the time to grab posts from HackerNews so I can mark them up.
And it automagically syncs everywhere across devices and lets me store way too many files on the web like the ADD packrat I am, without breaking a sweat.
In short, this software just works, and it works well. So when somebody behind Zotero talks about how to develop software, I listen.
And it's a fun post with some history. You should save this post to Zotero, and then read it.
Unrelated, but I love when people describe software with the phrase "just works".
It suggests a certain... simplicity that I chase in all of my developments. The amount of EFFORT to get software from "works but..." is interesting. You can get something that works 99% of the time, but that never quite fits the "just works" level of quality that we love to see.
Really, there's an immediate feeling of joy I get when I interact with software that "just works" knowing how much effort it takes to get something to that point.
It also depends what kind of software it is.
Like an application to book a haircut at a salon or log into the WiFi at a hotel that does something "on rails" and should work without any learning or training is one thing. A complex application for professional creative work is something else entirely.
Then there are the things that are just weird, like life got me looking at mobile games lately and I was really shocked to see how many of them have a horrific onboarding experience. The best games in the "clothes collecting" genre, for instance, like Love Nikki have a gentle onboarding process that teach you the game mechanics and get you into the story and maybe get you hooked. When I was developing consumer-facing social and entertainment apps (before Facebook!) I had my own belief that "easy onboarding is good for users and good for profits" and worked with founders who got it and pushed me to outdo myself. Most games in that genre have horrific UI and don't try to explain the mechanics because they could care less if users understand the currencies used in the system because they only care for the worst "whales" who click on any red dot and fold green at the slightest difficulty.
These days when it comes to "slow software", I think five years is an eternity from a platform perspective so if you start something you may need to rebuild it. Like I find it hard to justify developing on anything other than the web platform (like "write once and it runs on platforms you never thought of like VR headsets", "for christ's sake did you realize every Windows software vendor had at least one InstallShield engineer back in the 1990s and IT was pulling their hair out updating all their desktops?", "it took three weeks for the App Store to reject your update? what did you think would happen?") but the app I am working on right now depends on React components that haven't been updated since 2021 and the bill is coming due.
Oh man, the state of dress up games is a personal pet peeve of mine, poor onboarding tutorials is just one problem. The art direction is often simply atrocious, proper game loops barely exist (let alone creative ones), the microtransactions are the default business model. I honestly don't know what causes it, perhaps the games are just not that profitable so they don't attract highly skilled or at least conscientious game developers? I came to accept that they are just made by people who either dislike/don't care about games in general or dislike dress up games specifically. The best dress up games are normally made by Nintendo who have a strong game design and gave development culture in general.
P.s. As a side note, I started making small dress up games that I draw myself to scratch my own itch. I am not a programmer so they are absolutely horrendous and I only show them to friends. Making videogames is not as easy as making paper clothes for paper dolls like I did as a kid, but it is so fun and I feel 30 years younger somehow. :D
Yeah, I think those games have more potential than most people think. I love voting mechanics (spent a lot of time on ranking systems) and some have good stories ranging from "the heartwarming story of Nikki exploring a continent, making friends and proving her mad skills" to "a vicious fight for survival in an ancient court"
What's a good voting system? I remember checking out Dress to Impress (not a mobile but an incredibly popular Roblox game), and it had a somewhat unpleasant, imo, voting system -- players vote for each other's outfits and the score is a sum of the scores your co-players' gave you which incentivizies players to give everyone the lowest score. The game tries to "punish" this behavior by detecting "unfair" voters and lowering the weight of their votes, but I feel like you could tweak the original incentives instead by changing the mechanic. Having to rank outfits instead of giving each outfit a score could work I think? But there are probably other problems I'm missing. I dnn't play too many mutliplayer games.
(On the other hand, while I believe thinking about "gaming" the ranking takes away from the fun of the game, Dress to Impress is massively popular, so maybe this frustration is what makes it popular. IDK if kids play it because there are no competitors in the genre or because some competitive toxicity is addictive.)
Well I'm interested in voting theory from the viewpoint of political science and also from the viewpoint of developing ranking systems. It's actually a pretty depressing topic because you can easily come to the conclusion that the public doesn't have an interest at all.
In short, pairwise rankings are real and comparable between people whereas different people might use a scale differently. It's a difficult problem to deal with "i like the bagel shop a little bit more than the rice bowl restaurant but the party member who has celiac's opinion matters more than mine" See
https://brocku.ca/MeadProject/Thurstone/Thurstone_1927f.html
now if you have N options you don't have to have everyone evaluate N(N-1) pairs because preferences are correlated and should be transitive. There are many algorithms, like Elo and the methods in that paper that can be used to run tournaments and compute scores. The game Covet uses pairwise ranking and I think gets good scores.
To get back to social decision theory, it's probably not good for players to be in a team for purposes of winning and losing. Like I think sharing clothes with your friends is a really fun mechanic (might make you want to really buy or work for something!) but if people can help or hurt their friends you get into the whole can of worms of coalitions in N-person games.
Fascinating. Thank you or sharing.
I just quickly tried Covet and I can imagine how the act of comparing and picking one of the two options can feel fun. I can see myself just voting on outfits in the game without creating my own when I need some really simple "mindless" fun. Not a productive effect but also not addictive enough AFAIK to be a big problem.
I think teams can work as a concept but you need to design them with a great deal of care so that mechanics are pro-social and fascilitate friendships -- there's a nice article about it: https://www.gamedeveloper.com/design/game-design-patterns-fo...
What I dislike about Zotero is that it's a pain in the ass to self host. In my department I could spin up a local Zotero server for our PhD students and give them basically unlimited sync, but it's really finnicky
I use a locally hosted Wallabag, synchronized with all my computers, smartphone, etc.
Maybe Zotero does more than Wallabag, but I am happy with it so far.
What difficulties are you having? All you need is WebDAV. I use a nextcloud server for mine and my partner's zotero storage, and its entirely trivial to set up.
I have many issues with Zotero (which has been dumbing down at the expense of power users for some time) – but easy private sync is definitely not one of them.
WebDAV does not sync everything. With the amount of stuff faculty is syncing we would hit the storage limits with metadata alone
Really?
From Zotero's site:
https://www.zotero.org/support/syncLooking at my own Zotero account, I can confirm that all metadata storage is free and not counted (I am using none of my 300Mb storage in my account according to zotero.org, despite syncing metadata for several thousand bibliography items). The only storage Zotero seems to count is attachment storage, and that's all synced via my WebDAV.
What am I missing here?
The discussion is about self-hosting.
I thought the discussion was about self-hosting data, such that the provided free storage limits by Zotero remained sufficient? And for that use case, self-hosted WebDAV storage for file attachments is absolutely fine.
(The poster I was replying to was suggesting that their departmental users would go over their storage limits for metadata alone, even if they used WebDAV for attachments -- but that seems to be incorrect from my own experience, and everything I can find online.)
I agree. It's "open source" but I would put a huge asterisk on that label. It really reminds me most of AI models that are only open weights.
It’s a non-profit and the prices are very reasonable. I would pay them instead.
Why? I can serve it from my computer
I've been wanting to try it. In what way is it finnicky?
It is not supported natively (you can use a webdav server but it doesn't sync everything) and the third party options don't have the level of stability I'd need
In this day and age, I'd say this is hardly an excuse. Spin up your favorite harness and let the agents sort it out. :)
I was looking for software that would let me bookmark recipes (from web pages) and let me scrawl my notes on them. "Needs more of this or that ingredient" "Use less salt". This might be it.
It's baffling actually that Firefox doesn't let you annotate bookmarks, it seems such an obvious feature.
> You should save this post to Zotero, and then read it.
I might be dense - but I added this story as a URL to my library on Zotero on Android - and I don't see any way to read it "on" Zotero? I can apparently create a citation, however.
I could dig into their documentation but I'd rather ask an enthusiast. Is it still possible to get your own data out in a nice and easy format? I was once all-in on a note-taking app that slowly got worse and worse and it was a hassle to escape, so now I always consider it before starting.
If Zotero ever enshittifies or just disappears would I still be able to access my data and transfer it to something else?
User data exports are available as csv, xml, and latex, with original documents available in original input and also pdf.
It's also all open source and local-first. You can also add your own self-hosted WebDAV server for syncing across clients. Also, it's developed by a nonprofit organization.
> I was once all-in on a note-taking app that slowly got worse and worse and it was a hassle to escape
Taking a guess here, but Evernote? :)
Not the person you're replying it to. I had that happen with Microsoft's OneNote. I really liked it and I had no problems parsing the XML files it made so I could pipe data into other systems but then it went 100% cloud storage.
It was a classic example of "Microsoft made something surprisingly good, made people think it was crap because they stuffed five OneNote icons on your taskbar and three up your nose, and then they wrecked it", must have come from the same mind that thought buying Activision was a good idea.
Yes, but in addition to that, it is owned by a non-profit who hasn’t made a single controversial update in 20 years, so it’s got a pretty decent track record.
I have Zotero set up with some plugins that create a .bib file I create notes for in Emacs. I enjoy this setup because it allows me the convenience of saving files quickly using Zotero, but saves all the data it extracts to an open format with an auto export. If Zotero were to disappear tomorrow, I’d still have all my notes in org files and my citations in their bib file.
Don't suppose you have blogged about this? Or have recos for resources on combining Zotero with emacs? this sounds very intriguing!
Ty! I suspect this will replace some subset of this for me:
- Calibre Ebook reader - Browser bookmarks for article sites - Reference manuals/docs - Biology papers of interest
The guerilla advertising on HN is probably the most transparent lol
> But that slow formation led to software that was durable rather than ephemeral, with a strong foundation that could be built upon.
People say that agentic development is great because you can churn out so much so fast. But that doesn't mean that any of it will be truly good and reliable.
The things that are truly insightful and solid end up being used exponentially more, which makes the linear cost of extra development time (asymptotically) insignificant in the cost/benefit equation.
The emphasis on quantity while neglecting quality calls to mind a passage from EWD1175[0]:
> My second warning remark is that I shall refuse to discuss the academic enterprise in financial terms. The first reason is that the habit of trying to understand, explain, or justify in financial terms is unhealthy: it creates the ethics of the best-seller society in which saleability is confused with quality. The other day we had to discuss the professional quality of one of our colleagues, in whose favour it was then mentioned that one of his Ph.D.s had earned lots and lots of money in the computer business, and few people seemed to notice how ridiculous a recommendation this was. We also know that the financial success of a product can be totally independent of its quality (as everyone who remembers for instance the commercially successful IBM360 should know). The second reason for my refusal is that the value of money is a very fuzzy notion, so fuzzy in fact, that efforts to understand in financial terms always lead to greater confusion. [Remember this, for it is quite likely that this afternoon will give you the opportunity to observe the phenomenon. Note that money need not be mentioned explicitly for the nonsense to emerge, a reference to "the taxpayer" can do the job. The role of "the taxpayer" then invariably leads to the conclusion that of State Universities at least the undergraduate curriculum has to be second- or third-rate.] The final reason for my refusal is that the habit appeals to the quantitative mind and I come from a culture in which the primarily quantitative mind does not evoke admiration. [A major reason that we considered Roman Catholics to belong to a lower class was precisely their quantitative bent: they always counted, number of faithful, number of days in purgatory, you name it.....]
[0] https://www.cs.utexas.edu/~EWD/transcriptions/EWD11xx/EWD117...
Everyone I know who used the 360 has a lot of nostalgia for it.
It was well thought through. I mean, the 360 architecture is still with us because it avoided the traps that killed the PDP-11, VAX and the 68k -- as much as I loved the PDP-11. It's true when it came out that nobody had any idea what a general-purpose operating system looked like and it took a decade for them to productize VM so that you could run as many operating systems as you needed simultaneously.
The aside at the end seems a bit odd given the certainly "quantitative" bent of the Dutch Calvinist merchant class...
The trade-off between quality and speed of development has always been a tenet of software engineering. Agentic coding changes the equation a lot, but the equation is still there.
> The trade-off between quality and speed of development has always been a tenet of software engineering. Agentic coding changes the equation a lot, but the equation is still there.
The quality and speed tradeoff doesn't matter if other incentives aren't aligned. In today's product-driven world, where usage stats are compiled in real-time, features are added and then pop-ups, nudges, etc are added to software to get product usage up. These features may or may not add quality, relevance, etc. But the "success" is measured in usage, forced or not. This is essentially Microsoft today (especially with copilot), but even after every major apple OS update, I get "what's new" pop ups for every Apple app (notes, reminders, mail, etc) on every platform (iPad/iPhone/Mac) that I own. I opened the email app to check my email, don't get in my way!
Agile and/or web based app killed software quality.
/change my mind.
It may be so but the article does not claim that the strong foundation is the code, rather it seems to be product design, and design of other products, at that (the two predecessors, webapp and desktop app). No reason why you couldn't study existing products now and tell your agent to build something based on that.
This feels like a sleight of hand to me. The hard part of evolving Scribe and Web Scrapbook was discovering that a browser extension manipulating a local SQLite database was _the only_ architecture that could reconcile local offline persistence with live DOM scraping across arbitrary catalogs of academic data.
An agent can synthesize existing solutions but (because I see this failure mode at work constantly) it can't synthesize an architecture to resolve the sorts of tensions that the person prompting it doesn't yet understand (not that that is stopping anyone). You can't prompt it to build something if the operational primitives required to solve the problem haven't been mapped.
"Build a tool based on Scribe and Web Scrapbook" in 2003 would've made a fragile PHP wrapper because that's what the existing landscape looked like.
Yes, exactly. And this is why I don't think that the LLMs can make significant process beyond what humans have done and published.
"But the math proofs," people will say. A lot of those seem to be spam-solving things with a huge swath of existing lemmas, and a some of these are being debunked and retracted.
Just today I was quizzing ChatGPT about a basic grammar question for a language that has huge training data but for which the grammar was not well documented. It kept giving me confidently wrong answers until I drilled and drilled it and then finally it found/gave back an explanation that perfectly fit a pattern given in one particular grammar, citing that as a source. It doesn't appear to have been able to figure out the inner structure on it's own. It appears only able to pattern match and put things together from what humans have already discovered and written.
Right, there's a difference between statistical interpolation and semantic induction. The whole point is that LLMs can't reason from first principles to drive missing rules. It keeps confidently feeding you approximations until it collides with some source that already mapped it.
Interesting take, given that the whole reason LLMs are interesting is that they're the first system we have that can work in semantic space. Statistical interpolation, that we've solved long ago.
I think that's equivocating on the word semantic. Word embeddings map concepts like "king - man + woman = queen" or cluster synonyms together in high-dimensional vector space and the ML literature very loosely calls this "semantic space." But a high-dimensional topology of token co-occurrences isn't semantics in the sense of computation or formal semantics. It's still "just" measuring distributional similarity. Some vector that represents "thread deadlock" lives near tokens like "mutex" and "race condition" and "starvation," but the model itself has no concept of concurrency and contention.
Claiming we "solved" statistical interpolation long ago just means curve-fitting and basic regressions on structured data. Transformers are a truly impressive achievement, scaling all of this to unstructured high-dimensional text topologies, but it's fundamentally the same math operations on statistical proximity.
Like how do we explain hallucinations here? Tokens that are hallucinated are semantically "close" in that vector space but they're completely false in reality. If LLMs operated in a true semantic space they wouldn't hallucinate CLI flags that don't exist.
This guy knows what's what. Excellent input here on this thread.
> Some vector that represents "thread deadlock" lives near tokens like "mutex" and "race condition" and "starvation," but the model itself has no concept of concurrency and contention.
I propose that concepts of "concurrency" and "contention" are themselves vector in latent space. All concepts are. Recall that we're talking about a 10^4 - 10^5 dimensional space. You can fit in pretty much any conceivable association as some direction in there.
And try to zoom in on any concept you know. If you do, it should quickly become apparent that there's never any concept you can give a closed definition for. We can only define concepts, and we can only learn them, through generalizing from examples. Which is conceptually (pun not intended) regression - finding a vector along which examples live.
As somebody responsible for the acquisition of users and growth of a company in terms of customer and revenue, this is a VERY salient point:
> “… we could not have accelerated Zotero’s conception, because we did not know exactly what we wanted, and so could not have written coherent prompts for an LLM.”
A surprising proportion of software products, maybe even businesses today, are solutions in search of a problem.
Sometimes that’s okay, but only sometimes. And being a solution in search of a problem requires you to get everything /else/ pretty much perfect if you want to succeed.
The fact Zotero paid attention to what people wanted, and gave it to them, and were market oriented, is demonstrably a big part of their success.
It is MUCH easier to make something people want, than to make them want something you made.
Indeed, the solutions in search of a problem are teeming in the day and age of LLMs! I think the opposite framing is quite interesting aswell "solutions in search of a problem" -> "problem in search of a solution" as in solution are not always self evident and what might work for one might not work for someone else, striking the right balance takes time. And to me this article highlight this, they are academics that are well aware of problem at hand, but still, they can't easily formulate a solution to it.
Agreed, but it is also much easier to get something made in the first place. LLMs enable rapid prototyping and rapid shifting to solve real problems. My applications are morphing from mediocre to highly useful problem solving machines much more rapidly now.
That’s a really good point! But I am always surprised by how often people will avoid putting their prototypes out there and let the market shape their product.
Rapid prototyping is an amazing opportunity afforded to us by AI. But some people use that potential to spend even longer on a more developed prototype that they are too attached to to get feedback on!
> But I am always surprised by how often people will avoid putting their prototypes out there and let the market shape their product
I see the opposite. I see thousands and thousands of quickly produced apps that have been put out there, but have 0 users that could shape it. Not so much because of their quality, but rather distribution in a very crowded and shouty marketplace.
> Today, software can be created more or less instantly with AI, for a large user base or for yourself, for any purpose or for no serious purpose at all.
I don’t think this is true. I’m working on medium to large features in companies that have around 1K engineers. These features typically involve 50% of work within your domain plus 50% of work in dependant domains. You cannot get anything doing without previous alignment with such domains. There are discussions, tradeoffs, design docs, approval committees, etc. AI can (and does) help in every step, but it’s not a magic wand that can solve the whole thing with a well crafted prompt. Ithe hands of inexperienced people, AI slows down things (e.g., ai-generated robotic and lengthy slack messages that go nowhere, PRs that implement what a jira ticket says… but jira tickets without alignment are worthless, etc)
good things take time because they grow from something like seed. as that seed grows, it figures out its local and global context. a curious and patient caretaker of this seed will spend a lot of time looking at it, understanding it, trying to figure out the right way to give the small plant a steady foundation. with care and attention, it could grow into a tree and attract all sorts of other insects, animals, and all sorts of life.
speed kills quality. it’s literally impossible to make anything good fast.
we know this, and it still applies to software. while we may be able to make things faster, they will never become good (or great) without an incredible amount of care, patience, and joy from its maker.
there are no shortcuts to quality. it will always take a lot of time to make anything good.
I am not sure it IS impossible to make anything good fast. Look at how many incredible songs have been made in a single afternoon, or fantastic ideas for a simple product came to somebody in an instant.
Your comment made me think two things:
• Sometimes constraints make things better, and ‘speed’ can occasionally make you prioritise the things that actually matter so you deliver stuff that counts
• Sometimes, thinking longer about something doesn’t get you closer to the correct answer. You can rearrange and refactor and rewrite and redesign, but you won’t always get something objectively better than what you originally came up with. It’s still your thoughts, your brain and your ways of working that shape the output (and they haven’t changed).
Investing more time to make something better should be a conscious decision. Perhaps it’s one people decide against for the wrong reasons.
But sometimes things need something other than time and effort spent to make them better.
Keep up the good work. Your contrarian comments on this thread are saving me from wasting time being the contrarian myself.
Is Zotero good software? Yeah, it seems pretty good. I myself don’t use it, despite trying, because it didn’t solve my problems. It is free, has always been free, and its users are primarily academics who do not face market economics for their work output.
Speed is not the enemy. A good engineer knows that time is a precious resource, just like any other, and should be valued appropriately. And engineers are not academics.
Most good things are made fast, because once you slow down below certain degree, things won't get made at all.
Over the two decades of writing software professionally, I learned to appreciate the perspective of software companies: most software is crap, and without pressure to ship, smart programmers will forever keep polishing the turd, way beyond the point the software stops being relevant or useful.
The best, high quality, slow-developed software isn't the best because programmers knew when to stop. It's because the scope and timeline were bounded up front.
I agree with this deeply. I’d even go as far to say I don’t think it’s just software this is applicable too, it’s perhaps any body of collaborative, thoughtful and developed work
Regarding the song example, it may take an hour to write, but weeks, maybe years of material build up in the subconscious.
> Look at how many incredible songs have been made in a single afternoon
Ideas for songs. Good ideas for songs can come in an instant. And in the hands of an experienced songwriter, you can take that idea to a polished product very quickly. But that's because they spend years developing taste.
> fantastic ideas for a simple product came to somebody in an instant
What's the saying around here? Ideas are cheap. It's execution that matters? And the execution part often comes about through trial and error.
> But sometimes things need something other than time and effort spent to make them better.
I don't think anyone would disagree with this.
Nonsense. Plenty of VERY good software that has been built at immense speed. Look at Muse as an example.
> If AI had existed in the early aughts, we could not have accelerated Zotero’s conception, because we did not know exactly what we wanted, and so could not have written coherent prompts for an LLM. Instead, it took a great deal of time and collaboration to develop a clear vision for what Zotero should be.
AI doesn't preclude this. In fact, it can help accelerate parts of it.
He's describing the typical big project lifecycle:
- Examine the landscape
- User research (how they use existing software, what their frustrations are, etc)
- Brainstorming
- Early ideas and prototypes
- Refinement, user feedback
- Solidify the vision and high level process design
- Choose technologies
- Design & architecture
- Plan out phases
- Build phases, then test them with users
LLMs are great at research, and great at prototypes. Once you have your design, they're good at coding as well. They're also good at distilling user feedback.
> AI doesn't preclude this. In fact, it can help accelerate parts of it.
It can also slow it down as people get distracted experimenting with features they can build quickly.
That's a discipline and hubris failure, and has bitten many people even before LLMs were a thing.
Most V2 rewrites fall under a similar category.
The problem is that in the future, LLMs will change society so much that we won't even want to go after these things any more because the world will live and die on cheap prototypes and trends.
It's like all the food being replaced by MacDonalds food that is served from a replicator (from Star Trek). Sure, we can survive on it, sort of, but if all we ever know is this food and we're too busy to cook (or no one knows how to manufacture a stove) then we'll never know anything else. Maybe someone will learn how to tweak the recipe a little but our minds will be conditioned to context-switch so much that we'll never even think of learning to cook great meals.
That's what LLMs do. They're shit.
We've already gone through that many times with COBOL, Excel, Visual Basic, PHP, Javascript, etc.
Whenever a new technology makes what used to be hard easy, all the non-experts pile in and produce shit for awhile, until they're finally forced to admit that software engineering is hard (or die trying).
The same will happen with LLMs in a couple of years.
Star Trek isn't the best example, because many characters on the show like to whine about replicator food having bland taste, and treating cooking as important skill to have (though I'd argue it's for social reason).
I can't speak for everyone, but the software I've built via agentic coding still takes a long time to build, refine and iron out all the bugs. I still puzzle out how to solve issues that the agents can't deal with. I still find it challenging and enjoyable.
There seems to be this pervasive false dichotomy about software currently: that it's either lovingly hand-tooled and crafted and perfect; or that it's a vibe-coded one-shot sloppy-slop-slop-fest of spaghetti code.
There is a middle ground, where rigorous software project management and careful architectural guidance and oversight continues, but agentic coding takes away much of the coding drudgery. It takes time, and it's not even remotely one-shot, but it's still way faster than coding everything by hand. From what I can tell, this quiet approach is where most agentic coding effort is going currently -- it's not flashy, it's not sloppy, and mostly you won't even notice.
I love great software and agree that great software is evolved - not built. The whole point of building v0.1 is to figure out what's wrong and should be fixed in v0.2
At the same time, there are broadly three motions in the loop: - thinking/discussing (what should it do) - building (Make it do that) - using (Seeing whether that is actually what it SHOULD do)
And then of course you repeat until you run out of time, money, patience, volition, etc. For some software there is a terminal state - it truly does exactly what it should. For most you're always reaching for it.
LLMs definitely accelerate 2 and potentially can help accelerate 1 and 3. As such cycle time can be reduced. It still may take 100 turns to get what you want, but I'd be surprised if the clock time for the 100 turns would be unchanged using AI.
> LLMs definitely accelerate 2 and potentially can help accelerate 1 and 3.
People that do think of using LLMs to accelerate 2 usually don’t think enough about 1. And from what I’ve seen, they quickly get tired of 3.
Most quality software I’ve seen usually starts from a small subset for the cycle and then incrementally add to it. LLM projects usually rush the building part and do too big of a job and it’s become cumbersome to design (sunken cost) and evaluate (too many variables). If you want to build shelters, you start with a small hut, not with a cathedral.
> Most quality software I’ve seen usually starts from a small subset for the cycle and then incrementally add to it. LLM projects usually rush the building part and do too big of a job and it’s become cumbersome to design (sunken cost) and evaluate (too many variables). If you want to build shelters, you start with a small hut, not with a cathedral.
What on earth makes you think that you can't adopt this exact approach with agentic coding??
This is like saying that you shouldn't drive a car around suburban streets, because the maximum speed it can do is in excess of 250 kph. You absolutely can (and arguably should) take agentic coding slow and steady, and build software the way you always did. Like using a car to get to the shops at 50 kph, you'll still get there much faster than if you walk.
> What on earth makes you think that you can't adopt this exact approach with agentic coding??
It can be done, but it’s not where the hype and the practice is. It’s all about number of commits, number of PR, churns in LoC, which has no bearing on software quality and usefulness.
Something like caddy[0] is just 2700+ commits over 7 years. That’s like 32 commits a month in average. Even if you apply an exponential decay (going from greenfield to mature project), the latter years would have been way peaceful. I know AI can help in some cases, but is it such a pressing need. That would been like taking a car to go somewhere less than 10m away.
Most (mature) open source projects progress a few changes at a time. Most of the time is about ensuring that nothing breaks due to the change.
[0] https://gitlab.com/caddy/caddy
I completely agree. And yes, I think the current game-ification of repositories (more stars! more PRs!) is pretty counter-productive.
I suspect that projects that run hot and fast with LLMs will be very much like driving your car at 250 kph to the shops. You might get away with it briefly, but you'll ultimately crash and burn. As with all things -- the obsession will fade soon enough, and agentic coding will become just another tool, where those who use it well will be valued. At least, so I hope. We will see, I guess.
But if you want a cathedral…?
> But if you want a cathedral…?
How many cathedral does the world needs? And even if you wanted to build a new one, it’s no longer the middle age. No one is inventing UNIX all over again. Lots of hurdles have been solved. The difficulty of software is not technical or implementation related, it’s mostly about defining the problem.
In the Netherlands we have this saying: “Without friction, no shine”
Durability is mostly boring decisions compounded: open formats, exportable data, and no dependency on a vendor mood. Speed wins users, but those choices are what keep them.
The reason great artists spend so much time developing their skills is that the result no one knew they wanted comes from skill and the new possibilities emerging from that.
So as a software developer now might be the best time to throw system design on it's head and develop without a direction but make sure everything is as good as one can forge it.
Surprising functionality and stuff not found elsewhere then more or less simply emerges from that.
Granted - that has a certain freedom and no pressure to make money as a prerequisite but so do the 5 years noted here.
This seems like a highly romanticized retelling of the origin story. LLMs would have more likely made that 5 years into 5 months. LLMs are excellent at accelerating work where you don't know exactly what you are after.
They will also take you down a road you didn't want to go down at a breakneck speed too. And you don't realize you didn't want to go down that road. Sure once you figure out you didn't want that it is quick to go back and start again, but how many times do you need to do that until you get where you wanted to be? Are you really saving time in the end?
Yes, but you job is to know the difference.
I’m developing a software solution that integrates w/ Zotero (sortcite.com). Their API is very easy to work with. Some of their competitors (Endnote) don’t even have an API.
A very very strong point. I have personally detailed entire projects because adding a feature seemed easy with AI. It is very difficult to vibe code and not add a bunch of useless crap features.
Do you mean "derailed?"
You can also remove features with AI faster than before. During the initial development phase, that's the most important part IMO
Really? I find LLMs quite bad at deleting code. If you ask them to add a feature, then later take it out again, the codebase still grows. It always grows. Every time I've tried it, llms have failed to simplify code via refactoring. Even fable is incredibly bad at this for some reason.
LLMs are excellent at making prototypes though. And prototypes can be an excellent way to stop yourself from implementing the wrong features in the first place.
> the codebase still grows. It always grows.
Well, yes. I mean, it's in the name Generative Pre-trained Transformer: they're text generators!
To delete text using a text generator, you have to emit the original thing taking care to omit the deleted stuff during emission. It's more work.
We need ChatDPT
> The `rm` command was released on November 3, 1971, as part of the First Edition of Unix (Unix) by Ken Thompson and Dennis Ritchie
We've had that since 1971
> To delete text using a text generator, you have to emit the original thing taking care to omit the deleted stuff during emission. It's more work.
What are you talking about?
A small footnote regarding the intersection of this fascinating article and current(ish) events.
> That winter, with Roy as the principal investigator and Josh and me as co-directors, we applied for a grant from the Institute of Museum and Library Services.
Per https://en.wikipedia.org/wiki/Institute_of_Museum_and_Librar... :
> The Institute of Museum and Library Services (IMLS) is an independent agency of the United States federal government established in 1996. It is the main source of federal support for libraries and museums within the United States, having the mission to "advance, support, and empower America's museums, libraries, and related organizations through grantmaking, research, and policy development"
You can guess the next part, I’m sure:
> On March 14, 2025, President Trump issued an executive order virtually eliminating IMLS that directed that "the non-statutory components and functions ... shall be eliminated to the maximum extent consistent with applicable law, and such entities shall reduce the performance of their statutory functions and associated personnel to the minimum presence and function required by law", along with minimizing several other agencies. The entire 70 person staff was put on leave on March 31, 2025.
> On May 1, 2025, a lawsuit brought by the American Library Association and the American Federation of State, County and Municipal Employees resulted in the U.S. District Court for the District of Columbia granting a limited temporary restraining order to block any further actions to dissolve IMLS.
Couldn't this whole Institute of Museum and Library Services be replaced by an LLM?
No
This post smells a lot like academia. I say that as someone who has spent quite a bit of time there myself, so I understand where the author is coming from, but “ivory tower” is still an apt metaphor.
Slow software that works for 10 years beats fast software that breaks in 10 months. Choosing boring tech is underrated.
How about fast software that works for 10 years?
The difference between an academic and an engineer is that the engineer understands that time spent in research and design does not magically justify itself. Quality and time-to-market are both important and to be an engineer is to figure out how to deliver along the optimal frontier.
I don't think this is possible in this day and age, everyone expects the development to be instant.
I guess we'll only be able to verify this 10-15 years after coding agents arrived. Personally, I think there will always be a market for apps created with care and great attention to detail.
There is a large market for fine art. For everything else, there are printers and copiers. It's very difficult to maintain a career of painting and selling works for $100k. It's much easier to maintain a different career that happens to include printing things with little effort using an ordinary office printer.
That's where software development is going: of course you can still code by hand, but it will be increasingly difficult to get paid to do it. People will have a somewhat different career that happens to include coding things with little effort using an ordinary coding agent. Companies won't be able to afford to pay for expensive coders anymore (because otherwise their competition, who is getting along fine with a small software development budget, will take all their customers.)
Do it in your own projects.
Is "durable" the buzzword of the moment? I'm a bit lost here. I've seen it a lot lately but in wildly different contexts.
No, it's just a word for which LLMs have an inordinately large bias, so it's often found in blog posts written by one.
Durable as in having a durable place in the human lived experience rather than durable as in durability of state or workflows
So pleasant to remember a world where this photo:
https://assets.buttondown.email/images/93382906-4996-445c-81...
is just of some passionate academics working on a project with no real economic or social media incentives driving it.