I'll go out on a limb and say we _need_ Wikipedia and it's okay that traffic falls.
Physical print encyclopedias got replaced by Wikipedia, but AI isn't a replacement (can't ever see how either). While AI is a method of easier access for the end user, the purpose of Wikipedia stands on its own.
I've always scoffed at the Wikimedia Foundation's warchest and continuously increasing annual spending. I say now is the time to save money. Become self sustaining through investments so it can live for 1000 years.
To me, it is an existence for the common good and should be governed as such.
The Wikimedia Foundation is unlikely to last for 1000 years as an organisation because it doesn't exist within a social / economic system that will last for 1000 years. The US government is already actively plotting against it. Sure, they can try hopping from one country to another, but it won't be sustainable for that long.
It's not even certain if Wikipedia itself can exist for such a long period, given fragility of technological civilisation and data storage.
The Wikimedia Endowment [0] has been created for this. From it's Financials [1] it mentions that
> ... [its mission is...] to act as a permanent fund that can support in perpetuity the operations and activities of current and future Wikimedia projects, which are projects that are approved by and advance the purposes of the Foundation or its successor if the Foundation ceases to exist
The Wikimedia Endowment (which is sorta-kinda separate) is to drive solutions to that exact problem.
By having a separate fund that the Wikimedia Foundation can access to help Wikipedia to have the technical expertise and knowledge workers required to continue the work of the Wikimedia Foundation.
Should the Wikimedia Foundation cease to exist, the funds in the endowment can be redirected to a successor.
EDIT: this is similar in style to the UK's Guardian Foundation, who provides funding to The Guardian newspaper. https://theguardianfoundation.org/
The point is that without Wikipedians adding and improving articles, Wikipedia will die, even if the site could be kept up indefinitely. So it has to remain a relatively wide social phenomenon, an obscure Wikipedia that no students know about and care to use will knowledge rot, even if it doesn't but rot.
There are a million backups of wikipedia. That's not the main issue, it's the future of the project. If the main org goes down, there will be a million forks of it where none of them have the same legitimacy of the original, edits will die off, views will die off, and the content will become increasingly stale and vandalised.
I see that as an argument for serious research effort into systems of decentralized governance. A project like Wikipedia shouldn't need a centralized community or a singular centralized host. Unfortunately "web of trust" doesn't seem like it will be resolved in a robust manner anytime soon.
In 1000 years Sweden went from a Viking culture to an egalitarian, tolerant, multicultural society. Just imagine how much change you can have in the next 1000 years. 50 years ago you guys had basically an apartheid and women’s place was the kitchen.
The problems with that argument are that first, the Viking culture was mostly rural and agricultural, the raiding and pillaging was a tiny side-effect that mostly happened during times of economic hardship, not as a way of life ... and that today's Sweden is rapidly recessing back to a less-tolerant, anti-multicultural society for many reasons. You also conveniently ignore the much more bloody 'military superpower' phase in the 1600s.
It’s not Ragnar Lodbrok turning into ABBA. It’s a mostly peaceful peasant society getting tossed around by history until it became a mostly peaceful service economy - and the shaking never really stopped.
History is not a straight progress line. Ask the Romans.
If you really want to be accurate about history in what is now Sweden, let’s also mention the horrific mass graves discovered in Birka and surroundings which seems to imply that massacring whole villages, women and children included , was commonplace even before Viking times. The later imperialist period seems less brutal to me.
Pre-agricultural societies are generally much more brutal because you need a lot of territory to feed relatively few people. Agriculturalists can conquer other people and make them work for them in a way that doesn't scale for hunters-gatherers. The consequence is that the combatant/noncombatant distinction is much more blurry, and genocide is often an explicit goal of the war.
Yes! It annoys me to no end when people pretend that time is developing us into a perfect image - we're just the latest generation in a long string of improvements. Making it seem like development is predetermined.
Ask a medieval peasant and he wouldnt even be able to describe how his life is different from his father's.
35 years ago South Africa had the apartheid, and now they outrank the US on a The Economist's Democracy Index [1]. So yeah, things move fast.
Nitpick: The US Voting Rights Act became law in 1965 [2], which is more like 60 years ago. Not sure if that's what you were getting at with "basically an apartheid" but it was the closest concrete landmark near your 50 year timeline.
Yeah I think I should have been more clear that countries change a lot and sometimes even revert back to a much earlier state … it just sounds funny to me how the country of bloody thirsty warriors that pillaged faraway countries is now maybe the most peaceful place on Earth… at least if you ignore the bombings and gang culture that has developed in the last decade or so, but those are imported from elsewhere, which of course doesn’t mean it won’t become a feature of the country for decades ahead which may bring back some of the image of ruthlessness associated with the old times.
I agree with you. The main problem is that we don't have many politicians that want to tackle immigration in a serious and responsible way. So people end up giving votes to the far right.
Lol the pace things change. If there was an accurate time travel movie it would be more of a comedy. I don't care where you're from. Which language you speak. Even Chinese has changed. But any language especially English has changed so much it would be a foreign language if you went back.
> The US government is already actively plotting against it.
If we're going to take the actions of a couple of low level dope legislatures as, "the US government" and a toothless investigation as "actively plotting against", sure.
Even if this current plotting comes to nothing, there are plenty of ways for investments to fail over 1000 years. Even if we assume that the system continues intact so that the accumulated wealth retains its meaning and is protected, and it isn't directly targeted by the government at any time.
Investments can fail, such as bank deposits (bank collapses), shares (company goes bankrupt), government bonds (government defaults), commodities (price fluctuations as they go in or out of favour.)
Theft can also occur, including by corrupt insiders (sometimes even legally, just by inflating their salaries to ridiculous levels.)
The chance of remaining intact for 1000 years seems very low.
> Become self sustaining through investments so it can live for 1000 years.
I always wondered why more companies or organizations didn’t do this. Pile up money during the good years to allow themselves to not need continued outside income to keep going, so they can do what is right instead of compromising their vision for the sake of hitting quarterly earnings. That isn’t to say they can’t keep making money, but do it for the right reasons that will keep the core business around for the long run.
Unfortunately, Wikipedia doesn't seem interested in this. Their revenue is more than enough be able to invest and sustain the site forever, but they just increase expenses on non-core outgoings https://en.wikipedia.org/wiki/WP:CANCER
Am I misssing something or is Wikipedia net positive 200M? If this is true, that's eye opening. Weird given how much they beg for money on every article.
The amount of money moving through Wikipedia is absolutely mind blowing, and next to none of that is even being used to sustain the site. Hope those still donating to it feel smart about what they're supporting...
Because that's how you build up a foundation that will be self-sustaining when the donations run out.
It's not a bad strategy. I've looked at Wikimedia's financial statements and have no problem giving a small monthly amount to them considering how much value I get from the site.
I certainly prefer my money going to them than to Zuckerberg or Altman or MSFT shareholders.
You pay for Wikipedia because you want it to prosper. The political part of Wikipedia is vital to make it succeed. Even though everyone uses Wikipedia not everyone knows why it succeeded and why it is important.
The fluff is important to have a engaged super users. It is also important to get acceptance in certain circles.
> Pile up money during the good years to allow themselves to not need continued outside income to keep going, so they can do what is right instead of compromising their vision for the sake of hitting quarterly earnings.
Typically, individuals want to pile up money so that they don't need outside income to keep going, and the shareholders of a "quarterly earnings" company will squeeze the entity to get it for themselves in the form of dividends or higher share-price.
Money locked up in companies is managed at the leisure of its management, regardless of how good that might be for anyone else (ie. the shareholding owners). Money sitting there doing nothing is a poor use of it, so most companies are encouraged to run with only enough working capital to keep operations going.
It sounds weird, but this is better for shareholders and the economy (and companies can raise capital as needed down the line) than having all companies hold 3x the cash on the balance sheet.
The argument would be different for a foundation by wikipedia, albeit you still have problems between what the wikipedia management might want (high wages, little accountability) and everyone else.
> Money locked up in companies is managed at the leisure of its management, regardless of how good that might be for [...] the shareholding owners
But the management is aware that the shareholders can apply (direct or indirect) pressure for the money to be used in certain ways. Ultimately the shareholder can sue the management if they think the money is misused.
If a company retains earnings, it has to pay taxes on them (as profits), but the money is still at risk (from a shareholder’s point of view) if something bad happens to the company (lawsuit or market problem). Shareholders usually want to receive whatever money the company has saved up, to safeguard it from being lost for no reason, and so that they (the shareholder) can put it to use elsewhere. This would change if the government stopped taxing retained earnings.
Retained earnings are not taxed per se. A company pays taxes on profits. Whether the profits are distributed to shareholders or retained makes no difference whatsoever as far as taxes are concerned.
They are taxed; they are taxed because they are a subset of profits, which is a taxed category. They are not taxed more than other profits, but that doesn’t mean they’re ’not taxed’.
It would definitely change, because a shareholder could then take a loan against the non-taxed retained earnings they are owed. So then it comes down to whether the tax is higher or lower than the loan interest (adjust for the risk you mentioned).
I think some NGOs and privately owned companies actually do this. But most companies exist to increase the wealth of their shareholders, so if they don't consistently grow, they serve no purpose. If NVIDIA kept its exact current revenue and profits for the next two years, their CEO would be fired and their share price would be in the gutter.
Because unless the company is temporarily making losses because of temporarily macroeconomic conditions, what’s the point?
Having a war chest wasn’t going to help any of the retail companies, the technology companies who couldn’t pivot, etc. There is no reason for a company to live forever.
“A society grows great when old men plant trees whose shade they know they shall never sit in.”
I recently visited Scotland and on a visit to a distillery they mentioned they bought land in the US to grow trees that will make their barrels one day. The trees take over 100 years to grow (if I remember correctly). How is it we can invest ~200 years into a glass of scotch, yet we aren’t willing to take the same care and long term thinking in most other areas.
Even without being around for 1,000 years, I’d think doing this would de-stress and de-risk. Somewhere along way it became a bad thing to have a good, stable, long-lasting business. The only thing that seems to matter now is growth, even if they means instability, stress, excessive risk, and a short stay.
Craftsmen care about what they make, pursuing excellence even if it may take generations. Businessmen care about robbing you now in the most efficient way.
There is also an element of tradition with scotch, which is something that can survive even the occasional craftsman who may not care as much about what they do, as long as they respect the traditions.
Humans don't live that long, and there's a constant onslaught of fleeting fancies, especially in business (Wikipedia foundation should buy some crypto for it's treasury!)
Tradition is simply brand value to be monetized for most businessmen (to add to your criticism). Just look at scotch whisky and multinational conglomerate acquisitions. They would never plant trees in America, they simply order giant vats used for the strongest PX Sherry to get maximum flavor per euro for their blending process.
Some ancient family dynasty Scotch distillery perilous existential
vulnerabilities:
[] Indifferent, spoiled rotten progeny seeking maximum return upon
inheritance, selling distillery to Seagram's for an immediate
gratification windfall fortune.
[] John Cooper VII, the last barrel maker, skill lost at retirement,
or Master Cooper VIIth lost savings to Mister Market, or an expensive
clandestine affair, extorts 16X per barrel.
[] Seagram's hires brilliant Bill Burr(Breaking Bad car wash business) to pose
as EPA agent, threatening federal lawsuit for illegal violation of
dumping distillery toxic runoff for past centuries, and/or white oak barrels
cause cancer and the distillery has killed victims for hundreds of
years.
[] A society grows fallow when old men plant trees whose shade they
know they shall never sit in, while the president's daughter
threatens to enforce federal white oak forest's toxic(false, but that's ok!)
leaching prosecution if landowners do not immediately purchase a
fortune of President altCoin.
In this painfully craven hostile world, Benedictine liqueur would seem to be more durable spirit than any Scotch.
While that is technically true I have always thought hn as a mere place to find interesting content. Of course many of us, me included, dream about finding someone who pays millions for our code. But the attitude towards businessmen is generally hostile here and for a good reason. They can't make things and they aren't even interested in making things, and overall they are more often than not just lame and boring. They are interested in making money, ripping people off, being leeches, extracting every possible penny they just can. Weird geeks are what the hacker culture is all about and politically correct c-suite suckers can go pound sand.
Scotch is aged in previously used Bourbon barrels from the US. Those Bourbon barrels are harvested from local forests (in Kentucky) which are very large and healthy. The probable reason they purchased land was for an investment or perhaps an alternate income stream or even for some sort of government tax break or credit. They may have told you it was to make barrels but it isn't at all likely that will ever happen. Good story though...
I've always wondered if increasing everybody's lifespan would result in more people caring more about the long term future. Our lives are so short in the grand scheme of things that we never bother to really thing about the grand scheme of things.
I've heard that in the Tibetan tradiation where they ardently belive in reincarnation that thinking about the consequences of your actions for the next thousand years isn't unusual
There should be many offline copies of Wikipedia since it makes data dumps publicly available
Perhaps Wikimedia is only mentioning a drop in traffic as it suggests how www users may be gathering information, not because it is commercially significant, e.g., decline in audience for advertising
Indeed, it is okay for traffic to fall when a website is not trying sell out www users to advertisers. It does not mean the information offered by the website is any less valuable
Wikipedia, while not being reliant on ads, nor even on donations that much (given how much they have amassed) is still 100% dependent on having active users. People who get a ChatGPT answer will not go on to edit Wikipedia if that answer happens to be wrong. They won't curate articles to notice when someone is vandalizing them and put the right info back.
"Popular" information, evidenced by web traffic for example, may be important for so-called "tech" companies seeking to profit from intermediating access to free, public information, i.e., acting as a middleman. The information may be valuable to the so-called "tech" company middleman for purposes of supporting data collection, surveillance and online advertising services
However, the so-called "tech" company does not determine the value of the information to others. For example, a small group of people may find information on a web page to be valuable for their individual purposes. The web page may receive little traffic. The amount of traffic does not determine the value of the information to the small group
I think number of "active Wikipedia editors" is more important than number of "active Wikipedia users"
"AI search" might cause a decline in active users, but I cannot see how "AI search" would cause a decline in Wikipedia editors
Many Wikipedia editors are responsible for low traffic pages. I would imagine they are motivated to maintain these pages due to interest in the subject matter not interest in web traffic
In the world of so-called "tech" companies, who use public information found on the www as "bait" to attract potential ad targets for data collection, surveillance and sometimes programmatic, targeted advertising, the information may have a value to those companies, for that purpose. They might measure that value by how popular the information is amongst www users
But I'm referring to the value of information published on Wikipedia to www users, not so-called "tech" companies who seek to intermediate access to free, public information that they did not themselves produce
Wikipedia is the best thing ever. Anyone in the world can write anything they want about any subject, so you know you are getting the best possible information.
Have you actually tried editing wikipedia? It's extremely difficult to get anything approved. You have to provide multiple credible sources for things or else the mods just revert/delete your page.
Isn't is good that there are some forcing factors to help ensure the quality of the content? I get that there's plenty of drama and difficulties in building and moderating the content of Wikipedia, but it certainly does not appear to stagnate in terms of content if you are looking at e.g. the number of articles on English Wikipedia. The overall process appears to produce great outcomes and it is the greatest collection of knowledge created.
I took it as a statement that it is prohibitively difficult to contribute to Wikipedia, and wanted to point out that a large number of contributions are being made and the resulting quality being high, in part due to the difficulty of making contributions.
My comment was disputing the statement above that anyone can just stick junk in Wikipedia. While yes anyone can submit edits, it's pretty hard to get them accepted so the content on Wikipedia is more reliable than just a public notepad.
My statement was that the quality of Wikipedia overall is high, and that one of the reasons for that is because they set and enforce standards for contributions.
Certainly many people are put off by the process and will not have time to deal with it, but my belief would be that such cases are more likely on more controversial topics, and less likely for less controversial topics. Inherently, collaborating on difficult topics will be a difficult process, which also means that there are likely no easy answers for how to make this process not discourage anyone.
The forcing factors aren't what they are supposed to be though. "Credible" sources and citations are exclusively up to the article moderators personal tastes which are very rarely objective.
> The forcing factors aren't what they are supposed to be though.
Is it clear what they should rather be - and are there any examples of mechanisms that have worked better at a scale like this? How are you judging that they are not what they are supposed to be?
If the resulting body of work, which is the totality of Wikipedia, is able to be a curated and high signal collection of knowledge as a result of these mechanisms, how can it be said that they are not working? Having forcing factors, even if they are not ideally aligned or executed, which pushes contributors to increase the quality of their edits to pass, seems overall like a good thing. I'm not saying that its processes and mechanisms cannot be improved, I'm saying I believe it is incorrect to say that they are not working as a whole.
> "Credible" sources and citations are exclusively up to the article moderators personal tastes which are very rarely objective.
Overall I believe Wikipedia to be curated by a large group of people which coordinate through various rules and consensus mechanisms, where I don't believe it is correct to state that sources and citations are exclusively up to any specific article moderators, as they need to be able to build consensus and co-exist with other moderation.
Exactly because Wikipedia is such a large body of work it seems more resistant to corruption to have a large number of curators with different tastes and motivations. How would you determine that their selection of sources and citations is very rarely objective - especially when objectiveness itself seems quite hard to agree upon for many of the topics covered?
From my perspective it seems far more important to consider the quality and value of the totality of Wikipedia, which is massive and signs that many things are working, rather than insisting that it is not working, especially in times where knowledge is being broadly attacked, and where Wikipedia is one of the targets.
In countless ways, there's endless wars among admins.
An example though is that several historically relevant facts are edited out to favor some narrative.
E.g. Crimea, the Ukrainian autonomous republic that seceded/was annexed by Russia, tried to split 3 times from Ukraine, once even during the Soviet Union, but those events have been edited out on multiple pages to favor some western-centric narratives.
I've seen people cite an english book on a German topic and intentionally mistranslate words to construct a connection that doesn't exist and was never intended by the authors of the english book.
And editing with an account can be very dangerous if you manage to upset somebody with clout.
They can turn content disputes into conduct disputes and conduct disputes into social contests which are either shown on ANI or quietly adjudicated with an administrator block.
The content of Wikipedia is great. Its culture, not so much.
And the real reason that editing Wikipedia is difficult, is because of the ideological bias of the moderators so they support only the editors to conform to their ideology and reject any edits that go against their ideology even if those edit are literal proven truths.
Source: I am an editor on Wikipedia and my own informative useful edits to some important topics on my country have been rejected, while the misinformative and even malicious edits by agents of the enemy nation on those same topics were allowed and continue to persist and mislead whoever reads those articles.
Wikipedia has become a weapon of misinformative propaganda, and it's not a tool or repository of useful accurate information. This is why Wikipedia is banned in schools and universities, because its information may not be credible and Wikipedia has long ago lost the integrity that any worldwide free information repository should have had.
Larry Sanger is famously mad at Wikipedia for a very long time now because they wouldn't allow his pseudoscience on it. Later on that also extended to his (increasingly far right) politics.
Some of the garbage I've been involved with that I either had to avoid or bail out of an edit or revert war:
* A claim on "Fisting" that "seasoned fisters can insert their arm up to the shoulder into the anus", "supported" by a deleted PornHub video.
* Fighting on a Production Car top ten list when Tesla announced that Ludicrous Mode was coming the next year and "expected" to have certain performance stats, where multiple editors fell over each other to make sure it stayed at the top of the list, even when they eventually had to add a column just for Tesla where every other result had "Actual Results" and the Tesla had "Projected/Expected Results".
* A collation of John Deere tractors that described multiple models as "light years ahead of the competition".
* An article on an Australian drug smuggler where exhibits from court case were being removed as "biased".
* He's a quite interesting historical character popularized by the video game Assassin's Creed Shadows. However, this insertion has also been controversial. East Asian male protagonists are underrepresented in Western media already, and with Assassin's Creed there's the perception that now they snubbed East Asians again, following a long trend of forcefully inserting foreign perspectives in Asian settings, like The Last Samurai, Shōgun, etc.
* There aren't that many real historical accounts about him so people can argue all day about stuff like "Was Yasuke a real samurai?" without clear evidence about who's right.
Since the first games, many fans have clamoured for "Assassin's Creed in Japan"... I suspect they fantasised about being a male Japanese ninja running across rooftops in the moonlight, sneaking over nightinggale floors, and silently assassinating people in their beds, combining what they saw in Altaïr and Ezio, with every Japanese media trope they knew of.
But in the intervening years, Ubisoft and/or the entire AAA games industry is now seemingly driven by a need to conspicuously showcase diversity and inclusion (from some gamers' viewpoints). You could also view it as the gaming industry trying to broaden its audience and get out of the pigeonhole of catering to the base desires of sweaty manchildren, but either way, it's upsetting a certain type of game consumer.
So, when Ubisoft finally got around to setting Assassin's Creed in Japan, and they picked pretty much the only person around that time period who wasn't Japanese as the main protagonist, seemingly to meet diversity goals, capital-G Gamers went bananas over it, like it was a personal affront to them.
"In May 2024, it was announced that Yasuke would be a major character in an upcoming video game (Assassin's Creed Shadows). While onwiki disagreement about Yasuke's status as a samurai predates this announcement, the historical figure's Samurai status became part of a culture war around video games (J2UDY7r00CRjH evidence) that media sources have described as a continuation of or successor to Gamergate, leading to an increase in attention to the article. (Symphony Regalia evidence)"
I don't get your point. You are allowed to edit articles on contentious topics. Its just more likely to be reverted. Because the topic is... contentious.
My attempt at a point is that the controversial topics are a tiny percentage of the human knowledge stored on Wikipedia. If there were no controversies, then I would actually start to get worried. That would indicate that there was pure control of information on Wikipedia, like a theoretical CCPedia. [0]
Wikipedia is so open, that they even have their own "controversial" section! Is that not the coolest thing ever?
The chip on my shoulder is that there is a concerted effort to destroy and discredit Wikipedia.
The accomplishment of Wikipedia is not just beating the Library of Alexandria by many orders of magnitude, but doing so while keeping moderation logs in the open as well.
Ask @dang, or anyone that has ever had anything to do with forum moderation, if they would be cool with their moderation logs being completely open. Almost everyone with experience would say 100% no. They likely tried that and saw how much nutso drama it creates. Wikipedia actually does that, at the largest possible scale!
[0] Of course that exists, apparently it's called Baidu Baike
> if they would be cool with their moderation logs being completely open
It takes a certain mentality. That's rare but I think it makes for much better communities on the whole.
However I think most participants, not just moderators, don't like the environment that sort of mentality results in. When anything and everything, including the moderation itself, is up for civilized debate that tends to foster an environment in which it's acceptable to question core parts of people's worldviews. There's little shared doctrine beyond "argue any position you'd like" which most people seem to find intensely uncomfortable.
> The accomplishment of Wikipedia is not just beating the Library of Alexandria by many orders of magnitude, but doing so while keeping moderation logs in the open as well.
There is at least one exception to that rule. Users who attract the ArbCom's attention may get a general block. If they ask what they're blocked for, the ArbCom rep will tell them to read their email. These moderation decisions are not public, not even in a form with PII redacted.
There is also what's called "Oversight", which performs actions that are invisible to the public and administrators alike (though not invisible to some very privileged people)
There are also "office actions", where essentially the Wikimedia Foundation and/or its legal counsel have been required to do something. In most cases, the office actions are visible and logged, unless they've been required to use Oversight as well. But the main thing is that the office actions will generally not be explained to anyone, as it usually stems from some legal threat to Wikipedia.
I also have problems with Wikipedia's favoritism of insiders who have learned how to navigate its bureaucracy, but the fact that most edits of political and/or controversial topics are immediately reverted is not in itself evidence of a problem. A priori, I would expect that the majority of edits to political and controversial topics are bad and should be reverted.
I guess you picked “tax stuff” because the tax related thing you edited was a sort of dry tax related topic, but I’m sure we could find lots of controversial topics under the “tax stuff” umbrella.
I don't have an extensive wikipedia career, but I've found that even my few edits to political topics have been accepted.
What did get reverted was a trivial [citation needed] fix, for a musician's page, for a sentence stating they were involved in scoring a film. I found a relevant citation and this was promptly reverted, for reasons that were explained but, at least for me, utterly incomprehensible
You're not really considered a veteran editor until you've won at least 10 Request for Comments outquoting your detractors with at least 100 obscure Wikipedia guidelines and policies.
Many said (some still say) that Wikipedia is not a replacement for traditional encyclopedias with articles written by domain experts, such as Britannica.
Many more scoffed at that, saying those people were just stuck in their old ways and unable to adjust to the obviously superior new thing.
Is that you? AI applications are different than Wikipedia and are better in some ways: Coverage is much greater - you can get a detailed article on almost any topic. And if you have questions after reading a Wikipedia article, Wikipedia can't help you; the AI software can answer them. Also, it's a bit easier to find the information you want.
Personally, I'm with the first group, at the top if this comment. And now truth, accuracy, and epistemology, and public interest in those things, take another major hit in the post-truth era.
Right, and where are all those LLMs without the billions upon billions of lines of text written by humans? A not insignificant number coming from Wikipedia?
Also, LLMs don't produce truth. They don't have a concept of it. Or lies for that matter. If you are using LLMs do study something you know nothing about the information provided by them is as good as useless if you don't verify it with external sources written by a person. Wikipedia isn't perfect, nothing is, but I trust their model a shitload more then an LLM.
> where are all those LLMs without the billions upon billions of lines of text written by humans? A not insignificant number coming from Wikipedia?
Where is Wikipedia without all the learning and information from other sources, many of which it put out of business?
> Also, LLMs don't produce truth. They don't have a concept of it. Or lies for that matter. If you are using LLMs do study something you know nothing about the information provided by them is as good as useless if you don't verify it with external sources written by a person. Wikipedia isn't perfect, nothing is, but I trust their model a shitload more then an LLM.
Wikipedia produces consensus that correlates with truth to some degree. LLMs produce statistical output, which in a way is a automated consensus of the LLM's input, that also correlates with truth to some degree - and the correlation is hardly zero.
I agree that information has no value if you don't know its accuracy; it's always a sticking point for me. IMHO Wikipedia has the same problem: I have no idea how accurate it is without verifying it with an external source (and when I've done that, I've often been disappointed).
Has anyone researched the relative accuracy of Wikipedia and LLMs?
The comment about Wikipedia supposedly putting companies out of business is so goofy I'm not even gonna comment on it. I'm surprised you'd bother trying to make a point there.
The difference is humans have a concept of truth, humans have intent. A person, taking an aggregation of their research, expertise, and experience to produce an article is (presumably) trying to produce something factual. Other humans then come along, with similar intent, and verify it. Studies in the past have shown Wikipedia's accuracy rate is roughly on par with traditional encyclopedias, and more importantly sources are clearly documented. Making validation and further research fairly easy. And if something isn't sourced I know immediately it's more suspect.
LLMs have no concept of truth, they have no "intent". They just slap words down based on statistics. It is admittedly very impressive how good they are at doing that, but they don't produce truth in any meaningful way, it more a by product. On top of that all its sources get smashed together, making it much more difficult to verify the validity of any given claim. It's also unpredictable, so the exact same prompt could produce truth one time, and a hallucination another (a situation I have run into when it comes to engineering tasks). And worst of all. Not only will an LLM be wrong, but it will be confidently and persuasively wrong.
> The comment about Wikipedia supposedly putting companies out of business is so goofy I'm not even gonna comment on it.
I've learned that when people don't have any merits to argue, they turn to ridicule. Right back at you buddy.
> The difference is humans have a concept of truth, humans have intent. A person, taking an aggregation of their research, expertise, and experience to produce an article is (presumably) trying to produce something factual.
It's pretty naive to think that humans have intent and motivation for the truth, and no others. Just look around you in the world - most communication disregards the truth either carelessly or incidentally (because they are motivated to believe or claim something else) or intentionally (lots of that).
> LLMs have no concept of truth, they have no "intent".
My calculator app has no intent or concept of truth, but outputs truth pretty reliably.
To think that I'm saying all humans intend to produce truth you'd have to intentionally misread my comment. Wikipedia obviously has to deal with bad actors and vandalism, and they have processes in place for that. My point is that the intent matters.
Calculators aren't a useful analogy for LLMs. They produce a deterministic output based on a (relatively) narrow range of inputs. The calculations to produce those outputs follow very rigid and well defined rules.
LLMs by their very nature are non-deterministic, and the inputs/outputs are far more complicated.
It's not an insult when it's true. I don't think you've made one comment that actually added something useful. I did my best to reply to what was there, but you didn't give me anything to work with. Your last comment was so unrelated there was no where left to go.
If you have something actually relavent to say you're welcome to say it.
Here's an opportunity to talk about listening, epistemology, and human intercourse:
> It's not an insult when it's true.
It's not slander, but it's certainly an insult. If you tell someone they are fat and ugly, it's an insult regardless of its truth and you shouldn't say it, ever. There's never a good reason for personal insults.
> it's true
> you're welcome to
This assumes your perspective is truth. That is the case for nobody in the world; in fact, I also have a perspective that I'm confident in, as do many others. Your statements also assume that, perhaps as the arbiter of truth, you have some authorization or power to enforce it. Again, that's nobody's business.
We're in a world of peers, generally speaking, and none of us know who is right. We need strategies to navigate that world, not the one where truth is given to you.
> you didn't give me anything to work with
When I feel like you do, it's a signal I need to listen better - the other person probably does have something to say and I'm missing it. It's possible we're talking past each other, but that's never a reason for insults.
(human intercourse)
Note that the signal is that I need to do something, not the other person. That's not because I'm 'wrong' or 'right' - those are mostly unknowable and irrelevant because 1) We're in a world of peers, generally speaking, and none of us know who is right. Also, 2) I'm the only one I can control and am responsible for, and ...
3) Respecting other people is always more important. That's a strategy for, and wisdom in, a world of uncertainty (as described), as opposed to a world of certainty. Also, it's a strategy for social creatures in social groups - it keeps groups strong and functioning. Finally, it's strategy for both loving and respecting yourself - you deserve it. You're better than insults, I'm sure; and I sometimes say the wrong thing, but I'm better than that too.
> Where is Wikipedia without all the learning and information from other sources, many of which it put out of business?
Which businesses did Wikipedia put out of business? You will frequently see a 5k word article used for a couple of sentences in a Wikipedia page, with the entire Wikipedia page itself being smaller than one paper it cites for one small corner of said page. When I’m researching events, I frequently go to Wikipedia to find sources as search engines have a drastically larger recentism bias.
> Has anyone researched the relative accuracy of Wikipedia and LLMs
No comparative research on this specific topic has been conducted afaik, and most comparative research is aging (likely, to Wikipedia’s own detriment–general consensus is that Wikipedia’s reliability has increased over time). However at the time of research publication, the consensus seems to be that Wikipedia is generally only slightly less reliable than peers in a given field (ie textbooks or británica), although Wikipedia is often less in depth. The most frequently cited study is a 2005 comparison in Nature which found 4 major errors in both Wikipedia and Británica, and 130 minor errors on Británica whereas 160 on Wikipedia. All studies are documented on Wikipedia itself, see [[Reliability of Wikipedia]]. LLMs… do not have this same reputation.
> Which businesses did Wikipedia put out of business?
Just as a start, other sources of reference, including encylopedias, dictionaries, websites, etc. For example, I'm sure it impacts McGraw-Hill's AccessScience, which likely you've never heard of.
> This is documented on Wikipedia itself
Maybe there's a little bias there? Would Wikipedia accept Wikipedia's analysis of its own reliability as a valid source?
I've heard that claim, but having no knowledge of the accuracy of any particular article, it's not worth very much to me.
> LLMs… do not have this same reputation.
They don't with you, but many people obviously use them that way. Also, reputation does not correlate strongly with reality.
> Just as a start, other sources of reference, including encylopedias, dictionaries
This just seems like healthy competition. I thought we were talking about a situation where Wikipedia’s use of other encyclopedias is an instrument of their demise.
> Maybe there's a little bias there
Paradoxically, I suspect you’d be pleasantly surprised about how tough this article is on itself. A lot of attention is given to bias in this case.
> Would Wikipedia accept Wikipedia's analysis of its own reliability as a valid source?
First, it is not Wikipedia’s own analysis. Editors should not present their own conclusions from research, just what each paper says. See [[WP:SYNTH]]. Second, generally Wikipedia discourages anyone citing it as it is not a stable source of information. Much better is to use the sources the article itself conveniently cites inline. As a general policy citing any encyclopedia is discouraged.
> having no knowledge of the accuracy of any particular article, it's not worth very much to me.
Wikipedia does have internal metrics grading the quality of an article. [[WP:ASSESS]]. In general though, even entirely discounting the Wikipedia component of the británica comparison, based on británicas own failures it seems wise to verify each and every claim in an encyclopedia, which Wikipedia does an excellent job of helping you do.
> They don't with you, but many people obviously use them that way. Also, reputation does not correlate strongly with reality
OpenAIs own benchmarks show much higher hallucination rates than any study on Wikipedia. Wikipedia itself is quite close to a ban on LLMs for reliability issues. If you ask literally any layman “has ChatGPT ever been wrong for you” they will say yes, either in that moment or after only a little prompting. It is much harder to elicit such a response regarding Wikipedia in my experience
Correct. The amount of Wikipedia pages at any given moment with active vandalism is vanishingly small. The only time I have ever stumbled upon vandalism is as part of my work as a volunteer there actively looking for such cases. Looking at my feed of possibly problematic changes at the moment, about 3 entries are appearing per minute with the most recent revert being just 2 entries ago. It is significantly worse while school is in session in my experience, but vandalism very rarely lasts long. Talking to people, people frequently confess to vandalising wikipedia at one point or another. When I ask them "how long did it survive" they tell me answers ranging between "a few moments" to "5 minutes." So to answer your question, I believe it is unlikely the average person has seen vandalism on the site barring those looking at their own shit.
>> Just as a start, other sources of reference, including encylopedias, dictionaries
> This just seems like healthy competition. I thought we were talking about a situation where Wikipedia’s use of other encyclopedias is an instrument of their demise.
Somewhere above, someone complained that LLMs were harming Wikipedia, a source of its information. My point is that Wikipedia did the same to others.
> > Which businesses did Wikipedia put out of business?
> Just as a start, other sources of reference, including encylopedias, dictionaries, websites, etc. For example, I'm sure it impacts McGraw-Hill's AccessScience, which likely you've never heard of.
Your “for example” in response to a question about what businesses Wikipedia put out of business is a business that is...still in business?
this is pedantic, but I think its important. We can say that a truth and lie are exclusive. when we lie, we know the truth. we just make up something else because it fits our agenda better. post-truth implies that the distinction doesn't matter anymore. nothing is true, everything is true. no statement can be evaluated outside the intent in which its delivered. that's much more substantial shift than just presenting a set of lies as the truth. a post-truth society is almost compelled to devalue science or any other pursuit of knowledge. by making all the voices selfish and mundane, it explicitly rejects beauty and accomplishment.
Whenever I can, I try to think of modern events from the point of view of archaeologists digging through the layers, at some point in the distant future.
Given that perspective, my thought was: "Hey Bob, look at these morons, they called easily proven lies 'post-truth!' Can you believe that? In a civilization based on science, with AI, nuclear, and biological weapons?! No wonder they died out right after this. How did they not see this coming? Anyway, what's for lunch?"
It indicates that it's a follow-on to postmodernism. To a significant degree the post-truth era is built on a reactionary attack on postmodernism - you can see it on HN, where many people reflexively attack like a mob anything they perceive is postmodern. You can see it in so many people who will accept lies and disaster over postmodernism.
And post-truth is a postmodern term - ironic, ridiculing, makes you think, has some energy to it. How absurd to be literally //post-truth//.
> era of lies
That's a post-postmodern term. No irony or wit; a term of despair. :)
>It indicates that it's a follow-on to postmodernism.
How? it's just postmodernism itself.
There is no truth because everything is relative. There is no singular, objective truth, facts are intrinsically bound to their context, hence post-truth.
> How? it's just postmodernism itself. / There is no truth because everything is relative. There is no singular, objective truth, facts are intrinsically bound to their context, hence post-truth.
That's not postmodernism but a caricature of it by its critics.
Postmodernism cares deeply about truth. It is highly skeptical of power, bias, perception, etc. and provides tools to mitigate these risks to truth.
Post-truth is cares very little about truth and is especially non-skeptical, imho.
> Postmodernism cares deeply about truth. It is highly skeptical of power, bias, perception, etc. and provides tools to mitigate these risks to truth.
I think someone told you this and you believed it, but the track record of postmodernists in academia tells a different story.
I once saw a history lecture from a dean of the history department who had written of a book on WWII in the Philippines. This is one of the most heavily studied and well documented periods in human history. There are dozens of books and hundreds of papers on this topic.
My father-in-law, who was born there and lived through the war there, read the book. He called it historical fiction. The book cherry-picked facts badly, ignored events which countered the narrative of the book and in general made every effort to promote an entirely counter-factual narrative of events. In addition, the author had no real expertise in military or naval warfare and didn't seem to understand even the basics about how wars are fought.
When things like this happen on a regular basis, it is hard to say that postmodernism cares about objective truth in any way. In fact, they seem to actively dislike reality. This isn't a caricature, this is from the horse's mouth. If you want people to have a different perception of postmodernism, make events like this have some sort of penalty, because he's still the dean many years later.
> I think someone told you this and you believed it
Obviously you don't care about the truth but just fabricate bullshit about other people that's convenient to you. I know the topic well; you clearly don't.
Your father is one source; he no doubt has his experience and memory which are important to him. If you look at the sources of a scholarly history book, you'll see thousands of people - that's how history is done, by researching primary sources like your father. Of course not everyone will agree; people will have widely varying perspectives (which is something that postmodernism helps you understand and dealt with).
These cheap shots at me, scholarship, and postmodernism - which has its flaws, but not this stuff - are an insult to everyone's intelligence, including yours.
It's hilarious to read American history and read in every book how everyone who had something the US wanted, or went counter to US interests accidentally happened to have been evil in some way so the US could conveniently show up out of selfless desire just to right the wrongs and end up with the geopolitical result that it wanted in the first place.
People with decades long scholarly careers write this shit (some even having the highest credentials), and people eat it up.
History as written by (especially US) historians is just racist boomer fanfiction (pushed as propaganda to enforce the national myth) for your own country. This is especially true of the US.
And their errors and not subtle by any means. When they write something that is so wrong about you that every man on the street who even has cursory familiarity with the subject would reject as not even wrong, so far from the truth it's clear to the observer that the one who made these claims doesn't even have basic familiarity with the subject.
You've countered this argument with an appeal to authority (his word against thousands of researchers) - how fortunate was a person like Galileo who could just make people look into a telescope and show his numerous and highly distinguished opponents that they we wrong - unfortunately no such thing can exist for history. The next best thing I could recommend to US scholars is to have their work reviewed by top and highly respected local scholars for obvious errors, not biases of overarching narrative but, basic shit and continuity errors that common man on the street wold laugh at.
How could this be? Do I believe Americans to be specially dumb? Just like Big Tobacco pushing studies on the health benefits of smoking for pregnant mothers, and Coca Cola delegitimizing the view that sugar is bad, US historians have a vested interest in propping up US imperialism - or are acting as reactionaries saying everything the US (or white people) has ever done was purely evil, what you have is a partisan shouting match (also called activism), that is the exact opposite of scholarly work.
Caricature? Foucault talked about "regimes of truth" determining that context and society are what determine truth. Baudrillard straight up said "The simulacrum is true" putting it over objective fact. Derrida's whole deal was about how language constructs, rather than describes, reality.
If all of this sounds ridiculous that's another matter, if I actually wanted to cast shade upon them then gee, I'd just quote their stances on sex with minors.
First, I mean no personal insults. Let's discuss ideas.
As someone who grew up being de-programmed from Soviet propaganda by my parents every time I came home, starting from pre-school, I cannot even begin to communicate how allergic I am to this discourse. "There is no truth" is some grade-A bullshit to me. What's next? Maybe Stalin wasn't Hitler's military ally to start WWII? Maybe we live in a simulation? What's the point of anything? 1+1=3!
I could just be dumb, but my theoretical view from 30,000 feet, or 30,000 years in the future can be read here:
But “era of lies” doesn’t sound nice because nobody wants to be a liar… so “post-truth” sounds better: “I'm telling the truth. Almost. But I'm not lying.”
The inevitable AI investment bubble popping is going to be a big deal, but I believe that the larger bubble of political lies popping is much more interesting to think about.
What is that going to look like? How does one hedge against that eventuality?
Interesting reference. I had not heard of him, or that book from 2013. 10B is now projected to be our peak, and at 7B we have so much spare food that we burn it as automotive fuel. And now... we realize that population "collapse" is much more likely to happen than endless growth.
Obviously I have not read the book, but do you think it holds up in 2025?
Many books on the subject are very much "doomsayers" or "preppers" style.
The Roman Empire has been crumbling for 400 years, so it's likely that we won't experience the collapse of society as described in most history books either - life is too short for that. Unless a black swan comes along…
To answer your question, I like to come back to the book because it's written in the style of Dan Brown :) - short, punchy chapters. And it still makes sense (to me).
Worldwide population peaked sometime between 2015 and 2020 at somewhere between 7.2 and 7.4B. So my take on the book is that it isn't very good.
Also, biofuels based on most crops other than sugar cane, in addition to not being very helpful in the fight against AGW, triggered large price spikes and political turmoil in a dozen different countries at once. Perhaps you heard of this event, we call it the Arab Spring. We are still dealing with the fallout of it.
Someone made a similar argument to me recently about AI, talking about how programmers used to have stacks of reference books at their desks and how many of the old guard had to have trainings convincing them to use google as a reference.
I guess this argument was supposed to convince me to stop being such a luddite and accept the inevitable future, but really, in an increasingly post-truth world, it made me want to go and get myself a stack of reference books.
The production of factually accurate content is a pretty expensive job.
We who were born before this era really took off, are spoiled by the journalism standards and information purity levels of the past, especially post the fall of the USSR.
Wikipedia is impressive on what it manages to coordinate on a daily basis, especially given only 644 FT staff.
>Similar to last year, technology-related work represents nearly half of the Foundation's budget at 47% alongside priorities to protect volunteers and defend the projects of an additional 29% – a total of 76% of the Foundation's annual budget. Expenses for finance, risk management, fundraising, and operations account for the remaining 24%.
Yeah, probably. I understand there's a lot if logistics I simply don't know about. I was inquiring about more details on what the paid staff need to do.
A lot of it is engineers who work on improving the software that runs Wikipedia, and keeping the site running, which you can see happening at https://phabricator.wikimedia.org -- outside of security issues, all the dev work is done in the open. There's constant ongoing work on making Wikipedia and all the related projects work better.
There's also people who do the fundraising, community management, legal defense, etc. Then there's general HR infrastructure around employing hundreds of people.
Basically, that "Hosting was 3.4%. Facilities was 1.4%." point gets brought up, and neglects to mention that you then need to pay for a bunch of people to manage those servers and facilities.
(Disclaimer: I'm an employee of the WMF. I'm just an engineer, so I'm not speaking authoritatively about financial details.)
Why are they paying 6.4% on processing fees? What is "movement support" and where is the travel to? Do they have to publicly disclose these disbursements anywhere? This seems sketchy at best.
‘Processing fees’ likely includes the cost of administering the CRM, creating tax receipts and reports, donor support, and all the other ‘processing’ tasks that come with running a large fundraising effort. It wouldn’t just be the credit card fees.
Wikimedia accepts Paypal, Apple Pay, Google Pay, Visa, Mastercard, Amex, Check, ACH and Money Order.
Pretty hard to argue that mainstream processors don't like them.
Processors charge higher fees to merchants that are in lines of business with high fraud and chargeback risk, has nothing to do with whether they agree with them morally.
They refuse merchants with business they don't like.
If it were the case that processors didn't like what wikipedia publishes, they would not be able to accept payment, not have high fees.
I can't imagine that wikipedia has high chargeback rates, and clearly the processors don't mind doing business with them.
The processing line item probably includes not just the fees that they have to pay to processors, but FX fees, the cost of banking, the cost of paying people to open envelopes, the cost of accounting, etc.
> I can't imagine that wikipedia has high chargeback rates, and clearly the processors don't mind doing business with them.
Its actually somewhat common for people who steal credit cards to use non profits like wikipedia to "test" them. Typically such sites have no minimum donation, have donations from all over the world so fraud detection wont think its weird you're spending money half way across the world.
No. Why would a payment processor make an exception to its risk-based rules for an organization that increases exposure to that risk? Brand recognition is a liability in this case.
Wikipedia is a victim of it's own success, it was excellent at avoiding bias for quite awhile and the vast majority of articles are extremely well written.
LOL I just tried to search "U2" on it but it sent me to a Twitter post. That Twitter post required me to login to see if I wanted to see it. What a joke.
Wikipedia is fine. Bias cant be avoided, only made explicit, and the fact that you think it can means you should probably move to Conservapedia or whatever slop Musk will release.
The page on the Second law of thermodynamics has a chapter on "Liberal abuse of second law of thermodynamics". I wasn't sure if I was reading an imitation of The Onion that tries to take itself too seriously.
That said, it is indeed so insane that trolls sometimes become contributors just to see how far they can push it. There were several known cases of trolls going far enough undetected to get various admin rights (to block etc).
Shouldn't it be minimized? I think it's increasing on Wikipedia. I think there's too high of a barrier for actual experts to write/edit. What's left is primarily idealogues with a certain leanings resulting in a positive feedback loop to more bias. I don't think anyone should want an echo chamber where an encyclopedia is supposed to exist. It's not there yet, but this is exactly how they form - see reddit.
Yeah, yeah, let me guess, the truth has a liberal bias.
But continue to insist that everyone pointing should just move somewhere else, that will certainly make it less biased and more factual.
>Shouldn't it be minimized? I think it's increasing on Wikipedia. I think there's too high of a barrier for actual experts to write/edit. What's left is primarily idealogues with a certain leanings resulting in a positive feedback loop to more bias. I don't think anyone should want an echo chamber where an encyclopedia is supposed to exist. It's not there yet, but this is exactly how they form - see reddit.
I find this kind of a fascinating social phenomena.
I guess to give a personal example, I was trying to update a few pages about a country's Olympic history - that is, their Olympic bids, a few athletes, etc.
Unknowingly, I had stumbled across a particular power-editor's fiefdom, because they created all these pages and they were very aggressive in policing their articles to meet a certain style, tone and their beliefs.
Searching this person's username up (they went by their real name), they were closely related to that country's Olympic committee and an employee of a Ministry of Sport of sorts. The articles had lots of anonymous IP address edits from an university network this person was affiliated with in their program portfolio.
There was a clear conflict of interest, and I tried to point that out when they mass-reverted my edits, but they seemed committed to accusing me of edit warring by not sandboxing my changes and waiting for their personal review and approval, and quoted at least 12 different Wikipedia policies on notability, style, acceptable citations, etc. I still feel I was in the right, but I didn't have the willpower or stamina to fight against several requests for comments, speedy deletions, etc. They did get a warning from an administrator and some detractors in the discussion threads, but they weren't willing to let it go and at that point, it wasn't fun anymore for me. I have better things to do than to fight factual and nitpicky disputes on Wikipedia.
I get the sentiment of what you're saying, but I don’t think that’s totally fair to the parent.
Biases aren’t automatically "bad" or "failings". they’re akin to heuristics, and it’s practically impossible to eliminate them entirely. For example, we’re all here talking about how we should treat Wikipedia with skepticism. That's a sort of "neutral" bias that doesn't conjure a strong emotion, and is perhaps more acceptable for it, and probably leads to better informational hygiene overall.
In fact, the claim that “bias can be avoided and should be absolutely”, that is implicit in your resposne reflects a bias of its own: a bias toward moral or intellectual purity, as if the parent recognizing bias is equivalent to endorsing it. I get that this is a pedantic point to make but to come at the parent with such vigour for being realistic, again seems a bit unfair
It depends on if you take the parent (err GGP) literally versus in context. My personal experience has been that people claiming that bias is unavoidable (alternatively that objectivity is impossible, or that everything is political, or ...) have usually been attempting to justify a reasoning process that supports a political or religious slant of one form or another.
A helpless "we couldn't possibly do anything about that issue" sort of mentality.
That said if we're talking literally then I fully agree with you that heuristics are a form of bias and can sometimes be a very good thing on a case by case basis.
Yep fair point. I was taking them literally, though I didn't necessarily feel the context meant their post was bad faith or concessional, just a simple truth. Your elaboration adds a fair bit to your argument though and I am pretty sure I agree for the most part.
> In fact, the claim that “bias can be avoided and should be absolutely”, that is implicit in your resposne reflects a bias of its own: a bias toward moral or intellectual purity, as if the parent recognizing bias is equivalent to endorsing it.
Nonsense. Your definition of the word "bias" includes any assertion whatsoever. Bias is distortion from reality and truth. Saying that we can avoid distortion is not itself a distortion. I never claimed that all bias should be avoided, but the post I responded to said that bias can't be avoided.
Also,
> a bias toward moral or intellectual purity, as if the parent recognizing bias is equivalent to endorsing it.
There is no conceptual connection here between "purity" and the equivalence of recognizing bias with endorsing it, nor is saying "bias can be avoided" related to, or a kind of, "purity" in any useful sense. Stop using abstract words for effect and speak simply.
> I get that this is a pedantic point to make but to come at the parent with such vigour for being realistic, again seems a bit unfair
If the parent were being realistic, they'd say that we can't even recognize bias, which is actually more agreeable to me. But instead the parent admits that we can recognize bias. Since we've gotten that far, then I can say that failing to avoid it, when we should avoid it, is merely a lack of will and integrity rather than some inescapable fate.
The parent’s point was plainly that bias is unavoidable and making it overt is realistically all we can do. It’s a pragmatic take, albeit overly terse. They could have additionally included something like “and try to minimize its effects” sure, but that to me was implicit. I may be reading that fairly charitably, but perhaps that is just my own bias. I certainly have a bias to judge it more charitably than I do someone who leaps straight to moral outrage and judgement this early in the interaction.
In this case, I simply felt your judgment of the parent wasn’t fair, and showed a moral bias. Maybe they weren’t perfectly clear, and too absolute, but your response wasn’t proportionate either. It condemned more than it understood. I interpreted it as an epistemic observation and you interpreted it as an offense. The very fact that we came away with two completely different readings of the same short sentence rather proves the point.
Thank you for putting words in my mouth regarding my definition of the word bias, but lets use your own: "Bias is a distortion from reality and truth." If that is the case, we can never hope to avoid it, because we will never have perfect information. Using that definition, we are quite literally constantly in a state of bias. Your very own definition is far more broadly supportive of the notion that bias can't be avoided and consequently suggests bias is effectively ubiquitous. This to me is the primary point the parent was making.
I was perhaps too charitable, and you not enough. We are both biased, and going by the advice of the parent, I'm pointing it out. I don't think there is much more I can do.
> They could have additionally included something like “and try to minimize its effects” sure, but that to me was implicit.
I'm not interested in talking about what could have been implied, only about what was stated. I'm arguing against an idea that was articulated, not the person who articulated it.
One of the problems of the current-day liberals, in my opinion, is that they make universal statements that they don't mean in order to sound punchy and snag a few morality points. "Believe all women," "men are trash," "defund the police," "all cops are bastards" are all things you'd hear from a person who doesn't actually mean or want any of these things, even though the root of each of those is just and good. The idea that "bias can't be eliminated, only made explicit" is another one of these. If we don't believe it, then let's not say it.
> I certainly have a bias to judge it more charitably than I do someone who leaps straight to moral outrage and judgement this early in the interaction.
I'm not sure where you're reading outrage moral or otherwise. Was it that I used the word "so" in "so harmful"? And where's your bias against someone who tells another person to go to Conservapedia if they think bias can and should be avoided?
> Maybe they weren’t perfectly clear, and too absolute, but your response wasn’t proportionate either. It condemned more than it understood.
I merely stated that the belief was untrue and harmful, I don't think that's disproportionate at all. I can only understand what is stated, and according to my understanding we ought to condemn it.
> Thank you for putting words in my mouth regarding my definition of the word bias, but lets use your own: "Bias is a distortion from reality and truth." If that is the case, we can never hope to avoid it, because we will never have perfect information. Using that definition, we are quite literally constantly in a state of bias. Your very own definition is far more broadly supportive of the notion that bias can't be avoided and consequently suggests bias is effectively ubiquitous.
This is shifting the goalposts. First, I never claimed we could know the complete truth; it was the original post who stated that we couldn't course-correct upon learning new truth ("bias can't be avoided"). And second, the context of the original statement is bias in reporting, not epistemological certainty. We're not talking about positions of atoms here. We don't need perfect information to stop being biased against women in the workplace or against black people or whatever the subject. Even as individuals.
> This to me is the primary point the parent was making.
If that is their point then they can say it.
> We are both biased, and going by the advice of the parent, I'm pointing it out. I don't think there is much more I can do.
I have to ask, if I can't avoid my bias and you can't avoid yours, then what's the point of pointing out bias at all? Is it for other people to avoid our bias? How can they do that? I guess we're trying to minimize its effects, like you said.
> I was perhaps too charitable, and you not enough.
If I lack charity, it's in response to the original uncharitableness of the person telling someone to go to Conservapedia. If he would have mercy, let him show mercy.
Honestly, I think we are arguing around each other. We simply read this notion totally differently. You're taking it to mean that "we can't correct bias" but the statement was "we can't avoid bias". It actually makes no comment on being able to correct it (within, or without ourselves). To me it read as "when interacting with the world, we can't avoid encountering bias". If this is how it is interpreted, it actually doesn't do anything to rule out the ability to account and correct for it.
I'm not saying there's a definitive interpretation with how terse it is, just that we aren't necessarily on the same page and attempts to come to any sort of agreement with each other might be a waste of time as we are practically talking about two different ideas. I take this response as pretty fair, and I think the point you're making is totally valid, I just think our respective ideas would never converge as we are talking about 2 distinct things. (Interesting how much conversation a lack of clarity can generate).
Agree it is very bad for everyone to think like this. The problem is that on subject that divide you will always have two sides. Some times you need to tone down articles too much to not be biased to either way. So I also agree that there is a place for multiple articles on the same subject.
Also Wikipedia has the problem of being one type of article many subjects really need a other way to explain it.
Yes, biases which ideally you are aware of and try to account for.
For example, it would be stupid for me to entirely dismiss all of Wikipedia just because I know that there are some horribly biased articles and it'd be a disgrace if I shrugged such behavior off saying "hey, we all have biases, can't help it". That's what a child or even an animal would do.
They try to work hard and get rid of bias through various methods: peer reviews, precise methodology (randomization, blind tests etc.). And of course, even those sciences are not perfect, subject to biases, and evolve. This applies only to a subset of fields, in a lot of fields you either don't have the time to remove biases this way or it's impossible because it's inherently subjective. Besides, if you needed unbiased perfect opinions before taking any decision, the world would be stuck and slow.
I think we have a fundamental disagreement here. There is no bias in the statement "2 + 2 = 4". Nor is there bias in stating that "any two masses experience an attraction force proportional to the masses and inverse-square proportional to the distance separating them." Not less bias -- no bias.
There is a pretty strong bias in the statement "2 + 2 = 4" because math doesn't care about the encoding used to describe a sum. For example, in an alternative society the expression could be written with flipped number symbols, right to left or in a vertical layout. Axioms are also a form of bias.
Axioms would only be a form of bias if they were taken at random. It's not biased to take the axiom that repeatedly holds true in empiric testing, over others that do not.
1) The Monty Hall Problem (to be precise it hinges on ambiguity and the distinction between prior and posterior probabilities, but that is something most people aren't aware of and will get wrong the first time they see it, even people with knowledge of say Bayes' Theorem)
2) For several others, see Alon Amit's superb Quora answer to "What are the most interesting or popular probability puzzles in which the intuition is contrary to the solution?" ([2], login-walled). Mentions the very counterintuitive Penney's Game [0].
3) Berkson's Paradox, aka "People in hospital/getting treatment tend to have worse health indicators".
4) Asymmetric dice behavior is counterintuitive, when you first see it.
5) Benford's Law, on quantities occurring in nature (e.g. river lengths), as opposed to uniform distribution.
6) There are lots of counterintuitive things about Platonic solids.
7) Bayes' Theorem itself, superbly useful but possibly one of the things in probability most abused on a daily basis by bad journalism and bad statistics.
8) The Multiple Testing Problem/p-hacking/aka the xkcd "Green jelly beans cause acne"
and as a corollary:
8a) Most published (academic) findings aren't replicable, aka "Why Most Published Research Findings Are False", Joannidis (2005)
Great list! But I'm not sure the relevance? The intended point, which perhaps wasn't clear, was that logic / pure math as well as readily testable empirical theories are exceptions to the "everything is biased!" claim. There are in fact things that we can know with certainty, and that provides a foundation for a broader set of knowledge. The absolute relativist position is not just boring, but doesn't reflect reality either.
Traffic is how Wikipedia attracts future editors - people visit the site, some figure out that they can hit "edit", and some of those will go on to being productive editors on the platform.
Falling traffic is a problem. As editors turn over and fewer new editors show up, Wikipedia will become harder to maintain. For example, these days, defending against AI slop vandalism is a real problem that needs real humans to tackle - even as AI tools become more useful at detecting low-level vandalism.
I think Wikimedia needs expensive outreach and modernisation to stay relevant. And most of those attempts will be in vain.
A poor comparison is how much money coca cola spends on advertisement, even though it is one of the best known brands in the entire world. And most of their advertisement is simply "This is our name, we exists", not even a value proposition or call to action.
If Wikimedia sets themselves up to pay for servers and maintenance for perpetuity, they will fall into obscurity.
With that being said, I also don't think they are spending their money in a good way.
What about reciprocal cooperation with countries around the world? Foundation invests in country bonds, countries include small yearly payments (like dividends?) to the foundation.
There is more and more of these "essential global infrastructure" projects, many of them non-profit, yet I'm not sure we're seeing a lot of investment from the globe into those projects.
Dunno - it's worked ok for Oxford and Cambridge for many centuries, and the Church. I'd rather have land owned by vaguely public benefit bodies than some private equity fund.
What Wikipedia are you looking at? It's one of the things that has survived the longest on the Internet without being enshittified. It could live on in Cockroach Mode forever off the money from a modest solar farm or investment in an index fund. If someone sociopath bought the Wikimedia Foundation and tried to strip-mine Wikipedia, the Wikipedia editors could easily take a copy and go elsewhere.
Ironically, I was just listening to an interview with Jimmy Wales in which he said that, as individuals, most humans are basically good. They don't meet someone on the street and think "what is the rate of return on interacting with this thing?"
This is only the size of the text at one moment in time. The edit history is dozens of terabytes and the media is hundreds. All of this must be served to billions of requests a day. Almost every single request involves executing arbitrary lua and/or template syntax which was transcluded into the page. The media, particularly, introduces huge copyright problems. Ect ect. The WMF is undoubtably bloated, but it is not a walk in the park to run Wikipedia.
I would be extremely surprised if most articles that are regularly viewed aren't precached and thus served directly without involving any code execution.
Citing is not referencing links, it's finding primary sources and fact-checking. Kurzgesagt[1], an informational YouTube channel, has had issues with LLMs citing LLM generated content.
LLMs aren't arbiters of truth - they're just natural language search engines. In a way it's quite similar to Wikipedia. Something being on Wikipedia doesn't mean that it's true, but rather it means that a "reliable source", which are not infrequently less than reliable, said so. And even that often doesn't hold as there's another layer of indirection where it's an editor saying this is what a source said, which is again not infrequently not exactly accurate. And then there's 'citogenesis' [1] where imaginary facts can be circularly whisked into existence.
This is why Wikipedia is not a source, but can provide links to sources (which then, in turn, often send you down a rabbit hole trying to find their sources), and it's then up to you to determine the value and accuracy of those sources. For instance I enjoy researching historic economic issues and you'll often find there's like 5 layers of indirection before you can finally get to a first party source, and at each step along the road it's like a game of telephone of being played. It's the exact same with LLMs.
I've had LLMs cite me bullshit many times, links that don't exist and claiming it does. It even cited a very realistic git commit log entry about a feature that never existed.
Disagree. Say I'm looking for a list of countries and their populations.
Wikipedia almost certainly has this in a nice table, which I can sort by any column, and all the countries are hyperlinked to their own articles, and it probably links to the concept of population estimation too.
There will be a primary source - But would a primary source also have articles on every country? That are ad-free, that follow a consistent format? That are editable? Then it's just Wikipedia again. If not, then you have to rely on the LLM to knit together these sources.
I don't see wikis dying yet.
At work, I had rigged one of my internal tools so that when you were looking at a system's health report, it also linked to an internal wiki page where we could track human-edited notes about that system over time. I don't think an AI can do this, because you can't fine-tune it, you can't be sure it's lossless round-tripping, and if it has to do a web search, then it has to search for the wiki you said is obsolete.
OpenStreetMap does the same thing. Their UIs automatically deep-link every key into their wiki. So if you click on a drinking fountain, it will say something like "amenity:drinking_water" and the UI doesn't know what that is, but it links you to the wiki page where someone's certainly put example pictures and explained the most useful ways to tag it.
There has to be a ground truth. Wikipedia and alike are a very strong middle point on the Pareto frontier between primary sources (or oral tradition, for OSM) and LLM summary
AI companies should be donating large sums of money to Wikipedia and other such sites to keep them healthy. Without good sources, we’re going to have AI training off AI slop.
They should be, yes, but they won't. We already know this from the way those same companies have treated open source projects that they depend on.
One thing that I would really like to see is some kind of hefty tax on any kind of income derived from models trained on Wikipedia. Basically, make it legal to train, to share weights etc freely, and hosting them locally. But the moment you start charging people for subscription, the society should start charging you to maintain the commons that you are profiting from.
(This likely goes for more than Wikipedia, but that case is especially simple since there's a single legal entity that could be given the money.)
People cite this figure a lot, but its a little misleading because when you own your own servers a lot of the expenses that are typically hosting actually fall under a different category.
If you use AWS, the people hired to manage the servers is part of the price tag. When you own your own you have to actually hire those people.
I mean, it's not like you can get away with running with zero SREs if you're running in the cloud. The personnel costs for on-prem hosting are vastly exaggerated, especially if you contract out the actual annoying work to a colo.
Smart hands is more expensive than having dedicated datacenter staff, and the dedicated staff do a considerably better job. It's worth noting that WMF runs _very_ lean in terms of its datacenter staff.
You're also ignoring the need for infrastructure/network engineers, software engineers, fundraising engineers, product managers, community managers, managers, HR, legal, finance/accounting, fundraisers, etc.
1. I think their spending is a good thing. Charitable scholarships for kids and initiatives to have a more educated populous in general are things that I am happy to donate to.
2. As stated in the article, hosting is still a relatively simple expenditure compared to the rest of their operation. If Wikipedia really eats a huge loss, falling back to just hosting wouldn't be unrealistic, especially since the actual operations of Wikipedia are mostly volunteer run anyways. In the absolute worst case, their free data exports would lead to someone making a successor that can be moved to more or less seamlessly.
The only real argument in my eyes is that their donation campaigns can seem manipulative. I still think it's fine at the end of the day given that Wikipedia is a free service and donating at all is entirely optional.
AFAIK, they don't do any scholarships or really do any educational activities. By far their biggest spending item is just $105 million for salaries, mainly for all of its leadership, which is a majority of its expenses.
The second biggest line item is grants at $25 million, primarily for users to travel to meet up.
Then $10 million for legal fees, $7 million for Wikipedia-hosted travel.
I think it's pretty unethical to say you have to donate to keep Wikipedia running when you're practically paying for C-suite raises and politically-aligned contributors' vacations.
Paying the travel for a bunch of highly active volunteer contributors to meet up ocassionally and hash out complex community issues pays massive dividends. It keeps the site moving forward. Its also pretty cheap when you consider how much free labour those volunteers provide.
Whenever people criticize wikimedia finances, i think they miss the forest for the trees. I actually think there is a lot to potentially crticize, but in my opinion everyone goes for the wrong things.
What are the rights things to criticize in your opinion?
Also, asking out of ignorance, what things need to move forward? I thought wikipedia is a solved problem, the only work i would expect it to need is maintenance work, security patches etc.
> What are the rights things to criticize in your opinion?
I think criticism should be based on looking at what they were trying to accomplish by spending the money, was it a worthwhile thing to try and do and was the solution executed effectively.
Just saying they spent $X, X is a big number, it must be wasteful without considering the value that is attempting to be purchssed with that money is a bit meaningless.
> Also, asking out of ignorance, what things need to move forward? I thought wikipedia is a solved problem, the only work i would expect it to need is maintenance work, security patches etc.
I think the person who i was responding to was referring to volunteer travel not staff travel (which of course also happens but i believe would be a different budget line item). This would be mostly for people who write the articles but also for people who do moderation activity. In person meetings can help resolve intractable disputes, share best practises, figure out complex disagreements, build relationships. All the same reasons that real companies fly their staff to expensive offsites.
Software is never done, there are always going to be things that come up and things to be improved. Some of them may be worth it some not.
As an example, there are changes coming to how ip addresses are handled, especially for logged out users. Nobody is exactly saying why, but im 99% sure its GDPR compliance related. That is a big project due to some deeply held assumptions, and probably critical.
A more mid-tier example might be, last year WMF rolled out a (caching) server precense in Brazil. The goal was to reduce latency for South American users. Is that worth it? It was probably a fair bit of money. If WMF was broke it wouldn't be, but given they do have some money, it seems like a reasonable improvement to me. Reasonable minds could probably disagree of course.
And an example of stupid projects might be WMF's ill-fated attempt at making an AI summarizer. That was a pure waste of money.
I guess my point it, WMF is a pretty big entity, some of the things they do are good, some are stupid, and i think people should criticize the projects they embark on rather than the big sum of money taken out of context.
Isn't it true that only around 10% of Wikipedia massive budget is used to actually run the core website? The rest goes to bloated initiatives in the Wikimedia foundations orbit.
Page 21 of their 2024 annual report[1] has expenses listed. About $3,000,000 for web hosting, about $100,000,000 for salaries and benefits out of $178,000,000 total.
How many people are needed to maintain instagram, facebook, etc? Wikipedia isn't just a collection of static content. The page contents include static and dynamic content (via a lua scriptable set of templates), semantic data (wikidata), multimedia management (wikimedia commons), editing tools (WYSIWYG editing, with full support of the wiki markup and templating), global caching infrastructure, multiple datacenters for HA/DR, etc.
There's also the need to support the staff and volunteer developers, which includes wikimedia cloud services, git hosting, config management and orchestration, CI, community hosted tool/bot services, etc.
WMF has ~600 employees, and that's quite lean, for a service of their complexity.
That includes all of the Wikimedia websites and non-profit activities though, not just the functioning of Wikipedia.org proper. That is a much lower percentage of the total.
Infrastructure has long been a tiny portion of wikipedias costs.
I think Wikipedia even makes it easy to export all of its data, I don’t think AI scrapers would be a significant new cost
These are two very different cost factors. Scrapers don't use the available data dumps, that's why they scrape. They are also kind of dumb as they get lost in link structures constantly, which leads to unecessary traffic spikes.
I don't know -- as I said in another comment, my Wikipedia usage has gone down 90% thanks to LLMs.
But that means I'm still using it. Especially for more reference stuff like lists of episodes, filmographies, etc. As well as equations, math techniques, etc.
If you're the kind of person who donates to Wikipedia, you're probably still using it some even if less, and continue to recognize its importance. Possibly even more, as a kind of collaboratively-edited authority like Wikipedia only becomes more important as AI "slop" becomes more prevalent across blogs etc.
But were you originally convinced to donate by one of those giant Jimmy Wales banners that come up once a year? People won't see them anymore if they're using AI summaries.
scraping Wikipedia feels like the stupidest possible move. You can in fact download the entire encyclopedia at any time and take all the time in the world parsing offline.
For such purposes, I'd naively just setup some weekly job to download Wikipedia and then run a "scrape" on that. Even weekly may be overkill; a monthly snapshot may do more than enough.
You can download twice-monthly database dumps, but they consist of the raw wikitext, so you need to do a bunch of extra work to render templates and stuff. Meanwhile, if you write a generic scraper, it can connect to Wikipedia like it connects to any other website and get the correctly-rendered HTML. People who aren't interested in Wikipedia specifically but want to download pretty much the entire internet unsurprisingly choose the latter option.
as somebody that has wrassled with the wikipedia dumps a number of times, i don't understand why wiki doesn't release some sort of sdk that gives you the 'official' parse
I have wrestled with it too. I believe it's because wikitext is an ad-hoc format that evolved so that the only 100% correct parser/renderer is the MediaWiki implementation. It's like asking for an SDK that correctly parses Perl. Only Perl can do that.
There are a bunch of mainly-compatible third party parsers in various languages. The best one I've found so far is Sweble but even it mishandles a small percentage of rare cases.
This. I tried that a few years ago and fell off my chair when I started to realized how DYI the thing is. It's a bunch of unofficial scripts and half-assed out of date help pages.
At the time I though, well it's a bunch of hippies with a small budget, who can blame them? Now I learn that there is 600 of them with a budget in the hundreds of millions??
there are more complex tests in that benchmark. my point was, a decent server can serve tens or hundreds of thousands requests per second when it's a few kilobytes of static content from a RAM database.
as long as your web service isn't horribly mismanaged, even to a $5 VPS the bot traffic is infinitesimal background radiation.
> As Miller puts it, “With fewer visits to Wikipedia, fewer volunteers may grow and enrich the content, and fewer individual donors may support this work.”
Contributors are a tiny % of users. I'm sure they've got some room for improvement on incentivizing new contributors. But Wikipedia is a gift to humanity and I hope we find new ways for them to be paid for their contributions to AI.
> It means that now, people are paying for their AI subscriptions
Most people are not paying a cent. And the people that are, are paying for stuff like coding assistance or classification, not the kind of info you get on Wikipedia.
Looking up Wikipedia-style information on LLM's is not a driving factor in paid subscriptions to ChatGPT etc.
If nobody uses Wikipedia they won't get any donations, and unfortunately they wasted the last two decades blowing literally hundreds of millions of dollars on random community and outreach programs instead of building an endowment in case something exactly like this happened.
No really, it was in the news a few years ago but nothing changed as far as I know.
LLM's have definitely replaced 90% of what I used to look up on a Wikipedia, simply because they integrate from so many more additional sources.
But at the same time I continue to contribute edits to Wikipedia. Because it's the source of so much data. To me, it doesn't matter if the information I contribute gets consumed on Wikipedia or consumed via LLM. Either way, it's helping people.
Wikipedia isn't going away, even if its website stops being the primary way most people get information from it.
Wikipedia gives away your creation for free. The LLM companies do not. Google is operating a loss leader and not "helping people." In fact, quite the opposite.
Search results do not have as chilling an effect as LLM-generated text, because search results are in essence links back to content (meaning human authors benefit from knowing who reads them, people discover who those human authors are, etc.), whereas it is not a thing in LLM-generated text (which may have occasional links back, but they are not complete nor always correct).
AI seems obvious, but social video? Are they saying people watch TikToks instead of reading Wikipedia, or people who used to look things up don’t bother anymore because of TikTok?
I'm sure they do. Is that their primary source? Do they use nothing else? Is any of that information put to economic use in their lives or is it just curiosity?
The line of questioning was about news and knowledge, not ads. Ads in anything will affect people's purchases, so that's not evidence for what that question was actually going for.
It's all right. Wikipedia was a magical device for its time, and it's still a great aggregator of information. It will probably last forever as such a link aggregator. Read-time curation is obviously far better than write-time curation, but the former used to be very hard. Now we have the former for cheap so it makes sense for Wikipedia to be Yet Another Source into the read-time curator. And the existence of a source database like Wikipedia makes many of these tools work a lot better.
People rightfully get upset about individual editors having specific agendas on Wikipedia and I get it. Often that is the case. But the chat interface for LLMs allows for a back and forth where you can force them to look past some text to get closer to a truth.
For my part, I think it's nice to be part of making that base substrate of human knowledge in an open way, and some kinds of fixes to Wikipedia articles are very easy. So what little I do, I'll keep doing. Makes me happy to help.
Some of the fruit is really low-hanging, take a look at this garbage someone added to an article:
> Read-time curation is obviously far better than write-time curation, but the former used to be very hard. Now we have the former for cheap
It's _kinda_ cheap. Wikipedia is so cheap you can fit it all on a phone and search it instantly.
I agree overall but LLMs are just so heavy. I don't know if most people can afford to run one locally, and they're lossy. Both on a phone would be great. I fret a lot about data ownership, you know
Knowledge isn’t knowledge in itself — it’s about how people organize it.
With more AI tools mining existing knowledge and presenting it in increasingly accessible ways, I don’t think AI search fundamentally changes how information and knowledge are organized.
Of course, AI could reshape the organization of knowledge through areas like:
1. Fact-checking and sourcing
2. Drafting new pages
3. Editing and refining wording
…and more
---
Just like Wikipedia already has many bots running behind the scenes, if all these tasks were eventually handled by AI, there would still be things left for humans (or perhaps another AI) to decide:
1. When a fact has multiple perspectives, how should it be phrased to represent different viewpoints fairly?
> I still remember countless word battles on Wikipedia over this.
2. In the age of smartphones and social media, historical moments are documented not only by journalists or influencers but by thousands — even millions — of ordinary people. How should Wikipedia process and summarize such vast, distributed facts?
3. How do we properly incentivize contributors, whether human or AI?
> Wikipedia was born in an era when the Internet lacked reliable information, and building a shared, sustainable, independent knowledge base was a mission that resonated with its early contributors — traffic rewards came later.
4. And of course, geopolitics — Wikipedia must remain independent.
---
A bit of background: I once led a Chinese wiki product, but I eventually gave up on it — because almost no one cared why a wiki should exist beyond being just another searchable content platform.
I like to idea of using AI to make Wikipedia better.
There are many small tasks simple AI agents could handle.
For example:
Go to every article in the English Wikipedia of a Spanish city. Check data like inhabitants, major, etc. and compare them to the Spanish Wikipedia article. (The assumption here is that the Spanish Wikipedia will be more up-to-date for Spanish cities than the English Wikipedia.) Double-check what is written in the Spanish article. Update the English article accordingly.
If such an agents is only allowed to create drafts, human editors could review them and we would get a lot of small updates in.
You have to keep in mind that the original google search, by summarizing search results based on links between pages, essentially destroyed the web of interconnected links that existed at that time, so that the original pagerank algorithm doesn't even really work anymore. And the search results Google returns are worse than they were years ago. And you know, I use it, it is still super convenient, and the long term effects took some time to become apparent.
Now it is again feeding on and regurgitating Wikipedia but again in a way that will end up destroying the thing it is summarizing. Aggregators are parasitic on the thing they derive information from.
The TL;DR is that Wikipedia has a massive spending problem, and if they brought it back to something reasonable, their existing war chest would last them for at least a decade, even with no additional funding.
Remember that time in feb 2005 when Wikipedia went down for several days and lost about a week of data because there was a power failure and they were too shoe strings to have proper back ups, and also had disabled a bunch of ACID features in mysql to get the last bit of performance out of mysql causing the power failure to corrupt the db, because they didn't have enough money for more servers and were desperate to try anything to get the last bit of performance out of their servers?
No? That's because it doesn't happen anymore now that WMF has a reasonable budget.
Its actually kind of ironic given i don't think WMF is particulary well managed or effective in its spending. I just think most of the criticisms of WMF spending are kind of terrible.
There are like 3 orders of magnitude of budget difference; I'm sure a happy middle ground could be found. Even going back to 2010's budget would save $160M/year and still be 17x 2005's budget.
That doesn't mean their spending money on what the people donating want. Most people seem to want to donate to keep Wikipedia running, but that is not necessarily where the money actually goes.
So far the story with Wikipedia has always been that the number of contributors is declining, and I believe this will only get worse when fewer people read it (directly from the source). As much as I hate the idea, are people (in particular, Wikimedia) experimenting yet with using LLMs to contribute to/improve/check Wikipedia? Will page discussions and edit wars in the future get replaced with an arena of partially collaborating, partially competing LLM agents?
The system is off. Contributor would write a new piece. Spammers would go in and remove legitimate articles it links to and replace with spam links. Wikipedia bots go through and purge the new contributions. Contributors stop caring.
I stopped contributing money, and then stopped trusting and using Wikipedia after I saw how wrong they are on a subject I am well-informed on personally (ARM architecture), and how fast they revert properly-cited 100% factually correct changes (literally citing the arm architecture manual).
How could I trust them on things I do not know, if I know for a fact they are unrepentantly wrong about things I do know?
In my experience it's always better to only leave comments on the talk page and at most add template comments to the article, for a couple weeks; after that time passed you can go ahead with editing the page, and there's a better chance of the edits not getting reverted.
Most of them are trending public figures and media, which lines up with what I remembered from my prior Wikipedia analysis. So LLMs are likely replacing a lot of this.
It happens to all, means the good sites to that need add money and the AI crawlers add traffic to all so that even those without adds may get a cost problem.
Not to mention all the sites that will pop up to poison the LLMs in their favor.
On top of that LLMs will either cost more or will get adds too.
traffic falling means wikipedia will be cheaper to run. since they don’t rely on ads, it’d likely not affecting their revenues either (assuming those who don’t use it anymore weren’5 those givîng to it)
It’s probably not good long term for donations or attracting future editors. Possibly less people interested in editing when it becomes more unpaid work for AI companies than actually serving real viewers.
If I may speculate I would assume that the Venn diagram between people moving from wikipedia consumer to tiktok and AI generally don't overlap much with people prone to contribute or donate to wikipedia, so I'm not certain it's a net loss in that regard.
This is really the perfect point in the timeline- "in these uncertain times-" to posit themselves as a working repository of credible information.
Also I firmly believe the Wikipedia app is key to their sustained relevance. Users get forwarded to the app from web browsers with wiki links, this gets people in the dedicated interface.
from a design standpoint there need to be more avenues on each page inviting people to browse and explore, spend more time there.
Little off topic but does anyone else find AI summarise really misleading, almost dangerously misleading ? In a search box you can never rally provide anywhere near enough context to get a sufficient answer on anything of even small consequence.
Yes I do use the internet for “medical opinion” and information and seriously some of the falsehoods it’s provided…similarly anything related to construction. Steer clear.
For Wikipedia to last 1000 years, we need a publisher who sells a print of a snapshot of it every ten years or so. When the civilisation dies, and our civilisation will die eventually, sooner or later, just like any other civilisation, then there won‘t be no internet, no wikipedia foundation or anything. But printed paper might survive.
That little AI generated summary that seems to show up in every Google search is riddled with errors. I've learned that as often as not it's just making things up or confusing two different topics. I would not trust that thing under any circumstances. Its no better than a wild guess from a stranger on the street.
We can't just look up info using an LLM. We have no clue what its weights and biases are based on, and whatever other layers control the output. Just a total black box. It would be an irreplaceable loss if we lost wikipedia to LLMs.
Looking how editors can fabricate stuff OR argue like little kids whether original source from origin country should be mistrusted vs encyclopedia britanica is enough for me to know Wikipedia is no longer what it was and lost its purpose.
>It’s the dishonesty of Wikipedia that bothers me. The implication is that donations are urgently needed to keep the website running. In reality they have $300m in the bank and revenue is growing every year[0]. Even Wikipedia says only 43% of donations are used for site operations[1], and that includes all of their sites, not just Wikipedia. Fully 12% of the money they collect from you is. . . used to ask you for more money[1]
I find it a little annoying for a variety of reasons when universities I've been to ask me for money, but one of the main reason that I don't donate to universities is that I don't want the money I donate to be used for advertising, and especially advertising to solicit more donations instead of actually improving the school.
Yeah ... Some days it feels as though 98% of the net has turned into a giant begging-bowl. Long-time major sites are now saying stuff like "You're reading your last free article." Yeah? Don't count on that.
Many of them are sites that have built themselves without any original reporting. Where will they scrape the content they've used to grow if their sources take the same attitude?
So maybe they shouldn't depend on just search traffic. Their increased spend would suggest they got enough people to try and figure out alternative methods to attract editors and/or traffic.
How is it not conflict of interest when Google's AI summary (which is sometimes hilariously wrong) takes a click away from websites that pay for ads? Specially if it was trained on those websites
The sites paying for ads are usually offering products and services, not information. If you search "Plumbers near me", there will be no reason to have an AI summary, so they can show the ads. If you are searching "what temperature does copper melt", no one was buying ads on that anyway.
The point is that the sites from which the AI summary sourced its information might have ads, and if Google only linked to them they'd get ads impressions
> At this point, I am coming around to the argument that governments should make their own local/national AI.
I think this is a terrible idea. How do current Chinese-built LLMs respond when asked about Tianamen Square? How would American-built LLMs response when asked about Israel/Palestine?
If Wikipedia was the source for training data of the LLM then the answer would be quite horrible. There was a ring of Pro-Palestinian editors (possibly state sponsored) that were writing a lot of misinformation on that topic.
as much as I love Wikipedia and I think we need it, I just realise that I haven't opened Wikipedia in the past few months. My default workflow is now replaced to Option + Space and ask gpt questions.
WMF only spends a couple million on essentials like hosting though. They have over 700 employees, and their budget is pretty bloated with thing that aren't really related to running Wikipedia, including sizeable grants to other organizations. See their org chart [1] or this classic essay [2].
editing wikipedia was close to the most aggravating experience I've ever had. and that includes suing landlords for security deposits and arguing with HOAs.
My personal traffic to Wikipedia fell after around 2019, when activist editors took over, and the site ceased to be trustworthy for a lot of important topics.
Wiki has been dying such a slow painful death. I feel bad. Especially because I was the first generation to have the transition from encyclopedias to wiki. Rip wiki, it'll go down in history for sure.
(looks outside at the street) I'm wondering why would traffic be less because of AI search?? SO I asked AI. Its answer? "By analyzing data and demand in real-time, AI is able to predict peak travel times and adjust train, subway, and bus routes as needed, reducing overcrowding and, in the case of buses, traffic congestion."
In the past, you would type a question into Google. To get the full, detailed answer you had to click on the Wikipedia search result.
With ChatGPT or Google AI Mode, you get all answers directly in the chat. And you can even ask follow-up questions. There is no need to click on a link.
From the data I have seen, 40% of searches on Google used to lead to a click to another website. In ChatGPT and Google AI Mode, this number is lower than 5%. One study (with a small N) even came to the conclusion that the number is 0%.
> But it's a horribly boring "read" when it comes to human consumption.
I couldn't possibly disagree more. The website I visit most often after Reddit and HN is Wikipedia, and I often spend hours just diving through article after article. Learning facts is fun.
The other day I was reading the page about the Library of Alexandria.
The Wikipedia version virtually doesn't mention the stoning of Hypatia or the burning of the library, except for a different smaller fire caused by a war during Julius Caesar.
This page certainly reads like Christian apology, where the almost total destruction of the library by Christian fanatics didn't happen.
For me, this heavy bias in basically every article is the reason Wikipedia traffic is falling.
Cosmos does not make Hypatia’s death so much a religious issue as an anti-intellectual on, but the truth is that it was actually a political one. A second problem comes when Dr. Sagan links her death to the destruction of the Great Library. In fact, in the final episode of the original Cosmos, “Who Speaks for Earth”, Carl Sagan says, “The last remains of the library were destroyed within a year of Hypatia’s death.”
The problem with this is that the last remnant of the Library of Alexandria were almost certainly destroyed in 391, 24 years before Hypatia’s death, and most of the library was likely destroyed, by accident, centuries earlier.
It sounds strange, but we actually don’t have a very good idea of when the Library of Alexandria was destroyed. As best we can tell, much of it was burned unintentionally when a fire spread through the city during Julius Caesar’s invasion in 48 BC. While the majority of the library may have survived that war, it was almost certainly destroyed in the war between Emperor Aurelian and Queen Zenobia of Palmyra in the 270s AD. This also appears to have been unintentional, as a large part of the city was burned.
What little was left of the library was deliberately destroyed in 391, when Emperor Theodosius I banned Paganism. The remaining repository of books in Alexandria was destroyed along with the Pagan temple it was stored in. [Even this has no evidence as pointed out by Tim O'Neill in the blog comments]
I admire most of Dr. Sagan’s and Dr. Tyson work, but when they characterize Hypatia’s death and the burning of the Great Library as the deliberate (and linked) actions of an anti-intellectual mob, they are simply misrepresenting the history.
There are multiple new start ups and existing competition that's impacting them. They are facing competition.
The curious thing is that big LLM folks put together RAG systems which act much like wikipedia. But it's more than that. They built dictionaries, book repos(borderline illegal), news repos, and data knowledge base. These are bigger than wikipedia. Better because you dont have anonymous partisans.
Wikipedia is at a point where they have purged multiple perspectives and it has left an unreliable systemic bias in wikipedia. They are dealing with this problem and competitors are popping up because of these problems.
The hard truth is that LLMs will fully replace wikipedia.
Let's put aside wikipedia being rotten with bureaucracy and obsession-driven bias, which is similar to stackoverflow preexisting flows before LLMs streamrolled.
Fact is, wikipedia is a human driven summarization engine of secondary sources, hopefully in a way that echos the sources consensus.
This is exactly what LLMs are best for, summarizing huge amount of text, and training can easily focus on high quality books and thus exceed wikipedia in quality.
It's enough to read an AI summary where the first line talks about the subject in hand, compared to wikipedia where the first line is the product of some petty argument about a political disagreement
And wait what happens to their quality as soon as the AI companies habe to generate profit.
I doubt you can collect second source references at the same price Wikipedia does.
People aren’t as eager to help a billion dollar company for free to make profit.
LLMs lack the human nuance that a good Wikipedia article requires. Weighing quality sources and digesting them in the most useful way that a human would want and expect — that is very difficult for both humans and machines, and it is why Wikipedia as a whole is such a treasure: Because a community of editors take the time to tweak the articles and aim for perfection.
There are guidelines across all Wikipedia articles that make a good experience for the reader. We can’t even get the world’s greatest LLMs to follow a set of rules in a small conversation.
In my opinion simply using a dataset of high quality books and highly rated academic journals is enough to surpass current Wikipedia quality.
In my experience when using LLMs as a replacement for Wikipedia (learning about history), it is often of higher quality in niche topics and far less biased in political contentious areas
For me Wikipedia is only good for introductions and exploration. You don't have time to read a dense tome but also don't have enough experience in reading research papers in that area? Wikipedia it is then.
Wikipedia is the tabloid equivalent for scientific topics.
LLMs tend to be much more useful for niche topics, because they've most likely been trained directly on the source itself.
> and training can easily focus on high quality books and thus exceed wikipedia in quality.
Then it will never exceed Wikipedia. There are no high quality books to many of Wikipedia‘s articles and even if who qualifies those books as as high quality.
And less traffic is not only a problem of Wikipedia but also other websites.
So AI is killing the source it feeds on.
This is the golden age of AI.
The next age will be filled with less good sources, more AI generated content and sites that actively try to poison LLMs
As others have said, Wikipedia is a tertiary source. A primary source is the original materials, e.g. research papers for example. Secondary sources analyze or summarize, but also connect the primary sources into a bigger work. This is how comprehensive textbooks that cover a wide breadth of topics are written. Wikipedia is for those who are too lazy for even the secondary sources. I.e. blog post sized articles about a particular topic. That's what a tertiary source is and nobody really needs tertiary sources other than for convenience and time savings.
If if all this is true, LLMs would still need to create a repository of such aggregated "summarizations of secondary sources".
LLMs cannot in real time find and read thousands of secondary sources. Especially not if some of these might have already disappeared or are not digitalized.
I can see a future where LLM labs a) donate to Wikipedia and b) contribute to it with agents that suggest edits and review facts.
I think LLM companies will need to donate and contribute to more direct sources are newspapers, journalists, bloggers, coders posting on stack overflow etc and not Wikipedia (which is more of a tertiary source).
I'll go out on a limb and say we _need_ Wikipedia and it's okay that traffic falls.
Physical print encyclopedias got replaced by Wikipedia, but AI isn't a replacement (can't ever see how either). While AI is a method of easier access for the end user, the purpose of Wikipedia stands on its own.
I've always scoffed at the Wikimedia Foundation's warchest and continuously increasing annual spending. I say now is the time to save money. Become self sustaining through investments so it can live for 1000 years.
To me, it is an existence for the common good and should be governed as such.
The Wikimedia Foundation is unlikely to last for 1000 years as an organisation because it doesn't exist within a social / economic system that will last for 1000 years. The US government is already actively plotting against it. Sure, they can try hopping from one country to another, but it won't be sustainable for that long.
It's not even certain if Wikipedia itself can exist for such a long period, given fragility of technological civilisation and data storage.
https://www.usatoday.com/story/news/politics/2025/08/27/wiki...
The Wikimedia Endowment [0] has been created for this. From it's Financials [1] it mentions that
> ... [its mission is...] to act as a permanent fund that can support in perpetuity the operations and activities of current and future Wikimedia projects, which are projects that are approved by and advance the purposes of the Foundation or its successor if the Foundation ceases to exist
[0] https://wikimediaendowment.org/
[1] https://upload.wikimedia.org/wikipedia/foundation/f/f6/Wikim...
But Wikipedia only really exists as long as there are editors of a certain quality and dedication. Without those, what good is the foundation?
The Wikimedia Endowment (which is sorta-kinda separate) is to drive solutions to that exact problem.
By having a separate fund that the Wikimedia Foundation can access to help Wikipedia to have the technical expertise and knowledge workers required to continue the work of the Wikimedia Foundation.
Should the Wikimedia Foundation cease to exist, the funds in the endowment can be redirected to a successor.
EDIT: this is similar in style to the UK's Guardian Foundation, who provides funding to The Guardian newspaper. https://theguardianfoundation.org/
The point is that without Wikipedians adding and improving articles, Wikipedia will die, even if the site could be kept up indefinitely. So it has to remain a relatively wide social phenomenon, an obscure Wikipedia that no students know about and care to use will knowledge rot, even if it doesn't but rot.
Wikipedia seems like a pretty good candidate for a seed vault style “let’s make sure this gets really backed up” type project.
There are a million backups of wikipedia. That's not the main issue, it's the future of the project. If the main org goes down, there will be a million forks of it where none of them have the same legitimacy of the original, edits will die off, views will die off, and the content will become increasingly stale and vandalised.
I see that as an argument for serious research effort into systems of decentralized governance. A project like Wikipedia shouldn't need a centralized community or a singular centralized host. Unfortunately "web of trust" doesn't seem like it will be resolved in a robust manner anytime soon.
You are on the way to the religion pill
in my opinion, such a scenario would end up with a few weeks/months of chaos, before converging on two-three main forks.
[flagged]
The WF is not even 25 years old. It’s crazy to think it will exist another 100 years, let alone 1000.
Even the US isn’t 1000 years old. Who’s to say it will even exist in a remotely recognizable form in 1000 years.
The last 1000 years changed at a snails pace compared to the last 25 years.
In 1000 years Sweden went from a Viking culture to an egalitarian, tolerant, multicultural society. Just imagine how much change you can have in the next 1000 years. 50 years ago you guys had basically an apartheid and women’s place was the kitchen.
The problems with that argument are that first, the Viking culture was mostly rural and agricultural, the raiding and pillaging was a tiny side-effect that mostly happened during times of economic hardship, not as a way of life ... and that today's Sweden is rapidly recessing back to a less-tolerant, anti-multicultural society for many reasons. You also conveniently ignore the much more bloody 'military superpower' phase in the 1600s.
It’s not Ragnar Lodbrok turning into ABBA. It’s a mostly peaceful peasant society getting tossed around by history until it became a mostly peaceful service economy - and the shaking never really stopped.
History is not a straight progress line. Ask the Romans.
If you really want to be accurate about history in what is now Sweden, let’s also mention the horrific mass graves discovered in Birka and surroundings which seems to imply that massacring whole villages, women and children included , was commonplace even before Viking times. The later imperialist period seems less brutal to me.
Pre-agricultural societies are generally much more brutal because you need a lot of territory to feed relatively few people. Agriculturalists can conquer other people and make them work for them in a way that doesn't scale for hunters-gatherers. The consequence is that the combatant/noncombatant distinction is much more blurry, and genocide is often an explicit goal of the war.
Yes! It annoys me to no end when people pretend that time is developing us into a perfect image - we're just the latest generation in a long string of improvements. Making it seem like development is predetermined.
Ask a medieval peasant and he wouldnt even be able to describe how his life is different from his father's.
35 years ago South Africa had the apartheid, and now they outrank the US on a The Economist's Democracy Index [1]. So yeah, things move fast.
Nitpick: The US Voting Rights Act became law in 1965 [2], which is more like 60 years ago. Not sure if that's what you were getting at with "basically an apartheid" but it was the closest concrete landmark near your 50 year timeline.
[1]: https://en.wikipedia.org/wiki/The_Economist_Democracy_Index
[2]: https://en.wikipedia.org/wiki/Voting_Rights_Act_of_1965
Your link shows that South Africa ranks below the US. It is ranked 43rd in the world, vs the USA which is at 28 (in 2024 the latest year).
good point: I'd fix the post but it's too old. Well hopefully it gets downvoted into oblivion soon.
https://en.wikipedia.org/wiki/Racial_segregation_in_the_Unit...
(since posting Wikipedia links seems appropriate.)
[flagged]
Yeah I think I should have been more clear that countries change a lot and sometimes even revert back to a much earlier state … it just sounds funny to me how the country of bloody thirsty warriors that pillaged faraway countries is now maybe the most peaceful place on Earth… at least if you ignore the bombings and gang culture that has developed in the last decade or so, but those are imported from elsewhere, which of course doesn’t mean it won’t become a feature of the country for decades ahead which may bring back some of the image of ruthlessness associated with the old times.
I invite you to read a little bit of history.
These sort of hyper exaggerated comments help no one. Not to mention "other parts of Europe" were literally Islamic for hundreds of years.
I agree with you. The main problem is that we don't have many politicians that want to tackle immigration in a serious and responsible way. So people end up giving votes to the far right.
Lol the pace things change. If there was an accurate time travel movie it would be more of a comedy. I don't care where you're from. Which language you speak. Even Chinese has changed. But any language especially English has changed so much it would be a foreign language if you went back.
> The US government is already actively plotting against it.
the article you linked in no way supports that sentence.
(there's just a committee looking into claims that edits are used for propaganda in general.)
> The US government is already actively plotting against it.
If we're going to take the actions of a couple of low level dope legislatures as, "the US government" and a toothless investigation as "actively plotting against", sure.
Even if this current plotting comes to nothing, there are plenty of ways for investments to fail over 1000 years. Even if we assume that the system continues intact so that the accumulated wealth retains its meaning and is protected, and it isn't directly targeted by the government at any time.
Investments can fail, such as bank deposits (bank collapses), shares (company goes bankrupt), government bonds (government defaults), commodities (price fluctuations as they go in or out of favour.)
Theft can also occur, including by corrupt insiders (sometimes even legally, just by inflating their salaries to ridiculous levels.)
The chance of remaining intact for 1000 years seems very low.
Do we have any institutions to point to, other than the Roman Catholic Church, that have survived this long?
https://en.wikipedia.org/wiki/List_of_oldest_companies
> The US government is already actively plotting against it.
Surely, they can set up shop elsewhere on the globe?
> Become self sustaining through investments so it can live for 1000 years.
I always wondered why more companies or organizations didn’t do this. Pile up money during the good years to allow themselves to not need continued outside income to keep going, so they can do what is right instead of compromising their vision for the sake of hitting quarterly earnings. That isn’t to say they can’t keep making money, but do it for the right reasons that will keep the core business around for the long run.
Unfortunately, Wikipedia doesn't seem interested in this. Their revenue is more than enough be able to invest and sustain the site forever, but they just increase expenses on non-core outgoings https://en.wikipedia.org/wiki/WP:CANCER
Am I misssing something or is Wikipedia net positive 200M? If this is true, that's eye opening. Weird given how much they beg for money on every article.
The amount of money moving through Wikipedia is absolutely mind blowing, and next to none of that is even being used to sustain the site. Hope those still donating to it feel smart about what they're supporting...
Still a drop in the bucket compared to the money going in to AI companies off the back of wikipedia data.
Because that's how you build up a foundation that will be self-sustaining when the donations run out.
It's not a bad strategy. I've looked at Wikimedia's financial statements and have no problem giving a small monthly amount to them considering how much value I get from the site.
I certainly prefer my money going to them than to Zuckerberg or Altman or MSFT shareholders.
The real question, given how handy Wikipedia is as training data, is why it's your money going to them, and not Zuckerberg's or Altman's or Nadella's.
You aren't missing anything. This has been an increasingly popular criticism of Wikipedia. They are doing just fine financially.
The Mozilla playbook, I see.
Mozilla doesn't have any money... do they?
They have that much in assets. I'm not sure what that actually looks like in terms of cash vs. non-liquid assets vs. assets with strings attached.
If you look at revenue vs spend they are net positive by about 7mm last year.
https://en.wikipedia.org/wiki/Wikipedia:Fundraising_statisti...
You pay for Wikipedia because you want it to prosper. The political part of Wikipedia is vital to make it succeed. Even though everyone uses Wikipedia not everyone knows why it succeeded and why it is important.
The fluff is important to have a engaged super users. It is also important to get acceptance in certain circles.
Can you make the case as to why we should want a political, super user based version of Wikipedia to prosper?
what circles? I would like to have greater transparency into what this cabal of superusers doing, how are they recieving the cash flows, etc.
It just seems like every wiki results in defensive mod cabals
[dead]
> Pile up money during the good years to allow themselves to not need continued outside income to keep going, so they can do what is right instead of compromising their vision for the sake of hitting quarterly earnings.
Typically, individuals want to pile up money so that they don't need outside income to keep going, and the shareholders of a "quarterly earnings" company will squeeze the entity to get it for themselves in the form of dividends or higher share-price.
Money locked up in companies is managed at the leisure of its management, regardless of how good that might be for anyone else (ie. the shareholding owners). Money sitting there doing nothing is a poor use of it, so most companies are encouraged to run with only enough working capital to keep operations going.
It sounds weird, but this is better for shareholders and the economy (and companies can raise capital as needed down the line) than having all companies hold 3x the cash on the balance sheet.
The argument would be different for a foundation by wikipedia, albeit you still have problems between what the wikipedia management might want (high wages, little accountability) and everyone else.
> Money locked up in companies is managed at the leisure of its management, regardless of how good that might be for [...] the shareholding owners
But the management is aware that the shareholders can apply (direct or indirect) pressure for the money to be used in certain ways. Ultimately the shareholder can sue the management if they think the money is misused.
In the ideal case, but shareholders are distributed and often it's nontrivial to get enough involved to justify the effort and expense.
If a company retains earnings, it has to pay taxes on them (as profits), but the money is still at risk (from a shareholder’s point of view) if something bad happens to the company (lawsuit or market problem). Shareholders usually want to receive whatever money the company has saved up, to safeguard it from being lost for no reason, and so that they (the shareholder) can put it to use elsewhere. This would change if the government stopped taxing retained earnings.
That doesn't fully explain why this doesn't happen with non-profits.
Many nonprofits do this — it’s called an endowment fund.
Retained earnings are not taxed per se. A company pays taxes on profits. Whether the profits are distributed to shareholders or retained makes no difference whatsoever as far as taxes are concerned.
They are taxed; they are taxed because they are a subset of profits, which is a taxed category. They are not taxed more than other profits, but that doesn’t mean they’re ’not taxed’.
they are not
retained earnings by definition are the accumulation of net incomes, and net income by definition is post tax
what went into the produce the retained earnings (profit) has been taxed
but the retained earnings themselves are not subject to additional taxation (with a few exceptions)
> This would change if the government stopped taxing retained earnings.
No it wouldn't, because
> the money is still at risk (from a shareholder’s point of view) if something bad happens to the company (lawsuit or market problem).
It would definitely change, because a shareholder could then take a loan against the non-taxed retained earnings they are owed. So then it comes down to whether the tax is higher or lower than the loan interest (adjust for the risk you mentioned).
I think some NGOs and privately owned companies actually do this. But most companies exist to increase the wealth of their shareholders, so if they don't consistently grow, they serve no purpose. If NVIDIA kept its exact current revenue and profits for the next two years, their CEO would be fired and their share price would be in the gutter.
Because unless the company is temporarily making losses because of temporarily macroeconomic conditions, what’s the point?
Having a war chest wasn’t going to help any of the retail companies, the technology companies who couldn’t pivot, etc. There is no reason for a company to live forever.
It takes a pretty altruistic leadership team to plan for 1000 years; even the multi-billionaire tech bros only plan to live for ~200 or so.
“A society grows great when old men plant trees whose shade they know they shall never sit in.”
I recently visited Scotland and on a visit to a distillery they mentioned they bought land in the US to grow trees that will make their barrels one day. The trees take over 100 years to grow (if I remember correctly). How is it we can invest ~200 years into a glass of scotch, yet we aren’t willing to take the same care and long term thinking in most other areas.
Even without being around for 1,000 years, I’d think doing this would de-stress and de-risk. Somewhere along way it became a bad thing to have a good, stable, long-lasting business. The only thing that seems to matter now is growth, even if they means instability, stress, excessive risk, and a short stay.
Craftsmen care about what they make, pursuing excellence even if it may take generations. Businessmen care about robbing you now in the most efficient way.
There is also an element of tradition with scotch, which is something that can survive even the occasional craftsman who may not care as much about what they do, as long as they respect the traditions.
Humans don't live that long, and there's a constant onslaught of fleeting fancies, especially in business (Wikipedia foundation should buy some crypto for it's treasury!)
Tradition is simply brand value to be monetized for most businessmen (to add to your criticism). Just look at scotch whisky and multinational conglomerate acquisitions. They would never plant trees in America, they simply order giant vats used for the strongest PX Sherry to get maximum flavor per euro for their blending process.
Some ancient family dynasty Scotch distillery perilous existential vulnerabilities:
[] Indifferent, spoiled rotten progeny seeking maximum return upon inheritance, selling distillery to Seagram's for an immediate gratification windfall fortune.
[] John Cooper VII, the last barrel maker, skill lost at retirement, or Master Cooper VIIth lost savings to Mister Market, or an expensive clandestine affair, extorts 16X per barrel.
[] Seagram's hires brilliant Bill Burr(Breaking Bad car wash business) to pose as EPA agent, threatening federal lawsuit for illegal violation of dumping distillery toxic runoff for past centuries, and/or white oak barrels cause cancer and the distillery has killed victims for hundreds of years.
[] A society grows fallow when old men plant trees whose shade they know they shall never sit in, while the president's daughter threatens to enforce federal white oak forest's toxic(false, but that's ok!) leaching prosecution if landowners do not immediately purchase a fortune of President altCoin.
In this painfully craven hostile world, Benedictine liqueur would seem to be more durable spirit than any Scotch.
That's a bold and daring thing to say on a startup accelerator and venture capital forum, must I say.
While that is technically true I have always thought hn as a mere place to find interesting content. Of course many of us, me included, dream about finding someone who pays millions for our code. But the attitude towards businessmen is generally hostile here and for a good reason. They can't make things and they aren't even interested in making things, and overall they are more often than not just lame and boring. They are interested in making money, ripping people off, being leeches, extracting every possible penny they just can. Weird geeks are what the hacker culture is all about and politically correct c-suite suckers can go pound sand.
Efficient robbery is a craft too, just not one you (or most people) appreciate.
We do appreciate it! There's even posters for the most famous players, posted in prominent public places free of charge.
Now there you Ninja Academy guys go again...
Craftsman are businessesman.
Scotch is aged in previously used Bourbon barrels from the US. Those Bourbon barrels are harvested from local forests (in Kentucky) which are very large and healthy. The probable reason they purchased land was for an investment or perhaps an alternate income stream or even for some sort of government tax break or credit. They may have told you it was to make barrels but it isn't at all likely that will ever happen. Good story though...
I've always wondered if increasing everybody's lifespan would result in more people caring more about the long term future. Our lives are so short in the grand scheme of things that we never bother to really thing about the grand scheme of things.
I've heard that in the Tibetan tradiation where they ardently belive in reincarnation that thinking about the consequences of your actions for the next thousand years isn't unusual
Wake me up (from my cryo-sleep) when even a single one of the longevity tech-bros gets to 150.
There should be many offline copies of Wikipedia since it makes data dumps publicly available
Perhaps Wikimedia is only mentioning a drop in traffic as it suggests how www users may be gathering information, not because it is commercially significant, e.g., decline in audience for advertising
Indeed, it is okay for traffic to fall when a website is not trying sell out www users to advertisers. It does not mean the information offered by the website is any less valuable
Wikipedia, while not being reliant on ads, nor even on donations that much (given how much they have amassed) is still 100% dependent on having active users. People who get a ChatGPT answer will not go on to edit Wikipedia if that answer happens to be wrong. They won't curate articles to notice when someone is vandalizing them and put the right info back.
The term "relevance" usually refers to something, i.e., relevant to what
But it can also mean "popular"
https://en.wiktionary.org/wiki/relevant
"Popular" information, evidenced by web traffic for example, may be important for so-called "tech" companies seeking to profit from intermediating access to free, public information, i.e., acting as a middleman. The information may be valuable to the so-called "tech" company middleman for purposes of supporting data collection, surveillance and online advertising services
However, the so-called "tech" company does not determine the value of the information to others. For example, a small group of people may find information on a web page to be valuable for their individual purposes. The web page may receive little traffic. The amount of traffic does not determine the value of the information to the small group
I think number of "active Wikipedia editors" is more important than number of "active Wikipedia users"
"AI search" might cause a decline in active users, but I cannot see how "AI search" would cause a decline in Wikipedia editors
Many Wikipedia editors are responsible for low traffic pages. I would imagine they are motivated to maintain these pages due to interest in the subject matter not interest in web traffic
The value of information is determined by the person using it
https://en.wikipedia.org/wiki/Value_of_information
In the world of so-called "tech" companies, who use public information found on the www as "bait" to attract potential ad targets for data collection, surveillance and sometimes programmatic, targeted advertising, the information may have a value to those companies, for that purpose. They might measure that value by how popular the information is amongst www users
But I'm referring to the value of information published on Wikipedia to www users, not so-called "tech" companies who seek to intermediate access to free, public information that they did not themselves produce
> It does not mean the information offered by the website is any less valuable
Maybe not less valuable but definitely less relevant. I think both are important.
Wikipedia is the best thing ever. Anyone in the world can write anything they want about any subject, so you know you are getting the best possible information.
Have you actually tried editing wikipedia? It's extremely difficult to get anything approved. You have to provide multiple credible sources for things or else the mods just revert/delete your page.
Isn't is good that there are some forcing factors to help ensure the quality of the content? I get that there's plenty of drama and difficulties in building and moderating the content of Wikipedia, but it certainly does not appear to stagnate in terms of content if you are looking at e.g. the number of articles on English Wikipedia. The overall process appears to produce great outcomes and it is the greatest collection of knowledge created.
https://en.wikipedia.org/wiki/Wikipedia:Size_of_Wikipedia
GP wasn't complaining.
I took it as a statement that it is prohibitively difficult to contribute to Wikipedia, and wanted to point out that a large number of contributions are being made and the resulting quality being high, in part due to the difficulty of making contributions.
My comment was disputing the statement above that anyone can just stick junk in Wikipedia. While yes anyone can submit edits, it's pretty hard to get them accepted so the content on Wikipedia is more reliable than just a public notepad.
You are mistaking quality with difficulty. Many people have quality information for contributing but lack the time for politics.
Where do I mistake quality for difficulty?
My statement was that the quality of Wikipedia overall is high, and that one of the reasons for that is because they set and enforce standards for contributions.
Certainly many people are put off by the process and will not have time to deal with it, but my belief would be that such cases are more likely on more controversial topics, and less likely for less controversial topics. Inherently, collaborating on difficult topics will be a difficult process, which also means that there are likely no easy answers for how to make this process not discourage anyone.
The forcing factors aren't what they are supposed to be though. "Credible" sources and citations are exclusively up to the article moderators personal tastes which are very rarely objective.
> The forcing factors aren't what they are supposed to be though.
Is it clear what they should rather be - and are there any examples of mechanisms that have worked better at a scale like this? How are you judging that they are not what they are supposed to be?
If the resulting body of work, which is the totality of Wikipedia, is able to be a curated and high signal collection of knowledge as a result of these mechanisms, how can it be said that they are not working? Having forcing factors, even if they are not ideally aligned or executed, which pushes contributors to increase the quality of their edits to pass, seems overall like a good thing. I'm not saying that its processes and mechanisms cannot be improved, I'm saying I believe it is incorrect to say that they are not working as a whole.
> "Credible" sources and citations are exclusively up to the article moderators personal tastes which are very rarely objective.
Overall I believe Wikipedia to be curated by a large group of people which coordinate through various rules and consensus mechanisms, where I don't believe it is correct to state that sources and citations are exclusively up to any specific article moderators, as they need to be able to build consensus and co-exist with other moderation.
Exactly because Wikipedia is such a large body of work it seems more resistant to corruption to have a large number of curators with different tastes and motivations. How would you determine that their selection of sources and citations is very rarely objective - especially when objectiveness itself seems quite hard to agree upon for many of the topics covered?
From my perspective it seems far more important to consider the quality and value of the totality of Wikipedia, which is massive and signs that many things are working, rather than insisting that it is not working, especially in times where knowledge is being broadly attacked, and where Wikipedia is one of the targets.
It's too politicized and biased for my taste to give it any of my time.
Biased in what way? The traditional "reality has a distinct liberal bias" way or some other way I'm unaware of?
In countless ways, there's endless wars among admins.
An example though is that several historically relevant facts are edited out to favor some narrative.
E.g. Crimea, the Ukrainian autonomous republic that seceded/was annexed by Russia, tried to split 3 times from Ukraine, once even during the Soviet Union, but those events have been edited out on multiple pages to favor some western-centric narratives.
It literally has an article on this exact thing you're complaining about.
https://en.wikipedia.org/wiki/Republic_of_Crimea_(1992%E2%80...
> once even during the Soviet Union
That's in 1992, there was another occurrence in the previous decade.
It’s very bias too. If you cite a book from 1870 stating something and it doesn’t align with the persons beliefs it can be rejected.
I've seen people cite an english book on a German topic and intentionally mistranslate words to construct a connection that doesn't exist and was never intended by the authors of the english book.
And editing with an account can be very dangerous if you manage to upset somebody with clout.
They can turn content disputes into conduct disputes and conduct disputes into social contests which are either shown on ANI or quietly adjudicated with an administrator block.
The content of Wikipedia is great. Its culture, not so much.
And yet Wikipedia is biased (and hence may even be misinformative) on many topics, because its editors and moderators are biased.
Source: the Wikipedia founder himself..
https://www.dailysignal.com/2025/10/12/wikipedia-co-founder-...
And the real reason that editing Wikipedia is difficult, is because of the ideological bias of the moderators so they support only the editors to conform to their ideology and reject any edits that go against their ideology even if those edit are literal proven truths.
Source: I am an editor on Wikipedia and my own informative useful edits to some important topics on my country have been rejected, while the misinformative and even malicious edits by agents of the enemy nation on those same topics were allowed and continue to persist and mislead whoever reads those articles.
Wikipedia has become a weapon of misinformative propaganda, and it's not a tool or repository of useful accurate information. This is why Wikipedia is banned in schools and universities, because its information may not be credible and Wikipedia has long ago lost the integrity that any worldwide free information repository should have had.
>I am an editor on Wikipedia and my own informative useful edits to some important topics on my country have been rejected
This sounds really interesting, can you provide an example?
Larry Sanger is famously mad at Wikipedia for a very long time now because they wouldn't allow his pseudoscience on it. Later on that also extended to his (increasingly far right) politics.
Ironically, the supposedly-censored Wikipedia has a lengthy article covering all of it including Sanger's accusations: https://en.wikipedia.org/wiki/Larry_Sanger
Well, any writes are reverted within seconds by the gatekeepers, but sure, ok.
Of the dozens of contributions I've made over the decades, some recent, either zero or one of them have been reverted.
Some of the garbage I've been involved with that I either had to avoid or bail out of an edit or revert war:
* A claim on "Fisting" that "seasoned fisters can insert their arm up to the shoulder into the anus", "supported" by a deleted PornHub video.
* Fighting on a Production Car top ten list when Tesla announced that Ludicrous Mode was coming the next year and "expected" to have certain performance stats, where multiple editors fell over each other to make sure it stayed at the top of the list, even when they eventually had to add a column just for Tesla where every other result had "Actual Results" and the Tesla had "Projected/Expected Results".
* A collation of John Deere tractors that described multiple models as "light years ahead of the competition".
* An article on an Australian drug smuggler where exhibits from court case were being removed as "biased".
Were any on these topics?
https://en.wikipedia.org/wiki/Wikipedia:Contentious_topics#L...
I love how this list goes: Arab-Israel conflict, abortion, ...
All making sense so far. But then:
Article title capitalization.
Somehow this is peak wikipedia.
It's peak human. There's a reason why bikeshedding is called that.
Why the hell is Yasuke so controversial oO
* He's a quite interesting historical character popularized by the video game Assassin's Creed Shadows. However, this insertion has also been controversial. East Asian male protagonists are underrepresented in Western media already, and with Assassin's Creed there's the perception that now they snubbed East Asians again, following a long trend of forcefully inserting foreign perspectives in Asian settings, like The Last Samurai, Shōgun, etc.
* There aren't that many real historical accounts about him so people can argue all day about stuff like "Was Yasuke a real samurai?" without clear evidence about who's right.
Since the first games, many fans have clamoured for "Assassin's Creed in Japan"... I suspect they fantasised about being a male Japanese ninja running across rooftops in the moonlight, sneaking over nightinggale floors, and silently assassinating people in their beds, combining what they saw in Altaïr and Ezio, with every Japanese media trope they knew of.
But in the intervening years, Ubisoft and/or the entire AAA games industry is now seemingly driven by a need to conspicuously showcase diversity and inclusion (from some gamers' viewpoints). You could also view it as the gaming industry trying to broaden its audience and get out of the pigeonhole of catering to the base desires of sweaty manchildren, but either way, it's upsetting a certain type of game consumer.
So, when Ubisoft finally got around to setting Assassin's Creed in Japan, and they picked pretty much the only person around that time period who wasn't Japanese as the main protagonist, seemingly to meet diversity goals, capital-G Gamers went bananas over it, like it was a personal affront to them.
To quote from the page:
"In May 2024, it was announced that Yasuke would be a major character in an upcoming video game (Assassin's Creed Shadows). While onwiki disagreement about Yasuke's status as a samurai predates this announcement, the historical figure's Samurai status became part of a culture war around video games (J2UDY7r00CRjH evidence) that media sources have described as a continuation of or successor to Gamergate, leading to an increase in attention to the article. (Symphony Regalia evidence)"
What percentage of Wikipedia does that cover?
The entire world's knowledge has some controversial topics? Oh my! Burn it! Burn it all!!
The librarians of Alexandria would have killed for Wikipedia. It's easily our greatest digital achievement.
This is a hill on which I would happily fight to my metaphorical death, here and now. If you disagree, let's discuss please.
I don't get your point. You are allowed to edit articles on contentious topics. Its just more likely to be reverted. Because the topic is... contentious.
My attempt at a point is that the controversial topics are a tiny percentage of the human knowledge stored on Wikipedia. If there were no controversies, then I would actually start to get worried. That would indicate that there was pure control of information on Wikipedia, like a theoretical CCPedia. [0]
Wikipedia is so open, that they even have their own "controversial" section! Is that not the coolest thing ever?
The chip on my shoulder is that there is a concerted effort to destroy and discredit Wikipedia.
The accomplishment of Wikipedia is not just beating the Library of Alexandria by many orders of magnitude, but doing so while keeping moderation logs in the open as well.
Ask @dang, or anyone that has ever had anything to do with forum moderation, if they would be cool with their moderation logs being completely open. Almost everyone with experience would say 100% no. They likely tried that and saw how much nutso drama it creates. Wikipedia actually does that, at the largest possible scale!
[0] Of course that exists, apparently it's called Baidu Baike
> if they would be cool with their moderation logs being completely open
It takes a certain mentality. That's rare but I think it makes for much better communities on the whole.
However I think most participants, not just moderators, don't like the environment that sort of mentality results in. When anything and everything, including the moderation itself, is up for civilized debate that tends to foster an environment in which it's acceptable to question core parts of people's worldviews. There's little shared doctrine beyond "argue any position you'd like" which most people seem to find intensely uncomfortable.
> The accomplishment of Wikipedia is not just beating the Library of Alexandria by many orders of magnitude, but doing so while keeping moderation logs in the open as well.
There is at least one exception to that rule. Users who attract the ArbCom's attention may get a general block. If they ask what they're blocked for, the ArbCom rep will tell them to read their email. These moderation decisions are not public, not even in a form with PII redacted.
There is also what's called "Oversight", which performs actions that are invisible to the public and administrators alike (though not invisible to some very privileged people)
https://en.wikipedia.org/wiki/Wikipedia:Oversight
There are also "office actions", where essentially the Wikimedia Foundation and/or its legal counsel have been required to do something. In most cases, the office actions are visible and logged, unless they've been required to use Oversight as well. But the main thing is that the office actions will generally not be explained to anyone, as it usually stems from some legal threat to Wikipedia.
https://en.wikipedia.org/wiki/Wikipedia:Office_actions
I have contributed inconsequential lies to clown on a friend and they persisted for years.
In middle school, I pretended my friend was a famous rapper and snuck his name onto random articles. Some of those stuck around really long.
what topics did you make contributions to? From what I understand, it's mostly political and/or controversial topics that have this issue.
I also have problems with Wikipedia's favoritism of insiders who have learned how to navigate its bureaucracy, but the fact that most edits of political and/or controversial topics are immediately reverted is not in itself evidence of a problem. A priori, I would expect that the majority of edits to political and controversial topics are bad and should be reverted.
I've had tax stuff reverted even. eventually was allowed I think. Wikipedia has problems.
I guess you picked “tax stuff” because the tax related thing you edited was a sort of dry tax related topic, but I’m sure we could find lots of controversial topics under the “tax stuff” umbrella.
I don't have an extensive wikipedia career, but I've found that even my few edits to political topics have been accepted.
What did get reverted was a trivial [citation needed] fix, for a musician's page, for a sentence stating they were involved in scoring a film. I found a relevant citation and this was promptly reverted, for reasons that were explained but, at least for me, utterly incomprehensible
You're not really considered a veteran editor until you've won at least 10 Request for Comments outquoting your detractors with at least 100 obscure Wikipedia guidelines and policies.
Or you could make significant contributions, then get banned because of ignorant sociopaths like BrownHairedGirl and her coterie of followers.
She was banned a few years ago.
Indeed, and those same supporters are still there.
I have donated to them in the last two decades, and stopped because of this. Won't feed those beasts
should give them another chance
Working as intended
the free encyclopedia that ANYONE from Black Mesa can edit!
The free encyclopedia that NO ONE will ever edit.
"For those unaware, this is (exactly?) quoting Michael Scott from The Office." - Wayne Gretzky
Sounds like you're either a Wikipedia landlord or don't use it for anything other than STEM articles.
https://en.wikipedia.org/wiki/Wikipedia:Contentious_topics
one big reason why ai is garbage in garbage out
Many said (some still say) that Wikipedia is not a replacement for traditional encyclopedias with articles written by domain experts, such as Britannica.
Many more scoffed at that, saying those people were just stuck in their old ways and unable to adjust to the obviously superior new thing.
Is that you? AI applications are different than Wikipedia and are better in some ways: Coverage is much greater - you can get a detailed article on almost any topic. And if you have questions after reading a Wikipedia article, Wikipedia can't help you; the AI software can answer them. Also, it's a bit easier to find the information you want.
Personally, I'm with the first group, at the top if this comment. And now truth, accuracy, and epistemology, and public interest in those things, take another major hit in the post-truth era.
Right, and where are all those LLMs without the billions upon billions of lines of text written by humans? A not insignificant number coming from Wikipedia?
Also, LLMs don't produce truth. They don't have a concept of it. Or lies for that matter. If you are using LLMs do study something you know nothing about the information provided by them is as good as useless if you don't verify it with external sources written by a person. Wikipedia isn't perfect, nothing is, but I trust their model a shitload more then an LLM.
> where are all those LLMs without the billions upon billions of lines of text written by humans? A not insignificant number coming from Wikipedia?
Where is Wikipedia without all the learning and information from other sources, many of which it put out of business?
> Also, LLMs don't produce truth. They don't have a concept of it. Or lies for that matter. If you are using LLMs do study something you know nothing about the information provided by them is as good as useless if you don't verify it with external sources written by a person. Wikipedia isn't perfect, nothing is, but I trust their model a shitload more then an LLM.
Wikipedia produces consensus that correlates with truth to some degree. LLMs produce statistical output, which in a way is a automated consensus of the LLM's input, that also correlates with truth to some degree - and the correlation is hardly zero.
I agree that information has no value if you don't know its accuracy; it's always a sticking point for me. IMHO Wikipedia has the same problem: I have no idea how accurate it is without verifying it with an external source (and when I've done that, I've often been disappointed).
Has anyone researched the relative accuracy of Wikipedia and LLMs?
The comment about Wikipedia supposedly putting companies out of business is so goofy I'm not even gonna comment on it. I'm surprised you'd bother trying to make a point there.
The difference is humans have a concept of truth, humans have intent. A person, taking an aggregation of their research, expertise, and experience to produce an article is (presumably) trying to produce something factual. Other humans then come along, with similar intent, and verify it. Studies in the past have shown Wikipedia's accuracy rate is roughly on par with traditional encyclopedias, and more importantly sources are clearly documented. Making validation and further research fairly easy. And if something isn't sourced I know immediately it's more suspect.
LLMs have no concept of truth, they have no "intent". They just slap words down based on statistics. It is admittedly very impressive how good they are at doing that, but they don't produce truth in any meaningful way, it more a by product. On top of that all its sources get smashed together, making it much more difficult to verify the validity of any given claim. It's also unpredictable, so the exact same prompt could produce truth one time, and a hallucination another (a situation I have run into when it comes to engineering tasks). And worst of all. Not only will an LLM be wrong, but it will be confidently and persuasively wrong.
> The comment about Wikipedia supposedly putting companies out of business is so goofy I'm not even gonna comment on it.
I've learned that when people don't have any merits to argue, they turn to ridicule. Right back at you buddy.
> The difference is humans have a concept of truth, humans have intent. A person, taking an aggregation of their research, expertise, and experience to produce an article is (presumably) trying to produce something factual.
It's pretty naive to think that humans have intent and motivation for the truth, and no others. Just look around you in the world - most communication disregards the truth either carelessly or incidentally (because they are motivated to believe or claim something else) or intentionally (lots of that).
> LLMs have no concept of truth, they have no "intent".
My calculator app has no intent or concept of truth, but outputs truth pretty reliably.
To think that I'm saying all humans intend to produce truth you'd have to intentionally misread my comment. Wikipedia obviously has to deal with bad actors and vandalism, and they have processes in place for that. My point is that the intent matters.
Calculators aren't a useful analogy for LLMs. They produce a deterministic output based on a (relatively) narrow range of inputs. The calculations to produce those outputs follow very rigid and well defined rules.
LLMs by their very nature are non-deterministic, and the inputs/outputs are far more complicated.
> Wikipedia obviously has to deal with bad actors and vandalism
People produce false information for many reasons besides malice, and even when they intend to be honest.
Look how much bad information was out there before the advent of scientific method.
You're aware that if you don't have anything useful or interesting to contribute to the conversation you can just not comment right?
That's right: People turn to insults when they have nothing to say on the merits.
It's not an insult when it's true. I don't think you've made one comment that actually added something useful. I did my best to reply to what was there, but you didn't give me anything to work with. Your last comment was so unrelated there was no where left to go.
If you have something actually relavent to say you're welcome to say it.
Here's an opportunity to talk about listening, epistemology, and human intercourse:
> It's not an insult when it's true.
It's not slander, but it's certainly an insult. If you tell someone they are fat and ugly, it's an insult regardless of its truth and you shouldn't say it, ever. There's never a good reason for personal insults.
> it's true
> you're welcome to
This assumes your perspective is truth. That is the case for nobody in the world; in fact, I also have a perspective that I'm confident in, as do many others. Your statements also assume that, perhaps as the arbiter of truth, you have some authorization or power to enforce it. Again, that's nobody's business.
We're in a world of peers, generally speaking, and none of us know who is right. We need strategies to navigate that world, not the one where truth is given to you.
> you didn't give me anything to work with
When I feel like you do, it's a signal I need to listen better - the other person probably does have something to say and I'm missing it. It's possible we're talking past each other, but that's never a reason for insults.
(human intercourse)
Note that the signal is that I need to do something, not the other person. That's not because I'm 'wrong' or 'right' - those are mostly unknowable and irrelevant because 1) We're in a world of peers, generally speaking, and none of us know who is right. Also, 2) I'm the only one I can control and am responsible for, and ...
3) Respecting other people is always more important. That's a strategy for, and wisdom in, a world of uncertainty (as described), as opposed to a world of certainty. Also, it's a strategy for social creatures in social groups - it keeps groups strong and functioning. Finally, it's strategy for both loving and respecting yourself - you deserve it. You're better than insults, I'm sure; and I sometimes say the wrong thing, but I'm better than that too.
> Where is Wikipedia without all the learning and information from other sources, many of which it put out of business?
Which businesses did Wikipedia put out of business? You will frequently see a 5k word article used for a couple of sentences in a Wikipedia page, with the entire Wikipedia page itself being smaller than one paper it cites for one small corner of said page. When I’m researching events, I frequently go to Wikipedia to find sources as search engines have a drastically larger recentism bias.
> Has anyone researched the relative accuracy of Wikipedia and LLMs
No comparative research on this specific topic has been conducted afaik, and most comparative research is aging (likely, to Wikipedia’s own detriment–general consensus is that Wikipedia’s reliability has increased over time). However at the time of research publication, the consensus seems to be that Wikipedia is generally only slightly less reliable than peers in a given field (ie textbooks or británica), although Wikipedia is often less in depth. The most frequently cited study is a 2005 comparison in Nature which found 4 major errors in both Wikipedia and Británica, and 130 minor errors on Británica whereas 160 on Wikipedia. All studies are documented on Wikipedia itself, see [[Reliability of Wikipedia]]. LLMs… do not have this same reputation.
> Which businesses did Wikipedia put out of business?
Just as a start, other sources of reference, including encylopedias, dictionaries, websites, etc. For example, I'm sure it impacts McGraw-Hill's AccessScience, which likely you've never heard of.
> This is documented on Wikipedia itself
Maybe there's a little bias there? Would Wikipedia accept Wikipedia's analysis of its own reliability as a valid source?
I've heard that claim, but having no knowledge of the accuracy of any particular article, it's not worth very much to me.
> LLMs… do not have this same reputation.
They don't with you, but many people obviously use them that way. Also, reputation does not correlate strongly with reality.
> Just as a start, other sources of reference, including encylopedias, dictionaries
This just seems like healthy competition. I thought we were talking about a situation where Wikipedia’s use of other encyclopedias is an instrument of their demise.
> Maybe there's a little bias there
Paradoxically, I suspect you’d be pleasantly surprised about how tough this article is on itself. A lot of attention is given to bias in this case.
> Would Wikipedia accept Wikipedia's analysis of its own reliability as a valid source?
First, it is not Wikipedia’s own analysis. Editors should not present their own conclusions from research, just what each paper says. See [[WP:SYNTH]]. Second, generally Wikipedia discourages anyone citing it as it is not a stable source of information. Much better is to use the sources the article itself conveniently cites inline. As a general policy citing any encyclopedia is discouraged.
> having no knowledge of the accuracy of any particular article, it's not worth very much to me.
Wikipedia does have internal metrics grading the quality of an article. [[WP:ASSESS]]. In general though, even entirely discounting the Wikipedia component of the británica comparison, based on británicas own failures it seems wise to verify each and every claim in an encyclopedia, which Wikipedia does an excellent job of helping you do.
> They don't with you, but many people obviously use them that way. Also, reputation does not correlate strongly with reality
OpenAIs own benchmarks show much higher hallucination rates than any study on Wikipedia. Wikipedia itself is quite close to a ban on LLMs for reliability issues. If you ask literally any layman “has ChatGPT ever been wrong for you” they will say yes, either in that moment or after only a little prompting. It is much harder to elicit such a response regarding Wikipedia in my experience
https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f372...
> "It is much harder to elicit such a response regarding Wikipedia in my experience"
You're sincerely claiming that people can't think of times they've seen vandalism on Wikipedia?
Correct. The amount of Wikipedia pages at any given moment with active vandalism is vanishingly small. The only time I have ever stumbled upon vandalism is as part of my work as a volunteer there actively looking for such cases. Looking at my feed of possibly problematic changes at the moment, about 3 entries are appearing per minute with the most recent revert being just 2 entries ago. It is significantly worse while school is in session in my experience, but vandalism very rarely lasts long. Talking to people, people frequently confess to vandalising wikipedia at one point or another. When I ask them "how long did it survive" they tell me answers ranging between "a few moments" to "5 minutes." So to answer your question, I believe it is unlikely the average person has seen vandalism on the site barring those looking at their own shit.
>> Just as a start, other sources of reference, including encylopedias, dictionaries
> This just seems like healthy competition. I thought we were talking about a situation where Wikipedia’s use of other encyclopedias is an instrument of their demise.
Somewhere above, someone complained that LLMs were harming Wikipedia, a source of its information. My point is that Wikipedia did the same to others.
> > Which businesses did Wikipedia put out of business?
> Just as a start, other sources of reference, including encylopedias, dictionaries, websites, etc. For example, I'm sure it impacts McGraw-Hill's AccessScience, which likely you've never heard of.
Your “for example” in response to a question about what businesses Wikipedia put out of business is a business that is...still in business?
I bet your mind is more subtle than that.
> post-truth era.
I know it’s completely normalized and the official name, but this has to be the most dangerous euphemism of our time.
It’s the era of lies.
this is pedantic, but I think its important. We can say that a truth and lie are exclusive. when we lie, we know the truth. we just make up something else because it fits our agenda better. post-truth implies that the distinction doesn't matter anymore. nothing is true, everything is true. no statement can be evaluated outside the intent in which its delivered. that's much more substantial shift than just presenting a set of lies as the truth. a post-truth society is almost compelled to devalue science or any other pursuit of knowledge. by making all the voices selfish and mundane, it explicitly rejects beauty and accomplishment.
Whenever I can, I try to think of modern events from the point of view of archaeologists digging through the layers, at some point in the distant future.
Given that perspective, my thought was: "Hey Bob, look at these morons, they called easily proven lies 'post-truth!' Can you believe that? In a civilization based on science, with AI, nuclear, and biological weapons?! No wonder they died out right after this. How did they not see this coming? Anyway, what's for lunch?"
Post-truth means a couple of things to me:
It indicates that it's a follow-on to postmodernism. To a significant degree the post-truth era is built on a reactionary attack on postmodernism - you can see it on HN, where many people reflexively attack like a mob anything they perceive is postmodern. You can see it in so many people who will accept lies and disaster over postmodernism.
And post-truth is a postmodern term - ironic, ridiculing, makes you think, has some energy to it. How absurd to be literally //post-truth//.
> era of lies
That's a post-postmodern term. No irony or wit; a term of despair. :)
>It indicates that it's a follow-on to postmodernism.
How? it's just postmodernism itself.
There is no truth because everything is relative. There is no singular, objective truth, facts are intrinsically bound to their context, hence post-truth.
> How? it's just postmodernism itself. / There is no truth because everything is relative. There is no singular, objective truth, facts are intrinsically bound to their context, hence post-truth.
That's not postmodernism but a caricature of it by its critics.
Postmodernism cares deeply about truth. It is highly skeptical of power, bias, perception, etc. and provides tools to mitigate these risks to truth.
Post-truth is cares very little about truth and is especially non-skeptical, imho.
> Postmodernism cares deeply about truth. It is highly skeptical of power, bias, perception, etc. and provides tools to mitigate these risks to truth.
I think someone told you this and you believed it, but the track record of postmodernists in academia tells a different story.
I once saw a history lecture from a dean of the history department who had written of a book on WWII in the Philippines. This is one of the most heavily studied and well documented periods in human history. There are dozens of books and hundreds of papers on this topic.
My father-in-law, who was born there and lived through the war there, read the book. He called it historical fiction. The book cherry-picked facts badly, ignored events which countered the narrative of the book and in general made every effort to promote an entirely counter-factual narrative of events. In addition, the author had no real expertise in military or naval warfare and didn't seem to understand even the basics about how wars are fought.
When things like this happen on a regular basis, it is hard to say that postmodernism cares about objective truth in any way. In fact, they seem to actively dislike reality. This isn't a caricature, this is from the horse's mouth. If you want people to have a different perception of postmodernism, make events like this have some sort of penalty, because he's still the dean many years later.
> I think someone told you this and you believed it
Obviously you don't care about the truth but just fabricate bullshit about other people that's convenient to you. I know the topic well; you clearly don't.
Your father is one source; he no doubt has his experience and memory which are important to him. If you look at the sources of a scholarly history book, you'll see thousands of people - that's how history is done, by researching primary sources like your father. Of course not everyone will agree; people will have widely varying perspectives (which is something that postmodernism helps you understand and dealt with).
These cheap shots at me, scholarship, and postmodernism - which has its flaws, but not this stuff - are an insult to everyone's intelligence, including yours.
It's hilarious to read American history and read in every book how everyone who had something the US wanted, or went counter to US interests accidentally happened to have been evil in some way so the US could conveniently show up out of selfless desire just to right the wrongs and end up with the geopolitical result that it wanted in the first place.
People with decades long scholarly careers write this shit (some even having the highest credentials), and people eat it up.
History as written by (especially US) historians is just racist boomer fanfiction (pushed as propaganda to enforce the national myth) for your own country. This is especially true of the US.
And their errors and not subtle by any means. When they write something that is so wrong about you that every man on the street who even has cursory familiarity with the subject would reject as not even wrong, so far from the truth it's clear to the observer that the one who made these claims doesn't even have basic familiarity with the subject.
You've countered this argument with an appeal to authority (his word against thousands of researchers) - how fortunate was a person like Galileo who could just make people look into a telescope and show his numerous and highly distinguished opponents that they we wrong - unfortunately no such thing can exist for history. The next best thing I could recommend to US scholars is to have their work reviewed by top and highly respected local scholars for obvious errors, not biases of overarching narrative but, basic shit and continuity errors that common man on the street wold laugh at.
How could this be? Do I believe Americans to be specially dumb? Just like Big Tobacco pushing studies on the health benefits of smoking for pregnant mothers, and Coca Cola delegitimizing the view that sugar is bad, US historians have a vested interest in propping up US imperialism - or are acting as reactionaries saying everything the US (or white people) has ever done was purely evil, what you have is a partisan shouting match (also called activism), that is the exact opposite of scholarly work.
Caricature? Foucault talked about "regimes of truth" determining that context and society are what determine truth. Baudrillard straight up said "The simulacrum is true" putting it over objective fact. Derrida's whole deal was about how language constructs, rather than describes, reality.
If all of this sounds ridiculous that's another matter, if I actually wanted to cast shade upon them then gee, I'd just quote their stances on sex with minors.
First, I mean no personal insults. Let's discuss ideas.
As someone who grew up being de-programmed from Soviet propaganda by my parents every time I came home, starting from pre-school, I cannot even begin to communicate how allergic I am to this discourse. "There is no truth" is some grade-A bullshit to me. What's next? Maybe Stalin wasn't Hitler's military ally to start WWII? Maybe we live in a simulation? What's the point of anything? 1+1=3!
I could just be dumb, but my theoretical view from 30,000 feet, or 30,000 years in the future can be read here:
https://news.ycombinator.com/item?id=45661175
While we are in an era of lies, it's only because so much of the public no longer finds truth compelling. Hence, post-truth.
You are absolutely right.
But “era of lies” doesn’t sound nice because nobody wants to be a liar… so “post-truth” sounds better: “I'm telling the truth. Almost. But I'm not lying.”
The inevitable AI investment bubble popping is going to be a big deal, but I believe that the larger bubble of political lies popping is much more interesting to think about.
What is that going to look like? How does one hedge against that eventuality?
It'll probably be a black swan anyway. So it's best to stay calm, because to hedge against it...
Stephen Emmott ends his book Ten Billion with the line "Teach my son how to use a gun."
Interesting reference. I had not heard of him, or that book from 2013. 10B is now projected to be our peak, and at 7B we have so much spare food that we burn it as automotive fuel. And now... we realize that population "collapse" is much more likely to happen than endless growth.
Obviously I have not read the book, but do you think it holds up in 2025?
Many books on the subject are very much "doomsayers" or "preppers" style.
The Roman Empire has been crumbling for 400 years, so it's likely that we won't experience the collapse of society as described in most history books either - life is too short for that. Unless a black swan comes along…
To answer your question, I like to come back to the book because it's written in the style of Dan Brown :) - short, punchy chapters. And it still makes sense (to me).
Worldwide population peaked sometime between 2015 and 2020 at somewhere between 7.2 and 7.4B. So my take on the book is that it isn't very good.
Also, biofuels based on most crops other than sugar cane, in addition to not being very helpful in the fight against AGW, triggered large price spikes and political turmoil in a dozen different countries at once. Perhaps you heard of this event, we call it the Arab Spring. We are still dealing with the fallout of it.
>> I believe that the larger bubble of political lies popping is much more interesting to think about.
Does this mean that politics will finally get out of everything and give me back my sports that will be free of the constant political pandering?
When functional, politics should be boring af, and happen in the background, so yes.
Someone made a similar argument to me recently about AI, talking about how programmers used to have stacks of reference books at their desks and how many of the old guard had to have trainings convincing them to use google as a reference.
I guess this argument was supposed to convince me to stop being such a luddite and accept the inevitable future, but really, in an increasingly post-truth world, it made me want to go and get myself a stack of reference books.
I prefer the reference books myself - life is too short for anything but the highest quality information.
But that's not how the world has worked out recently ...
Life is too short to waste it leafing through printed documentation.
Use electronic versions.
>I've always scoffed at the Wikimedia Foundation's warchest and continuously increasing annual spending.
what are they increasing spending on? Are they still trying to branch out to other initiatives?
I understand, even with static pages, that hosting one of the largest websites in the world won't be cheap, but it can't be rising that much, right?
I imagine that, unfortunately, global legal counsel these days will likely be taking an increasing chunk of their change.
The production of factually accurate content is a pretty expensive job.
We who were born before this era really took off, are spoiled by the journalism standards and information purity levels of the past, especially post the fall of the USSR.
Wikipedia is impressive on what it manages to coordinate on a daily basis, especially given only 644 FT staff.
> I understand, even with static pages
Wikipedia's pages are not static.
Realistically though most of the budget is spent on improving the website not just treading water.
For 2023-2024, their budget was ~$177 million. Travel & events was 7.4% of their expenses. Processing fees on donations was 6.4%.
Grants & movement support was 25%.
Hosting was 3.4%. Facilities was 1.4%.
The Wikimedia Foundation is another Komen Foundation.
They reported a headcount of 644 for 2024–2025.
It's all very open if anyone wants to track down details themselves: https://meta.wikimedia.org/wiki/Category:Wikimedia_Foundatio...
2025–2026 is in-progress: https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_...
>Similar to last year, technology-related work represents nearly half of the Foundation's budget at 47% alongside priorities to protect volunteers and defend the projects of an additional 29% – a total of 76% of the Foundation's annual budget. Expenses for finance, risk management, fundraising, and operations account for the remaining 24%.
Maybe they can use the immense volunteer talent at their disposal to build their own AI/LLM.
I'm sure all those editors with decades of experience can do quickly outdo OpenAI and Grok and what have you.
So, as a non profit the other 55% or so (approximately 95m) goes to salaries? That's interesting considering how much work is performed by volunteers.
How so? I imagine there are many things involved with Wikipedia that volunteer editors don't do.
Yeah, probably. I understand there's a lot if logistics I simply don't know about. I was inquiring about more details on what the paid staff need to do.
You can get a general sense of it from the org chart here: https://meta.wikimedia.org/wiki/Wikimedia_Foundation/Organiz...
A lot of it is engineers who work on improving the software that runs Wikipedia, and keeping the site running, which you can see happening at https://phabricator.wikimedia.org -- outside of security issues, all the dev work is done in the open. There's constant ongoing work on making Wikipedia and all the related projects work better.
There's also people who do the fundraising, community management, legal defense, etc. Then there's general HR infrastructure around employing hundreds of people.
Basically, that "Hosting was 3.4%. Facilities was 1.4%." point gets brought up, and neglects to mention that you then need to pay for a bunch of people to manage those servers and facilities.
(Disclaimer: I'm an employee of the WMF. I'm just an engineer, so I'm not speaking authoritatively about financial details.)
Why are they paying 6.4% on processing fees? What is "movement support" and where is the travel to? Do they have to publicly disclose these disbursements anywhere? This seems sketchy at best.
‘Processing fees’ likely includes the cost of administering the CRM, creating tax receipts and reports, donor support, and all the other ‘processing’ tasks that come with running a large fundraising effort. It wouldn’t just be the credit card fees.
Wikipedia is full of the kind of stuff mainstream payment processors balk at, so they might have to use a higher risk processor with higher fees.
Easily Checked:
Wikimedia accepts Paypal, Apple Pay, Google Pay, Visa, Mastercard, Amex, Check, ACH and Money Order.
Pretty hard to argue that mainstream processors don't like them.
Processors charge higher fees to merchants that are in lines of business with high fraud and chargeback risk, has nothing to do with whether they agree with them morally.
They refuse merchants with business they don't like.
If it were the case that processors didn't like what wikipedia publishes, they would not be able to accept payment, not have high fees.
I can't imagine that wikipedia has high chargeback rates, and clearly the processors don't mind doing business with them.
The processing line item probably includes not just the fees that they have to pay to processors, but FX fees, the cost of banking, the cost of paying people to open envelopes, the cost of accounting, etc.
> I can't imagine that wikipedia has high chargeback rates, and clearly the processors don't mind doing business with them.
Its actually somewhat common for people who steal credit cards to use non profits like wikipedia to "test" them. Typically such sites have no minimum donation, have donations from all over the world so fraud detection wont think its weird you're spending money half way across the world.
Check the source on the donation page. It looks like they actually use something called Gr4vy above all that to handle payments.
https://gr4vy.com
Using a platform with its own fee on top of payment processor fees would explain the 6.4%.
Don't you think their brand recognition would be an easy way around that?
No. Why would a payment processor make an exception to its risk-based rules for an organization that increases exposure to that risk? Brand recognition is a liability in this case.
I agree, we will need Wikipedia and it is OK if the traffic falls, and that AI is not a replacement (and videos are not a replacement either).
Printed texts are still useful but so is Wikipedia (I continue to use both).
I think we need Wikipedia competitors.
Wikipedia is a victim of it's own success, it was excellent at avoiding bias for quite awhile and the vast majority of articles are extremely well written.
However it's massive popularity and dominance have also led to, well this guy put it best: https://en.wikipedia.org/wiki/John_Dalberg-Acton,_1st_Baron_...
Just fork it, the license is permissive :^)
Elon Musk is launching Grokipedia this week as a competition to Wikipedia.
LOL I just tried to search "U2" on it but it sent me to a Twitter post. That Twitter post required me to login to see if I wanted to see it. What a joke.
It hasn't launched yet.
The webpage was there which is what I tried last night. Is it gone?
There seem to be several fake websites already.
I thought you were joking, Was I wrong.
Ah yes, launched by the guy who keeps tweaking his own chatbot because it doesnt barf out the lies he wants it to tell.
I'm not sure why he feels the need to launch a competitor to Conservapedia.
If he's aware of it, he probably considered it to be insufficiently "based".
Wikipedia is fine. Bias cant be avoided, only made explicit, and the fact that you think it can means you should probably move to Conservapedia or whatever slop Musk will release.
> Conservapedia
This is the first time I've heard about it. Is that meant to be a satire website? I can't tell anymore.
No, it was a fork from like a decade ago.
https://www.conservapedia.com/Main_Page
It started out crazy but it only got crazier as time went on.
The biggest problem with wikipedia competitors, is the only people who tend to put in effort to make one are usually crazy.
I notice this with a lot of projects/communities. The only people who tend to create change are usually eccentric.
like the xorg fork
The page on the Second law of thermodynamics has a chapter on "Liberal abuse of second law of thermodynamics". I wasn't sure if I was reading an imitation of The Onion that tries to take itself too seriously.
It's founded by the son of https://en.wikipedia.org/wiki/Phyllis_Schlafly, so that brand of crazy is basically vintage.
That said, it is indeed so insane that trolls sometimes become contributors just to see how far they can push it. There were several known cases of trolls going far enough undetected to get various admin rights (to block etc).
Shouldn't it be minimized? I think it's increasing on Wikipedia. I think there's too high of a barrier for actual experts to write/edit. What's left is primarily idealogues with a certain leanings resulting in a positive feedback loop to more bias. I don't think anyone should want an echo chamber where an encyclopedia is supposed to exist. It's not there yet, but this is exactly how they form - see reddit.
Yeah, yeah, let me guess, the truth has a liberal bias.
But continue to insist that everyone pointing should just move somewhere else, that will certainly make it less biased and more factual.
>Shouldn't it be minimized? I think it's increasing on Wikipedia. I think there's too high of a barrier for actual experts to write/edit. What's left is primarily idealogues with a certain leanings resulting in a positive feedback loop to more bias. I don't think anyone should want an echo chamber where an encyclopedia is supposed to exist. It's not there yet, but this is exactly how they form - see reddit.
I find this kind of a fascinating social phenomena.
I guess to give a personal example, I was trying to update a few pages about a country's Olympic history - that is, their Olympic bids, a few athletes, etc.
Unknowingly, I had stumbled across a particular power-editor's fiefdom, because they created all these pages and they were very aggressive in policing their articles to meet a certain style, tone and their beliefs.
Searching this person's username up (they went by their real name), they were closely related to that country's Olympic committee and an employee of a Ministry of Sport of sorts. The articles had lots of anonymous IP address edits from an university network this person was affiliated with in their program portfolio.
There was a clear conflict of interest, and I tried to point that out when they mass-reverted my edits, but they seemed committed to accusing me of edit warring by not sandboxing my changes and waiting for their personal review and approval, and quoted at least 12 different Wikipedia policies on notability, style, acceptable citations, etc. I still feel I was in the right, but I didn't have the willpower or stamina to fight against several requests for comments, speedy deletions, etc. They did get a warning from an administrator and some detractors in the discussion threads, but they weren't willing to let it go and at that point, it wasn't fun anymore for me. I have better things to do than to fight factual and nitpicky disputes on Wikipedia.
> Bias cant be avoided, only made explicit
This is so untrue and this is a harmful belief. And I'm pretty much as liberal as they come.
I get the sentiment of what you're saying, but I don’t think that’s totally fair to the parent. Biases aren’t automatically "bad" or "failings". they’re akin to heuristics, and it’s practically impossible to eliminate them entirely. For example, we’re all here talking about how we should treat Wikipedia with skepticism. That's a sort of "neutral" bias that doesn't conjure a strong emotion, and is perhaps more acceptable for it, and probably leads to better informational hygiene overall.
In fact, the claim that “bias can be avoided and should be absolutely”, that is implicit in your resposne reflects a bias of its own: a bias toward moral or intellectual purity, as if the parent recognizing bias is equivalent to endorsing it. I get that this is a pedantic point to make but to come at the parent with such vigour for being realistic, again seems a bit unfair
It depends on if you take the parent (err GGP) literally versus in context. My personal experience has been that people claiming that bias is unavoidable (alternatively that objectivity is impossible, or that everything is political, or ...) have usually been attempting to justify a reasoning process that supports a political or religious slant of one form or another.
A helpless "we couldn't possibly do anything about that issue" sort of mentality.
That said if we're talking literally then I fully agree with you that heuristics are a form of bias and can sometimes be a very good thing on a case by case basis.
Yep fair point. I was taking them literally, though I didn't necessarily feel the context meant their post was bad faith or concessional, just a simple truth. Your elaboration adds a fair bit to your argument though and I am pretty sure I agree for the most part.
> In fact, the claim that “bias can be avoided and should be absolutely”, that is implicit in your resposne reflects a bias of its own: a bias toward moral or intellectual purity, as if the parent recognizing bias is equivalent to endorsing it.
Nonsense. Your definition of the word "bias" includes any assertion whatsoever. Bias is distortion from reality and truth. Saying that we can avoid distortion is not itself a distortion. I never claimed that all bias should be avoided, but the post I responded to said that bias can't be avoided.
Also,
> a bias toward moral or intellectual purity, as if the parent recognizing bias is equivalent to endorsing it.
There is no conceptual connection here between "purity" and the equivalence of recognizing bias with endorsing it, nor is saying "bias can be avoided" related to, or a kind of, "purity" in any useful sense. Stop using abstract words for effect and speak simply.
> I get that this is a pedantic point to make but to come at the parent with such vigour for being realistic, again seems a bit unfair
If the parent were being realistic, they'd say that we can't even recognize bias, which is actually more agreeable to me. But instead the parent admits that we can recognize bias. Since we've gotten that far, then I can say that failing to avoid it, when we should avoid it, is merely a lack of will and integrity rather than some inescapable fate.
The parent’s point was plainly that bias is unavoidable and making it overt is realistically all we can do. It’s a pragmatic take, albeit overly terse. They could have additionally included something like “and try to minimize its effects” sure, but that to me was implicit. I may be reading that fairly charitably, but perhaps that is just my own bias. I certainly have a bias to judge it more charitably than I do someone who leaps straight to moral outrage and judgement this early in the interaction.
In this case, I simply felt your judgment of the parent wasn’t fair, and showed a moral bias. Maybe they weren’t perfectly clear, and too absolute, but your response wasn’t proportionate either. It condemned more than it understood. I interpreted it as an epistemic observation and you interpreted it as an offense. The very fact that we came away with two completely different readings of the same short sentence rather proves the point.
Thank you for putting words in my mouth regarding my definition of the word bias, but lets use your own: "Bias is a distortion from reality and truth." If that is the case, we can never hope to avoid it, because we will never have perfect information. Using that definition, we are quite literally constantly in a state of bias. Your very own definition is far more broadly supportive of the notion that bias can't be avoided and consequently suggests bias is effectively ubiquitous. This to me is the primary point the parent was making.
I was perhaps too charitable, and you not enough. We are both biased, and going by the advice of the parent, I'm pointing it out. I don't think there is much more I can do.
> They could have additionally included something like “and try to minimize its effects” sure, but that to me was implicit.
I'm not interested in talking about what could have been implied, only about what was stated. I'm arguing against an idea that was articulated, not the person who articulated it.
One of the problems of the current-day liberals, in my opinion, is that they make universal statements that they don't mean in order to sound punchy and snag a few morality points. "Believe all women," "men are trash," "defund the police," "all cops are bastards" are all things you'd hear from a person who doesn't actually mean or want any of these things, even though the root of each of those is just and good. The idea that "bias can't be eliminated, only made explicit" is another one of these. If we don't believe it, then let's not say it.
> I certainly have a bias to judge it more charitably than I do someone who leaps straight to moral outrage and judgement this early in the interaction.
I'm not sure where you're reading outrage moral or otherwise. Was it that I used the word "so" in "so harmful"? And where's your bias against someone who tells another person to go to Conservapedia if they think bias can and should be avoided?
> Maybe they weren’t perfectly clear, and too absolute, but your response wasn’t proportionate either. It condemned more than it understood.
I merely stated that the belief was untrue and harmful, I don't think that's disproportionate at all. I can only understand what is stated, and according to my understanding we ought to condemn it.
> Thank you for putting words in my mouth regarding my definition of the word bias, but lets use your own: "Bias is a distortion from reality and truth." If that is the case, we can never hope to avoid it, because we will never have perfect information. Using that definition, we are quite literally constantly in a state of bias. Your very own definition is far more broadly supportive of the notion that bias can't be avoided and consequently suggests bias is effectively ubiquitous.
This is shifting the goalposts. First, I never claimed we could know the complete truth; it was the original post who stated that we couldn't course-correct upon learning new truth ("bias can't be avoided"). And second, the context of the original statement is bias in reporting, not epistemological certainty. We're not talking about positions of atoms here. We don't need perfect information to stop being biased against women in the workplace or against black people or whatever the subject. Even as individuals.
> This to me is the primary point the parent was making.
If that is their point then they can say it.
> We are both biased, and going by the advice of the parent, I'm pointing it out. I don't think there is much more I can do.
I have to ask, if I can't avoid my bias and you can't avoid yours, then what's the point of pointing out bias at all? Is it for other people to avoid our bias? How can they do that? I guess we're trying to minimize its effects, like you said.
> I was perhaps too charitable, and you not enough.
If I lack charity, it's in response to the original uncharitableness of the person telling someone to go to Conservapedia. If he would have mercy, let him show mercy.
Honestly, I think we are arguing around each other. We simply read this notion totally differently. You're taking it to mean that "we can't correct bias" but the statement was "we can't avoid bias". It actually makes no comment on being able to correct it (within, or without ourselves). To me it read as "when interacting with the world, we can't avoid encountering bias". If this is how it is interpreted, it actually doesn't do anything to rule out the ability to account and correct for it.
I'm not saying there's a definitive interpretation with how terse it is, just that we aren't necessarily on the same page and attempts to come to any sort of agreement with each other might be a waste of time as we are practically talking about two different ideas. I take this response as pretty fair, and I think the point you're making is totally valid, I just think our respective ideas would never converge as we are talking about 2 distinct things. (Interesting how much conversation a lack of clarity can generate).
Agree it is very bad for everyone to think like this. The problem is that on subject that divide you will always have two sides. Some times you need to tone down articles too much to not be biased to either way. So I also agree that there is a place for multiple articles on the same subject.
Also Wikipedia has the problem of being one type of article many subjects really need a other way to explain it.
Even facts can be slander.
It's true though, it's literally how your brain works. It's full of biases.
Yes, biases which ideally you are aware of and try to account for.
For example, it would be stupid for me to entirely dismiss all of Wikipedia just because I know that there are some horribly biased articles and it'd be a disgrace if I shrugged such behavior off saying "hey, we all have biases, can't help it". That's what a child or even an animal would do.
What about math and physics?
They try to work hard and get rid of bias through various methods: peer reviews, precise methodology (randomization, blind tests etc.). And of course, even those sciences are not perfect, subject to biases, and evolve. This applies only to a subset of fields, in a lot of fields you either don't have the time to remove biases this way or it's impossible because it's inherently subjective. Besides, if you needed unbiased perfect opinions before taking any decision, the world would be stuck and slow.
I think we have a fundamental disagreement here. There is no bias in the statement "2 + 2 = 4". Nor is there bias in stating that "any two masses experience an attraction force proportional to the masses and inverse-square proportional to the distance separating them." Not less bias -- no bias.
There is a pretty strong bias in the statement "2 + 2 = 4" because math doesn't care about the encoding used to describe a sum. For example, in an alternative society the expression could be written with flipped number symbols, right to left or in a vertical layout. Axioms are also a form of bias.
Axioms would only be a form of bias if they were taken at random. It's not biased to take the axiom that repeatedly holds true in empiric testing, over others that do not.
Definitely disagree.
1) The Monty Hall Problem (to be precise it hinges on ambiguity and the distinction between prior and posterior probabilities, but that is something most people aren't aware of and will get wrong the first time they see it, even people with knowledge of say Bayes' Theorem)
2) For several others, see Alon Amit's superb Quora answer to "What are the most interesting or popular probability puzzles in which the intuition is contrary to the solution?" ([2], login-walled). Mentions the very counterintuitive Penney's Game [0].
3) Berkson's Paradox, aka "People in hospital/getting treatment tend to have worse health indicators".
4) Asymmetric dice behavior is counterintuitive, when you first see it.
5) Benford's Law, on quantities occurring in nature (e.g. river lengths), as opposed to uniform distribution.
6) There are lots of counterintuitive things about Platonic solids.
7) Bayes' Theorem itself, superbly useful but possibly one of the things in probability most abused on a daily basis by bad journalism and bad statistics.
8) The Multiple Testing Problem/p-hacking/aka the xkcd "Green jelly beans cause acne" and as a corollary: 8a) Most published (academic) findings aren't replicable, aka "Why Most Published Research Findings Are False", Joannidis (2005)
[9] Almost-integers
---
[1]: https://en.wikipedia.org/wiki/Monty_Hall_problem
[2]: https://www.quora.com/What-are-the-most-interesting-or-popul...
[3]: https://en.wikipedia.org/wiki/Berkson%27s_paradox
[5]: https://en.wikipedia.org/wiki/Benford%27s_law
[8]: https://en.wikipedia.org/wiki/Multiple_comparisons_problem
[9]: https://mathworld.wolfram.com/AlmostInteger.html
[0]: https://en.wikipedia.org/wiki/Penney%27s_game
More: https://en.wikipedia.org/wiki/Category:Probability_theory_pa...
Great list! But I'm not sure the relevance? The intended point, which perhaps wasn't clear, was that logic / pure math as well as readily testable empirical theories are exceptions to the "everything is biased!" claim. There are in fact things that we can know with certainty, and that provides a foundation for a broader set of knowledge. The absolute relativist position is not just boring, but doesn't reflect reality either.
> Conservapedia
If you want a site that tells you that Jews launched a genocide of Nazis.
Conservapedia is intellectual cancer.
Traffic is how Wikipedia attracts future editors - people visit the site, some figure out that they can hit "edit", and some of those will go on to being productive editors on the platform.
Falling traffic is a problem. As editors turn over and fewer new editors show up, Wikipedia will become harder to maintain. For example, these days, defending against AI slop vandalism is a real problem that needs real humans to tackle - even as AI tools become more useful at detecting low-level vandalism.
Wikipedia is the best use of the Internet, the best reason & outcome of its existence.
Right up there with anime torrenting sites.
But seriously, AI trained on Wikipedia should donate to Wikipedia. Why are the AI companies not doing this, or are they?
I think Wikimedia needs expensive outreach and modernisation to stay relevant. And most of those attempts will be in vain.
A poor comparison is how much money coca cola spends on advertisement, even though it is one of the best known brands in the entire world. And most of their advertisement is simply "This is our name, we exists", not even a value proposition or call to action.
If Wikimedia sets themselves up to pay for servers and maintenance for perpetuity, they will fall into obscurity.
With that being said, I also don't think they are spending their money in a good way.
The WMF Australia gets a lot of money, but has barely any members. They have no intention of recruiting more.
That's a very naive view.
WHAT should the Wikimedia foundation invest in, that's viable for a thousand years?
That requires a Wall St/hedge fund and/or Buffett mindset.
The Wikimedia foundation is none of those, and they're not big enough to make even a ripple in the investment landscape.
What about reciprocal cooperation with countries around the world? Foundation invests in country bonds, countries include small yearly payments (like dividends?) to the foundation.
There is more and more of these "essential global infrastructure" projects, many of them non-profit, yet I'm not sure we're seeing a lot of investment from the globe into those projects.
Country bonds?? Have you looked at what they pay?
You’re not seeing investment into these kinds of “essential global infrastructure “ projects, because there are NO such things.
We can’t even agree on what “global” means.
Land has been a solid investment for over a thousand years. Going forward index investing normally does ok.
So, now you want the likes of the Wikipedia foundation to become a rent seeking landlord??
Dunno - it's worked ok for Oxford and Cambridge for many centuries, and the Church. I'd rather have land owned by vaguely public benefit bodies than some private equity fund.
Invest a little bit in everything / cut all the useless staff and just focus on longevity of the encyclopaedia
it’s certainly doable but requires selfless dedication at the top
“Selfless” - what world are you living in??
Money rules. There’s no ifs and buts about it.
If it doesn’t have a reasonable rate of return, nobody gives a shit.
What Wikipedia are you looking at? It's one of the things that has survived the longest on the Internet without being enshittified. It could live on in Cockroach Mode forever off the money from a modest solar farm or investment in an index fund. If someone sociopath bought the Wikimedia Foundation and tried to strip-mine Wikipedia, the Wikipedia editors could easily take a copy and go elsewhere.
Ironically, I was just listening to an interview with Jimmy Wales in which he said that, as individuals, most humans are basically good. They don't meet someone on the street and think "what is the rate of return on interacting with this thing?"
Yeah, it's good for them to save money for fewer traffic.
How much data does wikipedia take up?
25.60 GB
https://meta.wikimedia.org/wiki/Data_dump_torrents#English_W...
Wow they could easily maintain that for infinity at a low cost (less inflation and you know upkeep).
This is only the size of the text at one moment in time. The edit history is dozens of terabytes and the media is hundreds. All of this must be served to billions of requests a day. Almost every single request involves executing arbitrary lua and/or template syntax which was transcluded into the page. The media, particularly, introduces huge copyright problems. Ect ect. The WMF is undoubtably bloated, but it is not a walk in the park to run Wikipedia.
I would be extremely surprised if most articles that are regularly viewed aren't precached and thus served directly without involving any code execution.
Not for me.
Wikipedia had its day, in between print encyclopedias and quick query AI. Its place in history is now set.
Something else will come along soon enough.
Until LLMs gain the ability to cite their sources, they will be, at best, a search engine on top of Wikipedia, and not a replacement for it.
Most/all LLMs have already been able to cite sources for quite some time now.
Citing is not referencing links, it's finding primary sources and fact-checking. Kurzgesagt[1], an informational YouTube channel, has had issues with LLMs citing LLM generated content.
[1]: https://youtube.com/watch?v=_zfN9wnPvU0&t=175
LLMs aren't arbiters of truth - they're just natural language search engines. In a way it's quite similar to Wikipedia. Something being on Wikipedia doesn't mean that it's true, but rather it means that a "reliable source", which are not infrequently less than reliable, said so. And even that often doesn't hold as there's another layer of indirection where it's an editor saying this is what a source said, which is again not infrequently not exactly accurate. And then there's 'citogenesis' [1] where imaginary facts can be circularly whisked into existence.
This is why Wikipedia is not a source, but can provide links to sources (which then, in turn, often send you down a rabbit hole trying to find their sources), and it's then up to you to determine the value and accuracy of those sources. For instance I enjoy researching historic economic issues and you'll often find there's like 5 layers of indirection before you can finally get to a first party source, and at each step along the road it's like a game of telephone of being played. It's the exact same with LLMs.
[1] - https://xkcd.com/978/
I've had LLMs cite me bullshit many times, links that don't exist and claiming it does. It even cited a very realistic git commit log entry about a feature that never existed.
Haven't yet had the same issue with Wikipedia.
…. And it will be worse
Disagree. Say I'm looking for a list of countries and their populations.
Wikipedia almost certainly has this in a nice table, which I can sort by any column, and all the countries are hyperlinked to their own articles, and it probably links to the concept of population estimation too.
There will be a primary source - But would a primary source also have articles on every country? That are ad-free, that follow a consistent format? That are editable? Then it's just Wikipedia again. If not, then you have to rely on the LLM to knit together these sources.
I don't see wikis dying yet.
At work, I had rigged one of my internal tools so that when you were looking at a system's health report, it also linked to an internal wiki page where we could track human-edited notes about that system over time. I don't think an AI can do this, because you can't fine-tune it, you can't be sure it's lossless round-tripping, and if it has to do a web search, then it has to search for the wiki you said is obsolete.
OpenStreetMap does the same thing. Their UIs automatically deep-link every key into their wiki. So if you click on a drinking fountain, it will say something like "amenity:drinking_water" and the UI doesn't know what that is, but it links you to the wiki page where someone's certainly put example pictures and explained the most useful ways to tag it.
There has to be a ground truth. Wikipedia and alike are a very strong middle point on the Pareto frontier between primary sources (or oral tradition, for OSM) and LLM summary
Without Wikipedia, where will AIs get their (factual) training data? Reddit?
Torrented books
Backups of wikipedia.
LLMs are useless without source material.
AI companies should be donating large sums of money to Wikipedia and other such sites to keep them healthy. Without good sources, we’re going to have AI training off AI slop.
They should be, yes, but they won't. We already know this from the way those same companies have treated open source projects that they depend on.
One thing that I would really like to see is some kind of hefty tax on any kind of income derived from models trained on Wikipedia. Basically, make it legal to train, to share weights etc freely, and hosting them locally. But the moment you start charging people for subscription, the society should start charging you to maintain the commons that you are profiting from.
(This likely goes for more than Wikipedia, but that case is especially simple since there's a single legal entity that could be given the money.)
[dead]
As a 501(c)(3) nonprofit that is not dependent on web traffic for revenue, is a decline in traffic necessarily bad?
I always assumed the need for metastatic growth was limited to VC-backed and ad-revenue dependent companies.
They are highly dependent on web traffic for revenue.
And their costs are even increasing because while human viewers are decreasing they are getting hugged to death by AI scrapes.
If you look up their latest annual report (https://wikimediafoundation.org/wp-content/uploads/2025/04/W...) you can see that they're allocating ~1.7% of their expenses towards hosting.
I doubt that they're getting "hugged to death" by AI scrapers.
People cite this figure a lot, but its a little misleading because when you own your own servers a lot of the expenses that are typically hosting actually fall under a different category.
If you use AWS, the people hired to manage the servers is part of the price tag. When you own your own you have to actually hire those people.
But in that case your costs don't go up because of AI scrapers, as you don't need to scale employees with traffic.
I mean, it's not like you can get away with running with zero SREs if you're running in the cloud. The personnel costs for on-prem hosting are vastly exaggerated, especially if you contract out the actual annoying work to a colo.
Smart hands is more expensive than having dedicated datacenter staff, and the dedicated staff do a considerably better job. It's worth noting that WMF runs _very_ lean in terms of its datacenter staff.
You're also ignoring the need for infrastructure/network engineers, software engineers, fundraising engineers, product managers, community managers, managers, HR, legal, finance/accounting, fundraisers, etc.
this is a very eye-opening read on their financials: https://en.wikipedia.org/wiki/User:Guy_Macon/Wikipedia_has_C...
The easy counters to this article are:
1. I think their spending is a good thing. Charitable scholarships for kids and initiatives to have a more educated populous in general are things that I am happy to donate to.
2. As stated in the article, hosting is still a relatively simple expenditure compared to the rest of their operation. If Wikipedia really eats a huge loss, falling back to just hosting wouldn't be unrealistic, especially since the actual operations of Wikipedia are mostly volunteer run anyways. In the absolute worst case, their free data exports would lead to someone making a successor that can be moved to more or less seamlessly.
The only real argument in my eyes is that their donation campaigns can seem manipulative. I still think it's fine at the end of the day given that Wikipedia is a free service and donating at all is entirely optional.
AFAIK, they don't do any scholarships or really do any educational activities. By far their biggest spending item is just $105 million for salaries, mainly for all of its leadership, which is a majority of its expenses.
The second biggest line item is grants at $25 million, primarily for users to travel to meet up.
Then $10 million for legal fees, $7 million for Wikipedia-hosted travel.
I think it's pretty unethical to say you have to donate to keep Wikipedia running when you're practically paying for C-suite raises and politically-aligned contributors' vacations.
In person meetings move things forward.
Paying the travel for a bunch of highly active volunteer contributors to meet up ocassionally and hash out complex community issues pays massive dividends. It keeps the site moving forward. Its also pretty cheap when you consider how much free labour those volunteers provide.
Whenever people criticize wikimedia finances, i think they miss the forest for the trees. I actually think there is a lot to potentially crticize, but in my opinion everyone goes for the wrong things.
What are the rights things to criticize in your opinion?
Also, asking out of ignorance, what things need to move forward? I thought wikipedia is a solved problem, the only work i would expect it to need is maintenance work, security patches etc.
> What are the rights things to criticize in your opinion?
I think criticism should be based on looking at what they were trying to accomplish by spending the money, was it a worthwhile thing to try and do and was the solution executed effectively.
Just saying they spent $X, X is a big number, it must be wasteful without considering the value that is attempting to be purchssed with that money is a bit meaningless.
> Also, asking out of ignorance, what things need to move forward? I thought wikipedia is a solved problem, the only work i would expect it to need is maintenance work, security patches etc.
I think the person who i was responding to was referring to volunteer travel not staff travel (which of course also happens but i believe would be a different budget line item). This would be mostly for people who write the articles but also for people who do moderation activity. In person meetings can help resolve intractable disputes, share best practises, figure out complex disagreements, build relationships. All the same reasons that real companies fly their staff to expensive offsites.
Software is never done, there are always going to be things that come up and things to be improved. Some of them may be worth it some not.
As an example, there are changes coming to how ip addresses are handled, especially for logged out users. Nobody is exactly saying why, but im 99% sure its GDPR compliance related. That is a big project due to some deeply held assumptions, and probably critical.
A more mid-tier example might be, last year WMF rolled out a (caching) server precense in Brazil. The goal was to reduce latency for South American users. Is that worth it? It was probably a fair bit of money. If WMF was broke it wouldn't be, but given they do have some money, it seems like a reasonable improvement to me. Reasonable minds could probably disagree of course.
And an example of stupid projects might be WMF's ill-fated attempt at making an AI summarizer. That was a pure waste of money.
I guess my point it, WMF is a pretty big entity, some of the things they do are good, some are stupid, and i think people should criticize the projects they embark on rather than the big sum of money taken out of context.
Isn't it true that only around 10% of Wikipedia massive budget is used to actually run the core website? The rest goes to bloated initiatives in the Wikimedia foundations orbit.
Page 21 of their 2024 annual report[1] has expenses listed. About $3,000,000 for web hosting, about $100,000,000 for salaries and benefits out of $178,000,000 total.
[1]: https://wikimediafoundation.org/wp-content/uploads/2025/04/W...
A web server is useless if you aren't paying someone to plug it in.
There is probably a lot to criticize, but you need to go deeper than "salaries" are bad. You need some of those to actually run the website.
Even at a quarter million a year that's 400 salaries. How many people are needed to maintain a website?
How many people are needed to maintain instagram, facebook, etc? Wikipedia isn't just a collection of static content. The page contents include static and dynamic content (via a lua scriptable set of templates), semantic data (wikidata), multimedia management (wikimedia commons), editing tools (WYSIWYG editing, with full support of the wiki markup and templating), global caching infrastructure, multiple datacenters for HA/DR, etc.
There's also the need to support the staff and volunteer developers, which includes wikimedia cloud services, git hosting, config management and orchestration, CI, community hosted tool/bot services, etc.
WMF has ~600 employees, and that's quite lean, for a service of their complexity.
That includes all of the Wikimedia websites and non-profit activities though, not just the functioning of Wikipedia.org proper. That is a much lower percentage of the total.
[flagged]
> they are getting hugged to death by AI scrapes.
Wikipedia is not getting hugged to death by AI scrapers.
The source letter shows a relatively small portion of traffic was reclassified as bot traffic.
They get a lot of page views globally. It’s a popular website. The bot traffic is not crushing their servers.
Infrastructure has long been a tiny portion of wikipedias costs. I think Wikipedia even makes it easy to export all of its data, I don’t think AI scrapers would be a significant new cost
These are two very different cost factors. Scrapers don't use the available data dumps, that's why they scrape. They are also kind of dumb as they get lost in link structures constantly, which leads to unecessary traffic spikes.
How is their revenue traffic-dependent?
Their traffic is potential donations
Something tells me a person is way less likely to donate if they're consuming the content through an LLM middleman
I don't know -- as I said in another comment, my Wikipedia usage has gone down 90% thanks to LLMs.
But that means I'm still using it. Especially for more reference stuff like lists of episodes, filmographies, etc. As well as equations, math techniques, etc.
If you're the kind of person who donates to Wikipedia, you're probably still using it some even if less, and continue to recognize its importance. Possibly even more, as a kind of collaboratively-edited authority like Wikipedia only becomes more important as AI "slop" becomes more prevalent across blogs etc.
But were you originally convinced to donate by one of those giant Jimmy Wales banners that come up once a year? People won't see them anymore if they're using AI summaries.
My point is, using Wikipedia 10% as much is still using Wikipedia.
Does it matter if you see the banner 10 times or 100 times in a month?
In terms of the effectiveness of the banner, yes it absolutely does. Multiple exposures increase your propensity to convert.
Bandwidth is a ridiculously tiny portion of Wiki foundation's spending
Especially considering a considerably amount of their bandwidth is free, via peering agreements.
scraping Wikipedia feels like the stupidest possible move. You can in fact download the entire encyclopedia at any time and take all the time in the world parsing offline.
For such purposes, I'd naively just setup some weekly job to download Wikipedia and then run a "scrape" on that. Even weekly may be overkill; a monthly snapshot may do more than enough.
You can download twice-monthly database dumps, but they consist of the raw wikitext, so you need to do a bunch of extra work to render templates and stuff. Meanwhile, if you write a generic scraper, it can connect to Wikipedia like it connects to any other website and get the correctly-rendered HTML. People who aren't interested in Wikipedia specifically but want to download pretty much the entire internet unsurprisingly choose the latter option.
as somebody that has wrassled with the wikipedia dumps a number of times, i don't understand why wiki doesn't release some sort of sdk that gives you the 'official' parse
I have wrestled with it too. I believe it's because wikitext is an ad-hoc format that evolved so that the only 100% correct parser/renderer is the MediaWiki implementation. It's like asking for an SDK that correctly parses Perl. Only Perl can do that.
There are a bunch of mainly-compatible third party parsers in various languages. The best one I've found so far is Sweble but even it mishandles a small percentage of rare cases.
This. I tried that a few years ago and fell off my chair when I started to realized how DYI the thing is. It's a bunch of unofficial scripts and half-assed out of date help pages.
At the time I though, well it's a bunch of hippies with a small budget, who can blame them? Now I learn that there is 600 of them with a budget in the hundreds of millions??
This is becoming another Mozilla foundation...
There are also dumps of Wikipedia in html format.
Do you mean the discontinued Enterprise HTML dumps https://dumps.wikimedia.org/other/enterprise_html/ or the even older discontinued static HTML dumps https://dumps.wikimedia.org/other/static_html_dumps/current/... or is there another set of dumps I'm not aware of?
[flagged]
They also serve images.
Some of them even fairly often: https://news.ycombinator.com/item?id=26072025
[flagged]
> In this test, the framework responds with the simplest of responses: a "Hello, World" message rendered as plain text.
What a ridiculous comparison. You either don't know what you are talking about, or are a troll. Maybe even both.
there are more complex tests in that benchmark. my point was, a decent server can serve tens or hundreds of thousands requests per second when it's a few kilobytes of static content from a RAM database.
as long as your web service isn't horribly mismanaged, even to a $5 VPS the bot traffic is infinitesimal background radiation.
Bandwidth costs aren't the dominating cost.
> As Miller puts it, “With fewer visits to Wikipedia, fewer volunteers may grow and enrich the content, and fewer individual donors may support this work.”
Contributors are a tiny % of users. I'm sure they've got some room for improvement on incentivizing new contributors. But Wikipedia is a gift to humanity and I hope we find new ways for them to be paid for their contributions to AI.
> Contributors are a tiny % of users most of them were wikipedia users in some form before they were contributors I imagine
1/3 of all donations are from the banner. I just went and looked at their annual report, which disclosed this.
The warning sign is not traffic for ads, although this will result in a drop in donations eventually.
It means that now, people are paying for their AI subscriptions, while they don’t see Wikipedia at all.
The primary source is being intermediated - which is the opposite of what the net was supposed to achieve.
This is the piracy argument, except this time its not little old ladies doing it, but massive for profit firms.
> It means that now, people are paying for their AI subscriptions
Most people are not paying a cent. And the people that are, are paying for stuff like coding assistance or classification, not the kind of info you get on Wikipedia.
Looking up Wikipedia-style information on LLM's is not a driving factor in paid subscriptions to ChatGPT etc.
Wikipedia was never a primary source to begin with
Wait, when were little old ladies the perpetrators of piracy?
I believe that comment is referencing this recent news:
> Sony tells SCOTUS that people accused of piracy aren’t “innocent grandmothers”
https://arstechnica.com/tech-policy/2025/10/sony-tells-scotu...
Thank you!
If nobody uses Wikipedia they won't get any donations, and unfortunately they wasted the last two decades blowing literally hundreds of millions of dollars on random community and outreach programs instead of building an endowment in case something exactly like this happened.
No really, it was in the news a few years ago but nothing changed as far as I know.
LLM's have definitely replaced 90% of what I used to look up on a Wikipedia, simply because they integrate from so many more additional sources.
But at the same time I continue to contribute edits to Wikipedia. Because it's the source of so much data. To me, it doesn't matter if the information I contribute gets consumed on Wikipedia or consumed via LLM. Either way, it's helping people.
Wikipedia isn't going away, even if its website stops being the primary way most people get information from it.
> Either way, it's helping people.
Wikipedia gives away your creation for free. The LLM companies do not. Google is operating a loss leader and not "helping people." In fact, quite the opposite.
> Wikipedia gives away your creation for free. The LLM companies do not.
They LLM companies are so far. Just like Google has given search results away for free for decades now.
Search results do not have as chilling an effect as LLM-generated text, because search results are in essence links back to content (meaning human authors benefit from knowing who reads them, people discover who those human authors are, etc.), whereas it is not a thing in LLM-generated text (which may have occasional links back, but they are not complete nor always correct).
There are dozens of top-quality models with FOSS licenses you can download and run right now
Counterpoint: I'm fine paying for convenience.
Wikipedia should still exist, but I'm fine giving OpenAI 20 bucks a month to make my life easier.
AI seems obvious, but social video? Are they saying people watch TikToks instead of reading Wikipedia, or people who used to look things up don’t bother anymore because of TikTok?
Yes -- sadly, the hottest new search engines are YouTube and TikTok.
Yeah, I was very surprised by that. Article says:
> “younger generations are seeking information on social video platforms rather than the open web.”
YouTube I'll buy it, but the search function in Instagram and tiktok is beyond useless!
tiktok seems to be the primary medium by which Gen Alpha obtain their news and knowledge
No it's not. It's the primary medium by which they get their entertainment.
TikTok replaces YouTube and Tumblr before that. It's not like people are watching it instead of reading history books or the New Yorker.
And just because you use TikTok doesn't mean you don't check actual news headlines or listen to two-hour-long podcasts.
The younger generation absolutely use Tiktok search to find information.
I'm sure they do. Is that their primary source? Do they use nothing else? Is any of that information put to economic use in their lives or is it just curiosity?
They buy a ton of stuff from Tik-Tok and Instagram ads - yes.
The line of questioning was about news and knowledge, not ads. Ads in anything will affect people's purchases, so that's not evidence for what that question was actually going for.
Gen Z as well. Many well into their 20s heavily consume tiktok.
You probably mean Gen Z. Alpha's oldest members are 14-15. Middle schoolers have never been particularly well read.
Here's a direct quote from a WSJ article the other day: "She searched for the drug on TikTok, her go-to information source"
gift link: https://www.wsj.com/health/wellness/anti-depressants-lifesty...
It's all right. Wikipedia was a magical device for its time, and it's still a great aggregator of information. It will probably last forever as such a link aggregator. Read-time curation is obviously far better than write-time curation, but the former used to be very hard. Now we have the former for cheap so it makes sense for Wikipedia to be Yet Another Source into the read-time curator. And the existence of a source database like Wikipedia makes many of these tools work a lot better.
People rightfully get upset about individual editors having specific agendas on Wikipedia and I get it. Often that is the case. But the chat interface for LLMs allows for a back and forth where you can force them to look past some text to get closer to a truth.
For my part, I think it's nice to be part of making that base substrate of human knowledge in an open way, and some kinds of fixes to Wikipedia articles are very easy. So what little I do, I'll keep doing. Makes me happy to help.
Some of the fruit is really low-hanging, take a look at this garbage someone added to an article:
https://en.wikipedia.org/w/index.php?title=Salvadoran_gang_c...
> Read-time curation is obviously far better than write-time curation, but the former used to be very hard. Now we have the former for cheap
It's _kinda_ cheap. Wikipedia is so cheap you can fit it all on a phone and search it instantly.
I agree overall but LLMs are just so heavy. I don't know if most people can afford to run one locally, and they're lossy. Both on a phone would be great. I fret a lot about data ownership, you know
This made me curious enough to check the stats for my little site. According to Cloudflare’s AI Overview, over the last 24 hours the breakdown is:
665 ChatGPT-User
396 Bingbot
296 Googlebot
037 PerplexityBot
Fascinating.
Out of how many visits total?
About 80% of traffic to my sites (a few personal blogs and a community site) is from ai bots, search engine spiders or seo scrapers.
Knowledge isn’t knowledge in itself — it’s about how people organize it.
With more AI tools mining existing knowledge and presenting it in increasingly accessible ways, I don’t think AI search fundamentally changes how information and knowledge are organized.
Of course, AI could reshape the organization of knowledge through areas like:
1. Fact-checking and sourcing
2. Drafting new pages
3. Editing and refining wording
…and more
---
Just like Wikipedia already has many bots running behind the scenes, if all these tasks were eventually handled by AI, there would still be things left for humans (or perhaps another AI) to decide:
1. When a fact has multiple perspectives, how should it be phrased to represent different viewpoints fairly?
> I still remember countless word battles on Wikipedia over this.
2. In the age of smartphones and social media, historical moments are documented not only by journalists or influencers but by thousands — even millions — of ordinary people. How should Wikipedia process and summarize such vast, distributed facts?
3. How do we properly incentivize contributors, whether human or AI?
> Wikipedia was born in an era when the Internet lacked reliable information, and building a shared, sustainable, independent knowledge base was a mission that resonated with its early contributors — traffic rewards came later.
4. And of course, geopolitics — Wikipedia must remain independent.
---
A bit of background: I once led a Chinese wiki product, but I eventually gave up on it — because almost no one cared why a wiki should exist beyond being just another searchable content platform.
I like to idea of using AI to make Wikipedia better.
There are many small tasks simple AI agents could handle.
For example: Go to every article in the English Wikipedia of a Spanish city. Check data like inhabitants, major, etc. and compare them to the Spanish Wikipedia article. (The assumption here is that the Spanish Wikipedia will be more up-to-date for Spanish cities than the English Wikipedia.) Double-check what is written in the Spanish article. Update the English article accordingly.
If such an agents is only allowed to create drafts, human editors could review them and we would get a lot of small updates in.
You have to keep in mind that the original google search, by summarizing search results based on links between pages, essentially destroyed the web of interconnected links that existed at that time, so that the original pagerank algorithm doesn't even really work anymore. And the search results Google returns are worse than they were years ago. And you know, I use it, it is still super convenient, and the long term effects took some time to become apparent.
Now it is again feeding on and regurgitating Wikipedia but again in a way that will end up destroying the thing it is summarizing. Aggregators are parasitic on the thing they derive information from.
I still donate Wikipedia yearly. It's useful and I hope it stays afloat.
Why? They have hundreds of millions in assets to operate a relatively simple site.
That and the Internet Archive.
The Internet Archive deserve it more.
And they actually need it.
I considered donating to Wikipedia until I read this:
https://en.wikipedia.org/wiki/User:Guy_Macon/Wikipedia_has_C...
The TL;DR is that Wikipedia has a massive spending problem, and if they brought it back to something reasonable, their existing war chest would last them for at least a decade, even with no additional funding.
Remember that time in feb 2005 when Wikipedia went down for several days and lost about a week of data because there was a power failure and they were too shoe strings to have proper back ups, and also had disabled a bunch of ACID features in mysql to get the last bit of performance out of mysql causing the power failure to corrupt the db, because they didn't have enough money for more servers and were desperate to try anything to get the last bit of performance out of their servers?
No? That's because it doesn't happen anymore now that WMF has a reasonable budget.
How many comments did you write on this thread to justify Wikipedia's expenses? I counted at least 6.
Its actually kind of ironic given i don't think WMF is particulary well managed or effective in its spending. I just think most of the criticisms of WMF spending are kind of terrible.
This is by far the worst defense I have ever read for why wikipedia needs a $200m budget.
There are like 3 orders of magnitude of budget difference; I'm sure a happy middle ground could be found. Even going back to 2010's budget would save $160M/year and still be 17x 2005's budget.
That doesn't mean their spending money on what the people donating want. Most people seem to want to donate to keep Wikipedia running, but that is not necessarily where the money actually goes.
So far the story with Wikipedia has always been that the number of contributors is declining, and I believe this will only get worse when fewer people read it (directly from the source). As much as I hate the idea, are people (in particular, Wikimedia) experimenting yet with using LLMs to contribute to/improve/check Wikipedia? Will page discussions and edit wars in the future get replaced with an arena of partially collaborating, partially competing LLM agents?
The system is off. Contributor would write a new piece. Spammers would go in and remove legitimate articles it links to and replace with spam links. Wikipedia bots go through and purge the new contributions. Contributors stop caring.
Many things contributors are doing could be handled by AI agents.
We would just need a good system to a) keep humans in the loop and b) sort out bad actors.
The same is happening with almost all sites since google started providing AI summaries.
I stopped contributing money, and then stopped trusting and using Wikipedia after I saw how wrong they are on a subject I am well-informed on personally (ARM architecture), and how fast they revert properly-cited 100% factually correct changes (literally citing the arm architecture manual).
How could I trust them on things I do not know, if I know for a fact they are unrepentantly wrong about things I do know?
In my experience it's always better to only leave comments on the talk page and at most add template comments to the article, for a couple weeks; after that time passed you can go ahead with editing the page, and there's a better chance of the edits not getting reverted.
But I've always hated the way Wikipedia works.
I refuse to pay for the privilege of doing free work for someone else’s benefit. Pay — in time or in effort. Let it rot.
Here is a list of their most popular pages:
https://en.wikipedia.org/wiki/Wikipedia:Top_25_Report
Most of them are trending public figures and media, which lines up with what I remembered from my prior Wikipedia analysis. So LLMs are likely replacing a lot of this.
I really hope that the same thing is happening to all kinds of SEO-ridden websites out there.
https://news.ycombinator.com/item?id=44631662
It happens to all, means the good sites to that need add money and the AI crawlers add traffic to all so that even those without adds may get a cost problem.
Not to mention all the sites that will pop up to poison the LLMs in their favor.
On top of that LLMs will either cost more or will get adds too.
Source: https://diff.wikimedia.org/2025/10/17/new-user-trends-on-wik...
traffic falling means wikipedia will be cheaper to run. since they don’t rely on ads, it’d likely not affecting their revenues either (assuming those who don’t use it anymore weren’5 those givîng to it)
It’s probably not good long term for donations or attracting future editors. Possibly less people interested in editing when it becomes more unpaid work for AI companies than actually serving real viewers.
If I may speculate I would assume that the Venn diagram between people moving from wikipedia consumer to tiktok and AI generally don't overlap much with people prone to contribute or donate to wikipedia, so I'm not certain it's a net loss in that regard.
This is really the perfect point in the timeline- "in these uncertain times-" to posit themselves as a working repository of credible information.
Also I firmly believe the Wikipedia app is key to their sustained relevance. Users get forwarded to the app from web browsers with wiki links, this gets people in the dedicated interface.
from a design standpoint there need to be more avenues on each page inviting people to browse and explore, spend more time there.
Little off topic but does anyone else find AI summarise really misleading, almost dangerously misleading ? In a search box you can never rally provide anywhere near enough context to get a sufficient answer on anything of even small consequence.
Yes I do use the internet for “medical opinion” and information and seriously some of the falsehoods it’s provided…similarly anything related to construction. Steer clear.
For Wikipedia to last 1000 years, we need a publisher who sells a print of a snapshot of it every ten years or so. When the civilisation dies, and our civilisation will die eventually, sooner or later, just like any other civilisation, then there won‘t be no internet, no wikipedia foundation or anything. But printed paper might survive.
That little AI generated summary that seems to show up in every Google search is riddled with errors. I've learned that as often as not it's just making things up or confusing two different topics. I would not trust that thing under any circumstances. Its no better than a wild guess from a stranger on the street.
We can't just look up info using an LLM. We have no clue what its weights and biases are based on, and whatever other layers control the output. Just a total black box. It would be an irreplaceable loss if we lost wikipedia to LLMs.
Looking how editors can fabricate stuff OR argue like little kids whether original source from origin country should be mistrusted vs encyclopedia britanica is enough for me to know Wikipedia is no longer what it was and lost its purpose.
The further aways ppl stay from it the better.
Oh good, Jimmy can stop hounding me for money like a late night infomercial or televangelist.
https://news.ycombinator.com/item?id=34106982
>2022
>It’s the dishonesty of Wikipedia that bothers me. The implication is that donations are urgently needed to keep the website running. In reality they have $300m in the bank and revenue is growing every year[0]. Even Wikipedia says only 43% of donations are used for site operations[1], and that includes all of their sites, not just Wikipedia. Fully 12% of the money they collect from you is. . . used to ask you for more money[1]
I find it a little annoying for a variety of reasons when universities I've been to ask me for money, but one of the main reason that I don't donate to universities is that I don't want the money I donate to be used for advertising, and especially advertising to solicit more donations instead of actually improving the school.
Yeah ... Some days it feels as though 98% of the net has turned into a giant begging-bowl. Long-time major sites are now saying stuff like "You're reading your last free article." Yeah? Don't count on that.
Many of them are sites that have built themselves without any original reporting. Where will they scrape the content they've used to grow if their sources take the same attitude?
Here's looking at you, Wired
So maybe they shouldn't depend on just search traffic. Their increased spend would suggest they got enough people to try and figure out alternative methods to attract editors and/or traffic.
How is it not conflict of interest when Google's AI summary (which is sometimes hilariously wrong) takes a click away from websites that pay for ads? Specially if it was trained on those websites
The sites paying for ads are usually offering products and services, not information. If you search "Plumbers near me", there will be no reason to have an AI summary, so they can show the ads. If you are searching "what temperature does copper melt", no one was buying ads on that anyway.
The point is that the sites from which the AI summary sourced its information might have ads, and if Google only linked to them they'd get ads impressions
that's a good thing? Wikipedia is always worrying about server costs or am I wrong?
I’m a pro-market solution person, but markets are a tool.
This is the kind of capitalistic behavior that is repugnant to our idea of how things should work.
This is not what the commons is for - taking the work of creators, repackaging it, and using platform capability to re-sell it.
At this point, I am coming around to the argument that governments should make their own local/national AI.
> At this point, I am coming around to the argument that governments should make their own local/national AI.
I think this is a terrible idea. How do current Chinese-built LLMs respond when asked about Tianamen Square? How would American-built LLMs response when asked about Israel/Palestine?
Oh it’s absolutely a terrible idea.
I can think of an infinite ways why this is bad.
The alternative is what we have now. I don’t think the current situation is worth leaving uncontested.
Seriously - we’re copying all the art, music, lit, data and converting it into subscription revenue for the biggest firms on the planet ?
Which are based in the US?
And the alternative is Chinese open models ? Really ? This is the future.
In that case the only other groups left to enter the fray are national governments.
Frankly, after seeing what I am in Asia, I suspect they are already planning to do that anyway.
If Wikipedia was the source for training data of the LLM then the answer would be quite horrible. There was a ring of Pro-Palestinian editors (possibly state sponsored) that were writing a lot of misinformation on that topic.
Check the pro-Israel ones too
Is there any? I thought Paliz/Woke actors were in complete control over wikipedia. That's why the talk about the alternatives.
I'm not surprised you thought that.
as much as I love Wikipedia and I think we need it, I just realise that I haven't opened Wikipedia in the past few months. My default workflow is now replaced to Option + Space and ask gpt questions.
i feel like there's a postivie-sum game where the AI companies should start heavily subsidizing wikipedia.
wikipedia gets money to stay afloat, ai companies continue to get access to their huge human-curated knowledge graph.
WMF only spends a couple million on essentials like hosting though. They have over 700 employees, and their budget is pretty bloated with thing that aren't really related to running Wikipedia, including sizeable grants to other organizations. See their org chart [1] or this classic essay [2].
[1] https://meta.wikimedia.org/wiki/Wikimedia_Foundation/Organiz...
[2] https://en.wikipedia.org/wiki/User:Guy_Macon/Wikipedia_has_C...
I do not doubt this for a second. I hope that this does not alter Wikipedia.
Ok so is Wikipedia competing for traffic rankings?
editing wikipedia was close to the most aggravating experience I've ever had. and that includes suing landlords for security deposits and arguing with HOAs.
Will this also affect the search engines like Google?
Here's an interesting question: Wikipedia was a mind blowing improvement over a shelf full of Britannica encyclopedia, or several floppies of Encarta.
What's the next improvement over Wikipedia that changes things to a similar extent?
Guess that means they dont need that $1 donation anymore
The persecutors of notability are no longer notable.
So their hosting costs are going down, right?
My personal traffic to Wikipedia fell after around 2019, when activist editors took over, and the site ceased to be trustworthy for a lot of important topics.
[citation needed]
I was curious too and found these pretty quickly - I dunno if this applies to all topics, but at least the political ones:
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5022797
https://media4.manhattan-institute.org/wp-content/uploads/is...
https://www.researchgate.net/publication/241750096_Is_Wikipe...
[flagged]
openai, anthropic, gemini &co should pay wikipedia the rent.
Wikipedia offers paid services , like realtime and incremental update feeds . I know a few of the big companies do pay .
[dead]
[dead]
[dead]
[dead]
Wiki has been dying such a slow painful death. I feel bad. Especially because I was the first generation to have the transition from encyclopedias to wiki. Rip wiki, it'll go down in history for sure.
Good riddance to their constant begging for money they don't need.
https://en.wikipedia.org/wiki/Wikipedia:Suppress_display_of_...
I don't even begrudge them the money but I'm not giving them reasonable cause to bombard me with emails again.
Isn't this better for them in a way? Won't this reduce their hosting costs, whilst still being relevant to those who actually need deeper information?
Reduced traffic means less contributors and less donations.
(looks outside at the street) I'm wondering why would traffic be less because of AI search?? SO I asked AI. Its answer? "By analyzing data and demand in real-time, AI is able to predict peak travel times and adjust train, subway, and bus routes as needed, reducing overcrowding and, in the case of buses, traffic congestion."
In the past, you would type a question into Google. To get the full, detailed answer you had to click on the Wikipedia search result.
With ChatGPT or Google AI Mode, you get all answers directly in the chat. And you can even ask follow-up questions. There is no need to click on a link.
From the data I have seen, 40% of searches on Google used to lead to a click to another website. In ChatGPT and Google AI Mode, this number is lower than 5%. One study (with a small N) even came to the conclusion that the number is 0%.
wow, bunches of downvotes? Because of a little humor? My god people, what is it with you?
AI helped me realize why I have always read and referred to Wikipedia so little.
Don't get me wrong. Wikipedia is a GREAT resource that the world absolutely needs.
But it's a horribly boring "read" when it comes to human consumption.
It's an encyclopedia. A list of facts.
When I want to learn something about a particular topic, I rarely ever want to read a list of facts.
> But it's a horribly boring "read" when it comes to human consumption.
I couldn't possibly disagree more. The website I visit most often after Reddit and HN is Wikipedia, and I often spend hours just diving through article after article. Learning facts is fun.
Learning facts is extremely fun.
It's just not fun reading them as a list of facts.
I am not a computer. I enjoy reading prose.
> prose
I must be reading a different Wikipedia, then, because all the articles I read are prose, rather than a bulleted list of facts as you claim.
>Wikipedia is often described as the last good website on an internet
Oh who says that? Bloomberg.
https://www.bloomberg.com/news/newsletters/2023-10-24/why-wi...
So not people just bloomberg... trash article. Probably just stealth donation marketing.
The other day I was reading the page about the Library of Alexandria.
The Wikipedia version virtually doesn't mention the stoning of Hypatia or the burning of the library, except for a different smaller fire caused by a war during Julius Caesar.
This page certainly reads like Christian apology, where the almost total destruction of the library by Christian fanatics didn't happen.
For me, this heavy bias in basically every article is the reason Wikipedia traffic is falling.
What you say sounds influenced by the movie Agora: https://bookhaven.stanford.edu/2016/03/the-library-of-alexan...
I haven't fully watched that film.
It is definitely influenced by the book Cosmos by Carl Sagan. I feel I read it a hundred times.
Ok, the errors in Cosmos are covered here: https://sciencemeetsfiction.com/2014/06/15/cosmos-follow-up-...
Quote:
Cosmos does not make Hypatia’s death so much a religious issue as an anti-intellectual on, but the truth is that it was actually a political one. A second problem comes when Dr. Sagan links her death to the destruction of the Great Library. In fact, in the final episode of the original Cosmos, “Who Speaks for Earth”, Carl Sagan says, “The last remains of the library were destroyed within a year of Hypatia’s death.”
The problem with this is that the last remnant of the Library of Alexandria were almost certainly destroyed in 391, 24 years before Hypatia’s death, and most of the library was likely destroyed, by accident, centuries earlier.
It sounds strange, but we actually don’t have a very good idea of when the Library of Alexandria was destroyed. As best we can tell, much of it was burned unintentionally when a fire spread through the city during Julius Caesar’s invasion in 48 BC. While the majority of the library may have survived that war, it was almost certainly destroyed in the war between Emperor Aurelian and Queen Zenobia of Palmyra in the 270s AD. This also appears to have been unintentional, as a large part of the city was burned.
What little was left of the library was deliberately destroyed in 391, when Emperor Theodosius I banned Paganism. The remaining repository of books in Alexandria was destroyed along with the Pagan temple it was stored in. [Even this has no evidence as pointed out by Tim O'Neill in the blog comments]
I admire most of Dr. Sagan’s and Dr. Tyson work, but when they characterize Hypatia’s death and the burning of the Great Library as the deliberate (and linked) actions of an anti-intellectual mob, they are simply misrepresenting the history.
Thank you for your detailed reply.
There are multiple new start ups and existing competition that's impacting them. They are facing competition.
The curious thing is that big LLM folks put together RAG systems which act much like wikipedia. But it's more than that. They built dictionaries, book repos(borderline illegal), news repos, and data knowledge base. These are bigger than wikipedia. Better because you dont have anonymous partisans.
Wikipedia is at a point where they have purged multiple perspectives and it has left an unreliable systemic bias in wikipedia. They are dealing with this problem and competitors are popping up because of these problems.
Larry put out his theses on his user page: https://en.wikipedia.org/wiki/User:Larry_Sanger/Nine_Theses
Originally being heavily censored, vandalized, and deleted. It does seem to have been allowed, despite it being in user space.
Every one of those theses is correct.
The hard truth is that LLMs will fully replace wikipedia.
Let's put aside wikipedia being rotten with bureaucracy and obsession-driven bias, which is similar to stackoverflow preexisting flows before LLMs streamrolled.
Fact is, wikipedia is a human driven summarization engine of secondary sources, hopefully in a way that echos the sources consensus.
This is exactly what LLMs are best for, summarizing huge amount of text, and training can easily focus on high quality books and thus exceed wikipedia in quality.
It's enough to read an AI summary where the first line talks about the subject in hand, compared to wikipedia where the first line is the product of some petty argument about a political disagreement
This is the kind of attitude that I hope Wikipedia continues to exist and educate people away from falling for.
Being condescending is unwarranted really.
Is your argument that Wikipedia rules are not highly tailored to suppress original research and rely on secondary sources
For the latter LLMs are better than wikipedia today, the former arguably current alignment is better at suppressing bias than current editors
LLMs nowadays are like search engines before SEO.
They will habe there own kind of spam problem.
And wait what happens to their quality as soon as the AI companies habe to generate profit.
I doubt you can collect second source references at the same price Wikipedia does. People aren’t as eager to help a billion dollar company for free to make profit.
It most certainly will not.
LLMs lack the human nuance that a good Wikipedia article requires. Weighing quality sources and digesting them in the most useful way that a human would want and expect — that is very difficult for both humans and machines, and it is why Wikipedia as a whole is such a treasure: Because a community of editors take the time to tweak the articles and aim for perfection.
There are guidelines across all Wikipedia articles that make a good experience for the reader. We can’t even get the world’s greatest LLMs to follow a set of rules in a small conversation.
In my opinion simply using a dataset of high quality books and highly rated academic journals is enough to surpass current Wikipedia quality.
In my experience when using LLMs as a replacement for Wikipedia (learning about history), it is often of higher quality in niche topics and far less biased in political contentious areas
For me Wikipedia is only good for introductions and exploration. You don't have time to read a dense tome but also don't have enough experience in reading research papers in that area? Wikipedia it is then.
Wikipedia is the tabloid equivalent for scientific topics.
LLMs tend to be much more useful for niche topics, because they've most likely been trained directly on the source itself.
> and training can easily focus on high quality books and thus exceed wikipedia in quality.
Then it will never exceed Wikipedia. There are no high quality books to many of Wikipedia‘s articles and even if who qualifies those books as as high quality.
And less traffic is not only a problem of Wikipedia but also other websites.
So AI is killing the source it feeds on.
This is the golden age of AI. The next age will be filled with less good sources, more AI generated content and sites that actively try to poison LLMs
https://news.ycombinator.com/item?id=45529587
As others have said, Wikipedia is a tertiary source. A primary source is the original materials, e.g. research papers for example. Secondary sources analyze or summarize, but also connect the primary sources into a bigger work. This is how comprehensive textbooks that cover a wide breadth of topics are written. Wikipedia is for those who are too lazy for even the secondary sources. I.e. blog post sized articles about a particular topic. That's what a tertiary source is and nobody really needs tertiary sources other than for convenience and time savings.
If if all this is true, LLMs would still need to create a repository of such aggregated "summarizations of secondary sources".
LLMs cannot in real time find and read thousands of secondary sources. Especially not if some of these might have already disappeared or are not digitalized.
I can see a future where LLM labs a) donate to Wikipedia and b) contribute to it with agents that suggest edits and review facts.
I think LLM companies will need to donate and contribute to more direct sources are newspapers, journalists, bloggers, coders posting on stack overflow etc and not Wikipedia (which is more of a tertiary source).