I dislike those black and white takes a lot. It's absolutely true that most startups that just run an EC2 instance will save a lot of cash going to Hetzner, Linode, Digital Ocean or whatever. I do host at Hetzner myself and so do a lot of my clients.
That being said, the cloud does have a lot of advantages:
- You're getting a lot of services readily available. Need offsite backups? A few clicks. Managed database? A few clicks. Multiple AZs? Available in seconds.
- You're not paying up-front costs (vs. investing hundreds of dollars for buying server hardware) and everything is available right now [0]
- Peak-heavy loads can be a lot cheaper. Mostly irrelevant for you average compute load, but things are quite different if you need to train an LLM
- Many services are already certified according to all kinds of standards, which can be very useful depending on your customers
Also, engineering time and time in general can be expensive. If you are a solo entrepreneur or a slow growth company, you have a lot of engineering time for basically free. But in a quick growth or prototyping phase, not to speak of venture funding, things can be quite different. Buying engineering time for >150€/hour can quickly offset a lot of saving [1].
Does this apply to most companies? No. Obviously not. But the cloud is not too expensive - you're paying for stuff you don't need. That's an entirely different kind of error.
[0] Compared to the rack hosting setup described in the post. Hetzner, Linode, etc. do provide multiple AZs with dedicated servers.
[1] Just to be fair, debugging cloud errors can be time consuming, too, and experienced AWS engineers will not be cheaper. But an RDS instance with solid backups-equivalent will usually not amortize quickly, if you need to pay someone to set it up.
You don't actually need any of those things until you no longer have a "project", but a business which will allow you to pay for the things you require.
You'd be amazed by how far you can get with a home linux box and cloudflare tunnels.
On this site, I've seen these kind of takes repeatedly over the past years, so I went ahead and built a little forum that consists of a single Rust binary and SQLite. The binary runs on a Mac Mini in my bedroom with Cloudflare tunnels. I get continuous backups with Litestream, and testing backups is as trivial as running `litestream restore` on my development machine and then running the binary.
Despite some pages issuing up to 8 database queries, I haven't seen responses take more than about 4 - 5 ms to generate. Since I have 16 GB of RAM to spare, I just let SQLite mmap the whole the database and store temp tables in RAM. I can further optimize the backend by e.g. replacing Tera with Askama and optimizing the SQL queries, but the easiest win for latency is to just run the binary in a VPS close to my users. However, the current setup works so well that I just see no point to changing what little "infrastructure" I've built. The other cool thing is the fact that the backend + litestream uses at most ~64 MB of RAM. Plenty of compute and RAM to spare.
It's also neat being able to allocate a few cores on the same machine to run self-hosted GitHub actions, so you can have the same machine doing CI checks, rebuilding the binary, and restarting the service. Turns out the base model M4 is really fast at compiling code compared to just about every single cloud computer I've ever used at previous jobs.
The most consistent misunderstanding I see about the cloud, is disk I/O. Nobody understands how slow your standard cloud disk is under load. They see good performance and assume that will always be the case.
They don't realize that most cloud disks use a form of token tracking where they build up I/O over time and if you have bursts or sustained high I/O load you will very quickly notice that your disk speeds are garbage.
For some reason people more easily understand the limits of CPU and memory, but overlook disk constantly.
At one time I had a project to run a cryptocurrency node for BSC (this is basically a fork of Ethereum with all the performance settings cranked up to 11, and blocks centrally issued instead of being mined). It's very sensitive to random access disk throughput and latency. At the time I had a few tiny VPS on AWS and a spinning drive at home, so I evaluated running it there. Even besides the price, you simply cannot run it on AWS EBS because the disk is just too slow to validate each block before the next one arrives. I spent a few hundred dollars and bought an NVMe SSD for my home computer instead.
Even without that, you are still at the heart of it accessing over a SAN like interface with some sort of local cache. Getting an actual local drive on AWS the performance is night and day
I/O is hard to benchmark so it's often ignored since you can just scale up your disks. It's a common gotcha in the cloud. It's not a show stopper, but it blows up the savings you might be expecting.
that's what I always brag to my devs, why is our DB 1TB, and only 20 users are working in our app. They are collecting all garbage and saving it to DB. Poor development skills I would say. Our old app did the same thing, and after 15 years it was barely 100GB with tens of users. devs today are SELECT *. If it does not work, they say we need more resources. Thats why I hate cloud.
> Just one of the couple dozen databases we run for our product in the dev environment alone is over 12 TB.
> How could I not use the cloud?
Funnily enough, one of my side projects has its (processed) primary source of truth at that exact size. Updates itself automatically every night adding a further ~18-25 million rows. Big but not _big_ data, right?
Anyway, that's sitting running happily with instant access times (yay solid DB background) on a dedicated OVH server that's somewhere around £600/mo (+VAT) and shared with a few other projects. OVH's virtual rack tech is pretty amazing too, replicating that kind of size on the internal network is trivial too.
Not too push the point too hard, but a "dev environment" for a product is for a business (not an individual consumer). Having a server (rack) in an office is not that hard, but alas the cloud might be better here for ease of administration.
I don't think so. An organization so big and bureaucratic that needs 3 months to authorize a server purchase will for sure need a few weeks of paperwork to authorize a new AWS account creation, and will track the spending for OU and will cut budget and usage if they think you deserve it.
And plenty of datacenters will be happy to give you some space in one of their racks.
Not wanting to deal with backups or HA are decent reasons to put a database in the cloud (as long as you are aware how much you are overpaying). Not having a good place to put the server is not a good reason
If anyone's curious about the ballpark cost, a carrier-owned (?) DC near me that publishes prices (most don't) advertises a full rack for 650€ per month, including internet @ 20TB/month @ 1 Gbps, and 1kW power.
Though both of which are probably less than you'd need if you needed a full of rack of space, which I assume is part of the reason that pricing is almost always "contact us". I did not bother getting a quote just for the purpose of this comment. But another thing that people need to be less afraid of, when they're looking to actually spend a few digits of money and not just comment about it, is asking for quotes.
12 TB fits entirely into the RAM of a 2U server (cf. Dell PowerEdge R840).
However, I think there's an implicit point in TFA; namely, that your personal and side projects are not scaling to a 12 TB database.
With that said, I do manage approximately 14 TB of storage in a RAIDZ2 at my home, for "Linux ISOs". The I/O performance is "good enough" for streaming video and BitTorrent seeding.
However, I am not sure what your latency requirements and access patterns are. If you are mostly reading from the 12 TB database and don't have specific latency requirements on writes, then I don't see why the cloud is a hard requirement? To the contrary, most cloud providers provide remarkably low IOPS in their block storage offerings. Here is an example of Oracle Cloud's block storage for 12 TB:
Those are the kind of numbers I would expect of a budget SATA SSD, not "NVMe-based storage infrastructure". Additionally, the cost for 12 TB in this storage class is ~$500/mo. That's roughly the cost of two 14 TB hard drives in a mirror vdev on ZFS (not that this is a good idea btw).
This leads me to guess most people will prefer a managed database offering rather than deploying their own database on top of a cloud provider's block storage. But 12 TB of data in the gp3 storage class of RDS costs about $1,400/mo. That is already triple the cost of the NAS in my bedroom.
Lastly, backing up 12 TB to Backblaze B2 is about $180/mo. Given that this database is for your dev environment, I am assuming that backup requirements are simple (i.e. 1 off-site backup).
The key point, however, is that most people's side projects are unlikely to scale to a 12 TB dev environment database.
Once you're at that scale, sure, consider the cloud. But even at the largest company I worked at, a 14 TB hard drive was enough storage (and IOPS) for on-prem installs of the product. The product was an NLP-based application that automated due diligence for M&As. The storage costs were mostly full-text search indices on collections of tens of thousands of legal documents, each document could span hundreds to thousands of pages. The backups were as simple as having a second 14 TB hard drive around and periodically checking the data isn't corrupt.
Still missing the point. This is just one server and just in the dev enviornment?
How many pets do you want to be tending to? I have 10^5 servers I'm responsible for...
The quantity and methods the cloud affords me allow me to operate the same infrastructure with 1/10th as much labor.
At the extreme ends of scale this isn't a benefit, but for large companies in the middle this is the only move that makes any sense.
99% of posts I read talking about how easy and cheap it is to be in the datacenter all have a single digit number of racks worth of stuff. Often far less.
We operate physical datacenters as well. We spend multiple millions in the cloud per month. We just moved another full datacenter into the cloud and the difference in cost between the two is less than $50k/year. Running in physical DCs is really inefficient for us for a long of annoying and insurmountable reasons. And we no longer have to deal with procurement and vendor management. My engineers can focus their energy on more valuable things.
What is this ridiculous bait and switch. First you talk about a 12 TB dev databases and "How could I not use the cloud?". And you rightfully get challenged on that and then suddenly it's about the number of servers you have to manage and you don't have the energy to do that with your team. Those two have nothing to do with each other.
My friends run hundreds of TB:s serviced onto the Internet for hobby and pleasure reasons.
It's not all HA, NVMe, web scale stuff, but it's not like a few hundred TB:s is a huge undertaking even for individual nerds with a bit of money to spend or connections at corporations that monotonically decommission hardware and is happy to not have to spend resources getting rid of it.
This summer I bought a used server for 200 euros from an acquaintance, I plan on shoving 140 TB in it and expect some of my future databases to exceed 10 TB in size.
what are you doing that you have 12TB in dev??? my startup isn't even using a TB in production and we hands multiple millions of dollars in transactions every month.
Exactly! I've been self hosting for about two years now, on a NAS with Cloudflare in front of it. I need the NAS anyway, and Cloudflare is free, so the marginal cost is zero. (And even if the CDN weren't free it probably wouldn't cost much.)
I had two projects reach the front page of HN last year, everything worked like a charm.
It's unlikely I'll ever go back to professional hosting, "cloud" or not.
Is $0.55/GB not enough reason to avoid them? I guess not if your business is making more than that - bandwidth expense for a shopping site shouldn't be a problem when the customers are spending $100 for every 0.1GB - but that price should realistically be closer to $0.01/GB or even $0.002/GB. Sounds like they're forwarding you AWS's extremely excessive bandwidth pricing.
Where did you read that? The pricing page says 10 credits per GB, and extra credits can be purchased at $10 per 1500 credit. So it's more like $0.067/GB.
1. For small stuff, AWS et al aren't that much more expensive than Hetzner, mostly in the same ballpark, maybe 2x in my experience.
2. What's easy to underestimate for _developers_ is that your self hosted setup is most likely harder to get third party support for. If you run software on AWS, you can hire someone familiar with AWS and as long as you're not doing anything too weird, they'll figure it out and modify it in no time.
I absolutely prefer self hosting on root servers, it has always been my go to approach for my own companies, big and small stuff. But for people that can't or don't want to mess with their infrastructure themselves, I do recommend the cloud route even with all the current anti hype.
> 2. What's easy to underestimate for _developers_ is that your self hosted setup is most likely harder to get third party support for. If you run software on AWS, you can hire someone familiar with AWS and as long as you're not doing anything too weird, they'll figure it out and modify it in no time.
If you're at an early/smaller stage you're not doing anything too fancy either way. Even self hosted, it will probably be easy enough to understand that you're just deploying a rails instance for example.
It only becomes trickier if you're handling a ton of traffic or apply a ton of optimizations and end up already in a state where a team of sysadmin should be needed while you're doing it alone and ad-hoc. IMHO the important part would be to properly realize when things will get complicated and move on to a proper org or stack before you're stuck.
You'd think that, but from what I've seen, some people come up with pretty nasty self hosting setups. All the way from "just manually set it all up via SSH last year" to Kubernetes. Of course, people can and also definitely do create a mess on AWS. It's just that I've seen that _far_ less.
One way of solving for this is to just use K3s or even just plain docker. It is then just kuberneters/containers and you can hire alot of people who understand that.
>most startups that just run an EC2 instance will save a lot of cash going to Hetzner, Linode, Digital Ocean or whatever. I do host at Hetzner myself and so do a lot of my clients. That being said, the cloud does have a lot of advantages:
When did Linode and DO got dropped and not being part of the cloud ?
What used to separate VPS and Cloud was resources at per second billing. Which DO and Linode along with a lot of 2nd tier hosting also offer. They are part of cloud.
Scaling used to be an issue, because buying and installing your hardware or sending it to DC to be installed and ready takes too much time. Dedicated Servers solution weren't big enough at the time. And the highest Core count at the time was 8 core Xeon CPU in 2010. Today we have EPYC Zen 6c at 256 Core and likely double the IPC. Scaling issues that requires a Rack of server can now be done with 1 single server and fit everything inside RAM.
Managed database? PlanetScale or Neon.
A lot of issues for medium to large size project that "Cloud" managed to solve are no longer an issue in 2025. Unless you are top 5-10% of project that requires these sort of flexibilities.
> But the cloud is not too expensive - you're paying for stuff you don't need. That's an entirely different kind of error.
Agreed.
These sort of takedowns usually point to a gap in the author's experience. Which is totally fine! Missing knowledge is an opportunity. But it's not a good look when the opportunity is used for ragebait, hustlr.
And making sure you're not making a security configuration mistake that will accidentally leak private data to the open internet because of a detail of AWS you were unaware of.
Figuring out how to do db backups _can_ also be fairly time consuming.
There's a question of whether you want to spend time learning AWS or spend time learning your DB's hand-rolled backup options (on top of the question of whether learning AWS's thing even absolves you of understanding your DB's internals anyways!)
I do think there's value in "just" doing a thing instead of relying on the wrapper. Whether that's easier or not is super context and experience dependent, though.
> Figuring out how to do db backups _can_ also be fairly time consuming.
apt install automysqlbackup autopostgresqlbackup
Though if you have proper filesystem snapshots then they should always see your database as consistent, right? So you can even skip database tools and just learn to make and download snapshots.
nah filesystem snapshots may not lead to consistent DB backups. DB backup software usually use a plugin to tell the DB to coalesce data before taking a snapshot.
At least one OS you’ve heard of can quiesce the file system to allow taking a consistent snapshot; I’d be surprised if this wasn’t widely available everywhere.
Hmmm, I think you have to figure out how to do your database backups anyway as trying to get a restorable backup out of RDS to use on another provider seems to be a difficult task.
Backups that are stored with the same provider are good, providing the provider is reliable as a whole.
(Currently going through the disaster recovery exercise of, "What if AWS decided they didn't like us and nuked our account from orbit.")
Those buckets and IAM policies and roles also have to be managed.
There are also turnkeys solutions that allow one to spin up a DB, setup replication and backups inside or outside of big cñoud vendors. That is the point of db kubernetes operators for instance.
Plus the s3 bucket creation and definition commands, and the IAM role and attached policy commands. If you do all in the webUI it's not going to be 75 clicks either but 30 for sure.
> repeatable setups which you should be doing from the start
Yes, but not with
> TypeScript and CDK
Unless your business includes managing infrastructure with your product, for whatever reason (like you provision EC2 instances for your customers and that's all you do), there is no reason to shoot yourself in the foot with a fully fledged programming language for something that needs to be as stable as infrastructure. The saying is Infrastructure as Code, not with code. Even assuming you need to learn Terraform from scratch but already know Typescript, you would still save you time compared to learning CDK, figuring out what is possisble with it, and debugging issues down the line.
I think declarative is nicer too, but choosing a non mainstream tech here takes self-confidence in the matter that inexperienced AWSers are unlikely to have.
And learning something arguably better, like Cloudformation / Terraform / SST, is still a hurdle.
> You're getting a lot of services readily available. Need offsite backups? A few clicks
I think it is a lot safer for backups to be with an entirely different provider. It protects you in case of account compromise, account closure, disputes.
If using cloud and you want to be safe, you should be multi-cloud. People have been saved from disaster by multi-cloud setups.
> You're not paying up-front costs (vs. investing hundreds of dollars for buying server hardware)
Not true for VPSes or rented dedicated servers either.
> Peak-heavy loads can be a lot cheaper.
they have to be very spiky indeed though. LLMs might fit but a lot of compute heavy spiky loads do not. I saved a client money on video transcoding that only happened once per upload and only over a month or two an year by renting a dedi all ear round rather than using the AWS transcoding service.
> Compared to the rack hosting setup described in the post. Hetzner, Linode, etc. do provide multiple AZs with dedicated servers.
You have to do work to ensure things run across multiple availability zones (and preferably regions) anyway.
> But an RDS instance with solid backups-equivalent will usually not amortize quickly, if you need to pay someone to set it up.
You have more forced upgrades.
An unmanaged database will only need a lot of work if operating at large scale. If you are then its probably well worth employing a DBA anyway as an AWS or similar managed DB is not going to do all the optimising and tuning a DBA will do.
any serious business will(might?) have hundreds of Tbs of data. I store that in our DC and with a 2nd DC backup for about 1/10 the price of what it would cost in S3.
In my case we have a B2B SaaS where access patterns are occasional, revenue per customer is high, general server load is low. Cloud bills just don’t spike much. Labor is 100x the cost of our servers so saving a piddly amount of money on server costs while taking on even just a fraction of one technical employee’s worth of labor costs makes no sense.
I think compliance is one of the key advantages of cloud. When you go through SOC2 or ISO27001, you can just tick off entire categories of questions by saying 'we host on AWS/GCP/Azure'.
It's really shitty that we all need to pay this tax, but I've been just asked about whether our company has armed guards and redundant HVAC systems in our DC, and I wouldn't know how to do that apart from saying that 'our cloud provider has all of those'.
In my experience you still have to provide an awful lot of "evidence". I guess the advsntage of AWS/GCP/Cloud is that they are so ubiquitous you could literally ask an LLM to generate fake evidence to speed up the process.
> That being said, the cloud does have a lot of advantages:
Another advantage is that if you aim to provide a global service consumed throughout the world then cloud providers allow you to deploy your services in a multitude of locations in separate continents. This alone greatly improves performance. And you can do that with a couple of clicks.
I don’t feel like anything really changed? Fairly certain the prices haven’t changed. It’s honestly been pleasantly stable. I figured I’d have to move after a few months, but we’re a few years into the acquisition and everything still works.
Akamai has some really good infrastructure, and an extremely competent global cdn and interconnects. I was skeptical when linode was acquired, but I value their top-tier peering and decent DDoS mitigation which is rolled into the cost.
Guess you came for the hot take without actually using the service or participating in any intelligent conversation. All the sibling comments observe that nothing you are talking about happened.
Snarky ignorant comments like yours ruin Hacker News and the internet as a whole. Please reconsider your mindset for the good of us all.
To me DO is a cloud. It is pricey (for performance) and convenient. It is possibly a wiser bet than AWS for a startup that wants to spend less developer (read expensive!) time on infra.
It became much more expensive than AWS, because it bundled the hard drive space with the RAM. Couldn't scale one without scaling the other. It was ridiculous.
AWS has a bunch of startup credits you can use, if you're smart.
But if you want free hosting, nothing beats just CloudFlare. They are literally free and even let you sign up anonymously with any email. They don't even require a credit card, unlike the other ones. You can use cloudflare workers and have a blazing fast site, web services, and they'll even take care of shooing away bots for you. If you prefer to host something on your own computer, well then use their cache and set up a cloudflare tunnel. I've done this for Telegram bots for example.
Anything else - just use APIs. Need inference? Get a bunch of Google credits, and load your stuff into Vertex or whatever. Want to take payments anonymously from around the world? Deploy a dapp. Pay nothing. Literally nothing!
LEVEL 2:
And if you want to get extra fancy, have people open their browser tabs and run your javascript software in there, earning your cryptocurrency. Now you've got access to tons of people willing to store chunks of files for you, run GPU inference, whatever.
PS: For webrtc livestreaming, you can't get around having to pay for TURN servers, though.
LEVEL 3:
Want to have unstoppable decentralized apps that can even run servers? Then use pears (previously dat / hypercore). If you change your mindset, from server-based to peer to peer apps, then you can run hypercore in the browser, and optionally have people download it and run servers.
>It became much more expensive than AWS, because it bundled the hard drive space with the RAM. Couldn't scale one without scaling the other. It was ridiculous.
You can easily scale hard drive space independently of RAM by buying block storage separately and then mounting it on your Linode.
I think every VPS provider I have looked at any time recently (and I have been moving things in the last few weeks) offers some option for block storage separate from compute. Most offer an object storage option too.
I mean there are many places that sell multi AZ, hourly billed VPS/Bare Metal/GPU at a fraction of the cost of AWS.
I would personally have an account at one of those places and back up to there with everything ready to spin up instances and failover if you lose your rack, and use them for any bursty loads.
Best USDT Recovery Expert | Recover Lost or Stolen USDT Safely
Losing my USDT was one of the most painful moments of my life. I had invested with full trust, believing I was securing my future. Then, without warning, everything disappeared. My wallet showed nothing. My heart dropped. I felt sick. All I could think was, “How do I tell my family? How do I start again?”
I tried reaching out everywhere for help, but no one could do anything. The scammers were gone, and I felt like all hope was gone too.
Then someone recommended SWIFT RECOVERY SERVICES. From the very first moment I reached out to them, something felt different. They responded quickly, they listened to my story, and they treated my situation with seriousness and respect. They assured me that the stolen USDT could be traced and recovered.
They immediately began investigating the wallet addresses and tracking the scammer’s transactions. They updated me step by step. Every message gave me a little more hope than the last. I prayed the process would work.
The day they told me the funds had been successfully recovered, I was in shock. I stared at the message again and again with tears in my eyes. The USDT was finally returned back to me. After all the fear and stress, my money wasn’t gone forever after all.
I am beyond grateful to SWIFT RECOVERY SERVICES. They gave me back more than just my funds. They gave me back my confidence, my joy, and my peace of mind. I will never forget what they did for me.
If you have lost crypto due to fraud or a scam, don’t stay silent, and don’t give up. SWIFT RECOVERY SERVICES is truly reliable and capable. I thank God I found them at the right time. They are the reason I can smile again today.
This is all correct. I've been running my own servers for many years now, keeping things simple and saving a lot of money and time (setting anything up in AWS or Azure is horribly complicated and the UIs are terrible).
One thing to remember is that you do need to treat your servers as "cloud servers" in the sense that you should be able to re-generate your entire setup from your configuration at any time, given a bunch of IPs with freshly-installed base OSs. That means ansible or something similar.
If you insist on using cloud (virtual) servers, do yourself a favor and use DigitalOcean, it is simple and direct and will let you keep your sanity. I use DO as a third-tier disaster recovery scenario, with terraform for bringing up the cluster and the same ansible setup for setting everything up.
I am amused by the section about not making friends saying this :-) — most developer communities tend to develop a herd mentality, where something is either all the rage, or is "dead", and people are afraid to use their brains to experiment and make rational decisions.
Me, I'm rather happy that my competitors fight with AWS/Azure access rights management systems, pay a lot of money for hosting and network bandwith, and then waste time on Kubernetes because it's all the rage. I'll stick to my boring JVM-hosted Clojure monolith, deployed via ansible to a cluster of physical servers and live well off the revenue from my business.
I was a guy that built server clusters during the early 00's, for my own and others' web and other projects. When AWS really took off, it was like a spend all your money mania, and devs and companies treated my skills like dirt. I got a job writing facial recognition edge servers, with high performance many claim are impossible numbers (25M face compares per second per core) and my employer found itself a leader in the industry. But customers could not wrap their heads around just a single box capable of our numbers (800M face compares per second, plus ingestion of 32 video streams) and to get sales the company ended up moving everything into AWS because customers did not trust anything else.
Without targeting you directly, you wrote: "800M face compares per second". I creeped out when I read that. What is the real world application for something like that? My guess: Real-time, large scale surveillance.
That's interesting. Except for $GIGANTIC_CO (like, BofA, or the government), i'd expect a SLA that describes service resiliency and not "well, our service will be up because we're on AWS".
Why would you need to disclose your hosting provider? is that really a concern for hosted services (and if it is, why isn't the customer hosting it in their cloud?)
Most customers do not want to host anything if they can prevent it. My employer was selling the servers that host the entire shebang, and most did not want to host them. We'd explain they'd save a lot by hosting and viewing/streaming everything locally, but their IT people were not comfortable, and their execs wanted to see everything on their phones when not at work. We made it all plug and play, and still they wanted to pay 10-20X more for a web service.
Do you have any blog posts or something you could share on how facial recognition works? Specifically what a "face compare" is. That sounds interesting!
> If you insist on using cloud (virtual) servers, do yourself a favor and use DigitalOcean, it is simple and direct and will let you keep your sanity. I use DO as a third-tier disaster recovery scenario, with terraform for bringing up the cluster and the same ansible setup for setting everything up.
This is the general direction in which society is going. There is no dialogue. You are either with us or against us.
Sad.
Pre-cloud pets vs cattle approach. We ran a few pets, but had DC's worth of cattle thanks to PXE, TFPT and Ansible. No Terraform required, as there was no need to control the state of multiples of cloud cruft. Good times. Except when something would pack up in the middle of winter in the wee hours and it was a bollock-cracking motorbike ride to the DC to spit on the offending black box.
This is all true. But... But if you manage your own server, as the author advice, you need to figure out a lot of stuff and remember about a lot of stuff.
Are ulimits set correctly?
Shall I turn on syn cookies or turn them off because of performance?
What are the things I should know but I don't and Chat GPT has not told me them, as this is more than some intro tutorial on how to run VPS on DO, so it was never indexed by Chat GPT and alikes.
Is all of my software on the server up to date? Is there any library I use exploited, zero day attacks are on me too, blocking bots, etc. What if I do some update but it will turn out that my Postgres version is not working correctly anymore? This is all my problem.
What if I need to send emails? These days doing this ourselves is a dark art by itself (IP/domain address warming up, checking if my domain has not ended on some spam list, etc.).
What if I need to follow some regulations, like European Union GDPR compliance? Have I done everything what is needed to store personal data as GDPR requires? Is my DB password stored in a compliant way or I will face a fine up to 10% of my incomes.
This is not black/white situation as the author tries to present and those who use cloud services are not dumbards who are buying some IT version of snake oil.
Setting up the email server is the only thing I couldn't do with my own home hosted setup because you're at the mercy of your internet provider to give you the PTR record in their network, and lately many providers outright refuse to do it for "your own and their own safety" reasons. This thing alone could be the difference between deciding to host yourself or use a cloud service.
Just use the Postmark web service API, cheap, reliable, and far more if one really wants to lean into email with their online service. https://postmarkapp.com/email-api
I am aware of the tons of subscription based email services but that is not the point here. What good is self hosting when you still need to rely on some external paid service for a trivial thing like an email server? The costs add up.
>What if I need to send emails? These days doing this ourselves is a dark art by itself (IP/domain address warming up, checking if my domain has not ended on some spam list, etc.).
AFAIK, everyone sending automated emails just uses one of the paid services, like sendmail.
>What if I need to follow some regulations, like European Union GDPR compliance? Have I done everything what is needed to store personal data as GDPR requires? Is my DB password stored in a compliant way or I will face a fine up to 10% of my incomes.
What does this have to do with cloud vs non-cloud? You'll need to manage your data correctly either way.
All of this is true both for dedicated servers and cloud-hosted VMs.
This list looks like FUD, to be honest, trying to scare people. Yes, you should be scared of these things, but none of them are magically solved by hosting your stuff in AWS/Azure/Google or any other could provider du jour.
Some of them are solved by using managed-services that abstract away the messy config / security ops stuff.
The blog entry was wordy and repetitive for what it expressed (AI?), but the cloud argument should boil down to a few simple questions:
- Does it get you regulatory certifications you need?
- Do you need to rapidly scale-up / scale-down?
- Can you afford to hire the (minimal) necessary skills to self-administer servers?
- Can you stay on top of security updates?
Add all that together and you get a few customers who should be using the cloud:
- Regulated companies who want to punt on certs/attestations
- Small/medium growth-oriented startups (unknown needs, low headcount, focus on building product)
- Companies with hardware demand volatility that exceeds their ability to provision it
That's not "all companies" or "no companies" either way, but it is a very large number of companies who are paying cloud premiums without actually needing or benefiting from the cloud value add...
Yes, you will need to employ someone with basic system administration competence. That's a given.
Cloud infra is touted as obviating the need to hire system administrators, but that's a marketing fabrication. Trying to manage infrastructure without the necessary in-house skills is a recipe for disaster, whether it's in the cloud or on-prem.
I'm fully aware this is pedantic, but you can't save 10x. You can pay 1/10. You can save 90%. Your previous costs could have been 10x your current costs. But 10x is more by definition, not less. You can't save it.
In English, x or time(s) after a number marks a "unit" used by various verbs. A 10x increase. Increase by 10x. Go up 10x. Some of these verbs are negative like decrease or save. "Save 10x" is the same as "divide by 10". Four times less, 5 times smaller etc. are long attested.
No, x literally means multiply. It doesn't somehow also mean divide. They should use the percent sign, it's what it is for. 10x my costs means 10 x mycost, it's literally an equation
It's just inversion, like 2 to the power of 2 or 2 to the power of negative 2. These negative words inverse it just the same. You may dislike it, but millions of people have spoken this way for a long time.
> x literally means multiply
And some use the dot operator or even 2(3) or (2)(3). When programming, we tend to use *.
You can save 10x, but you need to have an x first. So if you've got formula A which costs 500 EUR, and formula B which costs 480, you're saving 20 EUR. If formula C costs 300, you could save 200, which is 10x what you would save using B. But just as people don't understand when to use 'fewer' or misuse 'literally' to mean 'figuratively' (which is rather the opposite meaning of the word they use); words don't mean anything any more, it's all about the feelings man.
People commonly use this expression in everyday conversation, such as, "you could save 10 times as much if you would just shop at Costco." So I agree with OP, their comment is correct but pedantic.
Cost of item = 10
First discounted cost of item = 9
=> First saving = 1
Second discounted cost of item = 6
=> Second saving = 4
Second saving is 4x first saving.
(Edit - formatting)
But that's 4x the savings compared to another saving. I suppose you've upped the pedantry and are technically correct, but that's a pretty narrow use case and not the one used in the article.
hmm.. if you reduce latency from one second to a hundred milliseconds, could you celebrate that you've made it 10x faster, or would you have the same quibble there too?
Edit: Thinking about this some more: You could say you are saving 9x [of the new cost], and it would be a correct statement. I believe the error is assuming the reference frame is the previous cost vs the new cost, but since it is not specified, it could be either.
> if you reduce latency from one second to a hundred milliseconds, could you celebrate that you've made it 10x faster
Yes you can, because speed has units of inverse time and latency has units of time. So it could be correct to say that cutting latency to 1/10 of its original value is equivalent to making it 10x the original speed - that's how inverses work.
Savings are not, to my knowledge, measured in units of inverse dollars.
Consider it as getting 10x the resources for the same price - that is, the resource-to-price ratio is 10x. Except you don't need 10x the resources so you choose to get 1x the resources for 0.1x the price instead.
I think you would be more accurate to say 'communicate the thought' as opposed to 'express the thought'. Communication carries the explicit desire to have an idea be shared. Expression carries no shared understanding and is personal.
The author expressed the thought in a valid way because we understood their words, but they did not communicate it well because we are confused about their understanding of those ideas.
I took care of IT for a startup hedge fund once. I was the quant's right-hand man, data engineer, visualization dashboard guy, everything. The quant needed to run a monolithic C++ program daily to chew through stock data and we decided a dual-Xeon server with 512 GB RAM would be great. OVH MG-512, for those curious.
Quant happy, boss happy, all good. Then the boss goes for lunch with someone and comes back slightly disturbed. We were not buzzword compliant. Apparently the other guy made him feel that he was using outdated tech by not being on AWS, using auto-scaling etc;
Here I am, from a background where my first language was 8086 assembly, and compactness was important to me. I remember thinking, "This whole thing could run on a powerful calculator, except for the RAM requirement".
It was a good lesson for me. Most CTOs know this bias and have unnecessarily huge and wasteful budgets but make sure they keep the business heads happy in the comfort that the firm is buzzword compliant. Efficiency and compactness are a marketing liability for IT heads!
I would think a quant would understand arithmetic.
Did you try crunching some of the numbers with him? I would hope a quant could also understand following the common wisdom can sometimes cost you more.
The author touches on it briefly, but I'd argue that the cloud is immensely helpful for building (and tearing down) an MVP or proving an early market for a new company using startup credits or free tiers offered by all vendors. Once a business model has been proven, individual components and the underlying infrastructure can be moved out of the cloud as soon as cost becomes a concern.
This means that teams must make an up-front architectural decision to develop apps in a server-agnostic manner, and developers must stay disciplined to keep components portable from day one, but you can get a lot of mileage out of free credits without burning dollars on any infrastructure. The biggest challenge becomes finding the time to perform these migrations among other competing priorities, such as new feature development, especially if you're growing fast.
Our startup is mostly built on Google Cloud, but I don't think our sales rep is very happy with how little we spend or that we're unwilling to "commit" to spending. The ability to move off of the cloud, or even just to another cloud, provides a lot of leverage in the negotiating seat.
Cloud vendors can also lead to an easier risk/SLA conversation for downstream customers. Depending on your business, enterprise users like to see SLAs and data privacy laws respected around the globe, and cloud providers make it easy to say "not my problem" if things are structured correctly.
Seems like nowadays people seem less concerned with vendor lockin than they were 15 years ago. One of the reason to want to avoid lockin is to be able to move when the price gouging gets just a little bit too greedy that the move is worth the cost. One of the drawbacks of all these built in services at AWS is the expense of trying to recreate the architecture elsewhere.
> This means that teams must make an up-front architectural decision to develop apps in a server-agnostic manner
Right. But none of the cloud providers encourage that mode of thinking, since they all have complete different frontends, API's, different versions of the same services (load balancers, storage) etc. Even if you standardize on k8s, the implementation can be chalk and cheese between two cloud providers. The lock in is way worse with cloud providers.
While I love a good cloud bashing, it's really not black and white. If you're really small, it probably doesn't matter much if you're using Hetzner or AWS, but co-location might be a bit to expensive. If you run an absolutely massive company, cloud vs. self-hosted comes down to whether or not you can build tooling as good as AWS, GCP or Azure, with all the billing infrastructure and reporting.
The issues are mostly in the SME segment and where it really depends on what your business is. Do you need completely separate system for each customer? In that case, AWS is going to be easier and probably cheaper. Are you running a constant low 24/7? Then you should consider buying your own servers.
It's really hard to apply a blanket conclusion to all industries, in regards to cloud cost and whether or not it's worth it. My criticism in regards to opting for cloud is that people want all the benefits of the cloud, but not use any of the features, because that would lock them into e.g. AWS. If you're using AWS as a virtual machine platform only, there's always going to be cheaper (and maybe better) options.
If you have high volume traffic depending on time of month, ie finance around ultimo/primo, you might need to scale your performance to 5-10x your normal idle load.
If running on your own data center, or renting physical/virtual machines from ie Hetzner, you will pay for that capability overhead for 30.5 days per month, when in reality you only need it for 2-3 days.
With the cloud you can simply scale dynamically, and while you end up paying more for the capacity, you only pay when you use it, meaning you save money for most of the month.
> If running on your own data center, or renting physical/virtual machines from ie Hetzner, you will pay for that capability overhead for 30.5 days per month, when in reality you only need it for 2-3 days.
I keep seeing this take on here and it just shows most people don't actually know what you can do off the cloud. Hertzner allows you to rent servers by the hour, so you can just do that and only pay for the 2-3 days you need them.
Also the GP take is kinda silly, because one pays 5-10x with the typical cloud providers in the first place, so it would be totally fine to pay that and already have it, if one is willing to spend that much.
More likely load will spread over time for most scenarios and the server will be ready to handle that with lower hardware specs.
There's a reason i specifically mentioned finance, where the end and start of a month is a lot more busy than the middle of the month, as in factor 10+.
People receive paychecks, pay bills, buy stuff, with holidays (christmas, x-mas, etc) being even busier.
Load does not even out, and when you have 3 million customers or more, the load is not really insignificant. Nor can you just delay it, or rely on eventual consistency.
The actual hardware (or virtual these days) is not the problem, but getting the 5000-10000 services properly connected with auditing, credentials and more is more trouble than most people expect.
Finance is a heavily regulated industry, so there’s a LOT of compliance that needs to happen, like segregation of duty, traceability, accountability, and other ilities.
Yes, it would probably cost less to run on Hetzner (provided their ISO audits are approved by financial authorities), but dynamically spinning up and down servers would cost more.
You also need fallback plans (regulated industry, critical infrastructure, etc).
It has literally taken years to get AWS and Azure approved in EU.
Anything that wants to adhere to GDPR should still be very careful, if not outright avoid, AWS and Azure. At most one could use an EU-isolated offspring of them, otherwise one runs into the insanity of US laws. The fact that many businesses don't care doesn't make it right.
Databases are rarely the bottleneck during access volume spikes in my experience. It’s really impressive how far some beefy servers and a read copy can take you.
But if that is your bottleneck you should be upgrading your DB system regardless of whether you’re on cloud or bare metal.
But if your one source of truth is a single huge database, maybe on a mainframe, as is often the case with finance, and eventual consistency is a huge no go, the mainframe can become a limit regardless when you're being "hammered" by 3+ million users all wanting to check their account balance.
Yes, you can solve the problem with sharding and other tricks, but for many banks, the mainframe is still their main data storage, and it has 60+ years of legacy code on it that is not easily or quickly migrated to modern architecture.
Banks running on mainframes don't shard. They just throw more hardware at the problem. The modern mainframes capacity for raw IO is insane, and much higher than what you can achieve with any PC setup. More hardware, more bandwidth, client facing applications in the cloud, database on the mainframe.
The major difference lies in infrastructure, particularly networking infrastructure. With cloud providers like Azure, AWS, etc, you can provision your vnet layout, and scale "indefinitely" on the same infrastructure. You don't need to provision new hosts, setup new secrets, or anything like that.
If a data center goes down, you can relatively easy switch to another one, though most financial institutions I know of uses hot/cold setups as hot/hot is essentially twice the money, and they rarely go down for long.
Of course it's all just regular servers underneath, so anything possible with AWS and Azure is also possible with other cloud providers, but the tooling simply isn't there (yet?).
Another issue is ISO auditor compliance. Being a regulated industry, finance (in EU at least) needs certain compliance to be fulfilled, not only regarding the services you consume, but also stuff like the physical locations, or being able to physically inspect the data center if auditors require it.
Microsoft and Amazon has this nailed. I've yet to experience a EU data center not run by FAANG meet the requirements, not that they can't. My best hope so far is probably "Lidl cloud" (forgot the name).
Right, the networking is the biggest hurdle in splitting up your infra.
The compliance issues are another big one though, for organizations that are still scaling up and don’t have that know how using cloud is a huge advantage as well.
I'd be more interested to understand (from folk who were there) what the conditions were that made AWS et al such a runaway hit. What did folks gain, and have those conditions meaningfully changed in some way that makes it less of a slam dunk?
My recollection from working at a tech company in the early 2010s is that renting rack space and building servers was expensive and time consuming, estimating what the right hardware configuration would be for your business was tricky, and scaling different services independently was impossible. also having multi regional redundancy was rare (remember when squarespace was manually carrying buckets of petrol for generators up many flights of stairs to keeps servers online post sandy?[1]).
AWS fixed much of that. But maybe things have changed in ways that meaningfully changes the calculus?
You're falling into the false dichotomy that always comes up with these topics: as if the choice is between the cloud and renting rack space while applying your own thermal paste on the CPUs.
In reality, for most people, renting dedicated servers is the goldilocks solution (not colocation with your own hardware).
You get an incredible amount of power for a very reasonable price, but you don't need to drive to a datacenter to swap out a faulty PSU, the on site engineers take care of that for you.
I ordered an extra server today from Hetzner. It was available 90 seconds afterwards. Using their installer I had Ubuntu 24.04 LTS up and running, and with some Ansible playbooks to finish configuration, all in all from the moment of ordering to fully operational was about 10 minutes tops. If I no longer need the server I just cancel it, the billing is per hour these days.
Bang for the buck is unmatched, and none of the endless layers of cloud abstraction getting in the way. A fixed price, predictable, unlimited bandwidth, blazing fast performance. Just you and the server, as it's meant to be.
I find it a blissful way to work.
I’d add this. Servers used to be viewed as pets; the system admins spent a lot time on snow flake configurations and managing each one. When we started standing up tens of servers to host the nodes of our app (early 2000s); the simple admin overhead was huge. One thing I have not seen mentioned here was how powerful ansible and similar tools were at simplifying server management. Iirc being able to provision and standup servers simply with known configurations was a huge win aws provided.
You were commonly given a network uplink and a list of public IP addresses you were to set up on your box or boxes. IPMI/BMC were not a given on a server so if you broke it, you needed to have remote hands and probably brains too.
Virtualisation was in the early days and most of the services were co-hosted on the server.
Software defined networks and Open vSwitch were also not a thing back then. There were switches with support for VLANs and you might have had a private network to link together frontend and backend boxes.
Servers today can be configured remotely. They have their own management interfaces so you can access the console and install OS remotely. The network switches can be reconfigured on the fly, making the network topology reconfigurable online. Even storage can be mapped via SAN. The only hands on issue is hardware malfunction.
If I was to compare with today, it was like having a wardrobe of Raspberry Pies on a dumb switch, plugging in cables when changes were needed.
Even if you don't go ansible/chef/puppet/salt, just having git is good. You can put your configs in git, use a git action to change whatever variables, and deploy to the target. No extra tools needed, and you get versioned configs.
> all in all from the moment of ordering to fully operational was about 10 minutes tops.
I think this is an important point. It's quick.
When cloud got popular, doing what you did could take upwards of 3 months in an organisation, with some being closer to 8 months. The organisational bureaucracy meant that any asset purchase was a long procedure.
So, yeah, the choices were:
1. Wait 6 months to spend out of capex budget
Or
2. Use the opex budget and get something in 10m.
We are no longer in that phase, so cloud services makes very little sense now because you can still use the opex budget to get a VPS and have in going in minutes with automation.
True, but I think you're touching on something important regarding value. Value is different depending on the consumer: for you, you're willing and able to manage more of the infrastructure than someone who has a more narrow skillset.
Being able to move the responsibility for areas of the service on to the provider is what we're paying for, and for some, paying more money to offload more of the responsibility actually results in more value for the organization/consumer
AWS also made huge inroads in big companies because engineering teams could run their own account off of their budget and didn’t have to go through to IT to requisition servers, which was often red tape hell. In my experience it was just as much about internal politics as the technical benefits.
Seconded. I was working for a storage vendor when AWS was first ascendant. After we delivered hardware, it was typically 6-12 weeks to even get it powered up, and often a few weeks longer to complete deployment. This is with professional services, e.g. us handling the setup once we had wires to plug in. Similar lead time for ordering, racking, and provisioning standard servers.
The paperwork was massive, too. Order forms, expense justifications, conversations with Legal, invoices, etc. etc.
And when I say 6-12 weeks, I mean that was a standard time - there were outliers measured in months.
Absolutely. At several startups, getting a simple €20–50/month Hetzner server meant rounds with leadership and a little dance with another department to hand over a credit card. With AWS, leadership suddenly accepted that Ops/Dev could provision what we thought was right. It isn’t logically compelling, but that’s why the cloud gained traction so quickly: it removed friction.
> At several startups, getting a simple €20–50/month Hetzner server meant rounds with leadership and a little dance with another department to hand over a credit card.
That's not a startup if you can't go straight to the founder and get a definite yes/no answer in a few minutes.
Computing power (compute, memory, storage) has increased 100x or more since 2005, but AWS prices are not proportionally cheaper. So where you were getting a reasonable value in ~2012, that value is no longer reasonable, and by an increasing margin.
In 2006 when the first EC2 instances showed up they were on par with an ok laptop and would take 24 months to pay enough in rent to cover the cost of hardware.
Today the smallest instance is a joke and the medium instances are the size of a 5 year old phone. It takes between 3 to 6 months to pay enough in rent to cover the cost of the hardware.
What was a great deal in 2006 is a terrible one today.
raises hand I ran a small SaaS business in the early 2000s, pre-AWS.
Renting dedicated servers was really expensive. To the extent that it was cheaper for us to buy a 1U server and host it in a local datacenter. Maintaining that was a real pain. Getting the train to London to replace a hard drive was so much fun. CDNs were "call for pricing". EC2 was a revelation when it launched. It let us expand as needed without buying servers or paying for rack space, and try experiments without shoving everything onto one server and fiddling with Apache configs in production. Lambda made things even easier (at the expense of needing new architectures).
The thing that has changed is that renting bare metal is orders of magnitude cheaper, and comparable in price to shared hosting in the bad old days.
> But maybe things have changed in ways that meaningfully changes the calculus?
I'd argue that Docker has done that in a LOT of ways. The huge draw to AWS, from what I recall with my own experiences, was that it was cheaper than on-prem VMware licenses and hardware. So instead of virtualizing on proprietary hypervisors, firms outsourced their various technical and legal responsibilities to AWS. Now that Docker is more mature, largely open source, way less resource intensive, and can run on almost any compute hardware made in the last 15 years (or longer), the cost/benefit analysis starts to favor moving off AWS.
Also AWS used to give out free credits like free candy. I bet most of this is vendor lock in and a lot of institutional brain drain.
The free credits... what a WILD time! Just show up to a hackathon booth, ask nicely, and you'd get months/years worth of "startup level" credits. Nothing super powerful - basically the equivalent of a few quad core boxes in a broom closet. But still for "free".
> although actually many people on here are American so I guess for you aws is legally a person...
Corporate legal personhood is actually older than Christianity, and it being applied to businesses (which were late to the game of being allowed to be corporations) is still significantly older than the US (starting with the British East India Company), not a unique quirk of American law.
What is unique in the US is the interaction between corporate personhood and our First Amendment and the way that our courts have applied that to limit political campaign finance laws, and a lot of “corporate personhood” controversy is really about that, not actually about corporate personhood as a broad concept.
People also get confused about the Citizens United ruling. It had nothing to do with corporate personhood.
The ruling said that since a person has first amendment rights, those same rights extend to a group of people—any group—whether it’s a non profit organization, a corporation, or something else.
Well, it's also what allows the executives, boards, and owners of companies to be divorced from the consequences of their actions in a legal sense much of the time.
I don't think that's a hard and fast rule? I think et al is for named, specific entities of any kind. You might say "palm trees, evergreens trees, etc" but "General Sherman, Grand Oak, et al"
The problem it really solved was your sysadmins were still operating by SSHing into the physical servers and running commands meticulously typed out in a releaae doc or stored on a local mediawiki instance, and acquiring new compute resources involved a battle with finance for the capex which would delay pretty much any project for weeks, while cloud vendors let engineers at many companies sidestep both processes.
Everything else was just reference material for how to sell it to your management.
TL;DR version - its about money and business balance sheets, not about technology.
For businesses past a certain size, going to cloud is a decision ALWAYS made by business, not by technology.
From a business perspective, having a "relatively fixed" ongoing cost (which is an operational expense ie OpEx ) even if it is significantly higher than what it would cost to do things with internal buy and build out (which is a capital expense cost ie CapEx), make financial planning, taxes and managing EBITDA much easier.
Note that no one on the business really cares what the tech implications are as long at "tech still sorta runs mostly OK".
It also, via financial wizardry, makes tech cost "much less" on a quarter over quarter and year over year basis.
Its not actually that hard to get your own server racked up in a data centre, I have done it. Since it was only one box that I built and installed at home I just shipped it and they racked it in the shared area and plugged the power and network in and gave me the IP address. It was cheaper than renting from someone like hetzner, was about £15 a month at the time for 1A and 5TB a month of traffic at 1gbps. Also had a one off install fee of £75.
At the time I did this no one had good gaming CPUs in the cloud, they are still a bit rare really especially in VPS offerings and I was hosting a gaming community and server. So I made a pair of machines in a 1U with dual machines in there and had a public and private server with raid 1 drives on both and redundant power. Ran that for a gaming server for many years until it was obsolete. It wasn't difficult and I think the machine was about £1200 in all, which for 2 computers running game servers wasn't too terrible.
I didn't do this because it was necessarily cheaper, I did it because I couldn't find a cloud server to rent with a high clockspeed CPU in it. I tested numerous cloud providers, sent emails asking for specs and after months of chasing it down I didn't feel like I had much choice. Turned out to be quite easy and over the years it saved a fortune.
We did this a lot in the early 2000's. At the time I worked for a company with offices in Bellevue and we put our own hardware in full sized racks at a datacenter in the komo4 building in Seattle.
Because of proximity it was easy to run over and service the systems physically if needed, and we also used modem based KVM systems if we really needed to reboot a locked up system quickly (not sure that ever actually happened!).
I'm sure customer owner hardware place in a datacenter rack is still a major business
"Remote hands" is the DC term for exactly what it sounds like. You write a list of instructions and someone hired by the DC will go over to your rack and do the thing.
the problem isn't setup. its maintaining it. Its not an easy job to do that some times. Im not trying to dissuade people from running there own servers, but its something to consider.
This isn’t a binary issue. I disagree with these “abandon the cloud” takes but do agree that most folks spend way way more than they should.
The biggest threat to cloud vendors is that everyone wakes up tomorrow and cost optimizes the crap out of their infrastructure. I don’t think it’s hyperbolic to say that global cloud spending could drop by 50% in 3 months if everyone just did a good audit and cleaned up their deployments.
> The biggest threat to cloud vendors is that everyone wakes up tomorrow and cost optimizes the crap out of their infrastructure
Well there's no danger of that. Even with AWS telling you exactly how to save money (they have a thousand different dashboards showing you where you can save, and even support will tell you), it'll still take you months to work through all the cost optimization changes. Since it's annoying and complicated to do, most people won't do it.
Their billing really is ridiculous. We have a TAM and use a reseller, and it's virtually impossible for us to see what we actually spend each month, what with the reseller discounts, enterprise PPA, savings plans, RIs, and multiple accounts. Their support reps even built us some kinda custom BI tool just to look at costs, and it's still not right.
We have to use cloud because we're at the low end of 10^5 servers. Once you hit the high end of 10^3 this is really where you need to be.
Everything we're doing is highly inefficient because of decades of fast and loose development by immature software engineers...and having components in the stack that did the same.
If I had 5 years to rewrite and redesign the product to reflect today's computing reality, I could eliminate 90%+ of the cost. But I'll never get that kind of time. Not with 1000 more engineers and 1000 more years and the most willing customers.
You might get lucky enough that you and a bunch of your customers are so fed up with your company that you get to create the competition.
You have over a 100,000 servers? How is this? This is wild to ponder. I can only guess your monthly bill is in the millions. Are you able to say the company?
"100.000 servers" sounds like the entirety of 2008-era Google, but with today's 10-100x performance per server. It could be bigger than Dropbox or Netflix.
Is your time worth more than what the fully managed services on AWS cost? And I mean that quite literally in the sense of your billable hours.
If you like spending time tinkering with manually configuring linux servers, and don't have anything else that generates better value for you to do - by all means go for it.
For small pet projects with no customers, a cheap Hetzner box is probably fine. For serious projects with customers that expect you to be able to get back on your feet quickly when shit hits the fan, or teams of developers that lose hours of productivity when a sandbox environment goes down, maybe not so much.
Is AWS more expensive than the salary of the infra guy you would need to hire to do all this stuff by hand? Probably not.
Well, this sounds wildly out of touch. Companies using Hetzner cloud or servers having thousands of customers and paying 50-100 people are running just fine, but you labeled them a "small pet project". Provisioning new servers works just fine via API when you need it.
The first couple of paragraphs of price comparisons are useful. Then there are many paragraphs of sheer waffle. The author doesn't even seem able to define what "the cloud" is:
> The whole debate of “is this still the cloud or not” is nonsense to me. You’re just getting lost in naming conventions. VPS, bare metal, on-prem, colo, who cares what you call it. You need to put your servers somewhere. Sure, have a computer running in your mom’s basement if that makes you feel like you’re exiting the cloud more, I’ll have mine in a datacenter and both will be happy.
I read the whole thing and I didn't see any waffle. Sure, undeniably some excess word count, some emotion in responding to critics. But no waffle.
The "is this cloud or not" debate in the piece makes perfect sense. Who cares whether Hetzner is defined as "the cloud" or not? The point is, he left AWS without going to Azure or some other obvious cloud vendor. He took a step towards more hands on management. And he saved a ton of money.
The cheap hosting service they switched to is arguably "cloud".
If you can't drive to the location where your stuff is running, and then enter the building blindfolded, yet put your hands on the correct machine, then it's cloud.
For this guy who spent thousands per month on AWS, using AWS makes no sense.
For a guy like me whose AWS bill was 38 cents last month, AWS makes a lot of sense. Currently I have a small KaiOS app with about 1000 unique visitors per month hosted on AWS. I used to self-host on an old laptop, but it costs less money to host on AWS (1 kwh cost me 19.89 cents and my laptop uses about 4.5kwh per month). Not to mention the convenience of not having to restart my server for software updates, or update my own certs every 3 months, or fuddling with nginx configurations.
And my AWS bill ever starts climbing, I will just self host once again.
Probably bad cache headers configuration. Even with Cloudflare in front it could be forwarding every request to the backend if the cache headers are misconfigured...
Cheap shot maybe, but the fact that the page takes 10 seconds to load when it hits the HN front page is a great, inadvertant illustration of why you might want to use the cloud sometimes.
The failure mode of self-hosting is that your site gets hugged to death, the failure mode of the cloud is that you lose a ton of money. For a blog that doesn't earn you anything, the choice is clear.
Besides, you can just put it behind cloudflare for free.
> the failure mode of the cloud is that you lose a ton of money
I'm perfectly happy to use cloud stuff for work. I will never ever give any cloud platform my personal credit card. I don't want to be the position of a misconfiguration leaving my finances at the mercy of the AWS forgiveness gods.
Absolutely. And I never said that everybody should use the cloud. Contrary to OP, it's more complicated than that. Not every site has the same needs. Some sites need scaling. Some sites are fine with being hugged to death, others would lose millions of dollars if they were. Some people are happy to tinker with their nginx config and package upgrades etc, others prefer to deploy and forget it (with cost controls enabled).
Disclosure: I work for Cloudflare, but 25 years ago served my own site from an iBook under the stairs.
OP here. As others have said, it loads immediately for me (tested on desktop + on mobile data + incognito)
The entire site is cached + Cloudflare sits on top of everything. I just ran a couple performance tests under the current HN traffic (~120 concurrent visitors) and everything looks good, all loads under 1 second. The server is quite happy at an average load of 0.06 right now, not even close to start breaking a sweat.
Turns out you can get off the cloud and hit the frontpage of HN and your site will be alright.
100% true, I've hit the front page of hn on a server with an old i5 (aka consumer hardware, and not even high end) with no cloudflare or similar caching, and had no problems. Computers are fast, and serving static html over https is a solved problem.
My guess is your caching was misconfigured in some way. It's fast for me now, but before commenting I tried several times and it always took at least 10s to load, often more. Others have also reported very slow load times. I was surprised because I saw at the time that it was behind Cloudflare. Disclosure: I work for Cloudflare.
That happens a lot to blogs deployed on the cloud too. They just need to put a small cache in front and they'll be able to serve one or two orders of magnitude more requests per second.
I have a VPS. It costs me £1.34 per month. It's way over-powered for what I need it for.
However, one situation where I think the cloud might be useful is for archive storage. I did a comparison between AWS Glacier Deep Storage and local many-hard-drive boxes, for storing PB-scale backups, and AWS just squeaked in as slightly cheaper, but only because you only pay for the amount you use, whereas if you buy a box then you have to pay for the unused space. And it's off-site, which is a resilience advantage. And the defrosting/downloading charge was acceptable at effectively 2.5 months worth of storage. However, at smaller scales you would probably win with a small NAS, and at larger scales you'd be able to set up a tape library and fairly comprehensively beat AWS for price.
Its a weird service because before that point AWS is crazy expensive for storage, especially down in the TB range its awful value compared to your box and drives. But once you get into that PB scale AWS actually seems to be competitive, I guess because the GB/TBs they are selling are from PB scale solutions and all the overhead that entails.
"To them, it’s way too convenient to be on AWS: not only it solves their problem, but it’s also a shiny object. It’s technically complex, it makes them look smart in front of other devs, "
why? why be so obnoxious to other people who you claim are being obnoxious to you. no need to read your blog post now/
It’s entirely possible for a rented server to host a site that gets millions of views. It’s also entirely possible to make an AWS setup that chokes with 100.
I've been at too many startups with a devops team that would rather provision 15 machines with 4GB RAM THAN ONE WITH 64GB.
I once got into an argument with a lead architect about it and it's really easy to twist the conversation into "don't you think we'll reach that scale?" To justify complexity.
The bottom line is for better or worse, the cloud and micro services are keeping a lot of jobs relevant and there's no benefit in convincing people otherwise
Many small boxes
Pros:
- Cost. Small boxes cost less. Spend less or spend more as needed. If one dies, cheaper to replace.
- Efficiency. Scaling *down* saves money when load is low. Can schedule specific loads to specific boxes.
- Redundancy. Multiple VMS, OSes, network paths, etc. No single point of failure.
- Zero-downtime. Rolling deployments, upgrades means changes with no user impact.
- System bandwidth. More network links, cpus, kernels, disks, etc = more bandwidth, capacity.
- Performance resilience. A heavily loaded app on one server doesn't affect others.
- Immutability. "Pets" rather than "cattle" uses automation to reduce maintenance/instability.
- Scalability. When you run out of resources, adding more is easy, zero impact.
Cons:
- Does not work with applications that require large memory/cpu.
- Inefficient for apps that require shared filesystem access (as opposed to database).
- Requires smarter architecture to reduce long tail of cross-host calls.
- More transient network path failures, troubleshooting issues.
One big box
Pros:
- Allows applications which require large memory/cpu.
- More efficient for apps that share a filesystem.
- Simpler architecture.
- Fewer network path failures.
Cons:
- Large cost that you can't easily reduce as needed.
- Waste (in unused resources) unless load is constant.
- Single point of failure, for reliability and security.
- Upgrades require reboots. App goes down; possibility the server might not boot up properly.
- Single network, cpu, kernel, disks(s), etc become bottlenecks.
- A single heavily-loaded process, excess interrupts, etc can bring down entire system performance.
- Often treated as "pet" rather than "cattle"; creates more maintenance, instability.
- Not scalable.
Big box VM host running several VMs, hardware starts to fail, brings the entire setup to its knees. That wouldn't happen if you had each VM running on its own physical hardware, unless the UPS/PDU died...
VMs are able to live migrate these days. Single VM has a lot more uptime guarantee than a distributed system with multiple over the network communication layers.
We've moved to Hetzner several years ago, and the savings will ensure we'll likely never consider the Cloud again (Exception is Cloudflare Workers, which is well done with generous quotas). Although it would be way easier to move now since automation, deployment, backup and retention scripts are much faster to write now with AI.
BTW in-case you've missed it [1] Anthropic is offering $250 free credits till Nov 18 for their new Claude Code on the Web [2] for Clause Pro/Max users.
How much is your labor worth though? The actual cost of a server is not expensive; the maintenance behind it is. If I have a fleet of 10 EC2 instances, I can keep them patched with Systems Manager without having to build any of that automation myself.
Sometimes I think I am out of the loop for using dedicated servers like OVH, DigitalOcean, and Hetzner, while others spend thousands of dollars for the things I spend barely a few hundred. This always made me think I am not a good developer enough to know the cutting-edge things others know.
Turns out most of the developers suck at handling barebones with a Linux distro + nginx and some other plugins to do the same things as the fancy-named aws stuff. If you are in the same boat, just know that most of these developers suck at what they are doing and don't care about the company budget.
You can get 99.99% of the things done with barebone + Cloudflare, including multiserver redundancy, at a fraction of AWS and Azure costs. Most of these technologies are just fancy words for basic Linux services.
I would really be interested in an actual comparison, where e.g. someone compares the full TCO of a mysql server with backup, hot standby in another data center and admin costs.
On AWS an Aurora RDS is not cheap. But I don't have to spend time or money on an admin.
Is the cost justified? Because that's what cloud is. Not even talking about the level of compliance I get from having every layer encrypted when my hosted box is just a screwdriver away from data getting out the old school way.
When I'm small enough or big enough, self managed makes sense and probably is cheaper. But when getting the right people with enough redundancy and knowledge is getting the expensive part...
But actually - I've never seen this in any if these arguments so far. Probably because actual time required to manage a db server is really unpredictable.
> Probably because actual time required to manage a db server is really unpredictable.
This, and also startups are quite heterogeneous. If you have an engineer on your team with experience in hosting their own servers (or at least a homelab-person), setting up that service with sufficient resiliency for your average startup will be done within one relaxed afternoon. If your team consists of designers and engineers who hardly ever used a command line, setting up a shaky version of the same thing will cost you days - and so will any issue that comes up.
Its a skillset that is out of favour at the moment as well but having someone who has done serverops and devops and can develop as well is a bit of a money saver generally because they open up possibilities that don't exist otherwise. I think its a skillset that no one really hired for past about 2010 when cloud was mostly taking off and got replaced with cloud engineers or pure devops or ops people but there used to be people with this mixed skillset in most teams.
I've never had a server go down. Most companies don't need a hot server because it's never going to be needed.
AWS + Azure have both gone down with major outages indivudually more over the last 10 years than any of the servers in companies I worked with in the 10 years before that.
And in comparable periods, not a single server failed or disk failed or whatever.
So I get SOME companies need hot standby servers, almost no company, no SaaS, no startup, actually does.
Because if it's that mission critical, then they would have already had to move off the cloud due to how frequently AWs/Azure/etc. have gone down over the last 10 years, often for 1/2 day or so,
I've had a lot of servers going down. I've had data centers going down. For various reasons - but normally not a failed disk but configuration errors due to human error.
And I've had enough cases where the company relied on just that one guy who knew how things worked - and when they retired or left, you had big work ahead understanding the systems that guy maintained and never let anyone else touch. Yes, this might also be a leadership issue - but it's also an issue if you have no one else with that specific knowledge. So I prefer standardized, prepackaged, off the shelf solutions that I can hire replacable people for.
The idea that it's cheaper not to use AWS is clear.
I was hoping to see more about porting AWS proprietary features into generic servers.
A big part of the problem isn't just monthly rent, it's vendor lock-in. When your whole system is implemented using AWS specific features, you're not going to run anywhere else.
AWS, and any other third party vendor, can and does obsolete features. Then you're having to port your system just to keep it running on the third party service.
Once you're implemented in a generic server, in a VPS, or your mom's basement, you're free to move to any other hosting provider, data center, whatever.
The loss of understanding that 3rd party dependencies are not good for your company or project, seems a bigger loss to the technical community than FTP hacking...
I worked for a small, local insurance company 10+ years ago that was not interested in the cloud in any way, shape, or form. They had their servers in two locations—the basement of our office building, and an offsite location ~300 miles away. They were professional server rooms, complete with proper cooling, backup power, and the like.
At first I remember thinking they were behind the times, but now I'm certain they've avoided wasting a fortune on the cloud.
After a few months of experimenting on AWS, I got scared seeing how fast small side projects could turn into expensive monthly bills.
I built my own DIY cloud, a minimal self-hosted setup that lets me deploy apps, databases, and backups on cheap VPS or bare-metal servers. At first I just wanted to save money, but I realized that managing it can easily become a full-time SRE job.
Still, for small, experimental, or hobby projects, this setup works perfectly. It keeps costs predictable and gives full control without needing large-scale infrastructure.
> To them, it’s way too convenient to be on AWS: not only it solves their problem, but it’s also a shiny object. It’s technically complex, it makes them look smart in front of other devs, it creates dependencies and lock-in effects so they’re not easily replaceable as employees, and most importantly – being on the cloud yields them a fat paycheck at the end of the month.
I recently read a fantastic comment or article on how developers, and IT workers in general, will be the first to be automated away, and groups like lawyers and doctors the last, entirely unrelated to how good LLMs and other AI models are at the specific tasks of each. Reason being that developers fall over themselves to prove how productive they are, happy to use all these tools. Anywhere else is concerned with job protection first and foremost. Metrics like SLOC having come from devs themselves in the chase of a raise.
This paragraph reminds me of that so much. Calling out those who push the cloud for job security. In basically every other sector - where this kind of behavior is 10x more prevalent - no one calls this out, there's a sense of solidarity and mutual understanding that this kind of thing is necessary.
This isn't judging one or the other to be good or bad. But it's going to cost devs very dearly.
I don't remember where I read it, they put it much better than I can - if anyone here recognizes it please drop a link.
A 2024 International LT Series semi-truck costs $130,000. That's very expensive compared to a $30,000 Ford Maverick.
Both of these trucks can technically be used to pick up groceries and commute. But, uh, if you bought the semi-truck to get groceries and commute? Nobody scammed you; you bought the wrong truck. You don't have to buy the biggest, most expensive truck to do small jobs. But also, just because there's a cheaper truck available, doesn't mean the semi-truck is overpriced or a scam. The semi is more expensive for a reason.
I wonder about people who write articles like these. I imagine at one point he believed he had to use the cloud, so he started using it without understanding what he was doing. As a result, he was charged a ton of money. He then found out there were cheaper, simpler ways to do what he wanted. And so, feeling hurt, embarrassed, and deceived, he writes this long screed, accusing people of scamming him - I mean scamming you - because you (not him!) could not possibly need to use the cloud, even though you (not him!) assumed you had to use the cloud.
Yes, dude. The cloud is expensive. Sorry you found out the hard way. And for what it's worth, you don't need a datacenter either; stick a 1U in your closet and call it a day.
I don't particularly like this article, mostly because he seems to have spent the great majority of it ranting about people on the internet supposedly yelling at him about not being "on the cloud" and semantics about exactly what constitutes "the cloud". I'd be much more interested in reading exactly what he was spending that money on at AWS and how he moved it all onto one or a few bare-metal boxes.
I don't think AWS is particularly expensive at all. In fact, it's rather cheap at quite a few things if you know what you're doing and use the right services. I've got all my server backups going to S3, for which I pay a whopping penny a month. I also host a handful of Docker images on ECR for 7 cents a month - way cheaper than any other private hosting service I know of.
Now RDS, and the equivalent at every other hosting provider I know of, gets quite pricey indeed. IMO, you shouldn't bother unless you have pretty serious needs for reliability and speed. Apt install your DB server of choice on bare metal goes quite far. Or possibly pull a container of it into whatever you're using to manage containers.
I think the main thing holding people back from leaving the cloud is simple inertia. There was a time when the cloud was obviously the right choice. Static IPv4 addresses were becoming scarce, IPv6 had not been deployed widely enough, and cloud providers made it easy to stand up a server and some storage with high speed links on the cheap. But over time, things have changed. Rate limits, data caps, and egress fees are now normal (and costly). IPv6 is now deployed widely enough that you might be willing to just run an IPv6-only stack, which greatly simplifies running a server on-premise. And of course, we've all seen time and again how providers will carelessly lock out your cloud account for arbitrary reasons with little to no recourse. The time has come to own your infrastructure again. But that won't happen until people realize it's easy to do.
If all you need is compute, than yeah, self hosting is easy. Otherwise, do you think just about every company under the sun is a sucker for being on the cloud? If it was so easy, companies would be either be constantly dropping prices to compete with all the self hosters, or new companies to fill in the price gaps.
I never understood the appeal of AWS like cloud for non corporate. Sure, when you run a BigCo, you have all the money and people to throw at big cloud providers, obtaining pointless certificates, and configuring obscure IAM roles.
But why small businesses like them, I can’t I understand. I host all my stuff on Hetzner. It’s easy, it’s affordable and I “move fast”. I don’t need certificates to be able to manage the servers, and there is tone of information available online. But maybe it’s because I’m old, and I used to ssh to a Linux machine, git pull the latest version, and call in a “deploy”. But modern developed seems to be afraid of raw machines.
The only thing I miss is a good transactional email provider with PAYG pricing, and EU data residency.
“Hey bob from finance, I heard you want a new bi and dashboarding system to help us make better strategy choices? here’s 200k, hire devs, infra spend can be billed to this project id”
I have some projects with a fairly steady load that I'm currently running on a VPS on Digital Ocean. I have some non-critical ones that I'd be interested in moving to "bare metal" for learning purposes, but having a hard time figuring out what the transition from VPS to dedicated server would look like.
Anybody have any guides or guidance on making the migration? I feel comfortable managing the VPS, but not sure what else I'd need to take on by moving to a dedicated server. I imagine hardware monitoring would be one, but what else?
Why it feels like the author is too young and just had breaking discovery that he can have servers without clouds!? Always been a thing, clouds were/are used in areas where it would be better, say some integration with already existing infrastructure, or some quick scaling. Just like everything, there’s always upside and downside, and it’s just about what suits your needs. The author should next try an on-prem approach where he even controls the hardware, even more cheaper but with extra maintenance. For example, I found a used server a while ago (44 Core HP Z840 WorkStation Dual Xeon E5-2699 V4 512GB RAM) for around $1000, that’s a one time pay.
I recently had the same realization and moved all my functions to a simple stand-alone server. Besides the normal AWS costs, what scares me most about AWS is the possibility that someone could try to DOS me, leaving me with a huge AWS bill, because there is no real way to limit AWS spending.
The main reason why I keep coming back to cloud providers is databases. I don't feel comfortable setting up a high-availability db setup, and I don't want the responsibility of managing backups. But if you go to, say, Hetzner, you won't be able to use a cloud database in the same network.
I've always found AWS IAM quite simple, but then again it is my job, so I might be biased. I haven't really dug into GCP well enough to understand it, but I did find it quite daunting to start the few times I messed with it. What's complex about it to you?
For personal projects, honestly, the built in roles AWS provides are okay enough for some semblance of least privilege x functionality IMO.
Plus, most of AWS's documentation tells you the specific policy JSON to use if you need to do XYZ thing, just fill in the blanks.
What if the data center that has your dedicated server burns down? You need to prepare to that buy renting another server at a different data center and synching them. Not a big problem, but needs to be mentioned
Correct AWS gives you availability zones. It is easy to spin up a cluster over them and have little or no downtime.
But a burning data centre is just one problem. Another is e.g. the USE1 issue AWS had. AZ didnt help here. Having compute in a other region an rerouting might. But maybe we aint talking startup concerns here. (If you have a simple enough "app" just deploy again in another region and update dns.)
>This happens with Hetzner all the time because they have no VLANs and all customers are on a single LAN and IPFS tries to discover other nodes in the same LAN by default.
I think a lot of teams using cloud are using SaaS rather than IaaS. They want a redis and a postgres and a S3 and a ... You can set all that up on a server, but it's not very fun if you've never done it before.
Its true that aws is very expensive for a lot of use cases, as long as you only count the cost of the servers.
However, in most (every?) large organisations, buying a server and putting it in a DC, and looking after it, is hugely time-consuming, lengthy and expensive.
You need to get quotes, approvals, purchase, rack, commission, maintain etc etc. It is usually close to 1 year to get a server.
In some companies, even getting a virtual server takes almost as long!
With AWS, once an account and service is approved, boom, you are done.
My experience is exactly the opposite (company with more than 10k employees). Getting anything done in Azure takes me 10x as long, as all of Azure is managed by one team, and everything requires approvals, lots of bureaucracy. Also, as it turns out, it is extremely expensive. Per our guidelines everything needs to be isolated within company intranet (unless really required to be external), which often means we need premium tier services in Azure. These are really, really pricey sometimes.
On the other hand, if I request a virtual server, it takes less than a week, and I can work with it much more freely.
Let’s not forget capex/opex. You can bill an aws project to a project, datacenter costs or shared across the entire org and it’s prohibitively complicated to budget these per project.
Sure you can do it cheaper, but money isn’t really the problem. Most cloud spend is less than the cost of a handful of senior devs a month.
In principle, you don't even need a "server", ie purpose-built hardware like what he's showing. An old laptop will be fine for many projects, although I'd be interested if someone can help put a perspective on how to decide whether your specific project/load will be too much for your specific hardware. My bigger concern with 'true' self-hosting would be how to get a good/reliable enough internet connection.
Even if you don't end up switching, the lesson from this person's experience and the comments here on HN is: Stop being afraid.
People make all sorts of excuses: it's not secure, it's not reliable, it doesn't scale. These are primarily people who have never touched a server. If they had, they would know actual tradeoffs instead of excuses. So go and touch a server and learn how it works.
Jeez, this was a painful read. I actually stopped after a few paragraphs and asked AI to make it more technically focused and remove the ranting so I could stomach it.
Strawman arguments, ad hominem attacks and Spongebob mocking memes, and the casual venturing into conspiracy theories and malicious intentions...
> Why do all these people care if I save more money or not? ... If they’re wrong, and if I and more people like me manage to convince enough people that they’re wrong, they may be out of a job soon.
I have a feeling AWS is doing fine without him. Cloud is one of the fastest growing areas in tech because their product solves a need for certain people. There is no larger conspiracy to keep cloud in business by silencing dissent on Twitter.
> You will hear a bunch of crap from people that have literally never tried the alternative. People with no real hands-on experience managing servers for their own projects for any sustained period of time.
This is more of a rant than a thoughtful technical article. I don't know what I was expecting, because I clicked on the title knowing it was clickbait, so shame on me, I guess...
Just a bit of nitpicking. I am AWS certified, I still tell people not to do cloud. I even recommended as much to my current employer, laid out the costs for their architecture project doing on-prem, colo, and cloud (with multiple different cloud vendors), and they still chose AWS. At this point, the choice of cloud is nearly religious (in my observation).
Jumping away from the article slightly, where would you go if you needed a ton of vanilla disk space, the ability to do compute near to that disk, and for it to be accessible with a reasonable pipe to the public internet?
As far as I can tell, you'd almost have to do something with a colo if you didn't want to pay 10x or more for the storage. Are there other options?
You'd colo if you didn't want to use public cloud. The savings are good over time, but the upfront cost can be huge depending upon how much disk space you need. The setup/config can also be rough (many 4U boxes packed with tons of disks each, then CephFS + Rados for S3 api). You also want to have a few configured nodes on cold stand by, and many extra disks. Because of this, I've seen some companies take compute off the cloud but leave bulk data storage on the cloud.
Well, is "a ton" a level that can fit in a dedicated server or many? Just looking at https://serversearcher.com you've got 72TB of storage on 10Gb pipe for $360 or so a month: 128 GB RAM 6c/12t E-2276G 2X 512GB NVME + 4X 18.0TB HDD 300TB / 10Gbps
Just a few PB, and sharding it across servers is fine.
It looks like all the disk-optimized examples on that site (still much more expensive than paying for raw disk, barely 5x cheaper than S3, when a disk-optimized colo solution only has ~3% overhead over the disks themselves) are through some no-name provider "HostKey". I suppose beggars can't be choosers, but in the contexst of storage (where systemic failures should be accounted for in the model) are you aware of more than one provider with reasonably priced storage?
I mean on that site you also have clouvider who are cheaper $423.79/mo for 72TB including bandwidth, compute and two extra boot SSDs. I just searched by a minimum disk size of 12TB.
Colo will be cheaper I'm sure but it's fundamentally a different comparison you have to pay for drive failures, networking, bandwidth, remote hands, network switches and so on and so forth.
Interesting, not sure how I missed that (edit: I missed the 4x on the drive count). Thanks for mentioning it.
And, yes it's a different comparison, but that's why I was asking; I was curious if it was even viable (and initial searches had indicated it wasn't). 4-5x cheaper than S3 as a substrate is potentially workable.
Yeah, I mean thats what like ~$4.9/TB/Month including 4TB+ of internet transfer at a substrate level. So with say 10 servers on 8+2 parity you're looking at ~$6/TB/Month including >5TB of internet. Probably makes sense until you can fill at least one whole rack and buy a 2X100Gb internet connection.
Having my own server in a datacenter would be cool, buy it's hard to imagine having side projects that I'm willingly spending $1400 US/month on AWS for. Right now I'm still using the free tier on most platforms, plus a $14/month database on digital ocean that I'm not even using
There are many scenarios in which cloud providers (especially AWS) make sense.
Ideally, your company has technical experts who can do quite a lot of things non-cloud, so you can make informed decisions about near-term costs, complexity, vendor lock-in, execution speed, etc.
I'm especially a fan of cloud providers for early startups, which tend to be high on velocity, and low on workers. And the free credits programs often solve the early problem of being low on dollars.
But as soon as you start dealing with data sovereignty for your SaaS app, and need co-lo's in multiple geographies, each with their own contract/billing cycles, HA and backup, the global reach and consistency of the cloud providers starts seeming quite compelling again.
Great write-up :) I find hosting VPS on DigitialOcean/Hetzner to be the best middle-ground: you save a lot, but it still convenient; you just SSH into the thing, no need to think about physical cables, cooling, electricity and all the being physical baggage
This exactly what was happening with me and I recently started a project called Postbase to setup my own cloud infrastructure. There is so much free OSS software out there which is 100x better than what Google, Amazon and other offer. github.com/umrashrf/postbase (this is in alpha and work in progress).
As with everything, the real answer as to whether or not the cloud is cheaper is "it depends". These simplistic one-sided tirades (intentionally?) leave out a lot of detail, use simple rage-bait titles, and are generally not very thoughtful or coherent.
The article featured The Server Store. It’s a north Texas company local to where I live. I have purchased all my personal computers from there for about the past decade.
1. "It’s difficult to say these things without coming across as arrogant, but I’ve been managing servers since 2006."
You have experience manager servers, so for you it's not a big deal. But maybe software developers who aren't devops don't want to worry about it and just want to write code and ship a project?
2. If you're launching a new product as a solo developer, isn't it faster to just build an MVP on the cloud, validate the idea, get some traction, and eventually migrate to cheaper hosting if you do become the x% that gets product/market fit and do need to scale?
3. "Look, first of all, you’re as unique as the other 1000 peanut gallery enjoyers that have made the same astute observation before you. Congratulations. But you’re absolutely missing the point."
What's with the tone? Is it supposed to be funny? Does it help getting the point across?
Not exactly like for like, but if you checkout comparison sites like https://serversearcher.com there are options that come close, like $6 for 4GB/2cores in many US locations.
I’m fundamentally confused. A server in a datacenter is still the cloud. Taking something off the cloud means you have hardware somewhere that you physically manage and pay the power bill.
This is simply collapsing the built for scale tools into a cacophony of open source tools.
You didn’t need it to begin with, you just didn’t know any better.
Sorry but my $3 AWS instance is still cheaper than all of those options.
If you need a lot of, well, anything, be it compute, memory, storage, bandwidth etc., of course cloud stuff is going to be more expensive... but if you don't need that, then IMO $3/mo on-demand pricing really can't be beat when I don't have to maintain any equipment myself. Oracle also offers perpetually free VM instances if you don't mind the glow.
With a quick LLM-assisted search, looks like the cheapest EC2 instance is t4g.micro, which comes in at $2.04/mo. It has 2 vCPUs and and only 512MiB of RAM. (I assume that doesn't include disk; EBS will be extra.)
I can certainly see a use for that small amount of compute & RAM, but it's not clear that your level of needs is common. I've been paying for a $16/mo VPS (not on AWS) for about 15 years. It started out at $9/mo, but I've upgraded it since then as my needs have grown. It's not super beefy with 2 vCPUs, 5GiB of RAM, and 60GiB of disk space (with free data ingress/egress), but it does the job, even if I could probably find it cheaper elsewhere.
But not at Amazon. Closest match is probably a t3.medium, with 2 vCPUs and 4GiB RAM. Add a 60GiB gp2 EBS volume, and it costs around $35/mo, and that's not including data transfer.
The point that you're missing is we're not looking for the cheapest thing ever, we're looking for the cheapest thing that meets requirements. For many (most?) applications, you're going to overpay (sometimes by orders of magnitude) for AWS.
You say "if you need a lot", but "lot" is doing a bit of work there. My needs are super modest, certainly not "a lot", and AWS is by far not the cheapest option.
I run heaps of services on AWS and my bill is ~$2-3 - I'm not running any EC2 instances at all. Some of the offerings these cloud providers offer are extremely affordable if you know how to play your cards right and use the right services.
Just get a raspberry pi and run it from your own home internet. You should already be paying for a VPN service and your regular internet service, so you should be able to trivially work out a self-hosted solution. You'll recover your costs inside of two years and come out the other end better off for it.
Don't give the big cloud companies an inch if you don't absolutely have to. The internet needs and deserves the participation of independent people putting up their own services and systems.
Amazon really doesn't care if your $10,000 bed folds up on you like a sandwich and cooks you when AWS us-east-1 goes down, or stops your smart toilet from flushing, or sets bucket defaults that allow trivial public access to information you assume to be secure, because nobody in their right mind would just leave things wide open.
Each and every instance of someone doing something independently takes money and control away from big corporations that don't deserve it, and it makes your life better. You could run pihole and a slew of other useful utilities on your self-hosted server that benefit anyone connected to your network.
AI can trivially walk you through building your own self-hosted setups (or even set things up for you if you entrust it with an automation MCP.)
Oracle and AWS and Alphabet and the rest shouldn't profit from eating the internet - the whole world becomes a better place every time you deny them your participation in the endless enshittification of everything.
If you’re going to write a post about why self-hosting is better than cloud*, then it’s probably a good idea to make sure your site loads in under a minute.
* at least I assume what this post is; I’m still waiting for it to load.
It's an informative post but I really dislike the language and style that are becoming common in this kind of posts, e.g.:
> Look, first of all, you’re as unique as the other 1000 peanut gallery enjoyers that have made the same astute observation before you. Congratulations. But you’re absolutely missing the point.
Local first! Someone else said, "You don't actually need any of those things until you no longer have a "project", but a business which will allow you to pay for the things you require." -- Similar response I have for clients and engineers when they waste time/energy/money on "scale" when we are building a prototype... "scale is a luxury that comes with revenue."
Im sure some of you may quip back, "it takes money to make money". No shit. Just dont get high on your own supply... Be smart, be scrappy, slow down.
Up to a point, cloud can actually be very cost effective. If you are a small company, you don't actually need a lot. I ran our current service for almost one and a half year on Google Cloud Run for free. It stayed below the freemium threshold. Eventually I transitioned it over to two simple VMs. Main reason for this was that I abused these servers to also do background processing and Cloud Run is not very suitable for that as it can shut down servers in between requests and/or starve them of resources. But a small vm only costs something like 20$/month. Add another 25 or so for the load balancer and a Redis, and a few other bits and we're looking at ~100$ per month. That's very reasonable. That's not a cost worth optimizing without a very good reason. A good devops freelance person asks more per hour than that.
Where a lot of companies go wrong is premature scaling and overengineering. As you might guess from the above, I don't run kubernetes. Reason, the smallest cluster I can manage would probably be more expensive than the entirety of my setup. Before I even deploy anything. And it might not be big enough even.
I also don't run a managed postgresql. I'd like to. But I can't justify dropping north of 300$/month on that currently. A single node vm with backups is good enough. For now. I might go active master - passive master. Do I need a 3 or more node cluster? Not right now. I might one day. I always have the option to upgrade. But until then my DB is going to not break the bank. So, I run with a more modest/risky setup. It's a managed risk. I can buy it off when that becomes relevant.
The other day, our CEO vibe coded a cost calculator thingy. He did a nice job. It had to be deployed somewhere. So, I used Cloud Run. Running a small thing with essentially no traffic is as close to free as you can get. I slapped some basic authentication on it and used a git hub action that we run from the freemium allocated build minutes each month. I reviewed our Github bills a few days ago over the years: The most we payed was a few months where we were running out of build minutes in 2024. I payed 55cents for one of those bills. Our setup is cloud based but frugal. I could probably lower our bills by double digit percentages by moving everything into Hetzner. But it's literally not worth my time.
Cloud on the cheap is just really good value. You can host low traffic static websites on Cloudflare. If you have a load balancer in Google cloud (25$/month or so) you can use buckets and their CDN for next to nothing as an alternative. A small Redis costs next to nothing. We run a slightly expensive vm for batch jobs for a maybe ~2 hours per month in total. Each job only takes a few minutes. The machine is suspended in between jobs and we only pay for the hours it runs. Next to nothing.
So, yes, you can get some bare metal in Hetzner or wherever for next to nothing. But then you need to manage that setup and you need to put the time in to do that. Your time even if you don't value it personally is just about the most scarce and expensive resource a startup has if you are the CTO. The difference between you doing valuable work that moves the company forward or you procrastinating and penny pinching over stuff that does not matter can literally make or break the company.
If you spend 5K on infrastructure per month, that's a the same as a very low salary for a full time person. If you are spending that much, definitely do something because it's an obscene amount. Especially if you are early stage and have no traffic. But if you need to put somebody on that that costs closer to 10K/month for a few weeks or months, that's a project that has a pretty high price tag as well. So cutting 5K down to 1K is worth doing. But spending 10K to cut 1K to 500/month is probably going to take you years to earn back. And you lose time doing it.
Our startup is bootstrapped. We run on a bit of revenue now and our savings for most of the past five years. Anything I spend on stuff that isn't needed literally is money that otherwise would go to my bank account (via a salary raise or bonus). I understand frugality. A lot of this stuff is people getting spoiled with investor cash and spending like there is no tomorrow. If you have no accountability you optimize the infrastructure for how good it makes your resume look, not for cost.
> Most people complaining about what I did happen to have “devops”, “cloud engineer”, “serverless guy”, “AWS certified”, or something similar in their bio.
“It is difficult to get a man to understand something, when his salary depends on his not understanding it.”
> You’ve been scammed, likely not out of your own money, but your employer’s money – and that’s why you don’t give a crap.
No, I've ran the numbers, and in doing so, realized that EC2 is the wrong solution for most problems. If you use the correct solution the price is highly competitive with your servers. One of the reasons being, I do not need to maintain servers.. I did that for 25 years and I absolutely hate it. The amount of time I save, which my company pays for, is ridiculous. I get a powerful security model, a unified API for managing everything, and expertly managed runtimes.
You're acting as if people only making $4k/mo in salary are making these decisions. You've got startup brain. Come back down to earth and reassess your position.
> you’re looking to save for the long run. Well, turns out buying your own servers is even cheaper in a long enough time frame
Yet another process I don't have to waste time on. "Capital purchasing" is extremely annoying (by design) in every single organization I've ever worked for or consulted with.
Like most "cloud cost analysis" this is extremely shallow and is apparently based upon the supposed "virality" of a single tweet.
yet another obsessive take on "cloud is bad and expensive" eh? I think they vastly forget the value of some SaaS offerings in terms of time saving for small companies. running and managing numerous DBs, k8s clusters, ci/cd pipelines and stateless container systems is simply impossible with a team of 1-2 people. sure if the setup is simple and only requires a few classic components, this is way cheaper and for a 99.9% SLA will work fine. otherwise it only makes sense if you had very large cloud bills and can dedicate multiple engineers to the newly created tasks.
> running and managing numerous DBs, k8s clusters, ci/cd pipelines and stateless container systems is simply impossible with a team of 1-2 people
Then don't. If your team and budget are small enough not to hire a sysadmin, then your workload is (almost certainly) small enough to fit on one server, one Postgres database, Jenkins or a bash script, and certainly no k8s.
I think we've gone a little nuts defining "production system" these days. I've worked for companies with zero-downtime deployments and quite a lot of redundancy for high availability, and for some applications it's definitely worthwhile.
But I think for many (most?) businesses, one nine is just fine. That's perfectly doable by one person, even if you want, say, >=96% uptime, which allows for 350 hours of downtime per year. Even two nines allows for ~88 hours of downtime per year, and one person could manage that without much trouble.
Most businesses aren't global. Downtime outside regular business hours for your timezone (and perhaps one or two zones to the west and east of you) is usually not much of a problem, especially if you're running a small B2B service.
For a small business that runs on 1-3 servers (probably very common!), keeping a hot spare for each server (or perhaps a single server that runs all services in a lower-supported-traffic mode) can be a simple way to keep your uptime high without having to spend too much time or money. And people don't have to completely opt out of the cloud; there are affordable options for e.g. managed RDBMS hosting that can make maintenance and incident response significantly easier and might be a good choice, depending on your needs.
(Source: I'm building a small one-person business that is going to work this way, and I've been doing my research and gaming it out.)
One thing that AWS, Google and Azure do that your own systems don't is release their updates whenever it suits them, often taking down your business down in the middle of the day with their own problems. You can't fix it, you can't rollback what you just did and get back up and running you just have to sit and wait.
That is quite different to a business that turns off its boxes for an hour at 0100 Sunday morning to do updates and release new software. The downtime isn't equivalent because it really matters when it is and if that hurts your use case or not. Your own system might be down for more hours a year than AWS, but its not down Monday to Friday on an evening when you do most your sales because you refuse to touch anything during that period and do all the work outside that and schedule your updates.
It also feels like AWS (or Azure) isn't really that much more reliable than your own thing. But half the internet is down at the same time so you don't get blamed as much.
Its the "No one gets blamed for going IBM" thing in the modern era. They are making it someone elses fault and absolves the blame. The problem is if your competitor is still up you could be loosing customers on average mid day outage, even if they are down for 3x as long its not when it matters.
I agree. "production system" isn't a useful term without context.
Either it's production system in the context of "it's a business serving customers", in which case, there are many homelabs out there which have received paying traffic, or it's a production system in terms of functionality, downtime, technical features etc. Again, there are many homelabs out there that can tick all the same technical and performance boxes as "production system".
In the context of the original comment I was responding to ("...running and managing numerous DBs, k8s clusters, ci/cd pipelines and stateless container systems is simply impossible with a team of 1-2 people.") you're more evidence for my comment that it's perfectly possible for 1 - 2 people to run a small production system, just as it's possible for 1 nerd to run a medium - large homelab with the same technical features.
Your production is not my production. 5 9's of businesses don't need 5 9's, and in my experience, people who talk like they do are talking from a position of managing 1 or 2 production systems in a single domain.
My experience is that the largest regulated production systems I've run have clearly been larger and more complex than my homelab, but my homelab has been significantly more resilient, featureful and robust than many of the smaller production systems I've been responsible for outside of regulated domains.
I'm sorry but whereever I've seen aws at work there was a sprawling terraform codebase to manage it.
This no different than puppet or ansible on bare metal complexoty-wise, you just pat extra for the shiny tools
The post is about that 99% of companies that will never go large scale. Its point is that they don't need cloud, buying a server or two is all they need.
I once worked at a bookmaker with millions of customers where they had extreme reliability demands during very popular events, since any bets that the punters couldn't place during live betting was lost money. They had one massive server with a big pipe and an identical hot spare in another location. It worked very well for them. That was a decade ago, you can get even bigger servers these days, there's a lot of room for scaling without needing to go beyond one machine.
I dislike those black and white takes a lot. It's absolutely true that most startups that just run an EC2 instance will save a lot of cash going to Hetzner, Linode, Digital Ocean or whatever. I do host at Hetzner myself and so do a lot of my clients.
That being said, the cloud does have a lot of advantages:
- You're getting a lot of services readily available. Need offsite backups? A few clicks. Managed database? A few clicks. Multiple AZs? Available in seconds.
- You're not paying up-front costs (vs. investing hundreds of dollars for buying server hardware) and everything is available right now [0]
- Peak-heavy loads can be a lot cheaper. Mostly irrelevant for you average compute load, but things are quite different if you need to train an LLM
- Many services are already certified according to all kinds of standards, which can be very useful depending on your customers
Also, engineering time and time in general can be expensive. If you are a solo entrepreneur or a slow growth company, you have a lot of engineering time for basically free. But in a quick growth or prototyping phase, not to speak of venture funding, things can be quite different. Buying engineering time for >150€/hour can quickly offset a lot of saving [1].
Does this apply to most companies? No. Obviously not. But the cloud is not too expensive - you're paying for stuff you don't need. That's an entirely different kind of error.
[0] Compared to the rack hosting setup described in the post. Hetzner, Linode, etc. do provide multiple AZs with dedicated servers.
[1] Just to be fair, debugging cloud errors can be time consuming, too, and experienced AWS engineers will not be cheaper. But an RDS instance with solid backups-equivalent will usually not amortize quickly, if you need to pay someone to set it up.
You don't actually need any of those things until you no longer have a "project", but a business which will allow you to pay for the things you require.
You'd be amazed by how far you can get with a home linux box and cloudflare tunnels.
On this site, I've seen these kind of takes repeatedly over the past years, so I went ahead and built a little forum that consists of a single Rust binary and SQLite. The binary runs on a Mac Mini in my bedroom with Cloudflare tunnels. I get continuous backups with Litestream, and testing backups is as trivial as running `litestream restore` on my development machine and then running the binary.
Despite some pages issuing up to 8 database queries, I haven't seen responses take more than about 4 - 5 ms to generate. Since I have 16 GB of RAM to spare, I just let SQLite mmap the whole the database and store temp tables in RAM. I can further optimize the backend by e.g. replacing Tera with Askama and optimizing the SQL queries, but the easiest win for latency is to just run the binary in a VPS close to my users. However, the current setup works so well that I just see no point to changing what little "infrastructure" I've built. The other cool thing is the fact that the backend + litestream uses at most ~64 MB of RAM. Plenty of compute and RAM to spare.
It's also neat being able to allocate a few cores on the same machine to run self-hosted GitHub actions, so you can have the same machine doing CI checks, rebuilding the binary, and restarting the service. Turns out the base model M4 is really fast at compiling code compared to just about every single cloud computer I've ever used at previous jobs.
Just one of the couple dozen databases we run for our product in the dev environment alone is over 12 TB.
How could I not use the cloud?
12TB is $960/month in gp3 storage alone. You can buy 12TB of NVMe storage for less than $960, and it will be orders of magnitude faster than AWS.
Your use case is the _worst_ use case for the cloud.
The most consistent misunderstanding I see about the cloud, is disk I/O. Nobody understands how slow your standard cloud disk is under load. They see good performance and assume that will always be the case. They don't realize that most cloud disks use a form of token tracking where they build up I/O over time and if you have bursts or sustained high I/O load you will very quickly notice that your disk speeds are garbage.
For some reason people more easily understand the limits of CPU and memory, but overlook disk constantly.
At one time I had a project to run a cryptocurrency node for BSC (this is basically a fork of Ethereum with all the performance settings cranked up to 11, and blocks centrally issued instead of being mined). It's very sensitive to random access disk throughput and latency. At the time I had a few tiny VPS on AWS and a spinning drive at home, so I evaluated running it there. Even besides the price, you simply cannot run it on AWS EBS because the disk is just too slow to validate each block before the next one arrives. I spent a few hundred dollars and bought an NVMe SSD for my home computer instead.
Even without that, you are still at the heart of it accessing over a SAN like interface with some sort of local cache. Getting an actual local drive on AWS the performance is night and day
Sure, you can work around it; but it blows up the savings alot of people expect when they don't include this in their math.
Also, SAN is often faster then local disk if you have a local SAN.
How is a SAN faster than a local disk? Any references / recommendations?
I would expect by the magic of parallelism?
Probably comparing a HDD SAN (with data spread across many drives) to a single local HDD.
What could I read to inform myself better on this topic? It is true I had not seen this angle before
This looks pretty informative. The terminology can be hard to follow. https://medium.com/@bounouh.fedi/understanding-iops-in-aws-w...
I/O is hard to benchmark so it's often ignored since you can just scale up your disks. It's a common gotcha in the cloud. It's not a show stopper, but it blows up the savings you might be expecting.
First of all, if you have a dev DB that’s 12 TB, I can practically guarantee that it is tremendously unoptimized.
But also, that’s extremely easily handled with physical servers - there are NVMe drives that are 10x as large.
that's what I always brag to my devs, why is our DB 1TB, and only 20 users are working in our app. They are collecting all garbage and saving it to DB. Poor development skills I would say. Our old app did the same thing, and after 15 years it was barely 100GB with tens of users. devs today are SELECT *. If it does not work, they say we need more resources. Thats why I hate cloud.
Nothing like piling transactions, analytics, and logs to the same database. /s
Eh, please find me a 120 TB NVMe.
https://www.solidigm.com/products/data-center/d5/p5336.html
> Just one of the couple dozen databases we run for our product in the dev environment alone is over 12 TB.
> How could I not use the cloud?
Funnily enough, one of my side projects has its (processed) primary source of truth at that exact size. Updates itself automatically every night adding a further ~18-25 million rows. Big but not _big_ data, right?
Anyway, that's sitting running happily with instant access times (yay solid DB background) on a dedicated OVH server that's somewhere around £600/mo (+VAT) and shared with a few other projects. OVH's virtual rack tech is pretty amazing too, replicating that kind of size on the internal network is trivial too.
https://www.seagate.com/products/enterprise-drives/exos/exos...
> one of the couple dozen databases
I guess this is one of those use cases that justify the cloud. It's hard to host that reliably at home.
Not too push the point too hard, but a "dev environment" for a product is for a business (not an individual consumer). Having a server (rack) in an office is not that hard, but alas the cloud might be better here for ease of administration.
My understanding is that aws exists because we can't get any purchase approved in under three months.
I don't think so. An organization so big and bureaucratic that needs 3 months to authorize a server purchase will for sure need a few weeks of paperwork to authorize a new AWS account creation, and will track the spending for OU and will cut budget and usage if they think you deserve it.
And plenty of datacenters will be happy to give you some space in one of their racks.
Not wanting to deal with backups or HA are decent reasons to put a database in the cloud (as long as you are aware how much you are overpaying). Not having a good place to put the server is not a good reason
If anyone's curious about the ballpark cost, a carrier-owned (?) DC near me that publishes prices (most don't) advertises a full rack for 650€ per month, including internet @ 20TB/month @ 1 Gbps, and 1kW power.
Though both of which are probably less than you'd need if you needed a full of rack of space, which I assume is part of the reason that pricing is almost always "contact us". I did not bother getting a quote just for the purpose of this comment. But another thing that people need to be less afraid of, when they're looking to actually spend a few digits of money and not just comment about it, is asking for quotes.
12 TB fits entirely into the RAM of a 2U server (cf. Dell PowerEdge R840).
However, I think there's an implicit point in TFA; namely, that your personal and side projects are not scaling to a 12 TB database.
With that said, I do manage approximately 14 TB of storage in a RAIDZ2 at my home, for "Linux ISOs". The I/O performance is "good enough" for streaming video and BitTorrent seeding.
However, I am not sure what your latency requirements and access patterns are. If you are mostly reading from the 12 TB database and don't have specific latency requirements on writes, then I don't see why the cloud is a hard requirement? To the contrary, most cloud providers provide remarkably low IOPS in their block storage offerings. Here is an example of Oracle Cloud's block storage for 12 TB:
https://docs.oracle.com/en-us/iaas/Content/Block/Concepts/bl...Those are the kind of numbers I would expect of a budget SATA SSD, not "NVMe-based storage infrastructure". Additionally, the cost for 12 TB in this storage class is ~$500/mo. That's roughly the cost of two 14 TB hard drives in a mirror vdev on ZFS (not that this is a good idea btw).
This leads me to guess most people will prefer a managed database offering rather than deploying their own database on top of a cloud provider's block storage. But 12 TB of data in the gp3 storage class of RDS costs about $1,400/mo. That is already triple the cost of the NAS in my bedroom.
Lastly, backing up 12 TB to Backblaze B2 is about $180/mo. Given that this database is for your dev environment, I am assuming that backup requirements are simple (i.e. 1 off-site backup).
The key point, however, is that most people's side projects are unlikely to scale to a 12 TB dev environment database.
Once you're at that scale, sure, consider the cloud. But even at the largest company I worked at, a 14 TB hard drive was enough storage (and IOPS) for on-prem installs of the product. The product was an NLP-based application that automated due diligence for M&As. The storage costs were mostly full-text search indices on collections of tens of thousands of legal documents, each document could span hundreds to thousands of pages. The backups were as simple as having a second 14 TB hard drive around and periodically checking the data isn't corrupt.
Still missing the point. This is just one server and just in the dev enviornment?
How many pets do you want to be tending to? I have 10^5 servers I'm responsible for...
The quantity and methods the cloud affords me allow me to operate the same infrastructure with 1/10th as much labor.
At the extreme ends of scale this isn't a benefit, but for large companies in the middle this is the only move that makes any sense.
99% of posts I read talking about how easy and cheap it is to be in the datacenter all have a single digit number of racks worth of stuff. Often far less.
We operate physical datacenters as well. We spend multiple millions in the cloud per month. We just moved another full datacenter into the cloud and the difference in cost between the two is less than $50k/year. Running in physical DCs is really inefficient for us for a long of annoying and insurmountable reasons. And we no longer have to deal with procurement and vendor management. My engineers can focus their energy on more valuable things.
What is this ridiculous bait and switch. First you talk about a 12 TB dev databases and "How could I not use the cloud?". And you rightfully get challenged on that and then suddenly it's about the number of servers you have to manage and you don't have the energy to do that with your team. Those two have nothing to do with each other.
Why do people think it takes "labor" to have a server up and running?
Multiple millions in the cloud per month?
You could build a room full of giant servers and pay multiple people for a year just on your monthly server bill.
Found the "AWS certified cloud engineer".
12 TB is easy. https://yourdatafitsinram.net/
Buy a pretty basic HDD? These days 12 TB isn’t all that much?
My friends run hundreds of TB:s serviced onto the Internet for hobby and pleasure reasons.
It's not all HA, NVMe, web scale stuff, but it's not like a few hundred TB:s is a huge undertaking even for individual nerds with a bit of money to spend or connections at corporations that monotonically decommission hardware and is happy to not have to spend resources getting rid of it.
This summer I bought a used server for 200 euros from an acquaintance, I plan on shoving 140 TB in it and expect some of my future databases to exceed 10 TB in size.
What's your cloud bill?
A high end laptop now can come with double that amount of storage.
Sounds more like your use case is like the 1~2% of the cases a simple server and sqlite is maybe not the correct answer.
what are you doing that you have 12TB in dev??? my startup isn't even using a TB in production and we hands multiple millions of dollars in transactions every month.
Exactly! I've been self hosting for about two years now, on a NAS with Cloudflare in front of it. I need the NAS anyway, and Cloudflare is free, so the marginal cost is zero. (And even if the CDN weren't free it probably wouldn't cost much.)
I had two projects reach the front page of HN last year, everything worked like a charm.
It's unlikely I'll ever go back to professional hosting, "cloud" or not.
If you have explosive growth, sure cloud.
The vast majority of us that are actually technically capable are better served self hosting.
Especially with tools like cloudflare tunnels and Tailscale.
You can get quite far without that box, even, and just use Cloudflare R2 as free static hosting.
CloudFlare Pages is even easier for static hosting with automatic GitHub pulls.
Happy Netlify customer here, same deal. $0.
(LOL 'customer'. But the point is, when the day comes, I'll be happy to give them money.)
Careful what you wish for. Netlify sent a guy a $104k bill from the free plan. Thankfully social media outage saved the guy.
https://news.ycombinator.com/item?id=39520776
Netlify changed their pricing after that so that free accounts are always free.
Could you give a reference please? I was literally going to recommend Netlify at work, but didn't after I saw that story.
Is $0.55/GB not enough reason to avoid them? I guess not if your business is making more than that - bandwidth expense for a shopping site shouldn't be a problem when the customers are spending $100 for every 0.1GB - but that price should realistically be closer to $0.01/GB or even $0.002/GB. Sounds like they're forwarding you AWS's extremely excessive bandwidth pricing.
> Is $0.55/GB not enough reason to avoid them?
Where did you read that? The pricing page says 10 credits per GB, and extra credits can be purchased at $10 per 1500 credit. So it's more like $0.067/GB.
https://www.netlify.com/pricing/#faq
> The free plan is always free, with hard monthly limits that cannot be exceeded or incur any costs.
FYI I just migrated from Netlify to Cloudflare pages and Cloudflare is massively faster across all metrics.
My pet peeves are:
1. For small stuff, AWS et al aren't that much more expensive than Hetzner, mostly in the same ballpark, maybe 2x in my experience.
2. What's easy to underestimate for _developers_ is that your self hosted setup is most likely harder to get third party support for. If you run software on AWS, you can hire someone familiar with AWS and as long as you're not doing anything too weird, they'll figure it out and modify it in no time.
I absolutely prefer self hosting on root servers, it has always been my go to approach for my own companies, big and small stuff. But for people that can't or don't want to mess with their infrastructure themselves, I do recommend the cloud route even with all the current anti hype.
> 2. What's easy to underestimate for _developers_ is that your self hosted setup is most likely harder to get third party support for. If you run software on AWS, you can hire someone familiar with AWS and as long as you're not doing anything too weird, they'll figure it out and modify it in no time.
If you're at an early/smaller stage you're not doing anything too fancy either way. Even self hosted, it will probably be easy enough to understand that you're just deploying a rails instance for example.
It only becomes trickier if you're handling a ton of traffic or apply a ton of optimizations and end up already in a state where a team of sysadmin should be needed while you're doing it alone and ad-hoc. IMHO the important part would be to properly realize when things will get complicated and move on to a proper org or stack before you're stuck.
You'd think that, but from what I've seen, some people come up with pretty nasty self hosting setups. All the way from "just manually set it all up via SSH last year" to Kubernetes. Of course, people can and also definitely do create a mess on AWS. It's just that I've seen that _far_ less.
One way of solving for this is to just use K3s or even just plain docker. It is then just kuberneters/containers and you can hire alot of people who understand that.
Absolutely recommend k3s. Start with a single node and keep on scaling as customer base increases.
> mostly in the same ballpark, maybe 2x in my experience.
2x is the same ballpark???
>most startups that just run an EC2 instance will save a lot of cash going to Hetzner, Linode, Digital Ocean or whatever. I do host at Hetzner myself and so do a lot of my clients. That being said, the cloud does have a lot of advantages:
When did Linode and DO got dropped and not being part of the cloud ?
What used to separate VPS and Cloud was resources at per second billing. Which DO and Linode along with a lot of 2nd tier hosting also offer. They are part of cloud.
Scaling used to be an issue, because buying and installing your hardware or sending it to DC to be installed and ready takes too much time. Dedicated Servers solution weren't big enough at the time. And the highest Core count at the time was 8 core Xeon CPU in 2010. Today we have EPYC Zen 6c at 256 Core and likely double the IPC. Scaling issues that requires a Rack of server can now be done with 1 single server and fit everything inside RAM.
Managed database? PlanetScale or Neon.
A lot of issues for medium to large size project that "Cloud" managed to solve are no longer an issue in 2025. Unless you are top 5-10% of project that requires these sort of flexibilities.
For a lot of people (not me), if it's not from AWS, Azure, GCP or Oracle then it's not cloud, it's just a sparkling hosting provider.
I had someone on this site arguing that Cloudflare isn't a cloud provider...
> But the cloud is not too expensive - you're paying for stuff you don't need. That's an entirely different kind of error.
Agreed. These sort of takedowns usually point to a gap in the author's experience. Which is totally fine! Missing knowledge is an opportunity. But it's not a good look when the opportunity is used for ragebait, hustlr.
> A few clicks.
Getting through AWS documentation can be fairly time consuming.
And making sure you're not making a security configuration mistake that will accidentally leak private data to the open internet because of a detail of AWS you were unaware of.
Figuring out how to do db backups _can_ also be fairly time consuming.
There's a question of whether you want to spend time learning AWS or spend time learning your DB's hand-rolled backup options (on top of the question of whether learning AWS's thing even absolves you of understanding your DB's internals anyways!)
I do think there's value in "just" doing a thing instead of relying on the wrapper. Whether that's easier or not is super context and experience dependent, though.
> Figuring out how to do db backups _can_ also be fairly time consuming.
apt install automysqlbackup autopostgresqlbackup
Though if you have proper filesystem snapshots then they should always see your database as consistent, right? So you can even skip database tools and just learn to make and download snapshots.
nah filesystem snapshots may not lead to consistent DB backups. DB backup software usually use a plugin to tell the DB to coalesce data before taking a snapshot.
Databases have to be ready for power loss, don't they? They might not be happy about it, but if that corrupts anything then the design has failed.
And again I'll emphasize proper snapshot, cutting off writes at an exact point in time. A normal file copy cannot safely back up an active database.
> filesystem snapshots may not lead to consistent DB backups
Only if your database files are split across multiple file systems, which is atypical.
At least one OS you’ve heard of can quiesce the file system to allow taking a consistent snapshot; I’d be surprised if this wasn’t widely available everywhere.
Hmmm, I think you have to figure out how to do your database backups anyway as trying to get a restorable backup out of RDS to use on another provider seems to be a difficult task.
Backups that are stored with the same provider are good, providing the provider is reliable as a whole.
(Currently going through the disaster recovery exercise of, "What if AWS decided they didn't like us and nuked our account from orbit.")
aws would never do that :) plus you can also do it in aws with like 75 clicks around UI which makes no sense even when you are tripping on acid
> 75 clicks
Well 2 commands...
Then copy it down The biggest effort would be then running the Apache Parquet to CSV tool on it.Those buckets and IAM policies and roles also have to be managed.
There are also turnkeys solutions that allow one to spin up a DB, setup replication and backups inside or outside of big cñoud vendors. That is the point of db kubernetes operators for instance.
Plus the s3 bucket creation and definition commands, and the IAM role and attached policy commands. If you do all in the webUI it's not going to be 75 clicks either but 30 for sure.
It could easily be 30 clicks.
But creating an S3 bucket, an IAM role and attaching policies isn't 30 commands.
most definitely do not want to spend time learning aws… would rather learn about typewriter maintenance
And learning TypeScript and CDK, if we're comparing scripted, repeatable setups which you should be doing from the start.
> repeatable setups which you should be doing from the start
Yes, but not with
> TypeScript and CDK
Unless your business includes managing infrastructure with your product, for whatever reason (like you provision EC2 instances for your customers and that's all you do), there is no reason to shoot yourself in the foot with a fully fledged programming language for something that needs to be as stable as infrastructure. The saying is Infrastructure as Code, not with code. Even assuming you need to learn Terraform from scratch but already know Typescript, you would still save you time compared to learning CDK, figuring out what is possisble with it, and debugging issues down the line.
I think declarative is nicer too, but choosing a non mainstream tech here takes self-confidence in the matter that inexperienced AWSers are unlikely to have.
And learning something arguably better, like Cloudformation / Terraform / SST, is still a hurdle.
gotta say, Amazon Q can do the details for you in many cases.
> You're getting a lot of services readily available. Need offsite backups? A few clicks
I think it is a lot safer for backups to be with an entirely different provider. It protects you in case of account compromise, account closure, disputes.
If using cloud and you want to be safe, you should be multi-cloud. People have been saved from disaster by multi-cloud setups.
> You're not paying up-front costs (vs. investing hundreds of dollars for buying server hardware)
Not true for VPSes or rented dedicated servers either.
> Peak-heavy loads can be a lot cheaper.
they have to be very spiky indeed though. LLMs might fit but a lot of compute heavy spiky loads do not. I saved a client money on video transcoding that only happened once per upload and only over a month or two an year by renting a dedi all ear round rather than using the AWS transcoding service.
> Compared to the rack hosting setup described in the post. Hetzner, Linode, etc. do provide multiple AZs with dedicated servers.
You have to do work to ensure things run across multiple availability zones (and preferably regions) anyway.
> But an RDS instance with solid backups-equivalent will usually not amortize quickly, if you need to pay someone to set it up.
You have more forced upgrades.
An unmanaged database will only need a lot of work if operating at large scale. If you are then its probably well worth employing a DBA anyway as an AWS or similar managed DB is not going to do all the optimising and tuning a DBA will do.
any serious business will(might?) have hundreds of Tbs of data. I store that in our DC and with a 2nd DC backup for about 1/10 the price of what it would cost in S3.
When does the cloud start making sense ?
In my case we have a B2B SaaS where access patterns are occasional, revenue per customer is high, general server load is low. Cloud bills just don’t spike much. Labor is 100x the cost of our servers so saving a piddly amount of money on server costs while taking on even just a fraction of one technical employee’s worth of labor costs makes no sense.
I think compliance is one of the key advantages of cloud. When you go through SOC2 or ISO27001, you can just tick off entire categories of questions by saying 'we host on AWS/GCP/Azure'.
It's really shitty that we all need to pay this tax, but I've been just asked about whether our company has armed guards and redundant HVAC systems in our DC, and I wouldn't know how to do that apart from saying that 'our cloud provider has all of those'.
In my experience you still have to provide an awful lot of "evidence". I guess the advsntage of AWS/GCP/Cloud is that they are so ubiquitous you could literally ask an LLM to generate fake evidence to speed up the process.
> That being said, the cloud does have a lot of advantages:
Another advantage is that if you aim to provide a global service consumed throughout the world then cloud providers allow you to deploy your services in a multitude of locations in separate continents. This alone greatly improves performance. And you can do that with a couple of clicks.
linode was better and had cheaper pricing before being bought by akamai
I don’t feel like anything really changed? Fairly certain the prices haven’t changed. It’s honestly been pleasantly stable. I figured I’d have to move after a few months, but we’re a few years into the acquisition and everything still works.
I concur with every word.
No longer getting DDOSed multiple years in a row on Christmas Eve is worth whatever premium Akamai wants to charge over old Linode.
Akamai has some really good infrastructure, and an extremely competent global cdn and interconnects. I was skeptical when linode was acquired, but I value their top-tier peering and decent DDoS mitigation which is rolled into the cost.
Whoa, an acquisition made things worse for everyone but the people who cashed out? Crazy, who could have seen that coming
Guess you came for the hot take without actually using the service or participating in any intelligent conversation. All the sibling comments observe that nothing you are talking about happened.
Snarky ignorant comments like yours ruin Hacker News and the internet as a whole. Please reconsider your mindset for the good of us all.
To me DO is a cloud. It is pricey (for performance) and convenient. It is possibly a wiser bet than AWS for a startup that wants to spend less developer (read expensive!) time on infra.
I started out with linode, a decade ago.
It became much more expensive than AWS, because it bundled the hard drive space with the RAM. Couldn't scale one without scaling the other. It was ridiculous.
AWS has a bunch of startup credits you can use, if you're smart.
But if you want free hosting, nothing beats just CloudFlare. They are literally free and even let you sign up anonymously with any email. They don't even require a credit card, unlike the other ones. You can use cloudflare workers and have a blazing fast site, web services, and they'll even take care of shooing away bots for you. If you prefer to host something on your own computer, well then use their cache and set up a cloudflare tunnel. I've done this for Telegram bots for example.
Anything else - just use APIs. Need inference? Get a bunch of Google credits, and load your stuff into Vertex or whatever. Want to take payments anonymously from around the world? Deploy a dapp. Pay nothing. Literally nothing!
LEVEL 2:
And if you want to get extra fancy, have people open their browser tabs and run your javascript software in there, earning your cryptocurrency. Now you've got access to tons of people willing to store chunks of files for you, run GPU inference, whatever.
Oh do you want to do distributed inference? Wasmcloud: https://wasmcloud.com/blog/2025-01-15-running-distributed-ml... ... but I'd recommend just paying Google for AI workloads
Want livestreaming that's peer to peer? We've got that too: https://github.com/Qbix/Media/blob/main/web/js/WebRTC.js
PS: For webrtc livestreaming, you can't get around having to pay for TURN servers, though.
LEVEL 3:
Want to have unstoppable decentralized apps that can even run servers? Then use pears (previously dat / hypercore). If you change your mindset, from server-based to peer to peer apps, then you can run hypercore in the browser, and optionally have people download it and run servers.
https://pears.com/news/building-apocalypse-proof-application...
>It became much more expensive than AWS, because it bundled the hard drive space with the RAM. Couldn't scale one without scaling the other. It was ridiculous.
You can easily scale hard drive space independently of RAM by buying block storage separately and then mounting it on your Linode.
I think every VPS provider I have looked at any time recently (and I have been moving things in the last few weeks) offers some option for block storage separate from compute. Most offer an object storage option too.
Not back then
You're literally playing into what the author is criticizing.
I want more examples of people running the admin interface on prem and the user visible parts on the cloud.
I mean there are many places that sell multi AZ, hourly billed VPS/Bare Metal/GPU at a fraction of the cost of AWS.
I would personally have an account at one of those places and back up to there with everything ready to spin up instances and failover if you lose your rack, and use them for any bursty loads.
[dead]
Best USDT Recovery Expert | Recover Lost or Stolen USDT Safely Losing my USDT was one of the most painful moments of my life. I had invested with full trust, believing I was securing my future. Then, without warning, everything disappeared. My wallet showed nothing. My heart dropped. I felt sick. All I could think was, “How do I tell my family? How do I start again?”
I tried reaching out everywhere for help, but no one could do anything. The scammers were gone, and I felt like all hope was gone too.
Then someone recommended SWIFT RECOVERY SERVICES. From the very first moment I reached out to them, something felt different. They responded quickly, they listened to my story, and they treated my situation with seriousness and respect. They assured me that the stolen USDT could be traced and recovered.
They immediately began investigating the wallet addresses and tracking the scammer’s transactions. They updated me step by step. Every message gave me a little more hope than the last. I prayed the process would work.
The day they told me the funds had been successfully recovered, I was in shock. I stared at the message again and again with tears in my eyes. The USDT was finally returned back to me. After all the fear and stress, my money wasn’t gone forever after all.
I am beyond grateful to SWIFT RECOVERY SERVICES. They gave me back more than just my funds. They gave me back my confidence, my joy, and my peace of mind. I will never forget what they did for me.
If you have lost crypto due to fraud or a scam, don’t stay silent, and don’t give up. SWIFT RECOVERY SERVICES is truly reliable and capable. I thank God I found them at the right time. They are the reason I can smile again today.
For contact information, please see below.
Whatsapp: +1 (9 78) 242-3160
Email: Info@swiftrecoveryservices.com
This is all correct. I've been running my own servers for many years now, keeping things simple and saving a lot of money and time (setting anything up in AWS or Azure is horribly complicated and the UIs are terrible).
One thing to remember is that you do need to treat your servers as "cloud servers" in the sense that you should be able to re-generate your entire setup from your configuration at any time, given a bunch of IPs with freshly-installed base OSs. That means ansible or something similar.
If you insist on using cloud (virtual) servers, do yourself a favor and use DigitalOcean, it is simple and direct and will let you keep your sanity. I use DO as a third-tier disaster recovery scenario, with terraform for bringing up the cluster and the same ansible setup for setting everything up.
I am amused by the section about not making friends saying this :-) — most developer communities tend to develop a herd mentality, where something is either all the rage, or is "dead", and people are afraid to use their brains to experiment and make rational decisions.
Me, I'm rather happy that my competitors fight with AWS/Azure access rights management systems, pay a lot of money for hosting and network bandwith, and then waste time on Kubernetes because it's all the rage. I'll stick to my boring JVM-hosted Clojure monolith, deployed via ansible to a cluster of physical servers and live well off the revenue from my business.
I was a guy that built server clusters during the early 00's, for my own and others' web and other projects. When AWS really took off, it was like a spend all your money mania, and devs and companies treated my skills like dirt. I got a job writing facial recognition edge servers, with high performance many claim are impossible numbers (25M face compares per second per core) and my employer found itself a leader in the industry. But customers could not wrap their heads around just a single box capable of our numbers (800M face compares per second, plus ingestion of 32 video streams) and to get sales the company ended up moving everything into AWS because customers did not trust anything else.
Without targeting you directly, you wrote: "800M face compares per second". I creeped out when I read that. What is the real world application for something like that? My guess: Real-time, large scale surveillance.
> to get sales the company ended up moving everything into AWS because customers did not trust anything else
This is a hidden cost of self-hosting for many in b2b. It's not just convincing management, it's convincing your clients.
That's interesting. Except for $GIGANTIC_CO (like, BofA, or the government), i'd expect a SLA that describes service resiliency and not "well, our service will be up because we're on AWS".
Why would you need to disclose your hosting provider? is that really a concern for hosted services (and if it is, why isn't the customer hosting it in their cloud?)
Most customers do not want to host anything if they can prevent it. My employer was selling the servers that host the entire shebang, and most did not want to host them. We'd explain they'd save a lot by hosting and viewing/streaming everything locally, but their IT people were not comfortable, and their execs wanted to see everything on their phones when not at work. We made it all plug and play, and still they wanted to pay 10-20X more for a web service.
Do you have any blog posts or something you could share on how facial recognition works? Specifically what a "face compare" is. That sounds interesting!
https://github.com/bsenftner/UnderstandingFacialRecognition/...
> If you insist on using cloud (virtual) servers, do yourself a favor and use DigitalOcean, it is simple and direct and will let you keep your sanity. I use DO as a third-tier disaster recovery scenario, with terraform for bringing up the cluster and the same ansible setup for setting everything up.
This is the general direction in which society is going. There is no dialogue. You are either with us or against us. Sad.
Pre-cloud pets vs cattle approach. We ran a few pets, but had DC's worth of cattle thanks to PXE, TFPT and Ansible. No Terraform required, as there was no need to control the state of multiples of cloud cruft. Good times. Except when something would pack up in the middle of winter in the wee hours and it was a bollock-cracking motorbike ride to the DC to spit on the offending black box.
Have you written about your app and revenue anywhere we can read about it? Looks interesting.
The only people writing about their revenue are online business influencers.
That seems to be quite true, as people who run a successful and enjoyable business are busy running their successful and enjoyable business :-)
"Building in public" is a reasonable marketing tactic, I wouldn't discount it entirely.
Marketing tactics only work if they reach your customers, and HN readers are generally not my customers.
I was addressing the "people who run a successful and enjoyable business" comment, not you specifically.
How many clients do you have?
This is all true. But... But if you manage your own server, as the author advice, you need to figure out a lot of stuff and remember about a lot of stuff.
Are ulimits set correctly?
Shall I turn on syn cookies or turn them off because of performance?
What are the things I should know but I don't and Chat GPT has not told me them, as this is more than some intro tutorial on how to run VPS on DO, so it was never indexed by Chat GPT and alikes.
Is all of my software on the server up to date? Is there any library I use exploited, zero day attacks are on me too, blocking bots, etc. What if I do some update but it will turn out that my Postgres version is not working correctly anymore? This is all my problem.
What if I need to send emails? These days doing this ourselves is a dark art by itself (IP/domain address warming up, checking if my domain has not ended on some spam list, etc.).
What if I need to follow some regulations, like European Union GDPR compliance? Have I done everything what is needed to store personal data as GDPR requires? Is my DB password stored in a compliant way or I will face a fine up to 10% of my incomes.
This is not black/white situation as the author tries to present and those who use cloud services are not dumbards who are buying some IT version of snake oil.
Isn't this long list applied to cloud hosted VMs as well?
Setting up the email server is the only thing I couldn't do with my own home hosted setup because you're at the mercy of your internet provider to give you the PTR record in their network, and lately many providers outright refuse to do it for "your own and their own safety" reasons. This thing alone could be the difference between deciding to host yourself or use a cloud service.
Just use the Postmark web service API, cheap, reliable, and far more if one really wants to lean into email with their online service. https://postmarkapp.com/email-api
I am aware of the tons of subscription based email services but that is not the point here. What good is self hosting when you still need to rely on some external paid service for a trivial thing like an email server? The costs add up.
Except email is not trivial, it's a time vampire I'm happy to pay $15 a month to not have to deal with.
>What if I need to send emails? These days doing this ourselves is a dark art by itself (IP/domain address warming up, checking if my domain has not ended on some spam list, etc.).
AFAIK, everyone sending automated emails just uses one of the paid services, like sendmail.
>What if I need to follow some regulations, like European Union GDPR compliance? Have I done everything what is needed to store personal data as GDPR requires? Is my DB password stored in a compliant way or I will face a fine up to 10% of my incomes.
What does this have to do with cloud vs non-cloud? You'll need to manage your data correctly either way.
All of this is true both for dedicated servers and cloud-hosted VMs.
This list looks like FUD, to be honest, trying to scare people. Yes, you should be scared of these things, but none of them are magically solved by hosting your stuff in AWS/Azure/Google or any other could provider du jour.
Some of them are solved by using managed-services that abstract away the messy config / security ops stuff.
The blog entry was wordy and repetitive for what it expressed (AI?), but the cloud argument should boil down to a few simple questions:
Add all that together and you get a few customers who should be using the cloud: That's not "all companies" or "no companies" either way, but it is a very large number of companies who are paying cloud premiums without actually needing or benefiting from the cloud value add...Yes, you will need to employ someone with basic system administration competence. That's a given.
Cloud infra is touted as obviating the need to hire system administrators, but that's a marketing fabrication. Trying to manage infrastructure without the necessary in-house skills is a recipe for disaster, whether it's in the cloud or on-prem.
I'm fully aware this is pedantic, but you can't save 10x. You can pay 1/10. You can save 90%. Your previous costs could have been 10x your current costs. But 10x is more by definition, not less. You can't save it.
In English, x or time(s) after a number marks a "unit" used by various verbs. A 10x increase. Increase by 10x. Go up 10x. Some of these verbs are negative like decrease or save. "Save 10x" is the same as "divide by 10". Four times less, 5 times smaller etc. are long attested.
No, x literally means multiply. It doesn't somehow also mean divide. They should use the percent sign, it's what it is for. 10x my costs means 10 x mycost, it's literally an equation
It's just inversion, like 2 to the power of 2 or 2 to the power of negative 2. These negative words inverse it just the same. You may dislike it, but millions of people have spoken this way for a long time.
> x literally means multiply
And some use the dot operator or even 2(3) or (2)(3). When programming, we tend to use *.
Nobody says times to mean divide. The word "negative" is what is doing the inversion for you in your example.
Agree to disagree.
It's basically multiplication with fractions, just like you get 1/4 from 1/2 * 1/2, which is once again a multiplication that turns into division
I agree that it's a suboptimal and click baity way to phrase it though...
Exactly, it's multiplication.
Example: He was saving $10 every month before the change. Then he switched and now he is saving 10 x $10, ie $100 every month.
But that's not the case here, right? And that's why the parent post is correct.
Thats not how English works.
Saved 10x would imply there was an amount being saved that they multipled.
You can save 10x, but you need to have an x first. So if you've got formula A which costs 500 EUR, and formula B which costs 480, you're saving 20 EUR. If formula C costs 300, you could save 200, which is 10x what you would save using B. But just as people don't understand when to use 'fewer' or misuse 'literally' to mean 'figuratively' (which is rather the opposite meaning of the word they use); words don't mean anything any more, it's all about the feelings man.
+1 words matter
Clarity of expression is a superpower
I don’t feel it’s pedantic at all.
Communication is a superpower. You say something and the other person understands.
Being pedantic about words means you think effective communication is somehow wrong. Be precise, don’t be pedantic.
People commonly use this expression in everyday conversation, such as, "you could save 10 times as much if you would just shop at Costco." So I agree with OP, their comment is correct but pedantic.
> People commonly
Your example them was a weasel worded advert that uses meaningless terminology to make something sound big (savings in this case).
People don't use that expression.
This becomes much clearer with a balance sheet in front of you.
What is saving? _Spending less_, that's all. Saving generates no income, it makes you go broke slower.
Independent of the price or the product, you can never save more than factor 1.0 (or 100%).
Wasn't there a guy on TV who wanted to make prices go down 1500%? Same BS, different flavor.
But that's 4x the savings compared to another saving. I suppose you've upped the pedantry and are technically correct, but that's a pretty narrow use case and not the one used in the article.
hmm.. if you reduce latency from one second to a hundred milliseconds, could you celebrate that you've made it 10x faster, or would you have the same quibble there too?
Edit: Thinking about this some more: You could say you are saving 9x [of the new cost], and it would be a correct statement. I believe the error is assuming the reference frame is the previous cost vs the new cost, but since it is not specified, it could be either.
> if you reduce latency from one second to a hundred milliseconds, could you celebrate that you've made it 10x faster
Yes you can, because speed has units of inverse time and latency has units of time. So it could be correct to say that cutting latency to 1/10 of its original value is equivalent to making it 10x the original speed - that's how inverses work.
Savings are not, to my knowledge, measured in units of inverse dollars.
Consider it as getting 10x the resources for the same price - that is, the resource-to-price ratio is 10x. Except you don't need 10x the resources so you choose to get 1x the resources for 0.1x the price instead.
Sure. Getting 10x the resources for the same price is another valid way to express the thought. Saving 10x isn't, though.
I think you would be more accurate to say 'communicate the thought' as opposed to 'express the thought'. Communication carries the explicit desire to have an idea be shared. Expression carries no shared understanding and is personal.
The author expressed the thought in a valid way because we understood their words, but they did not communicate it well because we are confused about their understanding of those ideas.
Colloquially, differences in powers of 10 would be better stated as differences of orders of magnitude.
I could care less.
So .... you DO care then? Or do you mean "I could NOT care less"?
On orders of the late Queen: https://youtube.com/watch?v=om7O0MFkmpw
Thanks - that is perfect.
Yes, that is why I replied with this to a pendant. :)
Apples and oranges, tbh.
I took care of IT for a startup hedge fund once. I was the quant's right-hand man, data engineer, visualization dashboard guy, everything. The quant needed to run a monolithic C++ program daily to chew through stock data and we decided a dual-Xeon server with 512 GB RAM would be great. OVH MG-512, for those curious.
Quant happy, boss happy, all good. Then the boss goes for lunch with someone and comes back slightly disturbed. We were not buzzword compliant. Apparently the other guy made him feel that he was using outdated tech by not being on AWS, using auto-scaling etc;
Here I am, from a background where my first language was 8086 assembly, and compactness was important to me. I remember thinking, "This whole thing could run on a powerful calculator, except for the RAM requirement".
It was a good lesson for me. Most CTOs know this bias and have unnecessarily huge and wasteful budgets but make sure they keep the business heads happy in the comfort that the firm is buzzword compliant. Efficiency and compactness are a marketing liability for IT heads!
I would think a quant would understand arithmetic.
Did you try crunching some of the numbers with him? I would hope a quant could also understand following the common wisdom can sometimes cost you more.
The quant was still happy, The Boss was the one with buzzword FOMO
Looks about right if you ask me
The author touches on it briefly, but I'd argue that the cloud is immensely helpful for building (and tearing down) an MVP or proving an early market for a new company using startup credits or free tiers offered by all vendors. Once a business model has been proven, individual components and the underlying infrastructure can be moved out of the cloud as soon as cost becomes a concern.
This means that teams must make an up-front architectural decision to develop apps in a server-agnostic manner, and developers must stay disciplined to keep components portable from day one, but you can get a lot of mileage out of free credits without burning dollars on any infrastructure. The biggest challenge becomes finding the time to perform these migrations among other competing priorities, such as new feature development, especially if you're growing fast.
Our startup is mostly built on Google Cloud, but I don't think our sales rep is very happy with how little we spend or that we're unwilling to "commit" to spending. The ability to move off of the cloud, or even just to another cloud, provides a lot of leverage in the negotiating seat.
Cloud vendors can also lead to an easier risk/SLA conversation for downstream customers. Depending on your business, enterprise users like to see SLAs and data privacy laws respected around the globe, and cloud providers make it easy to say "not my problem" if things are structured correctly.
Seems like nowadays people seem less concerned with vendor lockin than they were 15 years ago. One of the reason to want to avoid lockin is to be able to move when the price gouging gets just a little bit too greedy that the move is worth the cost. One of the drawbacks of all these built in services at AWS is the expense of trying to recreate the architecture elsewhere.
And yet, most of those success stories about ditching the cloud are from people that were not using services with hard vendor lock-ins.
Reading author's article:
> For me, that meant:
> RDS for the PostgreSQL database (my biggest monthly cost, in fact)
> EC2 for the web server (my 2nd biggest monthly cost)
> Elasticache for Redis
https://rameerez.com/how-i-exited-the-cloud/
> This means that teams must make an up-front architectural decision to develop apps in a server-agnostic manner
Right. But none of the cloud providers encourage that mode of thinking, since they all have complete different frontends, API's, different versions of the same services (load balancers, storage) etc. Even if you standardize on k8s, the implementation can be chalk and cheese between two cloud providers. The lock in is way worse with cloud providers.
While I love a good cloud bashing, it's really not black and white. If you're really small, it probably doesn't matter much if you're using Hetzner or AWS, but co-location might be a bit to expensive. If you run an absolutely massive company, cloud vs. self-hosted comes down to whether or not you can build tooling as good as AWS, GCP or Azure, with all the billing infrastructure and reporting.
The issues are mostly in the SME segment and where it really depends on what your business is. Do you need completely separate system for each customer? In that case, AWS is going to be easier and probably cheaper. Are you running a constant low 24/7? Then you should consider buying your own servers.
It's really hard to apply a blanket conclusion to all industries, in regards to cloud cost and whether or not it's worth it. My criticism in regards to opting for cloud is that people want all the benefits of the cloud, but not use any of the features, because that would lock them into e.g. AWS. If you're using AWS as a virtual machine platform only, there's always going to be cheaper (and maybe better) options.
If you have high volume traffic depending on time of month, ie finance around ultimo/primo, you might need to scale your performance to 5-10x your normal idle load.
If running on your own data center, or renting physical/virtual machines from ie Hetzner, you will pay for that capability overhead for 30.5 days per month, when in reality you only need it for 2-3 days.
With the cloud you can simply scale dynamically, and while you end up paying more for the capacity, you only pay when you use it, meaning you save money for most of the month.
> If running on your own data center, or renting physical/virtual machines from ie Hetzner, you will pay for that capability overhead for 30.5 days per month, when in reality you only need it for 2-3 days.
I keep seeing this take on here and it just shows most people don't actually know what you can do off the cloud. Hertzner allows you to rent servers by the hour, so you can just do that and only pay for the 2-3 days you need them.
Also the GP take is kinda silly, because one pays 5-10x with the typical cloud providers in the first place, so it would be totally fine to pay that and already have it, if one is willing to spend that much.
More likely load will spread over time for most scenarios and the server will be ready to handle that with lower hardware specs.
There's a reason i specifically mentioned finance, where the end and start of a month is a lot more busy than the middle of the month, as in factor 10+.
People receive paychecks, pay bills, buy stuff, with holidays (christmas, x-mas, etc) being even busier.
Load does not even out, and when you have 3 million customers or more, the load is not really insignificant. Nor can you just delay it, or rely on eventual consistency.
But most hosting providers allow you to just rent a server for a day or a few hours, so it's not really an argument why you have to use the cloud.
The actual hardware (or virtual these days) is not the problem, but getting the 5000-10000 services properly connected with auditing, credentials and more is more trouble than most people expect.
Finance is a heavily regulated industry, so there’s a LOT of compliance that needs to happen, like segregation of duty, traceability, accountability, and other ilities.
Yes, it would probably cost less to run on Hetzner (provided their ISO audits are approved by financial authorities), but dynamically spinning up and down servers would cost more.
You also need fallback plans (regulated industry, critical infrastructure, etc).
It has literally taken years to get AWS and Azure approved in EU.
Anything that wants to adhere to GDPR should still be very careful, if not outright avoid, AWS and Azure. At most one could use an EU-isolated offspring of them, otherwise one runs into the insanity of US laws. The fact that many businesses don't care doesn't make it right.
You can use the cloud to dynamically scale when needed while still running most of your own infra, best of both worlds.
Tricky networking though.
> Tricky networking though.
And data storage/locality/consistency.
Databases are rarely the bottleneck during access volume spikes in my experience. It’s really impressive how far some beefy servers and a read copy can take you.
But if that is your bottleneck you should be upgrading your DB system regardless of whether you’re on cloud or bare metal.
But if your one source of truth is a single huge database, maybe on a mainframe, as is often the case with finance, and eventual consistency is a huge no go, the mainframe can become a limit regardless when you're being "hammered" by 3+ million users all wanting to check their account balance.
Yes, you can solve the problem with sharding and other tricks, but for many banks, the mainframe is still their main data storage, and it has 60+ years of legacy code on it that is not easily or quickly migrated to modern architecture.
I agree with that, but I don’t understand how that’s particularly relevant to a bare metal vs cloud discussion.
Wouldn’t you need to do the work to shard regardless of where you’re running?
Banks running on mainframes don't shard. They just throw more hardware at the problem. The modern mainframes capacity for raw IO is insane, and much higher than what you can achieve with any PC setup. More hardware, more bandwidth, client facing applications in the cloud, database on the mainframe.
The major difference lies in infrastructure, particularly networking infrastructure. With cloud providers like Azure, AWS, etc, you can provision your vnet layout, and scale "indefinitely" on the same infrastructure. You don't need to provision new hosts, setup new secrets, or anything like that.
If a data center goes down, you can relatively easy switch to another one, though most financial institutions I know of uses hot/cold setups as hot/hot is essentially twice the money, and they rarely go down for long.
Of course it's all just regular servers underneath, so anything possible with AWS and Azure is also possible with other cloud providers, but the tooling simply isn't there (yet?).
Another issue is ISO auditor compliance. Being a regulated industry, finance (in EU at least) needs certain compliance to be fulfilled, not only regarding the services you consume, but also stuff like the physical locations, or being able to physically inspect the data center if auditors require it.
Microsoft and Amazon has this nailed. I've yet to experience a EU data center not run by FAANG meet the requirements, not that they can't. My best hope so far is probably "Lidl cloud" (forgot the name).
Right, the networking is the biggest hurdle in splitting up your infra.
The compliance issues are another big one though, for organizations that are still scaling up and don’t have that know how using cloud is a huge advantage as well.
I'd be more interested to understand (from folk who were there) what the conditions were that made AWS et al such a runaway hit. What did folks gain, and have those conditions meaningfully changed in some way that makes it less of a slam dunk?
My recollection from working at a tech company in the early 2010s is that renting rack space and building servers was expensive and time consuming, estimating what the right hardware configuration would be for your business was tricky, and scaling different services independently was impossible. also having multi regional redundancy was rare (remember when squarespace was manually carrying buckets of petrol for generators up many flights of stairs to keeps servers online post sandy?[1]).
AWS fixed much of that. But maybe things have changed in ways that meaningfully changes the calculus?
[1] https://www.squarespace.com/press-coverage/2012-11-1-after-s...
You're falling into the false dichotomy that always comes up with these topics: as if the choice is between the cloud and renting rack space while applying your own thermal paste on the CPUs. In reality, for most people, renting dedicated servers is the goldilocks solution (not colocation with your own hardware). You get an incredible amount of power for a very reasonable price, but you don't need to drive to a datacenter to swap out a faulty PSU, the on site engineers take care of that for you. I ordered an extra server today from Hetzner. It was available 90 seconds afterwards. Using their installer I had Ubuntu 24.04 LTS up and running, and with some Ansible playbooks to finish configuration, all in all from the moment of ordering to fully operational was about 10 minutes tops. If I no longer need the server I just cancel it, the billing is per hour these days.
Bang for the buck is unmatched, and none of the endless layers of cloud abstraction getting in the way. A fixed price, predictable, unlimited bandwidth, blazing fast performance. Just you and the server, as it's meant to be. I find it a blissful way to work.
I’d add this. Servers used to be viewed as pets; the system admins spent a lot time on snow flake configurations and managing each one. When we started standing up tens of servers to host the nodes of our app (early 2000s); the simple admin overhead was huge. One thing I have not seen mentioned here was how powerful ansible and similar tools were at simplifying server management. Iirc being able to provision and standup servers simply with known configurations was a huge win aws provided.
Also, it was a very very different landscape.
You were commonly given a network uplink and a list of public IP addresses you were to set up on your box or boxes. IPMI/BMC were not a given on a server so if you broke it, you needed to have remote hands and probably brains too.
Virtualisation was in the early days and most of the services were co-hosted on the server.
Software defined networks and Open vSwitch were also not a thing back then. There were switches with support for VLANs and you might have had a private network to link together frontend and backend boxes.
Servers today can be configured remotely. They have their own management interfaces so you can access the console and install OS remotely. The network switches can be reconfigured on the fly, making the network topology reconfigurable online. Even storage can be mapped via SAN. The only hands on issue is hardware malfunction.
If I was to compare with today, it was like having a wardrobe of Raspberry Pies on a dumb switch, plugging in cables when changes were needed.
Even if you don't go ansible/chef/puppet/salt, just having git is good. You can put your configs in git, use a git action to change whatever variables, and deploy to the target. No extra tools needed, and you get versioned configs.
> all in all from the moment of ordering to fully operational was about 10 minutes tops.
I think this is an important point. It's quick.
When cloud got popular, doing what you did could take upwards of 3 months in an organisation, with some being closer to 8 months. The organisational bureaucracy meant that any asset purchase was a long procedure.
So, yeah, the choices were:
1. Wait 6 months to spend out of capex budget
Or
2. Use the opex budget and get something in 10m.
We are no longer in that phase, so cloud services makes very little sense now because you can still use the opex budget to get a VPS and have in going in minutes with automation.
True, but I think you're touching on something important regarding value. Value is different depending on the consumer: for you, you're willing and able to manage more of the infrastructure than someone who has a more narrow skillset. Being able to move the responsibility for areas of the service on to the provider is what we're paying for, and for some, paying more money to offload more of the responsibility actually results in more value for the organization/consumer
> I ordered an extra server today from Hetzner. It was available 90 seconds afterwards.
Back when AWS was starting, this would have taken 1-3 days.
AWS also made huge inroads in big companies because engineering teams could run their own account off of their budget and didn’t have to go through to IT to requisition servers, which was often red tape hell. In my experience it was just as much about internal politics as the technical benefits.
> which was often red tape hell
Seconded. I was working for a storage vendor when AWS was first ascendant. After we delivered hardware, it was typically 6-12 weeks to even get it powered up, and often a few weeks longer to complete deployment. This is with professional services, e.g. us handling the setup once we had wires to plug in. Similar lead time for ordering, racking, and provisioning standard servers.
The paperwork was massive, too. Order forms, expense justifications, conversations with Legal, invoices, etc. etc.
And when I say 6-12 weeks, I mean that was a standard time - there were outliers measured in months.
Absolutely. At several startups, getting a simple €20–50/month Hetzner server meant rounds with leadership and a little dance with another department to hand over a credit card. With AWS, leadership suddenly accepted that Ops/Dev could provision what we thought was right. It isn’t logically compelling, but that’s why the cloud gained traction so quickly: it removed friction.
> At several startups, getting a simple €20–50/month Hetzner server meant rounds with leadership and a little dance with another department to hand over a credit card.
That's not a startup if you can't go straight to the founder and get a definite yes/no answer in a few minutes.
Computing power (compute, memory, storage) has increased 100x or more since 2005, but AWS prices are not proportionally cheaper. So where you were getting a reasonable value in ~2012, that value is no longer reasonable, and by an increasing margin.
This is the big one.
In 2006 when the first EC2 instances showed up they were on par with an ok laptop and would take 24 months to pay enough in rent to cover the cost of hardware.
Today the smallest instance is a joke and the medium instances are the size of a 5 year old phone. It takes between 3 to 6 months to pay enough in rent to cover the cost of the hardware.
What was a great deal in 2006 is a terrible one today.
raises hand I ran a small SaaS business in the early 2000s, pre-AWS.
Renting dedicated servers was really expensive. To the extent that it was cheaper for us to buy a 1U server and host it in a local datacenter. Maintaining that was a real pain. Getting the train to London to replace a hard drive was so much fun. CDNs were "call for pricing". EC2 was a revelation when it launched. It let us expand as needed without buying servers or paying for rack space, and try experiments without shoving everything onto one server and fiddling with Apache configs in production. Lambda made things even easier (at the expense of needing new architectures).
The thing that has changed is that renting bare metal is orders of magnitude cheaper, and comparable in price to shared hosting in the bad old days.
> But maybe things have changed in ways that meaningfully changes the calculus?
I'd argue that Docker has done that in a LOT of ways. The huge draw to AWS, from what I recall with my own experiences, was that it was cheaper than on-prem VMware licenses and hardware. So instead of virtualizing on proprietary hypervisors, firms outsourced their various technical and legal responsibilities to AWS. Now that Docker is more mature, largely open source, way less resource intensive, and can run on almost any compute hardware made in the last 15 years (or longer), the cost/benefit analysis starts to favor moving off AWS.
Also AWS used to give out free credits like free candy. I bet most of this is vendor lock in and a lot of institutional brain drain.
One factor was huge amounts of free credits for the first year or more for any startup that appeared above-board and bothered to ask properly.
Second, egress data being very expensive with ingress being free has contributed to making them sticky gravity holes.
The free credits... what a WILD time! Just show up to a hackathon booth, ask nicely, and you'd get months/years worth of "startup level" credits. Nothing super powerful - basically the equivalent of a few quad core boxes in a broom closet. But still for "free".
Et al is for people, Et cetera is for things.
Edit: although actually many people on here are American so I guess for you aws is legally a person...
As an American who studied Latin:
Et al. = et alii, "and other things", "among other things".
Etc. = et cetera, "and so on".
Either may or may not apply to people depending on context.
> although actually many people on here are American so I guess for you aws is legally a person...
Corporate legal personhood is actually older than Christianity, and it being applied to businesses (which were late to the game of being allowed to be corporations) is still significantly older than the US (starting with the British East India Company), not a unique quirk of American law.
Oh I didn't how that, thanks for the lesson.
Tbf it just sounds...so American, so I assumed, my bad. But East India Company was involved...whew I guess that does make sense, oof.
What is unique in the US is the interaction between corporate personhood and our First Amendment and the way that our courts have applied that to limit political campaign finance laws, and a lot of “corporate personhood” controversy is really about that, not actually about corporate personhood as a broad concept.
People also get confused about the Citizens United ruling. It had nothing to do with corporate personhood.
The ruling said that since a person has first amendment rights, those same rights extend to a group of people—any group—whether it’s a non profit organization, a corporation, or something else.
Well, it's also what allows the executives, boards, and owners of companies to be divorced from the consequences of their actions in a legal sense much of the time.
Source of it being older than christianity?
I don't think that's a hard and fast rule? I think et al is for named, specific entities of any kind. You might say "palm trees, evergreens trees, etc" but "General Sherman, Grand Oak, et al"
The problem it really solved was your sysadmins were still operating by SSHing into the physical servers and running commands meticulously typed out in a releaae doc or stored on a local mediawiki instance, and acquiring new compute resources involved a battle with finance for the capex which would delay pretty much any project for weeks, while cloud vendors let engineers at many companies sidestep both processes.
Everything else was just reference material for how to sell it to your management.
Answers here
https://learn.microsoft.com/en-us/azure/cloud-adoption-frame...
TL;DR version - its about money and business balance sheets, not about technology.
For businesses past a certain size, going to cloud is a decision ALWAYS made by business, not by technology.
From a business perspective, having a "relatively fixed" ongoing cost (which is an operational expense ie OpEx ) even if it is significantly higher than what it would cost to do things with internal buy and build out (which is a capital expense cost ie CapEx), make financial planning, taxes and managing EBITDA much easier.
Note that no one on the business really cares what the tech implications are as long at "tech still sorta runs mostly OK".
It also, via financial wizardry, makes tech cost "much less" on a quarter over quarter and year over year basis.
It was the accountants. CapEx vs. OpEx.
Its not actually that hard to get your own server racked up in a data centre, I have done it. Since it was only one box that I built and installed at home I just shipped it and they racked it in the shared area and plugged the power and network in and gave me the IP address. It was cheaper than renting from someone like hetzner, was about £15 a month at the time for 1A and 5TB a month of traffic at 1gbps. Also had a one off install fee of £75.
At the time I did this no one had good gaming CPUs in the cloud, they are still a bit rare really especially in VPS offerings and I was hosting a gaming community and server. So I made a pair of machines in a 1U with dual machines in there and had a public and private server with raid 1 drives on both and redundant power. Ran that for a gaming server for many years until it was obsolete. It wasn't difficult and I think the machine was about £1200 in all, which for 2 computers running game servers wasn't too terrible.
I didn't do this because it was necessarily cheaper, I did it because I couldn't find a cloud server to rent with a high clockspeed CPU in it. I tested numerous cloud providers, sent emails asking for specs and after months of chasing it down I didn't feel like I had much choice. Turned out to be quite easy and over the years it saved a fortune.
We did this a lot in the early 2000's. At the time I worked for a company with offices in Bellevue and we put our own hardware in full sized racks at a datacenter in the komo4 building in Seattle.
Because of proximity it was easy to run over and service the systems physically if needed, and we also used modem based KVM systems if we really needed to reboot a locked up system quickly (not sure that ever actually happened!).
I'm sure customer owner hardware place in a datacenter rack is still a major business
What is the mechanism for such services if you want to replace a component (ex. a failing hard drive or upgrade ram)?
"Remote hands" is the DC term for exactly what it sounds like. You write a list of instructions and someone hired by the DC will go over to your rack and do the thing.
the problem isn't setup. its maintaining it. Its not an easy job to do that some times. Im not trying to dissuade people from running there own servers, but its something to consider.
This isn’t a binary issue. I disagree with these “abandon the cloud” takes but do agree that most folks spend way way more than they should.
The biggest threat to cloud vendors is that everyone wakes up tomorrow and cost optimizes the crap out of their infrastructure. I don’t think it’s hyperbolic to say that global cloud spending could drop by 50% in 3 months if everyone just did a good audit and cleaned up their deployments.
> The biggest threat to cloud vendors is that everyone wakes up tomorrow and cost optimizes the crap out of their infrastructure
Well there's no danger of that. Even with AWS telling you exactly how to save money (they have a thousand different dashboards showing you where you can save, and even support will tell you), it'll still take you months to work through all the cost optimization changes. Since it's annoying and complicated to do, most people won't do it.
Their billing really is ridiculous. We have a TAM and use a reseller, and it's virtually impossible for us to see what we actually spend each month, what with the reseller discounts, enterprise PPA, savings plans, RIs, and multiple accounts. Their support reps even built us some kinda custom BI tool just to look at costs, and it's still not right.
Yes, the partner resale model of AWS has been a total mess for a while.
Exactly this.
We have to use cloud because we're at the low end of 10^5 servers. Once you hit the high end of 10^3 this is really where you need to be.
Everything we're doing is highly inefficient because of decades of fast and loose development by immature software engineers...and having components in the stack that did the same.
If I had 5 years to rewrite and redesign the product to reflect today's computing reality, I could eliminate 90%+ of the cost. But I'll never get that kind of time. Not with 1000 more engineers and 1000 more years and the most willing customers.
You might get lucky enough that you and a bunch of your customers are so fed up with your company that you get to create the competition.
You have over a 100,000 servers? How is this? This is wild to ponder. I can only guess your monthly bill is in the millions. Are you able to say the company?
"100.000 servers" sounds like the entirety of 2008-era Google, but with today's 10-100x performance per server. It could be bigger than Dropbox or Netflix.
Basically it boils down to:
Is your time worth more than what the fully managed services on AWS cost? And I mean that quite literally in the sense of your billable hours.
If you like spending time tinkering with manually configuring linux servers, and don't have anything else that generates better value for you to do - by all means go for it.
For small pet projects with no customers, a cheap Hetzner box is probably fine. For serious projects with customers that expect you to be able to get back on your feet quickly when shit hits the fan, or teams of developers that lose hours of productivity when a sandbox environment goes down, maybe not so much.
Is AWS more expensive than the salary of the infra guy you would need to hire to do all this stuff by hand? Probably not.
Well, this sounds wildly out of touch. Companies using Hetzner cloud or servers having thousands of customers and paying 50-100 people are running just fine, but you labeled them a "small pet project". Provisioning new servers works just fine via API when you need it.
Source: I worked at such a company.
The first couple of paragraphs of price comparisons are useful. Then there are many paragraphs of sheer waffle. The author doesn't even seem able to define what "the cloud" is:
> The whole debate of “is this still the cloud or not” is nonsense to me. You’re just getting lost in naming conventions. VPS, bare metal, on-prem, colo, who cares what you call it. You need to put your servers somewhere. Sure, have a computer running in your mom’s basement if that makes you feel like you’re exiting the cloud more, I’ll have mine in a datacenter and both will be happy.
I read the whole thing and I didn't see any waffle. Sure, undeniably some excess word count, some emotion in responding to critics. But no waffle.
The "is this cloud or not" debate in the piece makes perfect sense. Who cares whether Hetzner is defined as "the cloud" or not? The point is, he left AWS without going to Azure or some other obvious cloud vendor. He took a step towards more hands on management. And he saved a ton of money.
> He took a step towards more hands on management. And he saved a ton of money.
Then the article should be titled as
"Send this article to your friend who still thinks that AWS is a good idea"
or
"Save costs by taking a step towards more hands on management"
or
"How I saved money moving from AWS to Hetzner"
The cheap hosting service they switched to is arguably "cloud".
If you can't drive to the location where your stuff is running, and then enter the building blindfolded, yet put your hands on the correct machine, then it's cloud.
Engineering is all tradeoffs.
For this guy who spent thousands per month on AWS, using AWS makes no sense.
For a guy like me whose AWS bill was 38 cents last month, AWS makes a lot of sense. Currently I have a small KaiOS app with about 1000 unique visitors per month hosted on AWS. I used to self-host on an old laptop, but it costs less money to host on AWS (1 kwh cost me 19.89 cents and my laptop uses about 4.5kwh per month). Not to mention the convenience of not having to restart my server for software updates, or update my own certs every 3 months, or fuddling with nginx configurations.
And my AWS bill ever starts climbing, I will just self host once again.
You could probably bring that down to 30 cents on nearlyfreespeech.net
The irony is that I went to read this article and encountered the Cloudflare error "521: Web server is down".
I don't think it was designed to handle the volume of traffic that HN generates.
A Cloudflare fronted website can't handle HN frontpage levels of traffic?
Then why does anybody use cloudflare?
Probably bad cache headers configuration. Even with Cloudflare in front it could be forwarding every request to the backend if the cache headers are misconfigured...
yeah it was bad move to take it off the cloud
Cheap shot maybe, but the fact that the page takes 10 seconds to load when it hits the HN front page is a great, inadvertant illustration of why you might want to use the cloud sometimes.
The failure mode of self-hosting is that your site gets hugged to death, the failure mode of the cloud is that you lose a ton of money. For a blog that doesn't earn you anything, the choice is clear.
Besides, you can just put it behind cloudflare for free.
> the failure mode of the cloud is that you lose a ton of money
I'm perfectly happy to use cloud stuff for work. I will never ever give any cloud platform my personal credit card. I don't want to be the position of a misconfiguration leaving my finances at the mercy of the AWS forgiveness gods.
Absolutely. And I never said that everybody should use the cloud. Contrary to OP, it's more complicated than that. Not every site has the same needs. Some sites need scaling. Some sites are fine with being hugged to death, others would lose millions of dollars if they were. Some people are happy to tinker with their nginx config and package upgrades etc, others prefer to deploy and forget it (with cost controls enabled).
Disclosure: I work for Cloudflare, but 25 years ago served my own site from an iBook under the stairs.
> The failure mode of self-hosting is that your site gets hugged to death
Learn 2 load balance
If only it were that simple. Don't forget:
Learn 2 HA
Learn 2 MFA
Learn 2 backup
Learn 2 recover within RTO
Learn 2 ETL
Learn 2 queue
Learn 2 scale horizontally
Learn 2 audit log
Learn 2 SEIM
Learn 2 continuously gather SOC evidence
...
OP here. As others have said, it loads immediately for me (tested on desktop + on mobile data + incognito)
The entire site is cached + Cloudflare sits on top of everything. I just ran a couple performance tests under the current HN traffic (~120 concurrent visitors) and everything looks good, all loads under 1 second. The server is quite happy at an average load of 0.06 right now, not even close to start breaking a sweat.
Turns out you can get off the cloud and hit the frontpage of HN and your site will be alright.
100% true, I've hit the front page of hn on a server with an old i5 (aka consumer hardware, and not even high end) with no cloudflare or similar caching, and had no problems. Computers are fast, and serving static html over https is a solved problem.
You're right that serving static files is easy, but OP is using WordPress, not static files.
Disclosure: I work for Cloudflare.
My guess is your caching was misconfigured in some way. It's fast for me now, but before commenting I tried several times and it always took at least 10s to load, often more. Others have also reported very slow load times. I was surprised because I saw at the time that it was behind Cloudflare. Disclosure: I work for Cloudflare.
That happens a lot to blogs deployed on the cloud too. They just need to put a small cache in front and they'll be able to serve one or two orders of magnitude more requests per second.
You want to architect. Thats it. You can handle the 50krps that you wont get from HN from a dedicated box.
Took under a second. What part of the world are you located in?
Only took 1 second for me.
I have a VPS. It costs me £1.34 per month. It's way over-powered for what I need it for.
However, one situation where I think the cloud might be useful is for archive storage. I did a comparison between AWS Glacier Deep Storage and local many-hard-drive boxes, for storing PB-scale backups, and AWS just squeaked in as slightly cheaper, but only because you only pay for the amount you use, whereas if you buy a box then you have to pay for the unused space. And it's off-site, which is a resilience advantage. And the defrosting/downloading charge was acceptable at effectively 2.5 months worth of storage. However, at smaller scales you would probably win with a small NAS, and at larger scales you'd be able to set up a tape library and fairly comprehensively beat AWS for price.
Yeah, but in 800 months you'd come out ahead with a dedicated server in your closet.
I run a tiny local dedicated server 24/7 that consumes around 10W on average, which is about $2/mo in electricity costs where I live.
I meant the upfront cost of the machine.
Its a weird service because before that point AWS is crazy expensive for storage, especially down in the TB range its awful value compared to your box and drives. But once you get into that PB scale AWS actually seems to be competitive, I guess because the GB/TBs they are selling are from PB scale solutions and all the overhead that entails.
"To them, it’s way too convenient to be on AWS: not only it solves their problem, but it’s also a shiny object. It’s technically complex, it makes them look smart in front of other devs, "
why? why be so obnoxious to other people who you claim are being obnoxious to you. no need to read your blog post now/
Fittingly, his website was hugged to death
Loaded instantly for me (never visited, so not cached), in the central US.
It's almost like Clouds are really good at scaling and some rented server isn't! Perfect, almost poetic.
It’s entirely possible for a rented server to host a site that gets millions of views. It’s also entirely possible to make an AWS setup that chokes with 100.
I use Cloudflare in front of my personal stuff. Then it's just a quick DNS switch to go direct if I need to.
It's almost like nobody cares about scaling their blog.
Not trying to be dismissive of the article but, the way it's written, it reads like a lot of whining.
He could have summed up with "AWS is expensive, host your own server instead".
I've been at too many startups with a devops team that would rather provision 15 machines with 4GB RAM THAN ONE WITH 64GB.
I once got into an argument with a lead architect about it and it's really easy to twist the conversation into "don't you think we'll reach that scale?" To justify complexity.
The bottom line is for better or worse, the cloud and micro services are keeping a lot of jobs relevant and there's no benefit in convincing people otherwise
Multiple small boxes is actually better than one giant box, for a whole lot of reasons. Scaling isn't the issue.
This needs some context. I can think of several cases where this wouldn’t be the be right.
Here's a basic comparison:
Dangit, meant "cattle rather than pets" for small box pros
Big box VM host running several VMs, hardware starts to fail, brings the entire setup to its knees. That wouldn't happen if you had each VM running on its own physical hardware, unless the UPS/PDU died...
VMs are able to live migrate these days. Single VM has a lot more uptime guarantee than a distributed system with multiple over the network communication layers.
Not if you only have one VM host ;)
What I always say when given a false choice: ¿porque no las dos?
vcpu, iops, transfer fees, storage -- they are all resources going into a pool .
If Hetzner is giving you 10TB for $100 , then host your static files/images there and save $800.
Apps are very modular. You have services, asyncs, LBs, static files . Just put the compute where it is most cost effective.
You don't have to close your AWS account to stick it to the man. Like any utility, just move your resources to where they are most affordable.
We've moved to Hetzner several years ago, and the savings will ensure we'll likely never consider the Cloud again (Exception is Cloudflare Workers, which is well done with generous quotas). Although it would be way easier to move now since automation, deployment, backup and retention scripts are much faster to write now with AI.
BTW in-case you've missed it [1] Anthropic is offering $250 free credits till Nov 18 for their new Claude Code on the Web [2] for Clause Pro/Max users.
[1] https://x.com/adocomplete/status/1985766988724244839
[2] https://claude.ai/code
How much is your labor worth though? The actual cost of a server is not expensive; the maintenance behind it is. If I have a fleet of 10 EC2 instances, I can keep them patched with Systems Manager without having to build any of that automation myself.
Sometimes I think I am out of the loop for using dedicated servers like OVH, DigitalOcean, and Hetzner, while others spend thousands of dollars for the things I spend barely a few hundred. This always made me think I am not a good developer enough to know the cutting-edge things others know.
Turns out most of the developers suck at handling barebones with a Linux distro + nginx and some other plugins to do the same things as the fancy-named aws stuff. If you are in the same boat, just know that most of these developers suck at what they are doing and don't care about the company budget.
You can get 99.99% of the things done with barebone + Cloudflare, including multiserver redundancy, at a fraction of AWS and Azure costs. Most of these technologies are just fancy words for basic Linux services.
I always thought it was because they were working at a huge scale... but who knows.
I would really be interested in an actual comparison, where e.g. someone compares the full TCO of a mysql server with backup, hot standby in another data center and admin costs.
On AWS an Aurora RDS is not cheap. But I don't have to spend time or money on an admin.
Is the cost justified? Because that's what cloud is. Not even talking about the level of compliance I get from having every layer encrypted when my hosted box is just a screwdriver away from data getting out the old school way.
When I'm small enough or big enough, self managed makes sense and probably is cheaper. But when getting the right people with enough redundancy and knowledge is getting the expensive part...
But actually - I've never seen this in any if these arguments so far. Probably because actual time required to manage a db server is really unpredictable.
> Probably because actual time required to manage a db server is really unpredictable.
This, and also startups are quite heterogeneous. If you have an engineer on your team with experience in hosting their own servers (or at least a homelab-person), setting up that service with sufficient resiliency for your average startup will be done within one relaxed afternoon. If your team consists of designers and engineers who hardly ever used a command line, setting up a shaky version of the same thing will cost you days - and so will any issue that comes up.
Its a skillset that is out of favour at the moment as well but having someone who has done serverops and devops and can develop as well is a bit of a money saver generally because they open up possibilities that don't exist otherwise. I think its a skillset that no one really hired for past about 2010 when cloud was mostly taking off and got replaced with cloud engineers or pure devops or ops people but there used to be people with this mixed skillset in most teams.
I've never had a server go down. Most companies don't need a hot server because it's never going to be needed.
AWS + Azure have both gone down with major outages indivudually more over the last 10 years than any of the servers in companies I worked with in the 10 years before that.
And in comparable periods, not a single server failed or disk failed or whatever.
So I get SOME companies need hot standby servers, almost no company, no SaaS, no startup, actually does.
Because if it's that mission critical, then they would have already had to move off the cloud due to how frequently AWs/Azure/etc. have gone down over the last 10 years, often for 1/2 day or so,
I've had a lot of servers going down. I've had data centers going down. For various reasons - but normally not a failed disk but configuration errors due to human error.
And I've had enough cases where the company relied on just that one guy who knew how things worked - and when they retired or left, you had big work ahead understanding the systems that guy maintained and never let anyone else touch. Yes, this might also be a leadership issue - but it's also an issue if you have no one else with that specific knowledge. So I prefer standardized, prepackaged, off the shelf solutions that I can hire replacable people for.
every box is a screwdriver away
The idea that it's cheaper not to use AWS is clear.
I was hoping to see more about porting AWS proprietary features into generic servers.
A big part of the problem isn't just monthly rent, it's vendor lock-in. When your whole system is implemented using AWS specific features, you're not going to run anywhere else.
AWS, and any other third party vendor, can and does obsolete features. Then you're having to port your system just to keep it running on the third party service.
Once you're implemented in a generic server, in a VPS, or your mom's basement, you're free to move to any other hosting provider, data center, whatever.
The loss of understanding that 3rd party dependencies are not good for your company or project, seems a bigger loss to the technical community than FTP hacking...
Host is erroring out, so webpage is down, per cloudflare.
I'm quite sympathetic to on-prem but there's a possible argument against.
I worked for a small, local insurance company 10+ years ago that was not interested in the cloud in any way, shape, or form. They had their servers in two locations—the basement of our office building, and an offsite location ~300 miles away. They were professional server rooms, complete with proper cooling, backup power, and the like.
At first I remember thinking they were behind the times, but now I'm certain they've avoided wasting a fortune on the cloud.
After a few months of experimenting on AWS, I got scared seeing how fast small side projects could turn into expensive monthly bills.
I built my own DIY cloud, a minimal self-hosted setup that lets me deploy apps, databases, and backups on cheap VPS or bare-metal servers. At first I just wanted to save money, but I realized that managing it can easily become a full-time SRE job.
Still, for small, experimental, or hobby projects, this setup works perfectly. It keeps costs predictable and gives full control without needing large-scale infrastructure.
https://diy-cloud.remikeat.com
Would love to hear from others who tried leaving managed clouds behind.
Help convince me I should be confident taking responsibility for:
* off-site db backups
* a guaranteed db restore process
* auditable access to servers
* log persistence and integrity
* timely security patching
* intrusion detection
so that my employer can save money.
How many URLs of Google et al failing to provide per instance security (leaking your files etc) to other users would you like to see?
At least one?
https://www.cnbc.com/2020/02/04/google-accidentally-sent-som...
So security's too hard for Google.
But if this happens at work and we're using Google Cloud, I say "sorry boss, Google messed up".
If we're running our own servers, who do I blame?
Nobody ever got fired for buying IBM!
:) you definitely haven't changed my mind!
That wasn’t GCP. Their cloud computing business is operated very differently from their consumer businesses.
The cloud is a good idea. It becomes a bad idea when it is the only thing you know or, most likely, is the only cloud you know.
> To them, it’s way too convenient to be on AWS: not only it solves their problem, but it’s also a shiny object. It’s technically complex, it makes them look smart in front of other devs, it creates dependencies and lock-in effects so they’re not easily replaceable as employees, and most importantly – being on the cloud yields them a fat paycheck at the end of the month.
I recently read a fantastic comment or article on how developers, and IT workers in general, will be the first to be automated away, and groups like lawyers and doctors the last, entirely unrelated to how good LLMs and other AI models are at the specific tasks of each. Reason being that developers fall over themselves to prove how productive they are, happy to use all these tools. Anywhere else is concerned with job protection first and foremost. Metrics like SLOC having come from devs themselves in the chase of a raise.
This paragraph reminds me of that so much. Calling out those who push the cloud for job security. In basically every other sector - where this kind of behavior is 10x more prevalent - no one calls this out, there's a sense of solidarity and mutual understanding that this kind of thing is necessary.
This isn't judging one or the other to be good or bad. But it's going to cost devs very dearly.
I don't remember where I read it, they put it much better than I can - if anyone here recognizes it please drop a link.
A 2024 International LT Series semi-truck costs $130,000. That's very expensive compared to a $30,000 Ford Maverick.
Both of these trucks can technically be used to pick up groceries and commute. But, uh, if you bought the semi-truck to get groceries and commute? Nobody scammed you; you bought the wrong truck. You don't have to buy the biggest, most expensive truck to do small jobs. But also, just because there's a cheaper truck available, doesn't mean the semi-truck is overpriced or a scam. The semi is more expensive for a reason.
I wonder about people who write articles like these. I imagine at one point he believed he had to use the cloud, so he started using it without understanding what he was doing. As a result, he was charged a ton of money. He then found out there were cheaper, simpler ways to do what he wanted. And so, feeling hurt, embarrassed, and deceived, he writes this long screed, accusing people of scamming him - I mean scamming you - because you (not him!) could not possibly need to use the cloud, even though you (not him!) assumed you had to use the cloud.
Yes, dude. The cloud is expensive. Sorry you found out the hard way. And for what it's worth, you don't need a datacenter either; stick a 1U in your closet and call it a day.
I don't particularly like this article, mostly because he seems to have spent the great majority of it ranting about people on the internet supposedly yelling at him about not being "on the cloud" and semantics about exactly what constitutes "the cloud". I'd be much more interested in reading exactly what he was spending that money on at AWS and how he moved it all onto one or a few bare-metal boxes.
I don't think AWS is particularly expensive at all. In fact, it's rather cheap at quite a few things if you know what you're doing and use the right services. I've got all my server backups going to S3, for which I pay a whopping penny a month. I also host a handful of Docker images on ECR for 7 cents a month - way cheaper than any other private hosting service I know of.
Now RDS, and the equivalent at every other hosting provider I know of, gets quite pricey indeed. IMO, you shouldn't bother unless you have pretty serious needs for reliability and speed. Apt install your DB server of choice on bare metal goes quite far. Or possibly pull a container of it into whatever you're using to manage containers.
I think the main thing holding people back from leaving the cloud is simple inertia. There was a time when the cloud was obviously the right choice. Static IPv4 addresses were becoming scarce, IPv6 had not been deployed widely enough, and cloud providers made it easy to stand up a server and some storage with high speed links on the cheap. But over time, things have changed. Rate limits, data caps, and egress fees are now normal (and costly). IPv6 is now deployed widely enough that you might be willing to just run an IPv6-only stack, which greatly simplifies running a server on-premise. And of course, we've all seen time and again how providers will carelessly lock out your cloud account for arbitrary reasons with little to no recourse. The time has come to own your infrastructure again. But that won't happen until people realize it's easy to do.
If all you need is compute, than yeah, self hosting is easy. Otherwise, do you think just about every company under the sun is a sucker for being on the cloud? If it was so easy, companies would be either be constantly dropping prices to compete with all the self hosters, or new companies to fill in the price gaps.
I never understood the appeal of AWS like cloud for non corporate. Sure, when you run a BigCo, you have all the money and people to throw at big cloud providers, obtaining pointless certificates, and configuring obscure IAM roles.
But why small businesses like them, I can’t I understand. I host all my stuff on Hetzner. It’s easy, it’s affordable and I “move fast”. I don’t need certificates to be able to manage the servers, and there is tone of information available online. But maybe it’s because I’m old, and I used to ssh to a Linux machine, git pull the latest version, and call in a “deploy”. But modern developed seems to be afraid of raw machines.
The only thing I miss is a good transactional email provider with PAYG pricing, and EU data residency.
“Hey bob from finance, I heard you want a new bi and dashboarding system to help us make better strategy choices? here’s 200k, hire devs, infra spend can be billed to this project id”
It’s a paperwork thing, not a financial decision.
I have some projects with a fairly steady load that I'm currently running on a VPS on Digital Ocean. I have some non-critical ones that I'd be interested in moving to "bare metal" for learning purposes, but having a hard time figuring out what the transition from VPS to dedicated server would look like.
Anybody have any guides or guidance on making the migration? I feel comfortable managing the VPS, but not sure what else I'd need to take on by moving to a dedicated server. I imagine hardware monitoring would be one, but what else?
Why it feels like the author is too young and just had breaking discovery that he can have servers without clouds!? Always been a thing, clouds were/are used in areas where it would be better, say some integration with already existing infrastructure, or some quick scaling. Just like everything, there’s always upside and downside, and it’s just about what suits your needs. The author should next try an on-prem approach where he even controls the hardware, even more cheaper but with extra maintenance. For example, I found a used server a while ago (44 Core HP Z840 WorkStation Dual Xeon E5-2699 V4 512GB RAM) for around $1000, that’s a one time pay.
I recently had the same realization and moved all my functions to a simple stand-alone server. Besides the normal AWS costs, what scares me most about AWS is the possibility that someone could try to DOS me, leaving me with a huge AWS bill, because there is no real way to limit AWS spending.
The main reason why I keep coming back to cloud providers is databases. I don't feel comfortable setting up a high-availability db setup, and I don't want the responsibility of managing backups. But if you go to, say, Hetzner, you won't be able to use a cloud database in the same network.
For me, it is a lot simpler to host at Linode (or simpler) than figure out the AWS/GCP crazy complex IAM stuff.
However, there are cases where being able to spin down the server, and not pay for downtime is useful - like 36-core Yocto build machines.
I've always found AWS IAM quite simple, but then again it is my job, so I might be biased. I haven't really dug into GCP well enough to understand it, but I did find it quite daunting to start the few times I messed with it. What's complex about it to you?
For personal projects, honestly, the built in roles AWS provides are okay enough for some semblance of least privilege x functionality IMO.
Plus, most of AWS's documentation tells you the specific policy JSON to use if you need to do XYZ thing, just fill in the blanks.
What if the data center that has your dedicated server burns down? You need to prepare to that buy renting another server at a different data center and synching them. Not a big problem, but needs to be mentioned
Correct AWS gives you availability zones. It is easy to spin up a cluster over them and have little or no downtime.
But a burning data centre is just one problem. Another is e.g. the USE1 issue AWS had. AZ didnt help here. Having compute in a other region an rerouting might. But maybe we aint talking startup concerns here. (If you have a simple enough "app" just deploy again in another region and update dns.)
As a note hetzner has a lot of auction servers and I believe they lack the setup fee
They have also threatened to cancel my account more than once because I typed "ipfs daemon".
https://github.com/ipfs/kubo/issues/10327
https://discuss.ipfs.tech/t/moved-ipfs-node-result-netscan-d...
>This happens with Hetzner all the time because they have no VLANs and all customers are on a single LAN and IPFS tries to discover other nodes in the same LAN by default.
Hetzner is also sinkholed by lots of EDR products because they host a ton of malicious garbage. They are a bad actor.
Why is it their job to be the arbiters of what customers are allowed to do on their platform?
To be fair, most hosting platforms have those in T&S, some even explicitly say you can't torrent pirated movies and even monitor your activities.
Same as AWS. I've added quite a few AWS ip ranges to my firewall.
I think a lot of teams using cloud are using SaaS rather than IaaS. They want a redis and a postgres and a S3 and a ... You can set all that up on a server, but it's not very fun if you've never done it before.
Its true that aws is very expensive for a lot of use cases, as long as you only count the cost of the servers.
However, in most (every?) large organisations, buying a server and putting it in a DC, and looking after it, is hugely time-consuming, lengthy and expensive.
You need to get quotes, approvals, purchase, rack, commission, maintain etc etc. It is usually close to 1 year to get a server.
In some companies, even getting a virtual server takes almost as long!
With AWS, once an account and service is approved, boom, you are done.
My experience is exactly the opposite (company with more than 10k employees). Getting anything done in Azure takes me 10x as long, as all of Azure is managed by one team, and everything requires approvals, lots of bureaucracy. Also, as it turns out, it is extremely expensive. Per our guidelines everything needs to be isolated within company intranet (unless really required to be external), which often means we need premium tier services in Azure. These are really, really pricey sometimes.
On the other hand, if I request a virtual server, it takes less than a week, and I can work with it much more freely.
Let’s not forget capex/opex. You can bill an aws project to a project, datacenter costs or shared across the entire org and it’s prohibitively complicated to budget these per project.
Sure you can do it cheaper, but money isn’t really the problem. Most cloud spend is less than the cost of a handful of senior devs a month.
The article was talking about solo devs, small businesses, and startups. A few physical servers that can handle millions of request per day.
In principle, you don't even need a "server", ie purpose-built hardware like what he's showing. An old laptop will be fine for many projects, although I'd be interested if someone can help put a perspective on how to decide whether your specific project/load will be too much for your specific hardware. My bigger concern with 'true' self-hosting would be how to get a good/reliable enough internet connection.
Our infra runs on 2 12-Year old shitbox Poweredges. so yeah most is just overkill.
Cool! It's still purpose-built hardware, but cool to hear such stories. Do you use colocation in a data center?
No we don't. Demand isn't that high, we don't need to serve thousands of users. It's just manuals, firmware downloads, datasheets etc. For customers.
Even if you don't end up switching, the lesson from this person's experience and the comments here on HN is: Stop being afraid.
People make all sorts of excuses: it's not secure, it's not reliable, it doesn't scale. These are primarily people who have never touched a server. If they had, they would know actual tradeoffs instead of excuses. So go and touch a server and learn how it works.
Jeez, this was a painful read. I actually stopped after a few paragraphs and asked AI to make it more technically focused and remove the ranting so I could stomach it.
Strawman arguments, ad hominem attacks and Spongebob mocking memes, and the casual venturing into conspiracy theories and malicious intentions...
> Why do all these people care if I save more money or not? ... If they’re wrong, and if I and more people like me manage to convince enough people that they’re wrong, they may be out of a job soon.
I have a feeling AWS is doing fine without him. Cloud is one of the fastest growing areas in tech because their product solves a need for certain people. There is no larger conspiracy to keep cloud in business by silencing dissent on Twitter.
> You will hear a bunch of crap from people that have literally never tried the alternative. People with no real hands-on experience managing servers for their own projects for any sustained period of time.
This is more of a rant than a thoughtful technical article. I don't know what I was expecting, because I clicked on the title knowing it was clickbait, so shame on me, I guess...
Is this what I'm missing by not having Twitter?
Just a bit of nitpicking. I am AWS certified, I still tell people not to do cloud. I even recommended as much to my current employer, laid out the costs for their architecture project doing on-prem, colo, and cloud (with multiple different cloud vendors), and they still chose AWS. At this point, the choice of cloud is nearly religious (in my observation).
Jumping away from the article slightly, where would you go if you needed a ton of vanilla disk space, the ability to do compute near to that disk, and for it to be accessible with a reasonable pipe to the public internet?
As far as I can tell, you'd almost have to do something with a colo if you didn't want to pay 10x or more for the storage. Are there other options?
You'd colo if you didn't want to use public cloud. The savings are good over time, but the upfront cost can be huge depending upon how much disk space you need. The setup/config can also be rough (many 4U boxes packed with tons of disks each, then CephFS + Rados for S3 api). You also want to have a few configured nodes on cold stand by, and many extra disks. Because of this, I've seen some companies take compute off the cloud but leave bulk data storage on the cloud.
Well, is "a ton" a level that can fit in a dedicated server or many? Just looking at https://serversearcher.com you've got 72TB of storage on 10Gb pipe for $360 or so a month: 128 GB RAM 6c/12t E-2276G 2X 512GB NVME + 4X 18.0TB HDD 300TB / 10Gbps
Just a few PB, and sharding it across servers is fine.
It looks like all the disk-optimized examples on that site (still much more expensive than paying for raw disk, barely 5x cheaper than S3, when a disk-optimized colo solution only has ~3% overhead over the disks themselves) are through some no-name provider "HostKey". I suppose beggars can't be choosers, but in the contexst of storage (where systemic failures should be accounted for in the model) are you aware of more than one provider with reasonably priced storage?
I mean on that site you also have clouvider who are cheaper $423.79/mo for 72TB including bandwidth, compute and two extra boot SSDs. I just searched by a minimum disk size of 12TB.
Colo will be cheaper I'm sure but it's fundamentally a different comparison you have to pay for drive failures, networking, bandwidth, remote hands, network switches and so on and so forth.
Interesting, not sure how I missed that (edit: I missed the 4x on the drive count). Thanks for mentioning it.
And, yes it's a different comparison, but that's why I was asking; I was curious if it was even viable (and initial searches had indicated it wasn't). 4-5x cheaper than S3 as a substrate is potentially workable.
Yeah, I mean thats what like ~$4.9/TB/Month including 4TB+ of internet transfer at a substrate level. So with say 10 servers on 8+2 parity you're looking at ~$6/TB/Month including >5TB of internet. Probably makes sense until you can fill at least one whole rack and buy a 2X100Gb internet connection.
Having my own server in a datacenter would be cool, buy it's hard to imagine having side projects that I'm willingly spending $1400 US/month on AWS for. Right now I'm still using the free tier on most platforms, plus a $14/month database on digital ocean that I'm not even using
There are many scenarios in which cloud providers (especially AWS) make sense.
Ideally, your company has technical experts who can do quite a lot of things non-cloud, so you can make informed decisions about near-term costs, complexity, vendor lock-in, execution speed, etc.
I'm especially a fan of cloud providers for early startups, which tend to be high on velocity, and low on workers. And the free credits programs often solve the early problem of being low on dollars.
Webserver is down
It is hilarious to read the title: "I took all my projects off the cloud [...]" and then you click the link it says: "Webserver is down".
It works just fine?
Love a nice 'single box project'.
But as soon as you start dealing with data sovereignty for your SaaS app, and need co-lo's in multiple geographies, each with their own contract/billing cycles, HA and backup, the global reach and consistency of the cloud providers starts seeming quite compelling again.
Great write-up :) I find hosting VPS on DigitialOcean/Hetzner to be the best middle-ground: you save a lot, but it still convenient; you just SSH into the thing, no need to think about physical cables, cooling, electricity and all the being physical baggage
This exactly what was happening with me and I recently started a project called Postbase to setup my own cloud infrastructure. There is so much free OSS software out there which is 100x better than what Google, Amazon and other offer. github.com/umrashrf/postbase (this is in alpha and work in progress).
As with everything, the real answer as to whether or not the cloud is cheaper is "it depends". These simplistic one-sided tirades (intentionally?) leave out a lot of detail, use simple rage-bait titles, and are generally not very thoughtful or coherent.
a simple valid point wrapped in an enormous amount of garbage arguments from both sides. watching idiots argue is exhausting
yes but most of dude's websites are not loading and some are even without SSL in 2025.. https://imgur.com/a/irWADuq
You won't believe it, but I can host 100 websites for a maximum of $100 per year.
It may seem strange to you, but PHP servers work very well.
I think it's worth a try.
The article featured The Server Store. It’s a north Texas company local to where I live. I have purchased all my personal computers from there for about the past decade.
I'm not sold because:
1. "It’s difficult to say these things without coming across as arrogant, but I’ve been managing servers since 2006."
You have experience manager servers, so for you it's not a big deal. But maybe software developers who aren't devops don't want to worry about it and just want to write code and ship a project?
2. If you're launching a new product as a solo developer, isn't it faster to just build an MVP on the cloud, validate the idea, get some traction, and eventually migrate to cheaper hosting if you do become the x% that gets product/market fit and do need to scale?
3. "Look, first of all, you’re as unique as the other 1000 peanut gallery enjoyers that have made the same astute observation before you. Congratulations. But you’re absolutely missing the point."
What's with the tone? Is it supposed to be funny? Does it help getting the point across?
With the use of chat bots and agents, managing these on prem should be easier than ever right?
Hetzner is great if you’re in Europe, is there an equivalent in the US or East Asia?
Not exactly like for like, but if you checkout comparison sites like https://serversearcher.com there are options that come close, like $6 for 4GB/2cores in many US locations.
OVH Cloud’s dedicated Eco servers
It seems like a common thread for de-clouders is they run RoR.
I’m fundamentally confused. A server in a datacenter is still the cloud. Taking something off the cloud means you have hardware somewhere that you physically manage and pay the power bill.
This is simply collapsing the built for scale tools into a cacophony of open source tools.
You didn’t need it to begin with, you just didn’t know any better.
I'm going to go way out on a limb and offer: whatever it is that you are, it isn't "confused."
Your response clearly shows that you didn't read the article. There's literally a section where it goes through your point.
The article is hugged to death. Maybe it wasn't hosted in the cloud?
Right, because it’s not possible for cloud services to get hugged to death.
Loads just fine?
Sorry but my $3 AWS instance is still cheaper than all of those options.
If you need a lot of, well, anything, be it compute, memory, storage, bandwidth etc., of course cloud stuff is going to be more expensive... but if you don't need that, then IMO $3/mo on-demand pricing really can't be beat when I don't have to maintain any equipment myself. Oracle also offers perpetually free VM instances if you don't mind the glow.
With a quick LLM-assisted search, looks like the cheapest EC2 instance is t4g.micro, which comes in at $2.04/mo. It has 2 vCPUs and and only 512MiB of RAM. (I assume that doesn't include disk; EBS will be extra.)
I can certainly see a use for that small amount of compute & RAM, but it's not clear that your level of needs is common. I've been paying for a $16/mo VPS (not on AWS) for about 15 years. It started out at $9/mo, but I've upgraded it since then as my needs have grown. It's not super beefy with 2 vCPUs, 5GiB of RAM, and 60GiB of disk space (with free data ingress/egress), but it does the job, even if I could probably find it cheaper elsewhere.
But not at Amazon. Closest match is probably a t3.medium, with 2 vCPUs and 4GiB RAM. Add a 60GiB gp2 EBS volume, and it costs around $35/mo, and that's not including data transfer.
The point that you're missing is we're not looking for the cheapest thing ever, we're looking for the cheapest thing that meets requirements. For many (most?) applications, you're going to overpay (sometimes by orders of magnitude) for AWS.
You say "if you need a lot", but "lot" is doing a bit of work there. My needs are super modest, certainly not "a lot", and AWS is by far not the cheapest option.
I run heaps of services on AWS and my bill is ~$2-3 - I'm not running any EC2 instances at all. Some of the offerings these cloud providers offer are extremely affordable if you know how to play your cards right and use the right services.
Just get a raspberry pi and run it from your own home internet. You should already be paying for a VPN service and your regular internet service, so you should be able to trivially work out a self-hosted solution. You'll recover your costs inside of two years and come out the other end better off for it.
Don't give the big cloud companies an inch if you don't absolutely have to. The internet needs and deserves the participation of independent people putting up their own services and systems.
Amazon really doesn't care if your $10,000 bed folds up on you like a sandwich and cooks you when AWS us-east-1 goes down, or stops your smart toilet from flushing, or sets bucket defaults that allow trivial public access to information you assume to be secure, because nobody in their right mind would just leave things wide open.
Each and every instance of someone doing something independently takes money and control away from big corporations that don't deserve it, and it makes your life better. You could run pihole and a slew of other useful utilities on your self-hosted server that benefit anyone connected to your network.
AI can trivially walk you through building your own self-hosted setups (or even set things up for you if you entrust it with an automation MCP.)
Oracle and AWS and Alphabet and the rest shouldn't profit from eating the internet - the whole world becomes a better place every time you deny them your participation in the endless enshittification of everything.
How do you even get a static IP these days?
If you’re going to write a post about why self-hosting is better than cloud*, then it’s probably a good idea to make sure your site loads in under a minute.
* at least I assume what this post is; I’m still waiting for it to load.
Loaded instantly for me. :)
sounds like the site isn't able to guarantee reliable service but it works for some people in an unpredictable way
Vercel is my favorite.. They charge you to pay for AWS.
It's an informative post but I really dislike the language and style that are becoming common in this kind of posts, e.g.:
> Look, first of all, you’re as unique as the other 1000 peanut gallery enjoyers that have made the same astute observation before you. Congratulations. But you’re absolutely missing the point.
Local first! Someone else said, "You don't actually need any of those things until you no longer have a "project", but a business which will allow you to pay for the things you require." -- Similar response I have for clients and engineers when they waste time/energy/money on "scale" when we are building a prototype... "scale is a luxury that comes with revenue."
Im sure some of you may quip back, "it takes money to make money". No shit. Just dont get high on your own supply... Be smart, be scrappy, slow down.
Up to a point, cloud can actually be very cost effective. If you are a small company, you don't actually need a lot. I ran our current service for almost one and a half year on Google Cloud Run for free. It stayed below the freemium threshold. Eventually I transitioned it over to two simple VMs. Main reason for this was that I abused these servers to also do background processing and Cloud Run is not very suitable for that as it can shut down servers in between requests and/or starve them of resources. But a small vm only costs something like 20$/month. Add another 25 or so for the load balancer and a Redis, and a few other bits and we're looking at ~100$ per month. That's very reasonable. That's not a cost worth optimizing without a very good reason. A good devops freelance person asks more per hour than that.
Where a lot of companies go wrong is premature scaling and overengineering. As you might guess from the above, I don't run kubernetes. Reason, the smallest cluster I can manage would probably be more expensive than the entirety of my setup. Before I even deploy anything. And it might not be big enough even.
I also don't run a managed postgresql. I'd like to. But I can't justify dropping north of 300$/month on that currently. A single node vm with backups is good enough. For now. I might go active master - passive master. Do I need a 3 or more node cluster? Not right now. I might one day. I always have the option to upgrade. But until then my DB is going to not break the bank. So, I run with a more modest/risky setup. It's a managed risk. I can buy it off when that becomes relevant.
The other day, our CEO vibe coded a cost calculator thingy. He did a nice job. It had to be deployed somewhere. So, I used Cloud Run. Running a small thing with essentially no traffic is as close to free as you can get. I slapped some basic authentication on it and used a git hub action that we run from the freemium allocated build minutes each month. I reviewed our Github bills a few days ago over the years: The most we payed was a few months where we were running out of build minutes in 2024. I payed 55cents for one of those bills. Our setup is cloud based but frugal. I could probably lower our bills by double digit percentages by moving everything into Hetzner. But it's literally not worth my time.
Cloud on the cheap is just really good value. You can host low traffic static websites on Cloudflare. If you have a load balancer in Google cloud (25$/month or so) you can use buckets and their CDN for next to nothing as an alternative. A small Redis costs next to nothing. We run a slightly expensive vm for batch jobs for a maybe ~2 hours per month in total. Each job only takes a few minutes. The machine is suspended in between jobs and we only pay for the hours it runs. Next to nothing.
So, yes, you can get some bare metal in Hetzner or wherever for next to nothing. But then you need to manage that setup and you need to put the time in to do that. Your time even if you don't value it personally is just about the most scarce and expensive resource a startup has if you are the CTO. The difference between you doing valuable work that moves the company forward or you procrastinating and penny pinching over stuff that does not matter can literally make or break the company.
If you spend 5K on infrastructure per month, that's a the same as a very low salary for a full time person. If you are spending that much, definitely do something because it's an obscene amount. Especially if you are early stage and have no traffic. But if you need to put somebody on that that costs closer to 10K/month for a few weeks or months, that's a project that has a pretty high price tag as well. So cutting 5K down to 1K is worth doing. But spending 10K to cut 1K to 500/month is probably going to take you years to earn back. And you lose time doing it.
Our startup is bootstrapped. We run on a bit of revenue now and our savings for most of the past five years. Anything I spend on stuff that isn't needed literally is money that otherwise would go to my bank account (via a salary raise or bonus). I understand frugality. A lot of this stuff is people getting spoiled with investor cash and spending like there is no tomorrow. If you have no accountability you optimize the infrastructure for how good it makes your resume look, not for cost.
Another complement to easily moving data off cloud, check out https://github.com/slingdata-io/sling-cli. Use CLI, YAML or Python to run etl jobs.
> Most people complaining about what I did happen to have “devops”, “cloud engineer”, “serverless guy”, “AWS certified”, or something similar in their bio.
“It is difficult to get a man to understand something, when his salary depends on his not understanding it.”
― Upton Sinclair
Good topic, horrible ai emdash slop. There's so much fuss here that I wouldn't be surprised if it was just a prompt without any reviewing.
I'd much rather read an article to know when it's NOT relevant to switch. Someone other than "vendor/service lock-in". I already know that
I see articles likes these popping up recently quite a bit.
I love it!
FUCK CLOUD!
> You’ve been scammed, likely not out of your own money, but your employer’s money – and that’s why you don’t give a crap.
No, I've ran the numbers, and in doing so, realized that EC2 is the wrong solution for most problems. If you use the correct solution the price is highly competitive with your servers. One of the reasons being, I do not need to maintain servers.. I did that for 25 years and I absolutely hate it. The amount of time I save, which my company pays for, is ridiculous. I get a powerful security model, a unified API for managing everything, and expertly managed runtimes.
You're acting as if people only making $4k/mo in salary are making these decisions. You've got startup brain. Come back down to earth and reassess your position.
> you’re looking to save for the long run. Well, turns out buying your own servers is even cheaper in a long enough time frame
Yet another process I don't have to waste time on. "Capital purchasing" is extremely annoying (by design) in every single organization I've ever worked for or consulted with.
Like most "cloud cost analysis" this is extremely shallow and is apparently based upon the supposed "virality" of a single tweet.
yet another obsessive take on "cloud is bad and expensive" eh? I think they vastly forget the value of some SaaS offerings in terms of time saving for small companies. running and managing numerous DBs, k8s clusters, ci/cd pipelines and stateless container systems is simply impossible with a team of 1-2 people. sure if the setup is simple and only requires a few classic components, this is way cheaper and for a 99.9% SLA will work fine. otherwise it only makes sense if you had very large cloud bills and can dedicate multiple engineers to the newly created tasks.
> running and managing numerous DBs, k8s clusters, ci/cd pipelines and stateless container systems is simply impossible with a team of 1-2 people
Then don't. If your team and budget are small enough not to hire a sysadmin, then your workload is (almost certainly) small enough to fit on one server, one Postgres database, Jenkins or a bash script, and certainly no k8s.
Not agreeing/disagreeing with your core point, but this doesn't seem right:
> running and managing numerous DBs, k8s clusters, ci/cd pipelines and stateless container systems is simply impossible with a team of 1-2 people.
That's a medium to large homelab worth of stuff, which means it can be run by a single nerd in their spare time.
Homelab =/= Production systems
The gulf between these two insofar as what approach, technologies, and due-diligences are necessary is vast.
I think we've gone a little nuts defining "production system" these days. I've worked for companies with zero-downtime deployments and quite a lot of redundancy for high availability, and for some applications it's definitely worthwhile.
But I think for many (most?) businesses, one nine is just fine. That's perfectly doable by one person, even if you want, say, >=96% uptime, which allows for 350 hours of downtime per year. Even two nines allows for ~88 hours of downtime per year, and one person could manage that without much trouble.
Most businesses aren't global. Downtime outside regular business hours for your timezone (and perhaps one or two zones to the west and east of you) is usually not much of a problem, especially if you're running a small B2B service.
For a small business that runs on 1-3 servers (probably very common!), keeping a hot spare for each server (or perhaps a single server that runs all services in a lower-supported-traffic mode) can be a simple way to keep your uptime high without having to spend too much time or money. And people don't have to completely opt out of the cloud; there are affordable options for e.g. managed RDBMS hosting that can make maintenance and incident response significantly easier and might be a good choice, depending on your needs.
(Source: I'm building a small one-person business that is going to work this way, and I've been doing my research and gaming it out.)
One thing that AWS, Google and Azure do that your own systems don't is release their updates whenever it suits them, often taking down your business down in the middle of the day with their own problems. You can't fix it, you can't rollback what you just did and get back up and running you just have to sit and wait.
That is quite different to a business that turns off its boxes for an hour at 0100 Sunday morning to do updates and release new software. The downtime isn't equivalent because it really matters when it is and if that hurts your use case or not. Your own system might be down for more hours a year than AWS, but its not down Monday to Friday on an evening when you do most your sales because you refuse to touch anything during that period and do all the work outside that and schedule your updates.
It also feels like AWS (or Azure) isn't really that much more reliable than your own thing. But half the internet is down at the same time so you don't get blamed as much.
Its the "No one gets blamed for going IBM" thing in the modern era. They are making it someone elses fault and absolves the blame. The problem is if your competitor is still up you could be loosing customers on average mid day outage, even if they are down for 3x as long its not when it matters.
I agree. "production system" isn't a useful term without context.
Either it's production system in the context of "it's a business serving customers", in which case, there are many homelabs out there which have received paying traffic, or it's a production system in terms of functionality, downtime, technical features etc. Again, there are many homelabs out there that can tick all the same technical and performance boxes as "production system".
In the context of the original comment I was responding to ("...running and managing numerous DBs, k8s clusters, ci/cd pipelines and stateless container systems is simply impossible with a team of 1-2 people.") you're more evidence for my comment that it's perfectly possible for 1 - 2 people to run a small production system, just as it's possible for 1 nerd to run a medium - large homelab with the same technical features.
Your production is not my production. 5 9's of businesses don't need 5 9's, and in my experience, people who talk like they do are talking from a position of managing 1 or 2 production systems in a single domain.
My experience is that the largest regulated production systems I've run have clearly been larger and more complex than my homelab, but my homelab has been significantly more resilient, featureful and robust than many of the smaller production systems I've been responsible for outside of regulated domains.
I'm sorry but whereever I've seen aws at work there was a sprawling terraform codebase to manage it. This no different than puppet or ansible on bare metal complexoty-wise, you just pat extra for the shiny tools
"I FINALLY got everything off the cloud"
...
...
"P.S. follow me on Twitter"
So uh, not everything
It’s probably the whole point, so he can post later how he paid the $120 from twitter income and now running it for free :)
Idiotic piece - the purpose of 'the cloud' is to scale large demand applications. Rental hardware can't really do that.
The post is about that 99% of companies that will never go large scale. Its point is that they don't need cloud, buying a server or two is all they need.
I once worked at a bookmaker with millions of customers where they had extreme reliability demands during very popular events, since any bets that the punters couldn't place during live betting was lost money. They had one massive server with a big pipe and an identical hot spare in another location. It worked very well for them. That was a decade ago, you can get even bigger servers these days, there's a lot of room for scaling without needing to go beyond one machine.
An argument which begins by reducing an entire industry down to a single "purpose" is not convincing.
The vast majority of businesses are not "large demand applications".
> Idiotic piece
That's unnecessary; please don't do that here. Weird that you created an account just to post an unsubstantive comment.
You can automate scaling rental hardware too