I left digitalocean for fly because some of their tooling was excellent. I was pretty excited.
I’m back on digitalocean now. I’m not unhappy about it, they’re very solid. I don’t love some things about their services, but overall I’d highly recommend them to other developers.
I gave up on fly because I’d spontaneously be unable to automate deployments due to limited resources. Or I’d have previously happy deployments go missing with no automatic recovery. I didn’t realize this was happening to a number of my services until I started monitoring with 3rd party tools, and it became evident that I really couldn’t rely on them.
It’s a shame because I do like a lot of other things about them. Even for hobby work it didn’t seem worth the trouble. With digitalocean, everything “just works”. There’s no free tier, but the lower end of pricing means I can run several Go apps off of the same droplet for less than the price of a latte. It’s worth the sanity.
I adore DO. They’re seriously underrated. I love how they’ll just give you a server and say here, have at it. No abstractions, no fancy crap, just get out of my way and let me do my thing.
I'm using Digital Ocean App platform, which does pretty much everything for me. It's very simple to use. I can run my app as a single developer without caring about infrastructure for 99% of the time.
Part of what inspired me to give fly.io a shot was that I didn’t love the monorepo deployment story on the app platform. Fly doesn’t have a solution to that, but I suppose I felt less tied to DO at the time because I wasn’t totally content anyways. I’ve discovered since then that I was actually doing it wrong, so I’m way happier. I’m pretty big on monorepos so their whole system fits my workflow remarkably well now.
I’d like to figure out how to prevent deployments when my code doesn’t change in one app, but does in another. At the moment, pushing anything at all will trigger all apps to rebuild and deploy again. Not a huge deal and several orders of magnitude less painful than not being able to deploy at all, haha.
In addition to Supabase Auth the sibling mentions (which I played with very briefly) I've been using clerk.dev (no affiliation) and it's great. Depending on your definition of doing it yourself it could be just want you want. You have to set some things up, you're not going to get things like row-level permissions you get out of the box w/ Supabase, but if you're looking for a quick implementation where things like password reset etc. are handled for you, it might be a good fit.
I've been using Supabase for authentication/authorization in my recent side project.
The main app is node/express running on Digital Ocean and it connects to directly to the Supabase hosted Postgres for most operations, but then uses the Supabase auth API for auth related stuff.
Saves a lot of time sending password reset emails etc and the entire project costs less than $5/mo in hosting costs.
I love their high value content about dev ops, I have learned most of what I know in this field tinkering with a VPS with their great tutorials on how to set up stuff.
They filled the Slicehost vacuum nicely in this area. That's where I got my start in running my own servers about 15 years ago and the tutorials were the driving factor.
Seriously! They have an amazing article I followed one time to set up a k8s cluster to run any container I wanted with full automatic ssl provisioning/management and dns. Make a quick little yml file that includes what subdomain it wants to be and kubectl apply. The cluster was like $100 a month all-in and performed like a beast at huge traffic levels, and all I did was follow a tutorial.
I know that’s probably pretty easy for many, but I was pretty new to k8s and it felt like magic.
I wish I could say the same. My ISP and DO have absolutely terrible peering, unfortunately a lot of our internal stuff is hosted there. It’s always fun to git push/pull with 40kb/s on a gigabit connection.
When I’ve run into this in the past Cloudflare Warp has been a bit of a saviour. It’s a hassle free way to flick a switch and follow a different path over the network.
I went to DO's site due to your comment and I don't see anywhere where I can just get a server. Do you mean a VPS/Droplet? (I'm looking under Products and Solutions.)
The other commenter was correct - I meant a droplet. Should have been more explicit, apologies. But yeah if you're looking to learn how to work with backends, going through a droplet set up is by far the best way to get started IMO.
historically, I've used Vultr, but I don't see anyone talking about it—I'm curious if anyone else has thoughts on them? (I've been happy, but then again my usage has been exceedingly basic)
I've used Vultr for several years (hobby projects) with no issues. My favorite feature is having a BGP session from my VM, which is unusual among cloud providers. I have an AS and am able to advertise my own IPs from multiple Vultr instances (anycast).
Have used both DO and Vultr for years. Put simply, DO is better, but Vultr isn’t terrible.
Higher number of outages at Vultr over 5 years, but none longer than a few hours. I can’t remember the last DO outage lasting more than a few minutes.
Experienced a Vultr routing problem that lasted several hours; they communicated about it, but it was still a long time to fix.
DO once did an auto-migration of a server to another cluster with an attendant outage that lasted a few minutes at most. No IP changes, completely transparent.
I love DO for projects where I don't need control. For my side project, I eventually migrated to AWS after running into a lot of issues with DO.
Things like they don't give you the postgres root user on their managed postgres. And I ran into issues trying to capture the deployments in code. Their terraform providers are pretty good, but still leave something to be desired. For all its many warts, I'm much happier back on AWS. It did end up more expensive, but it's worth it for the fine grained control in my case.
But I spent the last 5 years as a DevOps/SRE, so... uh... I'm picky.
That's interesting, because granular control is why I enjoy DO, although I'm thinking about it from the server perspective. They set up a machine, give me root access, and that's literally it. I set up my own ssh keys, firewalls, and there's no additional abstraction that I have to learn. I might just be reminiscing because right now I'm on a team where we're writing terraform/helm/k8s in GCP and it makes me want to cry myself to sleep each night lol.
Those are good things to know. I’ve been wondering about their managed databases recently, so I’ll keep that in mind.
I’m nowhere near as picky as you are, but maybe I’ll need to be at some point. As it is I mostly just build stuff and send it to the internet. If it builds and it does what I expected, I’m pretty happy! I don’t often need anything too special.
Same! I've had my first server there for 10 years now. They added a lot of stuff in the meantime, they have AWS-like things you can do. But in terms of launching a VM that just works, they are a great choice.
I agree. I can either abstract with the app platform or kubernetes, or I can go straight into the box myself and do whatever needs doing. It has been a real pleasure.
I think fly’s tooling feels better than doctl, but the infrastructure is incomparable at the end of the day. doctl has improved over time too, and with added pressure from newcomers I don’t doubt that it’ll continue to improve.
I find myself going to DO docs on various setup things even when I'm not using said thing on DO (although I'm also a DO customer, and love them for the reasons you've stated).
I've been with them for a long time and my guesses would be:
1. Strict rules and strict customer verification. Crypto mining that wastes SSDs is not allowed. Portscans, mass emails, etc. are not allowed. They also don't offer GPUs to the general public because it has been abused in the past. You usually need to send in ID documents just to open an account. My guess is this allows them to avoid most bad actors and, thereby, waste less money on fraud.
2. Extremely long-term investments. They typically build their own hardware and then use it over 10 years. They have their own flea market where you can rent older server models for a steep discount. That means they will have a long time where the hardware is fully paid off and still generating revenue.
3. Great service. With a mid-sized company, I can call their technicians in the middle of the night. The fact that we could call them in case of a crisis has generated A LOT of good will. But I would be truly surprised if they didn't make a profit off those phone calls, as they charge roughly 4x the salary cost.
4. High-margin managed services. In addition to just the cheap servers, they also offer a managed service where they will do OS and security upgrades for you. It's roughly 2x the price of the server and it appears to be almost fully automated. I know some freelance web designers who will insist on using Hetzner Managed for deployment for their clients, because it is just so convenient. You effectively pass off all recurring maintenance for €300 a month and your client is happy to have an emergency phone number (see #3) in case the box goes down.
They run their own data centres and have for a while. There is a pretty big industry for that sort of thing as an alternative to “the cloud” here in Europe.
We used to use nianet to house our hardware in Denmark. Basically these companies does hardware renting and they also do hardware renting with more steps which is where you rent rack space but own the hardware. They provide the place for the hardware and they also have multiple locations so that you have both backup and redundancy, and while it doesn’t scale globally in 20 years I’ve literally never worked on anything that needed to beyond having some buffer caches for clients logging in on their vacations or something like that.
What Hetzner seems to be doing with the DO styled hosting, and this is just a guess, is that they are one or the many EU companies preparing for the big EU exodus from the non-EU cloud. Which is frankly a solid bet these days where both AWS and Azure are increasing prices and are becoming more and more unusable because of EU legislation. Part of this is privacy which Microsoft and Amazon are great with in terms of compliance, but part of it is also national security. I work in an investment bank that builds solar plants, since finance and energy are both critical sectors we risk being told that half of the finance/energy companies in the world can’t use Microsoft because the EU seems it as a single point of failure if our entire energy sector relies on Azure. Which is sort of reasonable right? But what this means for us is that we can’t vendor lock-in, not really, because we need to have up-to-date exit strategies for how we plan on being fully operation a month after leaving Azure. Which is easy when you just containerise everything and run it in VMs or similar, and really annoying if you go full in on things like AKS. Which doesn’t help our Azure costs.
Anyway, right now we are planning on leaving Azure because of cost. Not today, not next week but sometime in the next 5-10 years and a lot of these EU cloud alternatives that actually operate the hardware instead of renting it are likely going to be a very realistic alternative. And that is the private sector, I spend time in the EU public sector which is a massive amount of money and I’m guessing it’ll leave both AWS and Azure by 2050. Some of these EU cloud initiatives is going to explode when that happens, and right now, hetzner is one of the best bets.
To get back to your question, DO rents server space. I have no idea where they’d rent it in Germany but they could potentially be renting it from Hetzner.
Couldn't agree more, I think Hetzner is probably Europe's best bet on a hyperscaler. One of the more telling indicators IMO is their growing market share outside of the EU/DACH.
To add on to the comments about Hetzner building their own custom hardware, they also custom built their own software stack. They rejected the hype that was OpenStack and worked diligently on their own hypervisor platform (that they are incredibly secretive about) and that appears to be paying off in spades for them. Most sovereign cloud plays end up being suffocated by the complexity, and incoherence, of the OpenStack ecosystem. It just becomes impossible to ship.
Could you please elaborate how and what you know about managed Kubernetes on Hetzner?
I am asking for this since a while and was told there is no way Hetzner would offer such a service. Certain Posts on Social Media have also never been answered with any kind of indication that they are actually working on it.
They were in person recruiting at KubeCon EU this year and were advertising a good number of Kubernetes engineering roles. Definitely gave me the impression they were taking Kubernetes seriously but looking back a managed offering was just speculation on my part.
So huge grain of salt, you are totally right. It could be internal platform work only.
I think you might misunderstand me. The 2050 is a guesstimate and it's just my opinion on the matter. As far as planning ahead goes, you plan for 5-10 years when you try to figure out where to "iron" your enterprise IT. This is because that's how long your hardware will last if you go the route of renting rack space with your own hardware. I think we tend to plan for 8 years, with some space for "unintended" early failures on things like controllers after 4 years. So while you can contract big-cloud vendors for shorter, I think ours is on 3 year contracts right now, you still sort of do the business case for much longer. Maybe not every 3 years, but at least every 6 years.
You do the same on the other side of the table. Companies like Hetzner knows that EU cloud sollutions are likely to see growth, so it's only natural that they invest in the tech to put themselves in a prime position to jump on the opportunity. Selling a good product while you do so is the way I would do it personally, but you also have EU cloud initiatives backed by VC money going straight for the endgame.
I think multi-national energy sector should be working toward the goals without the regulations. The more prep done before the change the smoother the transition.
Hetzner also do some crazy-cool stuff, especially around the 7950X3D, cooling, AM5 etc. (https://www.youtube.com/watch?v=V2P8mjWRqpk). They also do some amazing stuff with ARM (their cloud offering is really solid for this).
Not to mention a German company that has price sensitivity in their DNA. Their first servers were just regular consumer tower PCs to drastically cut hardware costs. Now many years later it's a highly optimized mix of consumer, server and inhouse parts (e.g. they use their own racking system instead of 19", and the datacenters are built to make use of convection for a lot of the cooling). They also offer regular Dell servers for those that want them, but at 2x-4x the price of their homegrown boxes.
Me and my partner have paid a visit to their datacenter in Nüremberg. The answer is efficiency. They get more processing power than the other providers for the energy they have to put in
pardon my ingorance but i cannot quite see how cooling individual machines vs. the hole rack or row makes a difference in total heat production per machine
Simple, Hetzner mainly operates on Germany, the people are mostly Germans, and they automate the stuff to a point a small team could manage it well even if not remotely, so they have less cost on human resources.
> Simple, Hetzner mainly operates on Germany, the people are mostly Germans, and they automate the stuff to a point a small team could manage it well even if not remotely, so they have less cost on human resources.
I feel like there might be more to it, especially considering the situation with electricity prices in some places in EU recently.
I used (and still use) a Lithuanian platform called Time4VPS which was cheaper than Hetzner previously, yet had to increase their prices somewhat for that reason. Now only some of their plans are competitive with Hetzner, while Hetzner also provides some managed services as well.
And yet, I can't help but to wonder why they don't give in to the desire to maximize profit margins, like happened to say Scaleway (good platform, but as expensive as DigitalOcean).
Only complaint with Hetzner is they don't have some kind of OAuth setup for machines or scoped API tokens, just read/write. I'd like to use the former for doing Vault authentication from instances, and the latter for writing a dynamic Vault secret provider.
They require passport or some sort of ID on registration, and it is weird when compared to others. I was not happy with that part, but I am happy customer since (almost a decade now).
As far as I know, they do not require any ID when canceling the service.
Well I would appreciate that, since I was victim of russian hackers and they had access to all my servers and stuff on Hetzer, they even changed passwords and mail on Robot but i restored everything...
Yes and (unfortunately) no. Terraform providers are here [1] with the official documentation at [2]. Managed databases are not available, though. I think they have some sort of database offering if you select their web hosting options, but you can't just get a managed Postgres instance yourself.
EDIT: For what it's worth, I have had good experiences with app servers hosted on Hetzner Cloud and managed Postgres provided by ElephantSQL (https://www.elephantsql.com/) for Germany-based apps.
DO actually does have a free tier! If you use their “app platform” (their equivalent to fly/heroku/render/etc) you can host 3 “static” apps for free. So if you have a Hugo/Jekyll blog or something, it’ll set up a whole little CD system for it for free.
You’re totally right. I kind of forgot about this, in part because I’m over their free limit. I think their static sites are still dirt cheap once you hit that limit, though. I find their pricing totally reasonable for what I need.
I want to like Fly, but the reliability is one of those were I feel like every time I investigate moving workloads over I'm disappointed by these stories over and over again.
I’m back on digitalocean now. I’m not unhappy about it, they’re very solid. I don’t love some things about their services, but overall I’d highly recommend them to other developers.
I gave up on fly because I’d spontaneously be unable to automate deployments due to limited resources. Or I’d have previously happy deployments go missing with no automatic recovery. I didn’t realize this was happening to a number of my services until I started monitoring with 3rd party tools, and it became evident that I really couldn’t rely on them.
It’s a shame because I do like a lot of other things about them. Even for hobby work it didn’t seem worth the trouble. With digitalocean, everything “just works”. There’s no free tier, but the lower end of pricing means I can run several Go apps off of the same droplet for less than the price of a latte. It’s worth the sanity.