[ $davids.sh ] โ€” david shekunts blog

๐Ÿšข How Much Does It Cost to Play Around with Kubernetes? ๐Ÿšข

# [ $davids.sh ] ยท message #304

๐Ÿšข How Much Does It Cost to Play Around with Kubernetes? ๐Ÿšข

I set up my own hybrid k8s (that's cloud-based but on a VPS) for various projects and want to share the numbers I ended up with.

  • Minimum setup
  • Minimum resources
  • Price per single machine
  • Price for a cluster
  • What about managed solutions?
  • And, as always, how to beat the same old winner

The full article text is in the comments.

#k8s #devops #infra #cloud

  • @ [ $davids.sh ] ยท # 2006

    # Minimal Setup

    By my current calculations, to deploy:

    • 3 nodes (1 control plane + 2 workers)
    • Run a CNI on them
    • Prometheus stack
    • ArgoCD
    • 1 PG
    • 1 Redis
    • 1 NATS.io

    # Minimum Resources

    • 2 vCPUs, 4GB RAM for the control plane
    • 4 vCPUs, 8GB RAM for workers
    • ~200GB of attachable volumes
    • Internal subnet (thankfully, this is usually free)

    This will still leave 2 vCPUs and 4GB RAM for your applications.

    # Prices per Machine

    All shared CPU

    Control Panel (2vCPU, 4Gb)

    Hetzner โ€“ $4 (40Gb) Digital Ocean โ€“ $24 (80Gb) Vultr โ€“ $20 (80Gb) Linode โ€“ $24 (80Gb) AWS โ€“ $30 (! without storage) GCP (2vCPU 8Gb) โ€“ $50 (! without storage)

    Worker (4 vCPU, 8Gb)

    Hetzner โ€“ $8 (80Gb) DO โ€“ $48 (160Gb) Vultr โ€“ $40 (160Gb) Linode โ€“ $48 (160Gb) AWS โ€“ $120 (! without storage) GCP โ€“ $84 (! without storage)

    Volume 100Gb

    Hetzner โ€“ $5 Others โ€“ $10

    # Prices for 1 CP + 2 W

    Hetzner โ€“ $25 DO โ€“ $120 Vultr โ€“ $100 Linode โ€“ $120 AWS โ€“ $290 GCP โ€“ $250

    # Managed

    If you opt for managed solutions, on almost all platforms, you can easily reduce the cost by half to two-thirds.

    In Conclusion

    • If you want a DIY hybrid cluster โ€“ Hetzner is the absolute winner (even compared to managed).
    • If you need managed, I would try DO or Linode.

    I'd love to hear your stories about using on-premise / managed k8s servers, what you'd recommend, or if anyone has deployed on Hetzner and what your experience was like.

  • @ NP ยท # 2008

    Hetzner doesn't back up SSD storage, which is a big minus. I learned that the hard way once. Otherwise, I use and practice it.

  • @ Dima Web ๐ŸŒ ยท # 2009

    Hello to all foreigners) We use Yandex Cloud at home. It's expensive. ~$200 in total. Are we satisfied? โ€” Yep.

    Yandex Managed Service for Kubernetes

  • @ [ $davids.sh ] ยท # 2011

    Wow

    Do you mean Volumes? I remember this problem was N years ago, is it still happening? And you can't do both backups and snapshots?

  • @ [ $davids.sh ] ยท # 2012

    Ah yes, I completely forgot to mention that I was specifically talking about Bourgeois because I haven't touched Russian [services/companies] in a long time)

    But 3 years ago, we had managed k8s first on Yandex, then on Selectel, and it was absolute hell: from entire cluster crashes to RAID failures (hallelujah, we backed up to S3 on every release).

    Is it better now?

  • @ NP ยท # 2015

    Uh-huh, exactly them, and it seems like none of the listed things are allowed.

  • @ [ $davids.sh ] ยท # 2016

    So annoying...

  • @ Ivan ITK ๐Ÿšซ ยท # 2018

    Are you joking right now, or are you not being completely serious? What led you to those conclusions?

    Snapshots (also known as backups) are taken from the current main SSD disk (not including attached external block volumes) and are perfectly restorable to new instances or to the same one. This has always been the case as long as I've known Hetzner.

    Attached volumes do not have a built-in backup system because they have a different storage architecture. However, triple data replication is provided. Users must organize their own redundancy.

  • @ Dima Web ๐ŸŒ ยท # 2019

    Yes, everything is fine.

  • @ [ $davids.sh ] ยท # 2020

    The discussion is indeed about Volumes and the lack of snapshot functionality.

    Yes, triple replication greatly increases data storage reliability, but it doesn't, for example, allow for rollbacks.

    It's certainly possible to do it yourself, but it's interesting why they didn't provide a native way to snapshot volumes.

  • @ [ $davids.sh ] ยท # 2021

    And also, for example, it can be easier to make a snapshot of the DB storage during deployment, rather than making a dump from it (dumping a 250GB Pg is painful, and connecting and disconnecting a read replica is difficult/expensive).

  • @ NP ยท # 2022

    Yes, that's correct, we're talking about attachable Volumes.

  • @ Ivan ITK ๐Ÿšซ ยท # 2023

    And which provider has snapshots for block storage?

  • @ [ $davids.sh ] ยท # 2024

    Isn't there? I'll go double-check.

  • @ Ivan ITK ๐Ÿšซ ยท # 2025

    Only major ones like AWS, GCloud, etc.

  • @ Ivan ITK ๐Ÿšซ ยท # 2026

    This is an expensive pleasure in terms of infrastructure, and a heavy load on the channels to move hundreds of petabytes every day.