The Sunny Gardener - August 2024

Mastodon Update

On Aug 18 sunny.garden was updated to Mastodon v4.2.11, which contained mostly minor fixes.

One of the fixes was to an existing feature that allows mods and admins to message their own local users even if those users have blocked them; amusingly however, a bug in the fix resulted in it being impossible for admins or mods to send messages to their own local users at all.

The update was rolled back once the bug was discovered and, a day later, updated again… this time to v4.2.12 which included a fix for the local admin/mod messaging bug.

Hardware Upgrade

TL;DR

• After one aborted upgrade attempt which turned out to be more of a downgrade, sunny.garden has now upgraded from a shared virtual server to a dedicated server!

Back in July I added a secondary worker server to sunny.garden to take some of the CPU load off the main server, and save a little bit of cost on backups at the same time. This still left another problem to be addressed, which was the growing database filling up the server’s storage…

The server disk has about 160GB of disk space, with the database currently using about half of that, about 80 GB, plus 20 GB for caching media files, 7 GB for Elasticsearch, then there’s the operating system itself, Mastodon software, log files, etc… and we were down to about 28 GB of free space.

The database is currently growing by about 200MB per day (that is just the text of posts, boosts, favourites, etc., media files are stored separately), so it would be about 4 months until we were actually in danger of filling up the drive completely, but things start getting dicey earlier than that.

Running tasks like freeing up unused database space (“vacuuming” the database) can require many gigs of free storage in order to carry out, not to mention needing space temporarily while upgrading software, performing backups, and so on. Letting the drive fill up too far makes things more difficult overall.

Past Upgrades

Our hosting provider OVH has a few different kinds of servers available. There are dedicated servers (you rent an entire physical computer), classic “Virtual Private Servers” (you rent a “virtual” server which shares it’s RAM, CPU, disk etc. with other customers), and the more modern “Cloud Servers” which are also shared like VPS, but are billed by the minute, come in a wide variety of sizes, and can be created and deleted quickly, without any rental commitment.

sunny.garden has been running on a VPS service for the past couple of years. The pricing is pretty good, you get more capacity for your dollar than with the Cloud servers, and don’t need to pay for an entire dedicated server when that is more capacity than you need.

I originally started on one of the smallest VPS sizes, and as sunny.garden has grown, I’ve upgraded to larger sizes. The problem is, we’ve basically reached the highest offered VPS size, there’s no larger VPS configuration available.

Upgrade Attempt #1

OVH offers the ability to add an “additional disk” to a VPS, which sounds just like the kind of thing we need, however… they don’t provide any information on the specifications or performance that can be expected from this additional storage space. I was aware this was something that might be an issue, but didn’t realize just how big of an issue.

I ordered an “additional disk” to go along with a new VPS, and began setting up the new server with the intent to switch over on August 28.

When I had gotten far enough in the setup to begin running some tests, I quickly realized that the additional disk was much much too slow. Operations which took a few seconds on the old server took an hour. Certain pages in the Mastodon settings wouldn’t load at all, because the database took too long to respond and the web page timed out.

This wasn’t going to work, and I aborted the switch over to this new machine.

Upgrade Attempt #2

So we’ve outgrown the VPS services, it’s time to look at the alternatives: cloud and dedicated servers.

Cloud servers come in a wide range of sizes to serve nearly any purpose, so taking a look at the closest configuration to what we need (~ 8 CPU cores, ~ 500 GB high-speed disk space) that works out to… CAD $277/mo.

Compared to our current VPS costing CAD $69/mo. Yikes.

Ok, so that doesn’t seem so appealing, time to look into dedicated. The two main reasons I hadn’t considered dedicated servers until this point were:

1. Since you’re renting an entire computer, the lowest priced options are quite a bit higher than the smallest VPS or cloud servers.
2. There is a “setup fee” equal to one month service if you want to pay monthly instead of a yearly contract, which makes them quite expensive to evaluate on a trial basis.

But the time has come, sunny.garden has grown to the point where we can benefit from the full capacity of a dedicated server without that processing power being “wasted”.

OVH’s entry level dedicated server RISE-1 provides 12 CPU cores, 32 GB of RAM, and 512 GB of high performance disk space, for CAD $84/mo. (inc tax) on a 12 month commitment.

That’s much better value compared to the comparable cloud offering, and even compared to the current VPS, especially considering that with a dedicated server, those resources are not shared with other customers. When performing high CPU and disk use tasks on VPS and cloud servers, it’s often the case that you only end up able to use about 50-75% of what is theoretically available, and a lot of time is spent waiting for those shared resources to free up.

So that’s what I’ve gone with, on a 12 month contract. I’m relatively confident that it will be more than enough capacity for at least that period and possibly beyond, and I can stop spending time trying to figure out how to squeeze the database into too small a space. At least for another year or two.

I set up the new dedicated server – relieved to find that the performance overall was very much an improvement – and made the switch over on Sep 2nd.

Given the increased specs, and the fact that the resources are dedicated instead of shared, means we have a lot more breathing room than a month ago, system load wise.

Summary

August continued July’s trend of moderate user growth, and overall activity generally level.

Hosting Costs

Overall hosting costs have risen by about $20/mo. due to the expanded server capacity this month and last. I’m relatively confident the current setup will carry us over for at least the next year barring unforeseen circumstances.

As always, thank you to everyone that has donated on ko-fi! There’s no obligation or expectation that you donate in order to use the server, and I’m still able to cover any shortfall myself if needed.

– Brook