The Sunny Gardener - July 2024

TL;DR This month’s post is pretty techy, so if you don’t want to read all that:

• I was able to swap out an older and more costly backup server for one that is both cheaper and faster, to take care of both the backups, and reduce load on the main server.

Adding a Secondary Server

Around mid-month I noticed that system load on the server had increased to a point where it made sense to look into another possible upgrade.

The previous server upgrade was about six months ago, and while the local active user count is about the same, the server activity level has increased, likely due to a combination of gaining some more active users as well as overall increase in fediverse activity and connections between local and remote users.

There is some relief on the horizon, as the upcoming Mastodon v4.3 release is expected to bring optimizations in image processing, which should reduce CPU load on the server, based on what I’ve seen reported by other admins running nightly builds.

Since I don’t know when that release will be arriving, I still wanted to take some of the pressure off short term.

Main Server

Without going into too much detail “a Mastodon server” is actually a collection of multiple programs, which can run all on one computer, but they can also be separated out onto multiple computers so that each component has dedicated hardware to run on.

Those components are:

• the web server (runs the website and accepts connections from other servers)
• sidekiq workers (lots of things like sending messages and downloading media)
• the streaming server (sends new messages to users as they arrive)
• a redis database (coordination between components)
• a postgresql database (stores accounts and posts)
• a media server (stores all of the images and videos)
• optionally, an elasticsearch database (provides local searching)

Currently, the majority of the system load on sunny.garden is a combination of the web server and sidekiq workers.

New Worker Server

Since the sidekiq workers are one of the least disruptive components to separate out, that’s what I’ve moved onto a new secondary worker server.

The new worker server has only 2 CPU cores and 2 GB of RAM, compared to the 8 cores and 16 GB of the main server, but they are dedicated entirely to running sidekiq and not shared with the other components. This has made a considerable improvement to the system load on the main server, which is running comfortably once again.

Now that the separation has been set up, it’s possible to easily add or remove sidekiq resources without any interruption to the main server at all.

I’ll go into the costs for this down below.

Summary

Active users up somewhat, and overall activity noticeably higher this month.

Hosting Costs

I do my best to keep the cost of running the server down, not so low that things will bog down on a busy day, but not so high that there are wasted resources. I don’t want to pay for – or ask users to donate towards – server power that is not being put to good use.

It’s one server, Michael. What could it cost?

The additional cost of the new worker server is $7 per month, but conveniently after this month, this should be offset by a $7 reduction in the cost of running backups. This is because I was previously using a secondary backup server just to run backups, but the new worker server can do that job as well, with minimal effort.

The new worker server is one of a line of new offerings from our hosting provider OVH which provide twice as many CPU cores for similar or lower prices than were available before.

As always, thank you to everyone that has donated on ko-fi! There’s no obligation or expectation that you donate in order to use the server, and I’m still able to cover any shortfall myself if needed.

– Brook