Sending comments sometimes takes actual minutes and loading the site takes a bit long too… is this due to the Reddit migration or cuz I’m european?
Sending comments sometimes takes actual minutes and loading the site takes a bit long too… is this due to the Reddit migration or cuz I’m european?
Unfortunately, a Lemmy instance currently scales vertically.
To stop an instance from being overloaded, it needs more CPU/RAM on a single server.
Lemmy’s horizontal scaling comes from more instances federating with eachother. Which is why there are a lot of comments and posts about “please pick a quiet instance”.
I know people are trying to get a single instance to work across multiple servers, which would allow for load balancing and dynamic scaling.
However, at the scale Lemmy is currently operating at: throwing bigger hardware at it is easier than getting Lemmy to autoscale and use more hardware.
I imagine in a month or so, solutions will be developed to make horizontal scaling more accessible.
Do the protocols lemmy uses not allow for that?
The protocols Lemmy uses rely on additional instances and federation to scale horizontally.
That’s kind of the point of the fediverse. There shouldn’t be big instances.
The issue is a lot of users want to be on the busy instances.
Whilst it shouldn’t matter which instance you actually join and use, some instances might have community/moderation that aligns with how you want to experience Lemmy (eg beehaw)
Lemmy hasn’t been developed for a single instance to scale horizontally. Throwing bigger hardware at it is the correct way to implement scaling when a project is this size/maturity.
Having stateless middleware, running caches, sharding databases, database replication, read/write load balancing on databases, having the actual front end load balancers…
It’s a difficult problem. Companies have entire teams that work on this, and it requires a lot of skill and attention to keep all the parts working correctly, and ensure things are fault tolerant.
Most instances are run by volunteers and community funding.
Like I said, hopefully Lemmy will move to a format that allows for easier scaling. But it’s a lot of work.
There is probably more value in squashing bugs, improving user experience, adding some well-needed features, and any optimizations they find along the way - than there is in rebuilding the stack to support horizontal scaling.
Remember, Lemmy has a core team of 2 developers.
And this massive influx of users became apparent at the start of this month.
It’s going to take some time, and things will be rough round the edges.