When you upload an image in the URL bar when creating a post, where does it go? Are they hosted on the respective Instance of the community?
yes, they are hosted on the server that is running the Lemmy instance. But every other instance that is linked to that instance will ‘scrape’ all posts incl images and store them on they’re own server. So posts and images are served from the instance you yourself are a member of
Maybe it doesn’t matter so much because image files are usually relatively small, but it seems kind of inefficient. I wonder if BitTorrent could help somehow.
lots of traffic on many small files eventually uses more disk space and bandwidth. Depends on the growth of the instance
That seems weird. So the more people are here and longer Lemmy instances (any of them) are up the more storage each instance needs? Soon you can’t have small instances because you will need tons of storage for all pictures that have ever been posted in any of the instances?
only pictures and posts that the instance-member are subscribed to, and from the moment they interacted with it. But yes, I’m curious to see where this goes.
I noticed it when I was looking in the database and disk on my instance, where there are already thousands of posts and more then a thousand images - while my own instance only has a couple of posts and images
But it still means if a single user on a small instance is subscribed to a massive media instance, they alone will generate tons of traffic and storage requirement… Seems really weird
Thats the way the decentralized fediverse works (I think!) let me know if I’m wrong
I run a Mastodon instance and the amount of storage space I need is ridiculous. My own instance doesn’t have a lot of media, but all the other instances that mine federates with do — so I currently have to pay for 500 GB of object storage.
As far as I can see, images aren’t federated, only a link to the image.
When I look on my server in the volumes/pictrs folder, there are a LOT of images that I recognize from my feed. They are not from posts posted to my server but belong to posts of communities that I have subscribed to
Are they thumbnails? If not, perhaps this is configurable… I’ll check out my own images folder and see if it’s pulling them in, but it definitely shows the remote url for images when I’m viewing them on my instance.
no, a random image I just now check was 3.3mb even. But you’re right, the image source does point to the origins server location… Then why are all those images stored on the server ?
OK, I’ve just realised something new - images pulled from websites as thumbnails are uploaded to the server and appear to be basically full sized, i.e.:
https://agora.nop.chat/pictrs/image/59cfdd9c-2f69-4e8d-8afc-6fbc0c1cca87.jpeg
This image is on my server, but it’s just pulled from the website as a thumbnail. This doesn’t seem to be super consistent in how it’s handled - I’m continuing to look at what’s causing it and why it doesn’t seem to happen all the time.
EDIT: Seems like there’s more on there than I expected. Gonna continue digging.
If you run the normal Ansible setup, there’s one docker container running pictrs that’s going to handle the image uploads. By default, it stores it into a volume that comes from the server’s filesystem. So when you upload an image, it gets stored to the server’s file system with the default settings.
Pictrs supports object storage, and you can get it to store the images to any S3 compatible storage.
Probably not a bad idea to enable S3 support
Btw, if somebody here makes an instance with pictrs sending data to S3, it would be really cool if you would document it and write a guide how to enable it with a common Ansible deployment. It seems that the docker image doesn’t take any environment variables, or at least there is no documented way how to turn on the object storage using the pictrs docker image. Maybe it’s not implemented yet, and somebody needs to fork the docker image…
I will update this thread if/when I find out more information regarding this.
If you tend to upload images/memes etc., yes, it’s a good idea. And very cheap (read: free) if you go with Cloudflare R2…
I’ve never used CF for object storage, I personally use Backblaze B2, it’s pricing is extremely fair. If anyone needs object storage, it is something to consider, I am not sure how it compares to cloudflare though. There unfortunately isn’t much documentation on config vars for object storage on lemmy/pict-rs I am still looking into this.
For most of the self-hosters here, Cloudflare will be free. I know many Mastodon and Akkoma users using Backblaze, so it’s probably a good option too. I use it for my backups, and but not with their S3 API.
My understanding is that images don’t federate by default and you’ll load the image from whichever server it was uploaded to initially. Uploading an image here in the comment certainly does that:
This post that shows in my instance loads it’s image from feddit.uk, even when it shows on my own instance rather than on this URL:
Looking at the federation docs, I’m pretty confident that a link to the image is the only thing going through. If someone uploads that on the server directly, that’ll be a link to that server, but it may be a link to Imgur or something too.
the docs don’t explicitly specify but you can see in the image url that they do end up on the instance you upload to
Does that cost the instance creator anything?
i mean, it depends on how they are hosting the instance, i guess. it would certainly take up storage space wherever they host the instance. you could always try to upload images somewhere else like imgur and then link it to help not take up their instance storage
I see. Thanks!
np 💙
yes. running a server costs money. I have lemmyfly.org running just 2 days now on digitalocean using a cheap 1GB ram, 25GB diskspace and 1000GB bandwidth for $7,- a month. Storing images on the server will eventually take up all space and bandwidth meaning you have to upscale -> pay more.
Using a different location for storing images and/ or videos is best to offload the instance !
