Hello! I have been struggling through a few tutorials on getting a lemmy instance to work correctly when setup with Docker. I have it mostly done, but there are various issues each time that I do not have the knowledge to properly correct. I am familiar with Docker, and already have an Oracle VPS set up on ARM64 Ubuntu. I already have portainer and an NGINX proxy set up and working okay. I have an existing lemmy instance “running” but not quite working. My best guess here would be to have someone assist with setting up the docker-compose to work with current updates/settings, as well as the config.hjson.

TIA, and I cant wait to have my own entry into the fediverse working right!

  •  I eat words   ( @saint@group.lt ) 
    link
    fedilink
    English
    111 months ago

    try to look here for the config file:

    include /etc/nginx/conf.d/*.conf;
    	include /data/nginx/default_host/*.conf;
    	include /data/nginx/proxy_host/*.conf;
    	include /data/nginx/redirection_host/*.conf;
    	include /data/nginx/dead_host/*.conf;
    	include /data/nginx/temp/*.conf;
    

    btw, i think port in lemmy.bulwarkob.com:1236 in docker-compose is not needed for you, should be just lemmy.bulwarkob.com

    •  Jattatak   ( @Jattatak@beehaw.org ) OP
      link
      fedilink
      English
      111 months ago

      I appreciate your patience and clear assistance.

      conf.d/* has two configurations that appear to be some form of default. default.conf and production.conf. production.conf is only for the admin GUI. default.conf:

      Container has a volume set /lemmy/docker/nginx-proxy-manager/data:/data

      I have those folders and more, and they DO seem to have the correct custom item.

      Specifically, in the proxy_host folder I have a configuration for the proxy host I set up (1.conf) in the GUI:

      spoiler
      
      # ------------------------------------------------------------
      # lemmy.bulwarkob.com
      # ------------------------------------------------------------
      
      
      server {
        set $forward_scheme http;
        set $server         "172.24.0.5";
        set $port           1234;
      
        listen 80;
      listen [::]:80;
      
      listen 443 ssl http2;
      listen [::]:443 ssl http2;
      
      
        server_name lemmy.bulwarkob.com;
      
      
        # Let's Encrypt SSL
        include conf.d/include/letsencrypt-acme-challenge.conf;
        include conf.d/include/ssl-ciphers.conf;
        ssl_certificate /etc/letsencrypt/live/npm-1/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/npm-1/privkey.pem;
      
      
      
        # Block Exploits
        include conf.d/include/block-exploits.conf;
      
      
      
      
          # Force SSL
          include conf.d/include/force-ssl.conf;
      
      
        access_log /data/logs/proxy-host-1_access.log proxy;
        error_log /data/logs/proxy-host-1_error.log warn;
      
      
        location / {
      
      
      
      
      
          # Proxy!
          include conf.d/include/proxy.conf;
        }
      
      
        # Custom
        include /data/nginx/custom/server_proxy[.]conf;
      }
      
      

      The rest of the folders are empty:

        •  Jattatak   ( @Jattatak@beehaw.org ) OP
          link
          fedilink
          English
          311 months ago

          I actually started with this tutorial a few days ago after failing the official guide. I followed it but was unable to get it running due to unexpected errors. Im guessing this tutorial is somewhat out of date. Ive made progress since using that guide though so I will see if I can pull any useful bits out of it later today and continue.

          Worst case, I could also just ditch NPM if I can get another NGINX set up in a way that you might know how to do correctly.

        •  Jattatak   ( @Jattatak@beehaw.org ) OP
          link
          fedilink
          English
          111 months ago

          Hey, if you still feel like helping out :D

          Ive been through a boatload of changes today since earlier. Ive rebuilt using mostly the provided yml in the official guide, and after some tweaking, almost everything is working. The internal proxy is now working, and the containers are working amongst themselves fully as far as I can tell. I do not know how to setup a web facing reverse proxy in a way that works around the internal proxy already running (other than the already in place NPM). I turned the NPM back on, and was able to get it working to reach the site, however I cannot reach any other communities from within my site. I believe the reverse proxy NPM is just not set up right. Error message in lemmy:

          spoiler
          ERROR HTTP request{http.method=GET http.scheme="https" http.host=lemmy.bulwarkob.com http.target=/api/v3/ws otel.kind="server" request_id=69004ca6-7967-48c3-a4d2-583e961e34d3 http.status_code=101 otel.status_code="OK"}: lemmy_server::api_routes_websocket: couldnt_find_object: Request error: error sending request for url (https://midwest.social/.well-known/webfinger?resource=acct:projectzomboid@midwest.social): operation timed out
          
             0: lemmy_apub::fetcher::search::search_query_to_object_id
          
                       at crates/apub/src/fetcher/search.rs:17
          
             1: lemmy_apub::api::resolve_object::perform
          
                     with self=ResolveObject { q: "!projectzomboid@midwest.social", auth: Some(Sensitive) }
          
                       at crates/apub/src/api/resolve_object.rs:21
          
             2: lemmy_server::root_span_builder::HTTP request
          
                     with http.method=GET http.scheme="https" http.host=lemmy.bulwarkob.com http.target=/api/v3/ws otel.kind="server" request_id=69004ca6-7967-48c3-a4d2-583e961e34d3 http.status_code=101 otel.status_code="OK"
          
                       at src/root_span_builder.rs:16
          

          I would be happy to remove NPM from this stack if its not too difficult to get a correctly working reverse proxy set up. The documentation doesnt give much to work with in it.

          •  I eat words   ( @saint@group.lt ) 
            link
            fedilink
            English
            111 months ago

            from the log it seems that lemmy cannot reach https://midwest.social/ - if you have more such operation timed outs - probably there is some networking issue with outgoing requests - maybe you have some kind of firewall? i can reach your instance from other direction: https://group.lt/c/bulwarkob@lemmy.bulwarkob.com

            probably the easiest way to setup lemmy and another front facing reverse proxy is to use nginx that comes with lemmy on another port and setup simple reverse proxying with NPM to it. i myself using caddy for reverse proxying, using this config: https://join-lemmy.org/docs/en/administration/caddy.html

            •  Jattatak   ( @Jattatak@beehaw.org ) OP
              link
              fedilink
              English
              111 months ago

              I see that the instance can be reached, and posts are shown, however comments are not. I have found in the official docs that there is a config snippet for a web facing reverse proxy. https://join-lemmy.org/docs/en/administration/troubleshooting.html https://github.com/LemmyNet/lemmy-ansible/blob/main/templates/nginx.conf

              And this config appears quite different from the “Install with Docker” config instructions: https://join-lemmy.org/docs/en/administration/install_docker.html

              spoiler
              
              worker_processes 1;
              events {
                  worker_connections 1024;
              }
              http {
                  upstream lemmy {
                      # this needs to map to the lemmy (server) docker service hostname
                      server "lemmy:8536";
                  }
                  upstream lemmy-ui {
                      # this needs to map to the lemmy-ui docker service hostname
                      server "lemmy-ui:1234";
                  }
              
                  server {
                      # this is the port inside docker, not the public one yet
                      listen 80;
                      # change if needed, this is facing the public web
                      server_name localhost;
                      server_tokens off;
              
                      gzip on;
                      gzip_types text/css application/javascript image/svg+xml;
                      gzip_vary on;
              
                      # Upload limit, relevant for pictrs
                      client_max_body_size 20M;
              
                      add_header X-Frame-Options SAMEORIGIN;
                      add_header X-Content-Type-Options nosniff;
                      add_header X-XSS-Protection "1; mode=block";
              
                      # frontend general requests
                      location / {
                          # distinguish between ui requests and backend
                          # don't change lemmy-ui or lemmy here, they refer to the upstream definitions on top
                          set $proxpass "http://lemmy-ui";
              
                          if ($http_accept = "application/activity+json") {
                            set $proxpass "http://lemmy";
                          }
                          if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"") {
                            set $proxpass "http://lemmy";
                          }
                          if ($request_method = POST) {
                            set $proxpass "http://lemmy";
                          }
                          proxy_pass $proxpass;
              
                          rewrite ^(.+)/+$ $1 permanent;
                          # Send actual client IP upstream
                          proxy_set_header X-Real-IP $remote_addr;
                          proxy_set_header Host $host;
                          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                      }
              
                      # backend
                      location ~ ^/(api|pictrs|feeds|nodeinfo|.well-known) {
                          proxy_pass "http://lemmy";
                          # proxy common stuff
                          proxy_http_version 1.1;
                          proxy_set_header Upgrade $http_upgrade;
                          proxy_set_header Connection "upgrade";
              
                          # Send actual client IP upstream
                          proxy_set_header X-Real-IP $remote_addr;
                          proxy_set_header Host $host;
                          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                      }
                  }
              }
              
              

              Do you know if I should expect to have TWO unique NGINX proxy instances (assuming I use NGINX)? One in-stack, and one separate for web facing reverse proxy? Or do I need a combination of the two configs into one instance?

              I am going to see if I can get a caddy reverse proxy setup in the meantime and see how it performs given your configuration there.

              •  I eat words   ( @saint@group.lt ) 
                link
                fedilink
                English
                111 months ago

                you can have two nginx proxy instances, one as a front (serving other sites besides lemmy instance) and another - coupled with lemmy instance. in such case the first one can be configured minimally with basic proxy stuff to internal lemmy one, no need for this fancy lemmy and lemmy-ui proxying.

                location /{
                   proxy_pass http://nginx-lemmy-docker:someport;
                }
                
                •  Jattatak   ( @Jattatak@beehaw.org ) OP
                  link
                  fedilink
                  English
                  111 months ago

                  I believe I have the simple set up for the NPM reverse proxy. Just as you say, it points to the docker address of the lemmyInstance NGINX. I can get to my instance with HTTPS secured and good. Just getting errors when communicating to other instances. I can only imagine it is web socket related, but I am not familiar enough to look at the proxy configs and determine what might be wrong with it unfortunately. I might need to try and find someone to essentially look through it with me in real time.

                  spoiler
                  ERROR HTTP request{http.method=GET http.scheme="https" http.host=lemmy.bulwarkob.com http.target=/api/v3/ws otel.kind="server" request_id=67d75886-bf48-4444-a435-d98d8fc1e303 http.status_code=101 otel.status_code="OK"}: lemmy_server::api_routes_websocket: couldnt_find_object: Request error: error sending request for url (https://lemmy.ml/.well-known/webfinger?resource=acct:asklemmy@lemmy.ml): operation timed out
                  
                     0: lemmy_apub::fetcher::search::search_query_to_object_id
                  
                               at crates/apub/src/fetcher/search.rs:17
                  
                     1: lemmy_apub::api::resolve_object::perform
                  
                             with self=ResolveObject { q: "!asklemmy@lemmy.ml", auth: Some(Sensitive) }
                  
                               at crates/apub/src/api/resolve_object.rs:21
                  
                     2: lemmy_server::root_span_builder::HTTP request
                  
                             with http.method=GET http.scheme="https" http.host=lemmy.bulwarkob.com http.target=/api/v3/ws otel.kind="server" request_id=67d75886-bf48-4444-a435-d98d8fc1e303 http.status_code=101 otel.status_code="OK"
                  
                               at src/root_span_builder.rs:16
                  
                  spoiler
                   WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: Request error: error sending request for url (https://beehaw.org/u/Jattatak): operation timed out
                  
                     0: lemmy_server::root_span_builder::HTTP request
                  
                             with http.method=POST http.scheme="https" http.host=lemmy.bulwarkob.com http.target=/inbox otel.kind="server" request_id=f413d3e5-262a-4dac-bc2e-700b9a053954 http.status_code=400 otel.status_code="OK"
                  
                               at src/root_span_builder.rs:16
                  
                  LemmyError { message: None, inner: Request error: error sending request for url (https://beehaw.org/u/Jattatak): operation timed out
                  
                  Caused by:
                  
                      0: error sending request for url (https://beehaw.org/u/Jattatak): operation timed out
                  
                      1: operation timed out, context: "SpanTrace" }
                  
                  
                  •  I eat words   ( @saint@group.lt ) 
                    link
                    fedilink
                    English
                    111 months ago

                    from the logs it seems that lemmy docker does not communicate with outside servers.

                    also i have a bit different config for lemmy.hjson

                    {
                      # for more info about the config, check out the documentation
                      # https://join-lemmy.org/docs/en/administration/configuration.html
                    
                      setup: {
                        # username for the admin user
                        admin_username: "adminuser"
                        # password for the admin user
                        admin_password: "adminpassword"
                        # name of the site (can be changed later)
                        site_name: "group.lt"
                      }
                    
                      opentelemetry_url: "http://otel:4317"
                    
                    
                    
                      # the domain name of your instance (eg "lemmy.ml")
                      hostname: "group.lt"
                      # address where lemmy should listen for incoming requests
                      bind: "0.0.0.0"
                      # port where lemmy should listen for incoming requests
                      port: 8536
                      # settings related to the postgresql database
                      # address where pictrs is available
                    pictrs: {
                        url: "http://pictrs:8080/"
                        # api_key: "API_KEY"
                    }
                      database: {
                        # name of the postgres database for lemmy
                        database: "lemmy"
                        # username to connect to postgres
                        user: "lemmy"
                        # password to connect to postgres
                        password: "lemmy"
                        # host where postgres is running
                        host: "postgres"
                        # port where postgres can be accessed
                        port: 5432
                        # maximum number of active sql connections
                        pool_size: 5
                      }
                    #  # optional: email sending configuration
                      email: {
                    #    # hostname and port of the smtp server
                        smtp_server: "postfix:25"
                        smtp_from_address: "from@group.lt"
                        tls_type: false
                      }
                    
                    
                    }
                    

                    also check in admin interface if federation is enabled and you do not blacklist instances

                    (https://lemmy.bulwarkob.com/admin) and maybe you can try to enable federation debug mode for awhile