Deploying payload 3.0 with docker to Coolify

xn1cklaslast year

54

• 

  _mel.tslast year

  Ever solve this? Struggling with my docker build too.

  
  
  

  Mongo econn error while bulding but running locally works without issue.

  
  
  

  Used the same dockerfile you mention and only changed the if statement to run with the package manager I'm using (pnpm). I don't hit your error but still hit the mongodb econn error

• 

  anders2292last year

  I have not deployed to Coolify with docker, but have success with nixpacks. Then you do not have to worry about dockerfiles. Have you tried it?

  
  

  Sorry I can not help with the dockerfile itself.

• 

  xn1cklaslast year

  Hey guys!

  
  

  Yes nixpack worked instantly. Crazy how good it is.

  
  
  

  But it itched me that I couldn’t get the dockerfile running.

  
  
  

  I‘m using this script now, it works fine.

  
  
  https://github.com/vercel/next.js/blob/canary/examples/with-docker/Dockerfile
• 

  _mel.tslast year

  Awesome will give this a try next week. Noticed that the docs got updated with this too which is good as the original definitely had issues. Ty

• 

  xn1cklaslast year

  If any of you find a good solution for a docker compose with a mongo db, feel free to share here!

• 

  s.laamlast year

  I'm using docker with coolify

  
  

  my build is made on github actions , and I'm using the ghcr.io registry to host my image

  
  

  \

  
  

  the docker compose in payload docs is only for development

  
  
  

  what is your strategy for deployment on coolify ?

• 

  xn1cklaslast year

  for staging the same, but I want to hand over the project to a client soon and let them host it themselves just with docker-compose

• 

  s.laamlast year

  so in your docker compose you'll have the payload app and the database ?

• 

  xn1cklaslast year

  Yes, that was the plan. Does not work though. Will be taking a deeper look next week:

  
  
  version: '3' services: mongo: image: mongo:latest ports: - '27017:27017' environment: MONGO_INITDB_DATABASE: static-name command: - --storageEngine=wiredTiger volumes: - data:/data/db logging: driver: none healthcheck: test: ['CMD', 'mongosh', '--eval', "db.adminCommand('ping')"] interval: 5s timeout: 5s retries: 5 start_period: 10s payload: build: . ports: - '3000:3000' environment: NODE_ENV: production DATABASE_URI: mongodb://mongo/lernwelt-zukunft PAYLOAD_SECRET: ${PAYLOAD_SECRET} PAYLOAD_PUBLIC_SERVER_URL: ${PAYLOAD_PUBLIC_SERVER_URL} NEXT_PUBLIC_SERVER_URL: ${NEXT_PUBLIC_SERVER_URL} NEXT_PUBLIC_IS_LIVE: ${NEXT_PUBLIC_IS_LIVE} PAYLOAD_PUBLIC_DRAFT_SECRET: ${PAYLOAD_PUBLIC_DRAFT_SECRET} NEXT_PRIVATE_DRAFT_SECRET: ${NEXT_PRIVATE_DRAFT_SECRET} REVALIDATION_KEY: ${REVALIDATION_KEY} NEXT_PRIVATE_REVALIDATION_KEY: ${NEXT_PRIVATE_REVALIDATION_KEY} depends_on: mongo: condition: service_healthy volumes: data:
• 

  _mel.tslast year

  I had no issues with the one in the docs once I figured out the db uri...what errors do you get when you run this?

• 

  xn1cklaslast year

  It’s actually the db uri that’s causing errors. Is there a docker compose in the docs as well?

• 

  _mel.tslast year

  Yep there is. I'll be looking at this all day tomorrow trying to get my own dockerfile to deploy haha. Let me know how you get on

• 

  s.laamlast year

  check the docker file and docker compose in this project

  
  

  I'm using MinIO, so you can delete the services related to that

  
  
  https://github.com/elghaied/gshell-web
  
  

  in my case build is made through github actions with the docker file, so I pass the payload_secret as argument using github secrets .

  
  

  in coolify I have a project with 3 resources , payload type docker image , mangodb , minIO .

  
  

  docker compose is used to test production locally, but I think you can use it for deployment as well

  
  
  

  don't forget to add ( out: 'standalone' ) in you next config

  
  
  

  if your client will be using coolify as well

  
  

  look at coolify magic environment variables

  
  
  https://coolify.io/docs/knowledge-base/docker/compose/
  
  

  using them you could automatically generate the payload secret and revalidate key ...etc

• 

  zephury.last year

  you still need to provide a port to your database uri, ie;

  
  mongodb://mongo:27017/lernwelt-zukunft
• 

  xn1cklaslast year

  Used

  @245162072814387200

  dockerfile, env and docker compose but still running into database connection errors

  
  
  

  31 [14:29:17] ERROR: Error: cannot connect to MongoDB. Details: connect ECONNREFUSED 127.0.0.1:27017

  
  

  72.31 err: {

  
  

  72.31 "type": "c",

  
  

  72.31 "message": "connect ECONNREFUSED 127.0.0.1:27017",

  
  

  72.31 "stack":

  
  

  72.31 MongooseServerSelectionError: connect ECONNREFUSED 127.0.0.1:27017

• 

  _mel.tslast year

  which one are you running though? The docker compose or the docker file, as I understand it they are 2 separate build options

• 

  xn1cklaslast year

  I'm running the docker compose since I want to use the payload app with minio and mongo

• 

  zephury.last year

  hard to say without knowing exactly what you're doing;

  
  
  

  however, I don't think its necessary to host them all together in the same docker-compose either.

  
  
  

  Have you looked at coolify before? It's very easy to setup minio and mongodb or postgres and then just host your payload application as a simple nextjs application. you can run backups and worry about maintaining them completely separately

  
  
  

  edit; I reread the conversation history, I see.

  
  
  

  you should also take in to account backing up that database, backing up minio images and so fourth when its deployed. just handing over a docker-compose is either not ideal if the client is inexperienced, as it is not reliable in production, or if they have some experience, they will likely not even use it.

  
  
  

  did you try the exact string I sent for the uri?

• 

  xn1cklaslast year

  Yes I used the same one. Perhaps it would be easier to setup coolify for the client haha

• 

  zephury.last year

  its also possible to setup automatic database backups to s3 as well, or since you may use minio; if they are setting up two VPS to have a backup, you can have 2 minio instances with easy replication, then have your database backups get pushed to minio as well; so you end up with a pretty solid system that should keep their data pretty safe, without anything super complicated and in the off chance that something happens in prod, they just need someone to restore from backup.

  
  
  

  if you really do want to figure out that initial docker-compose setup, feel free to make a minimal reproduction and send the repo and I'd be happy to take a look. Based on what I can see, what you have

  should

  be fine

• 

  etjen_kavciclast year

  You added initial database name:

  
  

  - MONGO_INITDB_DATABASE: static-name. Did you try to change URI to mongodb://mongo/static-name ?

• 

  _mel.tslast year

  I ran this on my proj and it did not output a mongo image - do you have docker desktop? Once you run

  docker compose build

  does it output 2 images? What are they called?

  
  
  

  also you can explicitly name the container and reference it in the uri string

  
  
  

  services:
    mongo:
      container_name: my_mongo_container

  
  
  

  then your uri would be something llike

  mongodb://my_mongo_container:27017/static_name

  (try without the static name if that still fails)

• 

  xn1cklaslast year

  Thanks guys, I'm gonna give it a shot again and update soon!

  
  
  

  do you know if the .env.example is up to date in the gshell-web repo?

  
  
  

  Tried multiple solutions, but for some reason my dockerfile is not able to connect to the docker compose network where the mongo db lives. Even if I set up depends_on and healthchecks. No chance. When I build the app inside the docker-compose with a simple pnpm install and node server.js it works, but is not using the optimizations from the dockerfile recommended for nextjs applications

• 

  zephury.last year

  So, I went to try this out myself, as it's been a long time since I used mongo, or docker-compose for a Payload app. (I still don't really recommend it)

  
  
  

  I believe the issue many people may be facing is related to the fact that in your build processes, your NextJS application is attempting to generate static pages. During build time, particularly if you build your image inside of your

  docker-compose.yml

  , networking is not available during the build stage.

  
  
  

  A solution would be to gracefully handle errors within your code. For example, wrap your payload local api, or rest api calls inside of a try/catch. If there is any error, return things like undefined, or an empty array and instead of rendering data you expect to always be there, check if the data is defined, or if the array has any length to it and if it does not, call next's

  notFound()

  .

  
  
  

  It's essentially unhandled runtime errors that are causing this to happen. So, if you handle those, you should be able to build without the mongo connection, then when the application is actually running, it should attempt to query the data with the actual mongo connection.

• 

  xn1cklaslast year

  Thank you for the detailed response

  @132273173847736320

  
  
  

  What still confuses me, how this dockerfile can work with my coolify setup but fail with docker-compose.

  
  
  

  An approach I'm thinking of going is to use this docker-compose:

  https://payloadcms.com/docs/production/deployment#docker-compose

  and just replace the pnpm dev with build and a start command

• 

  zephury.last year

  I get what you're trying to do, but the reason that works in development and not in production, is due to the nextjs build process. Your options are to basically handle the errors for instances where the database will not be accessible during build, or just straight up separate the two, so that you can have Docker running during your build process.

  Maybe

  there is a way to make NextJS skip the static page generation during build, you can check the nextjs docs to see if they support that

• 

  xn1cklaslast year

  I see the issue. Basically the next js standalone output is not able to connect to a database in the build process

• 

  zephury.last year

  and again, it works in your coolify setup, likely because the mongodb instance is running during your build?

  
  
  

  Or do you mean that the mongodb is also in that docker-compose on Coolify?

  
  
  

  If so, it might be due to the way Coolify networking works. While Coolify uses docker-compose files, it also makes quite a few changes, adding some extra features. Perhaps Coolify does something that makes services connect to the network during the build process, which is possible, if I remember correctly, but pretty annoying to do on your own

  
  
  

  yeah, during that build process, its not connected to the docker network. only once its actually ran, can it communicate on the network; for docker specifically.

  maybe

  there is a workaround and if its really that important, thats what I would research.

  
  
  

  "How to make a docker-compose build step connect to the docker network during build" or something like that.

  
  
  

  If you just host mongodb on coolify and make the url publicly accessible, it would be able to connect to it during the build process. That again creates another issue though, where you'll have people trying to brute force your db 🤣

  
  
  

  I can't recall if through coolify's unique networking, if it'd be able to connect to the private network during the build process. I'd guess not

• 

  xn1cklaslast year

  Exactly, in coolify I have a separate mongodb which I just pass over in the build env file to the next dockerfile.

• 

  zephury.last year

  and is that url public or private?

• 

  xn1cklaslast year

  private, within the coolify network

• 

  zephury.last year

  oh thats interesting, so you don't need to make it publicly accessible to get it to hook on to it, during the build process?

  
  
  

  I think I had an issue with that, but I use a build server. So I'm guessing I wasn't able to connect during the build process due to the fact that the build was taking place on a different server all together

• 

  xn1cklaslast year

  Technically this setup should mirror how coolify works, shouldn't it?

  
  
  services: mongo: image: mongo:latest ports: - '27017:27017' environment: MONGO_INITDB_DATABASE: lernwelt-zukunft command: - --storageEngine=wiredTiger volumes: - data:/data/db logging: driver: none healthcheck: test: ['CMD', 'mongosh', '--eval', "db.adminCommand('ping')"] interval: 5s timeout: 5s retries: 5 start_period: 10s networks: - app-network payload: build: Dockerfile ports: - '3000:3000' environment: - .env depends_on: mongo: condition: service_healthy networks: - app-network volumes: data: networks: app-network: driver: bridge
• 

  zephury.last year

  do keep in mind that you generally need a beefier server if you run builds on the same server that you're deploying on; otherwise it'll crash and not only that, but during your builds, the site will be noticeable slower while its building.

  
  
  

  I think the best/easiest way to do your deployments with coolify, is to use github actions for the build, along with ghcr, so you just deploy on coolify with a finished docker image.

  
  
  

  It also means that you either have to make your database uri public, or you again have to go through and make sure that those runtime errors are caught when it attempts to run the static build, which is genuinely quite easy to do. Thats what I always do

• 

  xn1cklaslast year

  Afaik coolify has a default network and can also have sub networks. I have a coolify project with the github project connected and use the dockerfile for building and in the same project there is a mongo db. I use the uri as env variable at build time for the next js dockerfile

  
  
  

  I know what you're refering to. I feel quite confident handling coolify scenarios. The current context is that a client wans the docker-compose to host the app themselves.

• 

  zephury.last year

  maybe? I'm pretty good with Coolify, but I haven't exactly dug in to the code to know exactly what they're doing here.

  
  
  

  If you understood what I meant and what you're saying is true-- that you can build the Dockerfile and connect to a coolify mongodb service during the build process, then I believe Coolify

  must

  be doing something different behind the scenes to make this possible.

• 

  xn1cklaslast year

  I can't verfiy that the dockerfile can connect to the mongodb at build time, but I know it never had issues building on the server

• 

  zephury.last year

  that's a rough one; I think theres a lot of pros and cons 😅

  
  
  

  Theres also other factors for this client to be concerned about, which perhaps you should highlight. If I were you, I'd be trying to convince him to change his request. Not sure if thats possible for you.

  
  
  

  If they are technical, a Dockerfile alone for Payload should be enough.

  
  
  

  If they are not technical; then why do they want a docker-compose anyways? they would still need to setup ssl certificates, potentially worry about blue/green deployments, etc anyways.

  
  
  

  If they have their own server, I'd just try to get them to add your public coolify ssh key, so it can automate the ci/cd when you work on it, as well as if you have to do any maintenance/support and then they should understand that the

  Dockerfile

  provided in the project's code is what makes their application build; however a website is more than just a Dockerfile, or docker-compose; often times, you are managing the ssl cert, possibly a proxy, the database itself and the database backups, all sorts of stuff.

• 

  xn1cklaslast year

  You're right. I've been spending so many hours already on this. Coolify is our savior. We should not look away from it 😄

• 

  zephury.last year

  The way I like to "sell" coolify to people is that;

  
  
  

  When you are self-hosting, it's great, because you own your own data. No one can take it away from you.

  
  
  

  There is the concept of "CI/CD", as well as "Deployment" and I try to help clients understand that with various explanations, depending on complexity of the project and how technical they are.

  
  
  

  I let them know that we use a great tool called Coolify and it's not "black magic", it's more or less a "Free Devops" guy that accesses a server via ssh and follows a set of instructions in order to automate the deployment process, versus having to pay someone to do it.

  
  
  

  In my contracts, it explains that "ci/cd" is not something they have to pay for; we include it automatically and all of the code still runs on their own server. If they remove the ssh key, or change anything about the deployment, it voids their warranty. If they have another developer that wants to take over, they are free to setup their own ci/cd process at their own expense

• 

  xn1cklaslast year

  That's a great approach. I've thought about that, but couldn't find anything about whether it is possible to handle multiple domains with coolify. If I get you right you just add their server to your coolify right?

• 

  zephury.last year

  most vps services have a way to add a public key to the server when you purchase/rent it. you just add your coolify's public key and then in coolify you "add a server". you give it the server's ip and coolify is then able to ssh in, install docker and setup the proxy. then when you deploy a project, you can select the client's server as the deployment server.

  
  
  

  the proxy, dockerfile, etc; everything runs on their server

• 

  xn1cklaslast year

  gotcha. I even know they have a hetzner like us, but not arm but intel based

• 

  zephury.last year

  and as far as domains, you don't even need to do anything fancy; you just add an A record in their dns management, pointing towards the server ip for the domain itself, then an "*.theirdomain.com" pointing to the server as well

  
  
  

  everything is automatic past that

• 

  xn1cklaslast year

  And if I get the docs right the image needs to be pushed to a registry, right?

  
  
  https://coolify.io/docs/knowledge-base/server/multiple-servers
• 

  zephury.last year

  Yes. ghcr is free and makes it very easy to build and publish it with github actions. If you go for a private ghcr, or any private container registry, you’ll just have to run “docker login” on the client’s server as well, so they have permissions to pull images. I kinda wish coolify supported providing the registry login details as well, so that step can be skipped. Its very easy to just do with the integrated coolify console though

• 

  anders2292last year

  See this info from Alessio about building without prerendering

  
  https://discord.com/channels/967097582721572934/1215659716538273832/1270500923269054556
  

  Never tried it myself, but seems interesting.

• 

  mikecebullast year

  I think I had this exact issue using Dokploy instead of Coolify. The issue was how server actions worked in standalone. I could not use a private db connection string. As soon as I used the public connection string it built fine.

• 

  s.laamlast year

  hello sorry for the late reply

  
  

  do you have any content in your database during build ?

  
  

  docker compose works in my case because I'm not generating any SSG during build time , but on runtime when the page first visited

  
  
  https://nextjs.org/docs/app/api-reference/functions/generate-static-params#all-paths-at-runtime
  
  

  I will try to create a new project with the stable version of payload and add docker compose to it.

• 

  xn1cklaslast year

  Yes there‘s content in the db!

  
  
  

  Thanks for the info! What confuses me, how this exact same setup works fine in coolify and only has trouble with my local docker compose

• 

  mikecebullast year

  Building locally with docker while the db is in a different docker container, like docker desktop, the connection string needs to look like this:

  
  mongodb://host.docker.internal:27017/my-database
• 

  s.laamlast year

  hello

  @620311628679741447

  
  
  

  in this repository you'll find a working docker file and docker compose

  
  
  https://github.com/elghaied/payload-docker-compose-example
  
  

  instructions are in readme

  
  
  

  it's not that perfect but it's working ^^"

• 

  emot1onz.last year

  Hey

  @245162072814387200

  , thanks for your git repo. It was really helpful.

  
  
  

  Im using your first solution with host.docker.internal (on a VPS with plesk)

  
  
  

  What i noticed is that its not going to work if you dont specify the mongodb ports on the docker-compose.yml

  
  
  

  I was trying to find a solution where you dont make the db public.. Do you have any recommendations? If i open the ports what security measures should i take to make it secure? (I already use credentials for the db string)

• 

  max.deglast year

  Here's a simplified example of my dockerfile:

  
  
  

  # Base image with Node.js
  FROM node:22-alpine AS base
  
  # Install dependencies
  FROM base AS deps
  RUN apk add --no-cache libc6-compat
  WORKDIR /app
  
  # Copy package manager files and install dependencies
  COPY package.json package-lock.json* ./
  
  RUN \
    if [ -f package-lock.json ]; then npm install; \
    else echo "Lockfile not found." && exit 1; \
    fi
  
  # Build the Next.js application
  FROM base AS builder
  WORKDIR /app
  COPY --from=deps /app/node_modules ./node_modules
  COPY . .
  
  ENV NEXT_TELEMETRY_DISABLED=1
  RUN npm run build
  
  # Final stage: Set up the runtime environment
  FROM base AS runner
  WORKDIR /app
  
  # Create and set the application user
  RUN addgroup --system --gid 1001 nodejs
  RUN adduser --system --uid 1001 nextjs
  RUN apk add --no-cache curl
  
  # Secrets
  COPY --from=builder --chown=nextjs:nodejs /app/.env ./.env
  
  # Next.js
  RUN mkdir .next
  RUN chown nextjs:nodejs .next
  COPY --from=builder --chown=nextjs:nodejs /app/public ./public
  COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
  COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
  
  USER nextjs
  EXPOSE 3001
  
  ENV NODE_ENV=production
  ENV NEXT_TELEMETRY_DISABLED=1
  ENV NODE_OPTIONS=--no-deprecation
  ENV PORT=3001
  ENV HOSTNAME="0.0.0.0"
  # server.js is created by next build from the standalone output
  # https://nextjs.org/docs/pages/api-reference/next-config-js/output
  CMD ["sh", "-c", "node server.js"]

  
  
  

  Using a thing similar to coolify