Problems with deploying with docker

SuddenDev

2 years ago

1 2

Hi Everyone,

I'm trying to deploy payload with docker to a DigitalOcean Droplet. I push it to my gitlab ci/cd runner, which builds the docker image and then saves it to the container registry (also gitlab). I then pull the image on my droplet, which uses caddy 2 as my reverse proxy. I can access the api on my computer, but aparently, the admin panel can not.

This is my .env file on local, that I then override with docker env vars.

NODE_ENV=dev
ENVIRONMENT=dev
PORT=3000
PAYLOAD_PUBLIC_SERVER_URL=http://localhost:3000

MONGODB_URI=<redacted URI>
PAYLOAD_SECRET=<redacted secret>
PAYLOAD_LICENCE=<redacted licence>

On production I use the same, except PAYLOAD_PUBLIC_SERVER_URL

PAYLOAD_PUBLIC_SERVER_URL=https://cms.dtampe.com

When I try to access https://cms.dtampe.com/admin, it stucks at loading and I get the following error:
Failed to load resource: net::ERR_CONNECTION_REFUSED http://localhost:3000/api/users/me

This works though, so I really don't know where I went wrong. https://cms.dtampe.com/api/pages

I did already read through Issues and Discussions, but can't seem to figure it out. I also tried to use the next.js template or this repo here https://github.com/joshhills/joshhills-site, but to no avail.

My Docker FIle:

Dockerfile


From node:14-alpine
WORKDIR ./app
COPY package*.json  ./
RUN npm install
COPY . . 
RUN npm run build
#RUN rm -f .env
EXPOSE 3000
CMD ["npm", "start"]

gitlab ci

.gitlab-ci.yml


# This file is a template, and might need editing before it works on your project.
# Build a Docker image with CI/CD and push to the GitLab registry.
# Docker-in-Docker documentation: https://docs.gitlab.com/ee/ci/docker/using_docker_build.html
#
# This template uses one generic job with conditional builds
# for the default branch and all other (MR) branches.
docker-build:
  # Use the official docker image.
  image: docker:latest
  stage: build
  environment:
    name: production
    url: https://cms.dtampe.com
  services:
    - docker:dind
  before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  # Default branch leaves tag empty (= latest tag)
  # All other branches are tagged with the escaped branch name (commit ref slug)
  script:
    - |
      if [[ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]]; then
        tag=""
        echo "Running on default branch '$CI_DEFAULT_BRANCH': tag = 'latest'"
      else
        tag=":$CI_COMMIT_REF_SLUG"
        echo "Running on branch '$CI_COMMIT_BRANCH': tag = $tag"
      fi
    - echo "Running now docker build"
    - echo "$PAYLOAD_PUBLIC_SERVER_URL"
    - echo "$NODE_ENV"
    - docker build --pull -t "$CI_REGISTRY_IMAGE${tag}" .
    - docker push "$CI_REGISTRY_IMAGE${tag}"
  # Run this job in a branch where a Dockerfile exists
  rules:
    - if: $CI_COMMIT_BRANCH
      exists:
        - Dockerfile

I'd be greatful for any hints!
Cheers!

Selected Answer

SuddenDev

2 years ago

Okay, so I did manage to get everything working with Docker. I did already dip my toe into docker-compose, so I think it will follow, but for now I'm just gonna use MongoDB Atlas.

Like described before, the reason it did not work was, that I had to supply the final URL to the payload.config file without hardcoding it in. Using the method bellow works for me and using it dynamic with Docker & Gitlab CI/CD. To make it easier for people to also use docker with the gitlab-ci (or maybe github actions, or any other CI tool) here's the run down:

1. Push your code gitlab (if not already done)

2. (Optional: If you want to set the variables in the `.gitlab-ci.yml` file, you can skip this)
Go to your Repo -> Settings -> CI/CD. There find the Variables section and add the ones you need from your .env file. (Recommended is PAYLOAD_PUBLIC_SERVER_URL, since that's the only thing used for building the payload admin). You can add call them however you want. I did set mine to not protected, but I don't know if you need it.

3. In your code base, create a new File `.gitlab-ci.yml`

4. Use this file here or start with the template `(docker.gitlab-ci.yml)` gitlab provides. I comment the important parts that are different from the template.

.gitlab-ci.yml
----------------

docker-build:
  # Use the official docker image.
  image: docker:18
  stage: build
  services:
    - docker:dind
  before_script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
  # Default branch leaves tag empty (= latest tag)
  # All other branches are tagged with the escaped branch name (commit ref slug)
  script:
    - |
      if [[ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]]; then
        tag=""
        echo "Running on default branch '$CI_DEFAULT_BRANCH': tag = 'latest'"
      else
        tag=":$CI_COMMIT_REF_SLUG"
        echo "Running on branch '$CI_COMMIT_BRANCH': tag = $tag"
      fi
    - echo "Running now docker build"
    - echo $PAYLOAD_PUBLIC_SERVER_URL

    #
    # The next block is the important one: You have to pass your gitlab variables, 
    # that you can either set here in this file or in the project settings.  
    # Your environment variables may differ, but you'll need to set at least $PAYLOAD_PUBLIC_SERVER_URL,
    # since that's where you want to pass your serverUrl that should run on production / staging / whatever.
    # I recommend also caching from the previous registry image, since that reduces build time a bit.
    #
    - >
      docker build
      --pull
      --build-arg payload_public_server_url=$PAYLOAD_PUBLIC_SERVER_URL
      --cache-from $CI_REGISTRY_IMAGE:latest
      --tag $CI_REGISTRY_IMAGE${tag}
      .
    - docker push "$CI_REGISTRY_IMAGE${tag}"
  # Run this job in a branch where a Dockerfile exists
  rules:
    - if: $CI_COMMIT_BRANCH
      exists:
        - Dockerfile

5. After that, add this `Dockerfile` to your directory root as well

Dockerfile
------------

From node:14-alpine

# Setting up args to recieve from gitlab ci 
# or just --build-arg
# adjust to your case
ARG payload_public_server_url

# Mapping the argument to ENV for the build
ENV PAYLOAD_PUBLIC_SERVER_URL=$payload_public_server_url

## Logging
RUN echo "payload_public_server_url: ${payload_public_server_url}"

## Regular Setup
WORKDIR ./app
COPY package*.json  ./
RUN npm install
COPY . . 

## Run the admin build process 
RUN npm run build
EXPOSE 3000

# Payload Serve (renamed it because 'serve' didn't work)
CMD ["npm", "start"]

6. Push this to your gitlab repository

7. Check the CI/CD Sidepanel and find it in jobs.

8. Everything should go smoothly and you can pull you image from the gitlab container registry

Bonus: When running your Docker Container, you obviously also need to specify your .env files content.

SuddenDev
2 years ago
Okay, so I know I just started this discussion 15 minutes ago, but I finally found the culprit: my local .env file was used for the build process of the admin panel and by that was pointing to localhost instead of my URI. So I just need to figure out how to get the PAYLOAD_PUBLIC_SERVER_URL dynamic in gitlab ci.
2 replies
jmikrut
Payload Team
2 years ago
Ha I was just writing up a response!

Can you set that ENV variable in a serve NPM script?

package.json:
```
  "scripts": {
    "serve": "cross-env PAYLOAD_PUBLIC_SERVER_URL=https://site-goes-here.com node server.js"
  },
```
I used cross-env here but you may not need it.

Would this work?
SuddenDev
2 years ago
Ha, thanks for the swift response!
I don't think it would, since starting the service and fetching the /api/ route seems to work just fine, but I could not access the admin panel, because of the build process. Setting the SERVER_URL in Docker works fine, but I'm sure this should also work.