I'm coming from the more classical CMS world, where most things (structure, content, media, etc.) live in the database, content is built on publish and then cached in the database, ready to be delivered to the users. There's no real build step concerning the content. The CMS is deployed without any content whatsoever and then just uses the database.

Now, we're evaluating Payload which we'd like to deploy via Docker on Kubernetes with our own Postgres database. I'm having a really hard time understanding why Payload (or Next.js?) BY DEFAULT wants to access its database during BUILD TIME. Our Gitlab CI will not be able to access the database (and for good reasons). What if there are 10.000 articles in the CMS? Will they need to be regenerated on every build? That makes no sense to me. Do you guys all use cloud databases which can be accessed publicly from everywhere (including your CI pipeline)? That sounds like a security nightmare. 😉

I heard it's possible to deactivate the static generation of pages during build-time, but what happens at run-time? Will every page be generated when it is requested for the first time? That doesn't sound cool either, because most pages probably will not have changed between application upgrades. Is this where the new Next.js cache handler comes into play? Is the best-practice to put a Redis instance in place which is used as a cache handler for already generated content?

What piece of the puzzle am I missing? Thanks!

_elmgrenlast year
I really fail to understand what you are currently doing.

What is your current tech stack? Are you familiar with JS frameworks?

The build step is necessary in javascript frameworks to compile the code to javascript, and has nothing really to do with the CMS. Next.js upon this has included static generation in the build step.

I think for you, I'd use ISR. I.e if you have 10.000 posts, it means none of them will be prerendered or cached until a user has requested the post. Once the request has been made for that path, next.js will cache it and serve it to other users. You can control revalidations using revalidate:

https://nextjs.org/docs/app/building-your-application/data-fetching/incremental-static-regeneration

You could, in theory, spin up a payload CMS in a seperate express app and use PHP (or any other language) to render your content
soulch1ldlast year
I've been using JS for about 15 years now, so I'm very used to the fact that a build step might(!) be needed. Not every JS framework needs one though. You can run an Express or Fastify backend without any build step whatsoever. But I understand that it's a necessity when using frontend frameworks. So, that's not the source of my confusion. 😉

I fail to grasp why I would want to generate content (not application code) during build-time, which is based on stuff an editor put into a database during "run-time" (when using the Payload admin interface). To me, it's just unusual to access a live database during a CI/CD pipeline build to grab content in order to statically generate pages.

My frontend application is going to be Next.js as well, so Payload is a natural fit. I'll see how far I can get with the ISR approach and a Redis cache-handler.

Or to put it into other words: Isn't it weird to have a Docker image which not only contains your Next.js/Payload
application
but also every article ever published by your editors?
vielpekalast year
Hi there, prebuilding the content is more of a feature. You could also let the content be rendered on request. The website template of payload uses prebuilding afaik. If you are using nextjs you could get the content with on of the three apis payload is offering.
weetikveelmanlast year
Or to put it into other words: Isn't it weird to have a Docker image which not only contains your Next.js/Payload application but also every article ever published by your editors?

Not at all depending on the use case in my opinion, e.g. when you have a personal blog of at most 100 articles. But the story changes when you got 10.000 of them of course. Wouldn't want the artifacts of my build being in the gigabytes.
soulch1ldlast year
Yes, I'm indeed building upon the website template. Maybe that was not the best starting place for my use-case. I just have the feeling that prebuilding is somewhat of a default when it comes to Payload and I'm always a bit reluctant to go against defaults.
vielpekalast year
That makes sense. I started with the blank template and looked up best practices in the website template.
weetikveelmanlast year
The statically generated part of the template is just a template thing, it's not something that payload recommends you to do

thought depending on the use case, it might be very logical
soulch1ldlast year
Good to know!
vielpekalast year
The website template is also great for examples of hooks and revalidating. It is quite powerful but also a large boilerplate and optionated.
weetikveelmanlast year
And i'll admit, the part where you basically have to open your firewall to do SSG in some cases is also why I generally prefer ISR over SSG 😉
soulch1ldlast year
Thanks for all the insights, guys!

Are you self-hosting your site? I understand that caching is included out of the box when deploying to Vercel, but for my self-hosting use-case I believe I need to check out something like
https://github.com/caching-tools/next-shared-cache
, right?

Of course, caching will work without this, but won't be distributed among all Docker containers.

And invalidation gets hard.
_elmgrenlast year
I'm self hosting on an VPS in a docker container.

I use the ISR approach, in my
page.tsx
i export
force-static

I then use a revalidation tag
page-content
and have a hook on publishing a page to revalidate this tag, meaning upon publishing, the page gets revalidated but until then it is cached by next.js.

If you are located somewhere in europe you should get a quite fast response time from
https://stage.mysterylane.se/

This is my latest project built with payload hosted on a VPS in Finland, no CDN layers (which might give you a delay if you're far from europe)
soulch1ldlast year
Feels very snappy (from Germany) 😉
weetikveelmanlast year
i've got enough with 1 running instance (and i'm using nuxt) so it's not something i have to deal with

Open

Continue the discussion in Discord

Star on GitHub

Star

Chat on Discord

Discord

online

Can't find what you're looking for?

Get dedicated engineering support directly from the Payload team.