Next Payload boilerplate build

default discord avatar
jakehopking
2 months ago
11

Hi - I'm running into an issue when I try to build my site for production... which is based upon your nextjs boilerplate starter repo.



After a lot trial and error I can run a successful build, however the resulting

/dist

folder contains zero

*.module.scss

files. These files are correctly referenced throughout the generated

.js

files, however all component folders are empty of the referenced

*.module.scss

files.



Therefore the

yarn serve

command fails with

MODULE_NOT_FOUND

errors.



Do you have any ideas where I might start on fixing this issue? Feels like a Webpack issue, but I thought this kind of file processing was handled automatically by the next build process?



Happy to share access to the repo.



If I manually copy a few missing

*.scss

files into the respective folders within

/dist

, then the

yarn serve

moves on to the next missing scss file.



Tested this to sanity check that these files should actually be copied. Obviously I can't manually copy all of these missing scss files, so does anyone know how I might go about fixing this issue?



Looking into this further, I can see that next does indeed build the site correctly. I can run the build with

next start

and it loads the site with all styling in place. However nothing runs other than the 404 page....



So the problem seems to be coming from the build server script:


"build:server": "tsc --project tsconfig.server.json",


Which is what outputs to the

/dist

folder.



I'm really confused by what need to be done here.



Looking at your more recent examples… I can see the use of

copyfiles

script. I’ll try this and report back.

  • default discord avatar
    paulpopus
    2 months ago

    Hey @jakehopking ping me if you're still getting issues, happy to take a look at your repo

  • default discord avatar
    jakehopking
    2 months ago

    Thanks @paulpopus


    I'm making progress, but I'm going to have to move to a

    src/

    structure for my project I think.

    copyfiles

    nor

    rysnc

    seems to be able follow hairy exclude rules of dirs/files when copying a list of file extensions.



    Also, I use import alias rules for the front-end components e.g.

    import { some } from '@components/some/dir'

    which the custom

    next:build

    script doesn't like. It seems to need relative imports for eveything. I have done relative paths whenever I'm dealing payload specific files, but that's not enough anymore it seems.



    Really really wish I'd have separated out the front-end and the cms into individual repos from the outset.



    @paulpopus I have

    files

    and

    media

    folders which store uploaded files from the payload collections, should I move these into the

    src

    folder also?



    @dribbens what are your thoughts on how I should structure my folders going forward.


    Currently everything lives in the parent directory like so:


    .env
    .eslintrc.js
    .git/
    .gitignore
    .next/
    .npmrc
    .nvmrc
    .prettierignore
    .prettierrc
    access/
    assets/
    blocks/
    build/
    collections/
    components/
    contexts/
    css/
    declarations/
    declarations.d.ts
    dev.ts
    dist/
    emails/
    fields/
    files/
    globals/
    hooks/
    json/
    lib/
    media/
    mocks/
    next-env.d.ts
    next.config.js
    node_modules/
    nodemon.json
    package.json
    pages/
    payload.config.ts
    public/
    readme.md
    routes/
    scripts/
    seed/
    server.ts
    stores/
    stripe/
    tokens/
    tsconfig.json
    tsconfig.server.json
    utilities/
    yarn.lock


    I will move to a

    /src

    structure, however I'm not sure where the

    media

    and

    files

    folders should move to. The media folder has 1000s of images.



    can/should payload write to root

    /media, /files

    folders outside of the

    /src

    folder? or should everything be moved into the

    /src

    folder (excluding the config files i.e tsconfig.json, packaga.json etc)?

  • default discord avatar
    paulpopus
    2 months ago

    media/files usually sits in the gitroot



    the

    src

    is usually for anything related to building the app (not just for payload but in general) which is then built to a

    dist/

    directory



    you don't really want to copy thousands of files any time you build the app, it would take extra time, plus affect your system, disks etc. too much, so they sit in a shareable gitroot directory



    https://github.com/payloadcms/website-cms/

    this is a solid example of how your folder structure can be like

  • default discord avatar
    jakehopking
    2 months ago

    Thanks for that @paulpopus . Makes a lot of sense... will update as suggested πŸ‘



    I've moved to

    /src

    and updated all imports to be relative.



    I now have broken references to

    /media*

    and

    /files*

    within my front-end and payload admin. How can I fix this?



    Console is littered with these 404s


    upstream image response failed for http://10.0.1.236:3000/media/photos/


    and my payload admin looks like this for media collections



    @paulpopus @dribbens suggestions please πŸ™



    I also guess I need to add a

    staticDir

    to the

    upload

    object within collections that have uploads, right?



    or is that my answer? will try...



    Indeed, I answered it myslelf. Thanks for being a rubber duck πŸ™‚



    I'll try again the builds later when I have more time.

    Screenshot_2023-07-21_at_11.35.18.png
    Screenshot_2023-07-21_at_11.35.06.png
    Screenshot_2023-07-21_at_11.34.46.png
  • default discord avatar
    paulpopus
    2 months ago

    nice!

  • default discord avatar
    jakehopking
    2 months ago

    I can build, however I get this error when I try to serve:



    [12:35:36] INFO (payload): Connected to MongoDB server successfully!
    [12:35:36] INFO (payload): Starting Payload...
    - info Loaded env from /Users/jake/Development/personal/elkli-and-hart/.env.production.local
    - info Loaded env from /Users/jake/Development/personal/elkli-and-hart/.env
    - warn You have enabled experimental feature (scrollRestoration) in next.config.js.
    - warn Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use at your own risk.
    
    unhandledRejection Error: Could not find a production build in the '/Users/jake/Development/personal/elkli-and-hart/.next' directory. Try building your app with 'next build' before starting the production server. https://nextjs.org/docs/messages/production-start-no-build-id
        at NextNodeServer.getBuildId (/Users/jake/Development/personal/elkli-and-hart/node_modules/next/dist/server/next-server.js:349:23)
        at new Server (/Users/jake/Development/personal/elkli-and-hart/node_modules/next/dist/server/base-server.js:152:29)
        at new NextNodeServer (/Users/jake/Development/personal/elkli-and-hart/node_modules/next/dist/server/next-server.js:173:9)
        at NextServer.createServer (/Users/jake/Development/personal/elkli-and-hart/node_modules/next/dist/server/next.js:179:24)
        at async /Users/jake/Development/personal/elkli-and-hart/node_modules/next/dist/server/next.js:200:31
        at async NextServer.prepare (/Users/jake/Development/personal/elkli-and-hart/node_modules/next/dist/server/next.js:161:24)
        at async start (/Users/jake/Development/personal/elkli-and-hart/dist/server.js:58:9)
    error Command failed with exit code 1.
    info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.


    Specifically the


    Try building your app with 'next build' before starting the production server.


    Should this not be part of the

    next:build

    script in package.json?



    Due to my 404 page requiring access to the payload using fetch, running a regular

    next build

    returns an expected error:



    - info Collecting page data
    [    ] - info Generating static pages (0/1)
    Error occurred prerendering page "/404". Read more: https://nextjs.org/docs/messages/prerender-error
    TypeError: fetch failed
        at Object.fetch (node:internal/deps/undici/undici:11457:11)
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
        at async fetchCollection (/Users/jake/Development/personal/elkli-and-hart/.next/server/chunks/535.js:4982:22)
        at async Promise.all (index 0)
        at async MyApp.getInitialProps (/Users/jake/Development/personal/elkli-and-hart/.next/server/chunks/535.js:5248:71)
        at async loadGetInitialProps (/Users/jake/Development/personal/elkli-and-hart/node_modules/next/dist/shared/lib/utils.js:150:19)
        at async renderToHTMLImpl (/Users/jake/Development/personal/elkli-and-hart/node_modules/next/dist/server/render.js:426:13)
        at async /Users/jake/Development/personal/elkli-and-hart/node_modules/next/dist/export/worker.js:459:36
        at async Span.traceAsyncFn (/Users/jake/Development/personal/elkli-and-hart/node_modules/next/dist/trace/trace.js:103:20)
    - info Generating static pages (1/1)


    Looking in

    server.js

    I see the options being passed into the

    nextBuild

    function are (i've updated this from the example repo, because the previous version was missing required params, and throwing a type args error):



    if (process.env.NEXT_BUILD) {
        server.listen(process.env.PORT, async () => {
          console.log('Next.js is now building...');
          await nextBuild(
            path.join(__dirname, ''), // dir
            false, // reactProductionProfiling
            true, // debugOutput
            false, // runLint
            false, // noMangling
            false, // appDirOnly
            false, // turboNextBuild
            null, // turboNextBuildRoot
            'default', // buildMode
          );
          process.exit();
        });
      } else {
    ...


    What options should I be using here?



    My scrips in package.json:



    "scripts": {
        "build:next": "next build",
        "build:next:custom": "cross-env PAYLOAD_CONFIG_PATH=dist/payload.config.js NEXT_BUILD=true node dist/server.js",
        "build:payload": "cross-env PAYLOAD_CONFIG_PATH=src/payload.config.ts payload build",
        "build:server": "tsc --project tsconfig.server.json",
        "build": "cross-env NODE_ENV=production yarn build:payload && yarn build:server && yarn copyfiles && yarn build:next:custom",
        "copyfiles": "copyfiles -u 1 \"src/**/*.{html,css,scss,ttf,woff,woff2,eot,svg,jpg,png}\" dist/",
        "dev:basic": "cross-env NODE_ENV=development PAYLOAD_CONFIG_PATH=src/payload.config.ts ts-node src/dev.ts",
        "serve": "cross-env NODE_ENV=production PAYLOAD_CONFIG_PATH=dist/payload.config.js node dist/server.js",
      ...
      },


    The output from

    next:build:custom

    is:


    Route (pages)                              Size     First Load JS
    β”Œ   /_app                                  0 B             223 kB
    β”” Ξ» /404                                   182 B           223 kB
    + First Load JS shared by all              298 kB
      β”œ chunks/framework-d23a68e051eeb176.js   45.2 kB
      β”œ chunks/main-7728b542b54be24a.js        28.6 kB
      β”œ chunks/pages/_app-cda6d407303bf3de.js  149 kB
      β”œ chunks/webpack-4c367c758ffd7408.js     722 B
      β”” css/71bf26a48dc5ea11.css               74.4 kB
    
    Ξ»  (Server)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
    
    Redirects
    
    β”Œ source: /:path+/
    β”œ destination: /:path+
    β”” permanent: true
    
    β”Œ source: /checkout
    β”œ destination: /basket/checkout
    β”” permanent: true
    
    β”Œ source: /homepage
    β”œ destination: /
    β”” permanent: true
    
    β”Œ source: /collections/category
    β”œ destination: /collections
    β”” permanent: true
    
    β”Œ source: /essays/category
    β”œ destination: /essays
    β”” permanent: true
    
    β”Œ source: /journal/category
    β”œ destination: /journal
    β”” permanent: true
    
    β”Œ source: /products/category
    β”œ destination: /products
    β”” permanent: true
    
    β”Œ source: /email/order-confirmation
    β”œ destination: /
    β”” permanent: true


    Which is clearly wrong.... it's only generating the 404 page, and all of my other pages don't make the cut.



    IF I run my dev server, I CAN then run

    yarn next build

    becuase the

    fetch

    will resolve.



    When this runs, the generated output is correct:



    - info Generating static pages (1/1)
    - info Finalizing page optimization
    
    Route (pages)                              Size     First Load JS
    β”Œ Ξ» /                                      528 B           253 kB
    β”œ   /_app                                  0 B             165 kB
    β”œ Ξ» /[...slug]                             515 B           253 kB
    β”œ ● /404 (305 ms)                          413 B           211 kB
    β”œ   β”” css/7a63699efed55fb2.css             3.06 kB
    β”œ Ξ» /api/convertkit/subscribe              0 B             165 kB
    β”œ Ξ» /basket                                3.54 kB         256 kB
    β”œ   β”” css/4773fd37563d6b30.css             587 B
    β”œ Ξ» /basket/checkout                       4.89 kB         262 kB
    β”œ   β”” css/5399f81d4a5a3315.css             938 B
    β”œ Ξ» /collections                           2.05 kB         255 kB
    β”œ Ξ» /collections/[slug]                    792 B           253 kB
    β”œ Ξ» /collections/category/[slug]           2.09 kB         255 kB
    β”œ Ξ» /email/order-confirmation/[slug]       591 B           253 kB
    β”œ Ξ» /essays                                2.06 kB         255 kB
    β”œ Ξ» /essays/[slug]                         470 B           253 kB
    β”œ Ξ» /essays/category/[slug]                2.03 kB         255 kB
    β”œ Ξ» /journal                               2.05 kB         255 kB
    β”œ Ξ» /journal/[slug]                        474 B           253 kB
    ...


    β”œ Ξ» /journal/category/[slug]               2.06 kB         255 kB
    β”œ Ξ» /order/order-confirmation              8.67 kB         266 kB
    β”œ   β”” css/50084f88d6d052c2.css             529 B
    β”œ Ξ» /products                              2.1 kB          255 kB
    β”œ Ξ» /products/[slug]                       4.15 kB         257 kB
    β”œ   β”” css/e0377321d13df66e.css             1.22 kB
    β”œ Ξ» /products/category/[slug]              1.98 kB         255 kB
    β”” Ξ» /styles                                8.81 kB         228 kB
        β”” css/3077863a4748e15d.css             3.52 kB
    + First Load JS shared by all              239 kB
      β”œ chunks/framework-305cb810cde7afac.js   45.2 kB
      β”œ chunks/main-49c9bd953ebaf2bf.js        28.6 kB
      β”œ chunks/pages/_app-17c46d2ea32f7757.js  90.2 kB
      β”œ chunks/webpack-0e1e5104a50731a9.js     949 B
      β”” css/7703c8d58b79829f.css               74 kB
    
    Ξ»  (Server)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
    ●  (SSG)     automatically generated as static HTML + JSON (uses getStaticProps)
    
    ✨  Done in 35.44s.


    which is what I would expect from the

    build:next:custom


    After going through this rather convoluted build process, I AM ABLE to run

    yarn serve

    and all works there as expected... which is a win actually, just need to get to this stage from a single command, and not requiring a running dev instance to build next.



    @paulpopus @dribbens With this in mind, there's clearly something wrong with the

    nextBuild()

    inside the

    server.ts

    .


    Any ideas on how to fix this?

  • discord user avatar
    seanzubrickas
    Payload Team
    2 months ago

    Hey @jakehopking - we'll get some eyes on this ASAP and report back with some answers. Hold tight!

  • default discord avatar
    jakehopking
    2 months ago

    @seanzubrickas Thanks πŸ™



    @seanzubrickas β€” if someone needs to view the repo, then @dribbens already has access, with branch for latest prod updates:

    feat/production-updates
  • discord user avatar
    jacobsfletch
    Payload Team
    2 months ago

    Hey @jakehopking have you seen the latest Custom Server example? It's been recently updated, you might want to check it out:

    https://github.com/payloadcms/payload/tree/master/examples/custom-server
  • default discord avatar
    jakehopking
    2 months ago

    @jacobsfletch ❀️



    I had been looking at this updated example, but missed a tiny difference due to having moved to a

    /src

    structure, the

    nextBuild()

    dir was

    nextBuild(path.join(__dirname, ''));

    rather than

    nextBuild(path.join(__dirname, '../'));

    . This simple change was the golden bullet πŸš„ .



    Thanks for suggesting I look again... and this time I noticed. Suffice to say, all works now! πŸ₯³



    And @jacobsfletch as I'm typing, do you know more about the args that can be provided to

    nextBuild

    ? I've updated mine to same as new custom server (with the

    @ts-expect-error

    ) however I'd ideally like to know what options are best, rather than supressing this error.

  • discord user avatar
    jacobsfletch
    Payload Team
    2 months ago

    So happy that worked for you! Yea I know exactly what you’re talking about although I don’t know anything about the API beyond config path. Might be worth looking into though.

Open the post
Continue the discussion in Discord
Like what we're doing?
Star us on GitHub!

Star

Connect with the Payload Community on Discord

Discord

online

Can't find what you're looking for?

Get help straight from the Payload team with an Enterprise License.