Server side node packages cannot be used in PayloadCMS.

default discord avatar
KasparTr
8 months ago
23

I am using the following npm package


@google-cloud/secret-manager


This is not recommended to be used in a client-side browser environment which I was not intending to do.


Namely I am using this in the following file at a path

/src/core/third-party/google/index.ts

as follows



import { SecretManagerServiceClient } from '@google-cloud/secret-manager';

const PROJECT_NAME = process.env.GOOGLE_PROJECT_NAME
const getSecret = async (secretName: string): Promise<string> => {
    const client = new SecretManagerServiceClient();
    const name = `projects/${PROJECT_NAME}/secrets/${secretName}/versions/latest`;
    const [version] = await client.accessSecretVersion({ name });
    const secretValue = version.payload.data.toString();
    return secretValue;
}


When I run the project, I get the following error, suggesting that the package is being used in the browser environment.



Module not found: Error: Can't resolve '**zlib**' in '.../node_modules/request'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.


The solution is to install the browserify-zlib but my worry is in why does Payload expose this code in the browser. What am I doing wrong?



I tried to use the dependency in both dependencies as well as in devDependencies, same issue (I rebuilt package-lock each time).

  • default discord avatar
    linus
    8 months ago

    Details on aliasing server-side code so it isn't packed into the client-side is listed here:


    https://payloadcms.com/docs/admin/webpack
  • default discord avatar
    KasparTr
    8 months ago

    I have done exactly that. My hook uses a proxy service which ultimately, through other services uses the google-secret-manager. I have aliased the hook but that doesn't help. Do I need to alialise all the files participating in the call-stack?



    here is my payload.config part


     
    const walletsBeforeOperationHookEncryptPath = path.resolve(__dirname, 'collections/Wallets/hooks/walletsBeforeOperationHookEncrypt.js');
    const mockModulePath = path.resolve(__dirname, 'mocks/emptyObject.js');
    
    webpack: (config) => ({
      ...config,
      resolve: {
        ...config.resolve,
        fallback: {
          ...config.resolve.fallback,
          util: require.resolve('util'),
          stream: require.resolve('stream-browserify'),
          fs: false,
          url: false,
          querystring: false,
          child_process: false,
          assert: false,
          tls: false,
          net: false,
          os: false
        },
        alias: {...config.resolve.alias,[walletsBeforeOperationHookEncryptPath]: mockModulePath,}
      }
    })


    And I have cleared the webpack cache as suggested in the docs.

  • default discord avatar
    linus
    8 months ago
    *let me rephrase one sec

    Is Payload not launching at all or is the admin panel just blank white?

  • default discord avatar
    KasparTr
    8 months ago

    Its a compilation error, the Payload is not launching and the following error is seen both in console and browser.



    ERROR in ./node_modules/request/request.js 8:11-26
    Module not found: Error: Can't resolve 'zlib' in '/.../node_modules/request'
    
    BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
    This is no longer the case. Verify if you need this module and configure a polyfill for it.
    
    If you want to include a polyfill, you need to:
            - add a fallback 'resolve.fallback: { "zlib": require.resolve("browserify-zlib") }'
            - install 'browserify-zlib'
    If you don't want to include a polyfill, you can use an empty module like this:
            resolve.fallback: { "zlib": false }


    Its was used by another hook also, I aliased this as well. Didn't help. I then removed the hooks completely from the collection and the same issue. Now the @google/secret-manager is not used at all but I still get the error.



    This error goes away when I remove the package from the dependencies completely.



    I have isolated the usage to only one hook. If I comment this out from Collection, all works, if I use this hook, I get the zlib error.



    The hook is aliased in the payload.config and I have cleared the node-modules .cache.

  • default discord avatar
    linus
    8 months ago

    Can you post your current webpack config again?



    I'm not using the same libs as you of course, but I had to alias server-code today as well so I could maybe check if anything differs between our configs

  • default discord avatar
    KasparTr
    8 months ago

    payload.config.ts




    const walletsBeforeOperationHookEncryptPath = path.resolve(__dirname, 'collections/Wallets/hooks/walletsBeforeOperationHookEncrypt.js');
    const mockModulePath = path.resolve(__dirname, 'mocks/emptyObject.js');
    
    ...
    webpack: (config) => ({
      ...config,
      resolve: {
        ...config.resolve,
        fallback: {
          ...config.resolve.fallback,
          util: require.resolve('util'),
          stream: require.resolve('stream-browserify'),
          fs: false,
          url: false,
          querystring: false,
          child_process: false,
          assert: false,
          tls: false,
          net: false,
          os: false
        },
        alias: { ...config.resolve.alias,[walletsBeforeOperationHookEncryptPath]: mockModulePath }
      }
    })
    ...
  • default discord avatar
    linus
    8 months ago

    Is walletsBeforeOperationHookEncrypt a

    .js

    or

    .ts

    file?



    Except for the mockModulePath I have not provided file endings when resolving. So in my project it'd be


    const walletsBeforeOperationHookEncryptPath = path.resolve(__dirname, 'collections/Wallets/hooks/walletsBeforeOperationHookEncrypt');
  • default discord avatar
    KasparTr
    8 months ago

    It's a .ts file but I believe there, it pointing to the name used.


    I tried all the below variants with to no prevail.



    /walletsBeforeOperationHookEncrypt


    /walletsBeforeOperationHookEncrypt.ts


    /walletsBeforeOperationHookEncrypt.js


    /WalletsBeforeOperationHookEncrypt.ts (actual filename is with Capital)


    /WalletsBeforeOperationHookEncrypt

  • default discord avatar
    linus
    8 months ago

    Have you tried


    /WalletsBeforeOperationHookEncrypt

    too?

  • default discord avatar
    KasparTr
    8 months ago

    Yes, and here is the Wallets.ts collection file


    import walletsBeforeOperationHookEncrypt from '../hooks/WalletsBeforeOperationHookEncrypt';
    hooks: {
        beforeOperation: [walletsBeforeOperationHookEncrypt]
    }


    Also, the hook is not directly using the server-side code, as you would want to structure your code following appropriate design patterns. I'm just wondering if I need to do more aliasing for all the files used throughout the call-stack?

  • default discord avatar
    linus
    8 months ago

    I can't guarantee anything, but I don't think you need to. The function I've aliased is used in a hook inside my Users.ts and I only aliased the function and nothing else

  • default discord avatar
    KasparTr
    8 months ago

    I see, and is there any other way to check if the alias is working in the config file? For some reason this doesn't seem to be doing the trick.



    What worked for a momenr. Then after a rebuild, back to square one.



    const walletsBeforeOperationHookEncryptDirectPath = path.resolve(__dirname, 'hooks/WalletsBeforeOperationHookEncrypt');
  • default discord avatar
    linus
    8 months ago

    Try removing the .ts at the end this time again. I have encountered the same error when I first tried aliasing and I believe that fixed it iirc

  • default discord avatar
    KasparTr
    8 months ago

    Looks like all this is random anyway. I tryed to remove .ts and it reintroduced the zlib error. I then added the .ts back and the zlib is still there.


    node-modules cache is cleared with every change. 🙈 This works in mysterious ways



    I removed all usage of the secret-manager and the error still keeps popping up. This is now doing rounds on me.



    Server side node packages cannot be used in PayloadCMS.



    Turns out you need to add alias for each individual file using node package even if the files are not in use.


    Adding this to payload.config helps, though turns up a bunch of warning messages.



    payload.config


    const walletsBeforeOperationHookEncryptDirectPath = path.resolve(__dirname, 'hooks/WalletsBeforeOperationHookEncrypt');
    const assetAfterChangePublishAssetDirectPath = path.resolve(__dirname, 'hooks/AssetAfterChangePublishAsset');
    const secoCallbackPath = path.resolve(__dirname, 'core/third-party/seco/callback/index')
    const secoAuthPath = path.resolve(__dirname, 'core/third-party/seco/auth/index')
    
        webpack: (config) => ({
          ...config,
          externals: {
            './core/third-party/google/secret-manager': `() => []`,
            './core/third-party/seco/callback/auth': `() => []`,
            './core/third-party/seco/auth': `() => []`,
          },
          resolve: {
            ...config.resolve,
            fallback: {
              ...config.resolve.fallback,
              util: require.resolve('util'),
              stream: require.resolve('stream-browserify'),
              fs: false,
              url: false,
              querystring: false,
              child_process: false,
              assert: false,
              tls: false,
              net: false,
              os: false
            },
            alias: {
              ...config.resolve.alias,
              [walletsBeforeOperationHookEncryptDirectPath]: mockModulePath,
              [assetAfterChangePublishAssetDirectPath]: mockModulePath,
              [secoCallbackPath]: mockModulePath,
              [secoAuthPath]: mockModulePath
            }
          }
        })

    Half of those files aren't even in use in the code, go figure.

Open the post
Continue the discussion in Discord
Like what we're doing?
Star us on GitHub!

Star

Connect with the Payload Community on Discord

Discord

online

Can't find what you're looking for?

Get help straight from the Payload team with an Enterprise License.