Server side node packages cannot be used in PayloadCMS.

KasparTr

8 months ago

23

@google-cloud/secret-manager

import { SecretManagerServiceClient } from '@google-cloud/secret-manager';

const PROJECT_NAME = process.env.GOOGLE_PROJECT_NAME
const getSecret = async (secretName: string): Promise<string> => {
    const client = new SecretManagerServiceClient();
    const name = `projects/${PROJECT_NAME}/secrets/${secretName}/versions/latest`;
    const [version] = await client.accessSecretVersion({ name });
    const secretValue = version.payload.data.toString();
    return secretValue;
}

Module not found: Error: Can't resolve '**zlib**' in '.../node_modules/request'
BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.

• 

  linus

  8 months ago

  Details on aliasing server-side code so it isn't packed into the client-side is listed here:

  
  https://payloadcms.com/docs/admin/webpack
• 

  KasparTr

  8 months ago

  I have done exactly that. My hook uses a proxy service which ultimately, through other services uses the google-secret-manager. I have aliased the hook but that doesn't help. Do I need to alialise all the files participating in the call-stack?

  
  
  

  here is my payload.config part

  
  

   
  const walletsBeforeOperationHookEncryptPath = path.resolve(__dirname, 'collections/Wallets/hooks/walletsBeforeOperationHookEncrypt.js');
  const mockModulePath = path.resolve(__dirname, 'mocks/emptyObject.js');
  
  webpack: (config) => ({
    ...config,
    resolve: {
      ...config.resolve,
      fallback: {
        ...config.resolve.fallback,
        util: require.resolve('util'),
        stream: require.resolve('stream-browserify'),
        fs: false,
        url: false,
        querystring: false,
        child_process: false,
        assert: false,
        tls: false,
        net: false,
        os: false
      },
      alias: {...config.resolve.alias,[walletsBeforeOperationHookEncryptPath]: mockModulePath,}
    }
  })

  
  
  

  And I have cleared the webpack cache as suggested in the docs.

• 

  linus

  8 months ago

  *let me rephrase one sec
  

  Is Payload not launching at all or is the admin panel just blank white?

• 

  KasparTr

  8 months ago

  Its a compilation error, the Payload is not launching and the following error is seen both in console and browser.

  
  
  

  ERROR in ./node_modules/request/request.js 8:11-26
  Module not found: Error: Can't resolve 'zlib' in '/.../node_modules/request'
  
  BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
  This is no longer the case. Verify if you need this module and configure a polyfill for it.
  
  If you want to include a polyfill, you need to:
          - add a fallback 'resolve.fallback: { "zlib": require.resolve("browserify-zlib") }'
          - install 'browserify-zlib'
  If you don't want to include a polyfill, you can use an empty module like this:
          resolve.fallback: { "zlib": false }

  
  

  
  

  Its was used by another hook also, I aliased this as well. Didn't help. I then removed the hooks completely from the collection and the same issue. Now the @google/secret-manager is not used at all but I still get the error.

  
  
  

  This error goes away when I remove the package from the dependencies completely.

  
  

  
  

  I have isolated the usage to only one hook. If I comment this out from Collection, all works, if I use this hook, I get the zlib error.

  
  
  

  The hook is aliased in the payload.config and I have cleared the node-modules .cache.

• 

  linus

  8 months ago

  Can you post your current webpack config again?

  
  

  
  

  I'm not using the same libs as you of course, but I had to alias server-code today as well so I could maybe check if anything differs between our configs

• 

  KasparTr

  8 months ago

  payload.config.ts

  
  
  
  

  const walletsBeforeOperationHookEncryptPath = path.resolve(__dirname, 'collections/Wallets/hooks/walletsBeforeOperationHookEncrypt.js');
  const mockModulePath = path.resolve(__dirname, 'mocks/emptyObject.js');
  
  ...
  webpack: (config) => ({
    ...config,
    resolve: {
      ...config.resolve,
      fallback: {
        ...config.resolve.fallback,
        util: require.resolve('util'),
        stream: require.resolve('stream-browserify'),
        fs: false,
        url: false,
        querystring: false,
        child_process: false,
        assert: false,
        tls: false,
        net: false,
        os: false
      },
      alias: { ...config.resolve.alias,[walletsBeforeOperationHookEncryptPath]: mockModulePath }
    }
  })
  ...

• 

  linus

  8 months ago

  Is walletsBeforeOperationHookEncrypt a

  .js

  or

  .ts

  file?

  
  

  
  

  Except for the mockModulePath I have not provided file endings when resolving. So in my project it'd be

  
  

  const walletsBeforeOperationHookEncryptPath = path.resolve(__dirname, 'collections/Wallets/hooks/walletsBeforeOperationHookEncrypt');

• 

  KasparTr

  8 months ago

  It's a .ts file but I believe there, it pointing to the name used.

  
  

  I tried all the below variants with to no prevail.

  
  
  

  /walletsBeforeOperationHookEncrypt

  
  

  /walletsBeforeOperationHookEncrypt.ts

  
  

  /walletsBeforeOperationHookEncrypt.js

  
  

  /WalletsBeforeOperationHookEncrypt.ts (actual filename is with Capital)

  
  

  /WalletsBeforeOperationHookEncrypt

• 

  linus

  8 months ago

  Have you tried

  
  /WalletsBeforeOperationHookEncrypt

  too?

• 

  KasparTr

  8 months ago

  Yes, and here is the Wallets.ts collection file

  
  

  import walletsBeforeOperationHookEncrypt from '../hooks/WalletsBeforeOperationHookEncrypt';
  hooks: {
      beforeOperation: [walletsBeforeOperationHookEncrypt]
  }

  
  

  
  

  Also, the hook is not directly using the server-side code, as you would want to structure your code following appropriate design patterns. I'm just wondering if I need to do more aliasing for all the files used throughout the call-stack?

• 

  linus

  8 months ago

  I can't guarantee anything, but I don't think you need to. The function I've aliased is used in a hook inside my Users.ts and I only aliased the function and nothing else

• 

  KasparTr

  8 months ago

  I see, and is there any other way to check if the alias is working in the config file? For some reason this doesn't seem to be doing the trick.

  
  

  
  

  What worked for a momenr. Then after a rebuild, back to square one.

  
  
  

  const walletsBeforeOperationHookEncryptDirectPath = path.resolve(__dirname, 'hooks/WalletsBeforeOperationHookEncrypt');

• 

  linus

  8 months ago

  Try removing the .ts at the end this time again. I have encountered the same error when I first tried aliasing and I believe that fixed it iirc

• 

  KasparTr

  8 months ago

  Looks like all this is random anyway. I tryed to remove .ts and it reintroduced the zlib error. I then added the .ts back and the zlib is still there.

  
  

  node-modules cache is cleared with every change. 🙈 This works in mysterious ways

  
  

  
  

  I removed all usage of the secret-manager and the error still keeps popping up. This is now doing rounds on me.

  
  

  
  

  Server side node packages cannot be used in PayloadCMS.

  
  

  
  

  Turns out you need to add alias for each individual file using node package even if the files are not in use.

  
  

  Adding this to payload.config helps, though turns up a bunch of warning messages.

  
  
  

  payload.config

  
  

  const walletsBeforeOperationHookEncryptDirectPath = path.resolve(__dirname, 'hooks/WalletsBeforeOperationHookEncrypt');
  const assetAfterChangePublishAssetDirectPath = path.resolve(__dirname, 'hooks/AssetAfterChangePublishAsset');
  const secoCallbackPath = path.resolve(__dirname, 'core/third-party/seco/callback/index')
  const secoAuthPath = path.resolve(__dirname, 'core/third-party/seco/auth/index')
  
      webpack: (config) => ({
        ...config,
        externals: {
          './core/third-party/google/secret-manager': `() => []`,
          './core/third-party/seco/callback/auth': `() => []`,
          './core/third-party/seco/auth': `() => []`,
        },
        resolve: {
          ...config.resolve,
          fallback: {
            ...config.resolve.fallback,
            util: require.resolve('util'),
            stream: require.resolve('stream-browserify'),
            fs: false,
            url: false,
            querystring: false,
            child_process: false,
            assert: false,
            tls: false,
            net: false,
            os: false
          },
          alias: {
            ...config.resolve.alias,
            [walletsBeforeOperationHookEncryptDirectPath]: mockModulePath,
            [assetAfterChangePublishAssetDirectPath]: mockModulePath,
            [secoCallbackPath]: mockModulePath,
            [secoAuthPath]: mockModulePath
          }
        }
      })

  
  

  Half of those files aren't even in use in the code, go figure.