Error when implementing plugin-cloud-storage

default discord avatar
Step
2 months ago
28

Hello, someone has implemented a bucket in GCS using plugin-cloud-storage, because when I invoke: credentials: JSON.parse(process.env.GCS_CREDENTIALS), I have error when parsing the JSON declared as environment variable.

  • discord user avatar
    jesschow
    Payload Team
    2 months ago

    Hi @Step - does it work if you prefix your environment variable with

    PUBLIC_PAYLOAD_

    ? See docs here

    https://payloadcms.com/docs/admin/webpack#admin-environment-vars
  • default discord avatar
    Step
    2 months ago

    based on the Payload Cloud Storage Plugin documentation, when using GCS Adapter I don't see that they prefix PUBLIC

    PAYLOAD

    with the environment variables:

    https://github.com/payloadcms/plugin-cloud-storage#gcs-adapter
  • discord user avatar
    jesschow
    Payload Team
    2 months ago

    What do you get when you console.log

    JSON.parse(process.env.GCS_CREDENTIALS)

    ?

  • default discord avatar
    Step
    2 months ago

    Through console.log I get the following error:

    SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>).

    My credentials are defined as follows:


    GCS_CREDENTIALS =. { "type": "xxx", "project_id": "xxx", "private_key_id": "xx", "private_key": "xxx", "client_email": "xxx", "client_id": "xxx", "auth_uri": "xxx", "token_uri": "xxx", "auth_provider_x509_cert_url": "xxxx", "client_x509_cert_url": "xxx". }

    The error occurs when parsing the JSON:

    JSON.parse(process.env.GCS_CREDENTIALS)
  • discord user avatar
    jesschow
    Payload Team
    2 months ago

    So if you console.log(

    process.env.GCS_CREDENTIALS

    ) does it return the above object?



    If that is the exact object you're using, this error could be due to invalid json, you have a period on the end of the last line here "client_x509_cert_url": "xxx". which should be removed

  • default discord avatar
    Step
    2 months ago

    There is no period, not if I have correctly declared my credentials in JSON object as variable environment.

    image.png
  • discord user avatar
    jesschow
    Payload Team
    2 months ago

    Is that screenshot directly from your

    .env

    ? If so, you need to put the entire object on one line

  • default discord avatar
    Step
    2 months ago

    ok, but in JSON format? Because the plugin

    plugin-cloud-storage

    plugin parse a JSON:

    import { gcsAdapter } from '@payloadcms/plugin-cloud-storage/gcs'; const adapter = gcsAdapter({ options: { // you can choose any method for authentication, and authorization which is being provided by

    @google-cloud/storage

    keyFilename: './gcs-credentials.json', //OR credentials: JSON.parse(process.env.GCS_CREDENTIALS) // this env variable will have stringify version of your credentials.json file }, bucket: process.env.GCS_BUCKET, }) // Now you can pass this adapter to the plugin
  • discord user avatar
    jesschow
    Payload Team
    2 months ago

    Yes, so it looks like this screenshot.



    In the terminal I am console logging twice, first box is

    process.env.CREDS

    then

    JSON.parse(process.env.CREDS)

    and you can see it come through as expected

    Screen_Shot_2023-03-20_at_7.17.25_PM.png
  • default discord avatar
    Step
    2 months ago

    now I get the following error 😢

    image.png
  • discord user avatar
    jesschow
    Payload Team
    2 months ago

    @Step have you run

    yarn

    in your project?

  • default discord avatar
    Step
    2 months ago

    yes, I use

    yarn

    to compile my project

  • discord user avatar
    jesschow
    Payload Team
    2 months ago

    @Step the errors look to be coming from other dependencies now

  • default discord avatar
    Step
    2 months ago

    it seems to require

    stream-browserify

    to be able to process the JSON.

  • default discord avatar
    iamlinkus
    2 months ago

    I've run into the same problem. @Step have you managed to solve this?

  • default discord avatar
    Step
    2 months ago

    Hi, yes I got it, the credentials must be declared as public and in one line:

    PAYLOAD_PUBLIC_GCS_CREDENTIALS ={"type": "xx",....}

    and

    PAYLOAD_PUBLIC_GCS_BUCKET=xxx

    , also, you must parse it in the following way, credentials:

    JSON.parse(process.env.PAYLOAD_PUBLIC_GCS_CREDENTIALS.replace(/\n/g, ''))
  • default discord avatar
    Step
    2 months ago

    Yes, and I am using the following versions:


    "@google-cloud/storage":"^6.9.4", "@payloadcms/plugin-cloud-storage": "^1.0.14", "payload": "^1.6.26".
  • default discord avatar
    iamlinkus
    2 months ago

    got it, thanks!

  • default discord avatar
    Step
    2 months ago

    Hi @iamlinkus, case update, I had problems when deploying my project, the error occurred when reading:

    credentials: JSON.parse(process.env.PAYLOAD_PUBLIC_GCS_CREDENTIALS.replace(/\n/g, ''))

    The solution is to read the json file directly:

    keyFilename: path.resolve(__dirname, "./gcs-credentials.json")
  • default discord avatar
    iamlinkus
    2 months ago

    I’ve managed to get it working with default code from the readme, the problem specifically about the gcs creds was within the stringified version of my service account key. Locally the key string stored in the .env file had to be wrapped in single quotes, but when deploying and using cloud run variables I had to remove the quotes and all worked well



    I wouldn’t advise keeping the gcs creds inside the repo @Step , better use cloud run env variables, that way anyone with access to your repo/code will not get access to your gc services

  • default discord avatar
    Step
    2 months ago

    @iamlinkus I understand, and how did you manage to read the

    .env

    variables? I'm doing it this way locally and I'm getting an error:

    credentials: JSON.parse(process.env.PAYLOAD_PUBLIC_GCS_CREDENTIALS)
  • default discord avatar
    iamlinkus
    2 months ago

    are you asking how I managed to read it locally?



    if you’re asking how I read it on the cloud run deployment, in cloud run go into your service, click “edit revision” and there you can add all your env variables



    Cloud run -> YOUR SERVICE -> Edit & deploy new revision -> Environment variables

  • default discord avatar
    Step
    2 months ago

    exactly, how did you do to read them locally?

  • default discord avatar
    iamlinkus
    2 months ago

    I didn’t change anything with regards to the reading locally. Default .env file with all the variables (without the PUBLIC_ prefix).

Open the post
Continue the discussion in Discord
Like what we're doing?
Star us on GitHub!

Star

Connect with the Payload Community on Discord

Discord

online

Can't find what you're looking for?

Get help straight from the Payload team with an Enterprise License.