Media, S3 and Additional Image Formats

default discord avatar
58bits
7 months ago
17

Hi All - we enabled the cloud storage plugin for S3 today (

https://github.com/payloadcms/plugin-cloud-storage

), and I have to say it was a breeze to setup and configure. The

generateFileURL

collection option was especially helpful - allowing us to 'point' to a CloudFront distribution.



The only thing that we're missing and getting ready to look at is an option for creating additional image formats - in particular Webp. We've looked at

https://github.com/chladog/payload-webp

- which is very nice too - but we'd also like to experiment with a custom collection hook - possibly

beforeOperation

? - where we intercept the incoming file, and perform the conversion based on the received mimeType.



Before we dig into this - was wondering if anyone could offer a likely approach (at a high level) for how this might work when combined with the Cloud Storage plugin. I see from the docs that one of the use cases described for collection hooks is "Send a copy of uploaded files to Amazon S3 or similar" - and so feel pretty sure this is all possible.



Thoughts, suggestions, pointers all greatly appreciated.

  • default discord avatar
    Tinouti
    7 months ago

    Isn't this what the

    imageSizes

    option in your upload collection config is for? 🤔


    (I might be misunderstanding what you're trying to achieve though)



    For example, I have the cloud-storage plugin enabled and configured with GCS, and in my

    imageSizes

    I've got a bunch of different sizes and formats including webp, like:


    {
      width: 720,
      name: 'thumbnail',
      formatOptions: {
        format: "webp",
      },
    },


    So out-of-the-box, whenever I add a new doc in my Media collection, Payload generates all the various sizes and formats I've specified in my config and upload them to my GCS. No hook or anything required. 🙌

  • default discord avatar
    58bits
    7 months ago

    Ouch - I completely misunderstood formatOptions. I thought this was a level up - for

    all

    sizes. Thanks a bunch @Tinouti



    Okay so this works, but I

    think

    I may have found bug @Tinouti @denolfe @jmikrut . Depending on how you order the image sizes, the last formatOptions setting will apply to any subsequent imageSize. For example....



     imageSizes: [
          {
            name: 'thumbnail',
            height: 400,
            width: 400,
            position: 'center',
          },
          {
            name: 'medium',
            width: 900,
            height: 450,
            position: 'center',
          },
          {
            name: 'medium_webp',
            width: 900,
            height: 450,
            position: 'center',
            formatOptions: {
              format: 'webp',
            },
          },
          {
            name: 'large', // <------- this will output as webp
            width: 1200,
            height: 675,
            position: 'center',
          },
          {
            name: 'large_webp',
            width: 1200,
            height: 675,
            position: 'center',
            formatOptions: {
              format: 'webp',
            },
          },
        ],


    The 'switch' to webp at

    medium_webp

    above - will cause normal

    large

    to output as webp.



    Whereas if you place all of the webp formats at the bottom - the

    large

    format comes out in the source format as expected...



     imageSizes: [
          {
            name: 'thumbnail',
            height: 400,
            width: 400,
            position: 'center',
          },
          {
            name: 'medium',
            width: 900,
            height: 450,
            position: 'center',
          },
          {
            name: 'large',
            width: 1200,
            height: 675,
            position: 'center',
          },
          {
            name: 'medium_webp',
            width: 900,
            height: 450,
            position: 'center',
            formatOptions: {
              format: 'webp',
            },
          },
          {
            name: 'large_webp',
            width: 1200,
            height: 675,
            position: 'center',
            formatOptions: {
              format: 'webp',
            },
          },
        ],


    @Tinouti @jmikrut - I just went back and looked at the docs again, as well as took a quick look at the Payload source .

    formatOptions

    is only referred to under 'Collection Upload Options' -

    https://payloadcms.com/docs/upload/overview#enabling-uploads

    - and although I really only spent a few minutes looking at the source - it wasn't clear to me how

    formatOptions

    is working inside the imagesSizes array. Wondering if the the fact that

    formatOptions

    works above at all is a quirk of how the settings are read? For example in the Joi schema here

    https://github.com/payloadcms/payload/blob/db7acb4eddb43200ef00cb3a442779b80723313d/src/collections/config/schema.ts#L159

    -

    formatOptions

    are definitely at the upload collection config level and not inside the imageSizes schema. ¯\_(ツ)_/¯ Hope some of this helps - and apologies in advance if I'm miles out.



    Okay I've just seen the config test here

    https://github.com/payloadcms/payload/blob/faef4d5f8eddddecc491c0f6c3a95be701678b58/test/uploads/config.ts

    - and

    formatOptions

    is at both the upload config and imagesSizes levels.



    And I've just seen the source here -

    https://github.com/payloadcms/payload/blob/1fd61fb9898a08b28b22be61d9b64aa58aa61088/src/uploads/imageResizer.ts#L76

    - so - apologies for the spammy messages. I do however, think there might still be a bug as described in the example imageSizes setting above - with the solution for the moment to place all the 'different' formats together, and after the defaults.

  • discord user avatar
    denolfe
    Payload Team
    7 months ago

    This is a bit hard to follow. Can you create a reproducible example and open an issue?

  • default discord avatar
    58bits
    7 months ago

    Yeah sure. Will open an issue in the morning our time.



    The issue is here

    https://github.com/payloadcms/payload/issues/2526
  • discord user avatar
    denolfe
    Payload Team
    7 months ago

    Fixed in main branch, will be in the next release 👍



    https://github.com/payloadcms/payload/pull/2547
  • default discord avatar
    58bits
    7 months ago

    Awesome - thank you!

Open the post
Continue the discussion in Discord
Like what we're doing?
Star us on GitHub!

Star

Connect with the Payload Community on Discord

Discord

online

Can't find what you're looking for?

Get help straight from the Payload team with an Enterprise License.