Simplify your stack and build anything. Or everything.
Build tomorrow’s web with a modern solution you truly own.
Code-based nature means you can build on top of it to power anything.
It’s time to take back your content infrastructure.

How does one handle syncing uploads between environments?

default discord avatar
jakey___2 years ago
7

I've noticed that if you just copy an asset into the corresponding upload folder, it doesn't change anything in the CMS to allow you to see it / use it.



This makes me wonder how the use case of syncing your local images up to a remote host should be handled. Is there a "sync-to-remote" functionality in Payload I'm not aware of? Or will I have to write up a script that also merges in the data from local to remote?



Any feedback welcome, thank you.



So far the most common response I'm getting is to utilize a CDN instead of relying on local storage, which is probably why this use case isn't accounted for in Payload?

  • default discord avatar
    thgh.2 years ago
  • default discord avatar
    see.ess2 years ago

    Are you talking uploading media in a local vs remote environment? Or are you asking for a way for Payload to detect you've copied a

    .jpg

    into your uploads folder and create a record for it?

  • default discord avatar
    jakey___2 years ago

    at my company, it's common to have a local, qa, and prod environment. local for code changes, qa for review of code changes, then prod is typically left for content management. We never sync data to prod since its data is most up to date. So as you might imagine that's an annoying scenario to deal with at times.



    In this workflow, we'll typically sync our media (images/videos/etc) to the media folder on the other two environments (since typically they don't override existing data, just add on). However, in both Payload and WP, the db state doesn't recognize the new images that were synced up and thus you can't just pick the new items for future content up dates either.



    I'm definitely in favor of instead relying on a CDN solution like that which

    @476523795251855360

    shared. I was just curious if Payload had some way of addressing this since it is one of those annoying things that happens in WP for instance.

  • default discord avatar
    see.ess2 years ago

    Yeah, kinda figured that's what you're referring to... and I agree it's a pain (especially when building first-class features that depend on the presence of a database record). I've dealt with it in WordPress pretty easily with some abhorrent hacks... not as easy when you're trying to do things the right way. I'm working towards v1 of a site for my client now and am getting close to having to deal with this issue on Payload.



    Not sure if a more general mongo diff/migration tool would help?

    https://github.com/datacoon/diffmongo

    or here's a gist of something relatively simple that you might be able to build on:

    https://gist.github.com/niccottrell/3ff2efc2146090aa8c5b73c1eb75ba2c
  • default discord avatar
    jakey___2 years ago

    Interesting…. This seems like it’ll be useful anyway for comparing environment differences. Thanks!!

  • default discord avatar
    see.ess2 years ago

    fwiw, I managed to hack together this typescript version of the above gist...

    https://gist.github.com/marpstar/b257ceb628a428ec66aabde72c0cdbc4

    it runs on the entire collection, not just a sample. Also allows you to specify different DB names for each environment. I've included instructions for launching (and debugging) for

    ts-node
  • default discord avatar
    jakey___2 years ago

    much appreciated!

Star on GitHub

Star

Chat on Discord

Discord

online

Can't find what you're looking for?

Get dedicated engineering support directly from the Payload team.