Would moving localisation to its own collections similar to versions reduce fear of migration?

default discord avatar
mailaneel11 months ago
11

Currently if we want to migrate a field that is not localized before, we have to write a script to migrate so as to not lose data. What if we do localisations similar to versions where a new collection is created or new record is created within the same collection?



I assume we can do a similar approach for variants too?

https://discord.com/channels/967097582721572934/1132826511280459876

I am new to payload, so exploring options on how to do several things. I am newbie so I might not have full context on these things so please excuse my navieness

  • discord user avatar
    seanzubrickas
    11 months ago

    Hey @Zero I think @denolfe or @Dan Ribbens might be able to lend a hand here. I'll surface this and we should be able to get you some answers. Hold tight!

  • discord user avatar
    dribbens
    11 months ago

    We're working on changes to both localization and migrations so this question comes at an interesting time!



    Migrations will be available so that you have a way of dealing with the moving of existing data as your config evolves.


    I think that alone should be enough to provide a good path forward, but there might be more we can do to simplify this as we go into our localization (roadmap item)[

    https://github.com/payloadcms/payload/discussions/1234

    ].



    It might be that when adding localization where data exists we could simply use the existing string value instead of dropping the value if it isn't an object.



    As for handling localization in the same pattern as versions, I'm not sure exactly on this idea.



    With how we plan to do this in relational databases, if I understand what you're suggesting, we are working towards doing this right now. For MongoDB, I don't think it makes as much sense.

  • default discord avatar
    mailaneel11 months ago

    Thanks that makes sense,



    It might be that when adding localization where data exists we could simply use the existing string value instead of dropping the value if it isn't an object.
  • default discord avatar
    pgrts6 months ago

    Hey @Dan Ribbens or @denolfe, I'll follow up on this topic as I want to localize a payload project that wasn't previously localized. If I understand correctly from this post, I need to create a migration script, but I honestly have no idea where to start. Can anyone help me get started or is there an example of a migration script to prevent me from losing all my data? Thanks!

  • default discord avatar
    dion_jrny6 months ago

    @Zero can you share this script please? 🙏

  • discord user avatar
    dribbens
    6 months ago

    Suppose your collection slug is

    posts

    and you've added

    localization: true

    to a field with

    name: 'title'

    (if you have more just include more in your code)



    If you're doing it in a migration, the field will be localized by Payload, but the db only has a

    string

    instead of object with languages keyed. Because of tht you need to read each document using the database model directly, then save the fields that are now localized using

    payload.updateByID

    .



    To get the model you can use

    payload.db.collections.posts

    or whatever the slug you need is.



    export async function up({ payload }: MigrateUpArgs): Promise<void> {
      const docs = await payload.db.collections.posts.find()
        docs.forEach(async (doc) => {
          await payload.update({
            id: doc._id,
            collection: 'posts',
            data: {
              title: doc.title,
              // any other fields that are needing to be localized
            },
            locale: 'en', // this could be your current locale saved, optional if your defaultLocale is the same as your existing locale (likely the case)
          })
        })
    };


    @PG 👋 I hope this helps ya!



    We should add a sample to the docs



    Also, if you have thousands of docs on your collection, this might be too much memory for your server to find and loop over all docs at once. Instead you could do some simple pagination and filter to fewer docs at time with some extra looping.

  • default discord avatar
    pgrts6 months ago

    Thank you @Dan Ribbens. There is one strange thing happening when running this code. I wonder if this is a bug or that i'm missing something. I'm trying to migrate a rich text field (Slate). If i log const docs in the migration up function i can see that my rich text field exists:

    foo: [ { children: [Array] } ]

    . However, once i add

    localized: true

    to that field, it's not visible anymore in the docs when running

    payload.db.collections.posts.find()

    . It still exists in my mongo db. This only occurs with a rich text field, i tested it with a normal text field and that works just fine. Any idea why payload.db.collections.posts.find() does not show the rich text field?

  • discord user avatar
    dribbens
    6 months ago

    You've saved the data to it and it still isn't in the doc?

  • default discord avatar
    pgrts6 months ago

    @Dan Ribbens Yes, this happens only with a rich text field. And i can see it if i look directly in the mongo db but payload.db.collections.posts.find() doesn't show it.

  • default discord avatar
    ps4masterrace4 months ago

    Did you get it to work on rich text fields?

  • default discord avatar
    pgrts3 months ago

    unfortunately not, still the same issue as mentioned above.

Star on GitHub

Star

Chat on Discord

Discord

online

Can't find what you're looking for?

Get help straight from the Payload team with an Enterprise License.