Diffing two collections

default discord avatar
jakey___11 months ago
6

I've got two identical collections:

MutableData

and

ImmutableData

. I would like to get a diff between what has changed in

MutableData

since the initial cloning.



I'm learning how to do this via mongoose aggregations at the moment, but I'm curious if there is some solution out there to this that someone has already encountered.



Thank you

  • discord user avatar
    dribbens
    11 months ago

    You're already ahead of my knowledge it seems with using aggregations for this. It is essentially just comparing JSON for differences?


    My approach might be to write out the diff as the changes are written instead of trying to compare them in a read operation.

  • default discord avatar
    jakey___11 months ago

    Oh, thanks for responding @dribbens ! I forgot I posted this. I did end up getting an aggregate going that allowed me to tell the docs that changed between two collections.



    [
        {
            $lookup: { // provides the mutable data to compare against
                from: "immutable-data-collection-name",
                localField: "_id",
                foreignField: "_id",
                as: "mutableData",
            },
        },
        {
            $addFields: { // Filters the data down to only that which has changed and puts it in a new field called "diff"
                diff: {
                    $filter: {
                        input: "$mutableData",
                        cond: {
                            $or: [
                                { $ne: ["$$this.Name", "$Name"] },
                                // ...and so on for all the other relevant fields you want to check against when determining an change has happened.
                            ],
                        },
                    },
                },
            },
        },
        {
            $match: { // further filters out any documents that have no changes, leaving only the ones that have changed
                $expr: { $gt: [ { $size: "$diff", }, 0, ], },
            },
        },
    ]


    Then to actually use this in a payload endpoint, I ran code that looked like the following:



    const getDiff = () => {
        return new Promise((resolve, reject) => {
            payload.collections['mutable-data-collection-name'].Model.aggregate(diffAggregate)
            .then((result) => {
                resolve(result)
            })
            .catch((err) => {
                reject(err)
            })
        })
    }


    diffAggregate

    being the initial array of objects code I pasted.



    Props to chatGPT for helping me understand how this could work, lol. Also to @jmikrut for responding with how to access the mongoose model in this channel

    https://discord.com/channels/967097582721572934/1041606968755896370

    . Very helpful!

  • discord user avatar
    dribbens
    11 months ago

    Marking as Answered so it shows up in our site's community help. Great work @jakey___!

Star on GitHub

Star

Chat on Discord

Discord

online

Can't find what you're looking for?

Get help straight from the Payload team with an Enterprise License.