V8::FatalProcessOutOfMemory and Unable to List Docs

default discord avatar
58bits
4 months ago
25

So this just happened - we were updating a document on our AWS ECS hosted Payload installation, and the container suffered a V8::FatalProcessOutOfMemory error and shut down. So we increased the memory allocation for this instance, and then restarted the application - but now every time we try to list the Docs collection.... we generate another V8::FatalProcessOutOfMemory. The Docs collection list just sits there with loading skeletons like the attached. So we've connected a local build (on a local machine) to the MongoDB Atlas Serverless instance we're using, and we still see the same - we're unable to list our docs in the collection. The V8::FatalProcessOutOfMemory occured while we were trying to save an update a document. Is it possible the list is now somehow corrupt? Compas, and the MongoDB Atlas web-ui show all 12 document that are present - and at a glance they seem to be okay - although hard to tel. Thoughts?



As an update - we're also generating v8::internal::V8::FatalProcessOutOfMemory errors on our local host trying to list Docs - and this machine has plenty of RAM.



(we're connecting to the same MongoDB Atlas server)



okay found it - it's our fancy

afterRead: [populateLexicalRelationships],

hook - and it happened immediately after creating an internal link (although we'd created several already). Time to debug ;-)



@jacobsfletch @denolfe - okay so this one is definitely on us, but... is it possible to prevent a field

afterRead

hook from executing in a collection list view? And only executing when a document is requested (details view)?



hah -

findMany

-

https://payloadcms.com/docs/hooks/fields

- you guys are amazing :-)



@alessiogr so the problem was caused by the Lexical

afterRead

hook - which is trying to collect additional data for an internal link, but, if two or more documents refer to each other, the the recursion in the hook will trigger infinite reads between the two (or more) documents since each call to

payload.findByID

to find the relationship data will trigger another

afterRead

hook call. @jacobsfletch @denolfe - I don't suppose there's a way to query document data without triggering the field

afterRead

hook?

  • discord user avatar
    jacobsfletch
    Payload Team
    4 months ago

    @58bits just catching up on this thread now, sounds like you've narrowed down the issue to a bad hook. The infinite loop is a common problem, one workaround for this is to set a

    skipHook

    or similar field which will allow you to toggle it off after your first hook fires. The Stripe plugin does something similar with a

    skipSync

    field. You could also query the database directly by using the model. This avoids hooks but it not a widely suggested approach because it's non-standard and might lead to other gotchas.

  • discord user avatar
    alessiogr
    Payload Team
    4 months ago

    One thing Im wondering there: how does the internal slate richtext field do it? Does it have the same issue?

  • default discord avatar
    58bits
    4 months ago

    @jacobsfletch - thanks for the reply - and yup - it's the hook. Could you elaborate a little on where the

    skipHook

    flag would go?

  • default discord avatar
    58bits
    4 months ago

    good question ;-)



    I see - you place a field on the document, and then after the first read or hook set it to true?

  • discord user avatar
    jacobsfletch
    Payload Team
    4 months ago

    You got it 👍

  • default discord avatar
    58bits
    4 months ago

    Heh - okay I've 'almost' got it - but it's late, and a long day of fighting with my computer - but.. how do you turn the flag off after you're done?



    So say we have two documents - Document A and Document B - and each has an internal link to the other. The

    afterRead

    hook fires when we open Document A - and then as we're getting the related link information for Document B - Document B's

    afterRead

    hook fires - which then of course discovers Document A again, and so on.

  • discord user avatar
    jacobsfletch
    Payload Team
    4 months ago
    oic

    it will be more difficult using

    afterRead

    since we're not updating any documents, I'm wondering if

    maxDepth

    might work for you by preventing recursive relationships

  • default discord avatar
    58bits
    4 months ago

    yeah I was looking at that - but we're actually traversing the json document structure and child nodes (Lexical nodes) - since a link could appear one, two, or more levels down in the tree.



    The only way I could see out of it was to somehow perform a read that doesn't trigger the hook - but... as you say we'd have to use the model (Mongoose?) for that?



    (although it's entirely possible I've missed something)



    Okay so I think this works @jacobsfletch @alessiogr - since we have a

    findMany

    Boolean that tells us if the hook was triggered by a findMany query - we can both ignore the collection list view, and only process the

    afterRead

    hook when and individual document is requested. And then change our

    findByID

    query for relationships to a 'many' -

    find

    query with a where clause - which shows up in the hook as a

    findMany === true

    - and so we can ignore it. The only hooks we respond to therefore - are the ones called with

    findMany

    false or undefined.



    It's a bit 'bendy' - but I think it works.



    Will test it over the next few days.



    @alessiogr - it's working - so okay for now - but I suspect we should probably handle internal links a little differently - since as you pointed out previously it would also be nice to have either the slug or the title in the floating link editor for internal links.



    There might also be another more robust way to do this if we only need to traverse one level to get the data we need - buy tracking the IDs of each related document request. Will try this soon.



    @alessiogr @jarrod69420 As an epilogue to this one - for internal links, we moved our hook to

    beforeChange

    - which for the moment is fine, since we're only storing the docID, relationTo, title, and slug (i.e. not a completely nested child document) [edited -

    beforeChange

    not

    afterChange

    ]

Open the post
Continue the discussion in Discord
Like what we're doing?
Star us on GitHub!

Star

Connect with the Payload Community on Discord

Discord

online

Can't find what you're looking for?

Get help straight from the Payload team with an Enterprise License.