Set `depth` selectively for rich text field with Slate

We're using Slate as the rich text editor. In there we have two types of relations, one for embedded images (i.e. a relation to our media collection) and one for links (i.e. a relation to the pages collection). As far as I can tell you can only set one value for

when querying this field via GraphQL. But this means the frontend will receive both the full image (which it needs)

the full contents of all pages which are linked to (which it doesn't need, it only needs the ID to generate the link). As far as I can tell there is no way to improve this situation. Am I correct in that?

Continuation and POC: