Simplify your stack and build anything. Or everything.
Build tomorrow’s web with a modern solution you truly own.
Code-based nature means you can build on top of it to power anything.
It’s time to take back your content infrastructure.

How to download / fetch a file bypassing access controls as the server?

default discord avatar
mobeigilast year
2

Howdy,



My use case is I have a custom route and want to fetch a raw file from within the server itself.



I use payload to get the

global

that has my file with no issues:


{
  id: 1,
  resumeFile: {
    id: 2,
    title: 'Test',
    updatedAt: '2024-09-24T12:13:05.928Z',
    createdAt: '2024-09-24T12:13:05.928Z',
    url: '/api/private-files/file/test.pdf',
    thumbnailURL: null,
    filename: 'test.pdf',
    mimeType: 'application/pdf',
  },
}


My

private-files

collection:


export const PrivateFiles: CollectionConfig = {
  slug: 'private-files',
  labels: { singular: 'Private File', plural: 'Private Files' },
  access: {
    create: authenticated,
    delete: authenticated,
    read: authenticated,
    update: authenticated,
  },
  fields: [
    {
      name: 'title',
      type: 'text',
    },
  ],
  upload: {
    staticDir: path.resolve('private/uploads/files'),
  },
};


So by design only authenticated users can view the file. However, the 'server' should also be able to get the file.



How exactly do I go from:

/api/private-files/file/test.pdf

to a raw binary file. Using

fetch

will run into auth issues. I can't seem to find any sort of

payload

wrapper around fetch that can do this (something like

payload.fetch(url)

with free auth baked in for the server would do the trick.



I don't want to use the hardcoded file path

private/uploads/files

obvioulsy because I want to use payload to abstract away that implementation detail.



Okay so from what I gathered, there is no built in way to do this so you have to authenticate with Payload even if the request is local to the server.



I believe there are 2 options:


1. Tweak your access controls so they detect the IP address in the request as local.


2. Use a system user account designed to do this and authenticate with user/pass or API key.



I opted for 2nd approach as it seems cleaner.



Docs:


https://payloadcms.com/docs/authentication/config

Add to

User

collection:


  auth: {
    useAPIKey: true,
  },


Then make a new user in UI to be the

System

account.


Make an API KEY for this user, and use that in your app for local HTTP requests you need to authenticate.



Request looks like:


import User from '../collections/User'

const response = await fetch('http://localhost:3000/api/private-files/file/test.pdf', {
  headers: {
    Authorization: `${User.slug} API-Key ${SYSTEM_USER_API_KEY}`,
  },
})


Its a little weird using an API Key to authentication as a user for a local server request, ideally I'd want some way to bypass auth locacally directly via payload but I couldn't figure out how to do that.



@360823574644129795

Sorry to ping you directly. Do you know if the above approach is current best practice? I'd hate to be missing something obvious for the above use case.

  • discord user avatar
    alessiogr
    last year

    Since this is all on the server, you can just use

    fs

    to access the file directly. Much more efficient than doing a fetch call when you're already on the same server.


    Could access

    PrivateFiles.upload.staticDir

    to get the base path. Calling the exposed api wouldn't abstract anything useful. The only useful thing it would add is access control, which you don't need



    What you might find useful, this is how (in v3) api calls to the files are handled:

    https://github.com/payloadcms/payload/blob/beta/packages/next/src/routes/rest/files/getFile.ts#L57

    Copy as much as you need (e.g. how we are constructing the file path). If some upload collections use handlers, e.g. through using the plugin cloud storage plugin, it might make sense to check for

    collection.config.upload.handlers

    as well, just like we are doing in our codebase

  • default discord avatar
    mobeigilast year

    Thanks for the response! 👍



    Importing the

    PrivateFiles.upload.staticDir

    makes it much cleaner to avoid hard coding a path.



    I ended up implementing it with the system API key approach (that I also might reuse in another scenarios where I want to hit the API internally as authenticated). The one up side as you mention is the ability to enforce access controls which I may add in the future.



    I do wonder if a

    payload.fetch

    wrapper or some sort of

    payload.systemToken

    value that was available on the server itself to allow full authenticated bypass access on the server would be valuable to have. However, with the exception of getting a file you can just use the

    payload.find

    methods etc to get collections/globals and

    overrideAccess

    so it might be too niche.



    Anyway, its working for me now. One more blocker gone and one step closer to release 😄



    Thank you!

Star on GitHub

Star

Chat on Discord

Discord

online

Can't find what you're looking for?

Get dedicated engineering support directly from the Payload team.