Logging Payload API Requests

default discord avatar
58bits
5 months ago
34

Very cool to see Payload using Pino logger, but was curious to know if it's possible to log all Payload's api requests? Or would this have to be configured in separately in Express? I currently have

loggerOptions

set to

info

in

server.ts

- and see startup messages via Pino, but I'm not seeing any api requests.

  • default discord avatar
    thisisnotchris
    5 months ago

    @58bits Good question - I wonder if Pino's config can be extended to catch the missing requests?



    I know the default level is "info", how about debug or trace?



    If that exposes the payload requests, you could check the level of those and add a custom logging level



    That's my best guess, I'll keep my eye open for other possibilities, though another community member or a team member may have more insight

  • default discord avatar
    58bits
    5 months ago

    Hi! @thisisnotchris I've tried debug and trace but those don't add request logging.



    For context - we use CloudWatch Logs Insights extensively with all of our deployed APIs (typically with custom 'topic' properties in log messages), and if we were to deploy Payload CMS to AWS we'd ideally like to do the same.

  • default discord avatar
    thisisnotchris
    5 months ago

    Hmm



    @58bits Did you say you tried to add a middleware to express, prior to Payload?

  • default discord avatar
    58bits
    5 months ago

    No not yet.

  • default discord avatar
    thisisnotchris
    5 months ago
    const requestLoggerMiddleware = ({ logger }) => (req, res, next) => {
      console.log('Should run on every request')
      next();
    };


    app.use(requestLoggerMiddleware)

  • default discord avatar
    58bits
    5 months ago

    Thanks - yup - was about to try that.

  • default discord avatar
    thisisnotchris
    5 months ago

    Hopefully that exposes all the requests!



    Let us know

  • default discord avatar
    58bits
    5 months ago

    @thisisnotchris close....



    const requestLoggerMiddleware =
      ({ logger }) =>
      (req, res, next) => {
        logger.info(`request: ${req.method} ${req.url}`)
        next()
      }
    
    const start = async () => {
      // Initialize Payload
      await payload.init({
        secret: process.env.PAYLOAD_SECRET,
        mongoURL: process.env.MONGODB_URI,
        express: app,
        loggerOptions: {
          level: 'debug',
        },
        onInit: async () => {
          payload.logger.info(`Payload Admin URL: ${payload.getAdminURL()}`)
        },
      })
    
      app.use(requestLoggerMiddleware({ logger: payload.logger }))
    
      // Add your own express routes here
    
      app.listen(process.env.PORT, async () => {
        payload.logger.info(`Server listening on port ${process.env.PORT}`)
      })
    }
    
    start()
  • default discord avatar
    thisisnotchris
    5 months ago

    Ooooo

  • default discord avatar
    58bits
    5 months ago

    I can pass in a configured instance of payload - but for any route that matches the payload runtime, the middleware will not log the request



    for example if I pass an API request to

    /api/foo

    (not a payload collection) then I

    will

    get a middleware logged request - since express is handling this route now



    So I suspect Payload needs to decide if it's going to log its own handled requests.

  • default discord avatar
    thisisnotchris
    5 months ago

    Hmmm



    @jacobsfletch Any ideas on how they could get some extended logging?

  • discord user avatar
    jacobsfletch
    Payload Team
    5 months ago

    @58bits @notchris Using custom middleware is the right approach here but I think you need to define it at the

    router

    level, like this:



    import express from "express";
    import payload from "payload";
    
    const app = express();
    
    const start = async () => {
      // Initialize Payload
      await payload.init({
        secret: process.env.PAYLOAD_SECRET,
        mongoURL: process.env.MONGODB_URI,
        express: app,
        loggerOptions: {
          level: 'debug',
        },
        onInit: async () => {
          payload.logger.info(`Payload Admin URL: ${payload.getAdminURL()}`)
        },
      })
    
      const router = express.Router();
    
      router.use(requestLoggerMiddleware({ logger: payload.logger });
    
      app.listen(process.env.PORT, async () => {
        payload.logger.info(`Server listening on port ${process.env.PORT}`)
      })
    }
    
    start()
  • default discord avatar
    58bits
    5 months ago

    @jacobsfletch @thisisnotchris <plays sad trombone> - sorry Jacob but this didn't work either. Highly likely I'm doing something wrong here, as we're not very familiar with Express (we're a Fastify shop).

  • discord user avatar
    denolfe
    Payload Team
    5 months ago

    It appears this is an undocumented feature but might work for you - You could try sending in some logging middleware into

    preMiddleware

    in your Payload config via the

    express

    options



    postMiddleware

    is available as well

    CleanShot_2023-04-14_at_10.12.02.png
  • default discord avatar
    58bits
    5 months ago

    @denolfe thanks a bunch - will give it a try. Was also going to

    guess

    that Payload's middleware / handlers are not calling

    next()

    - and so we might try registering our own logger middleware

    before

    payload.

  • discord user avatar
    denolfe
    Payload Team
    5 months ago

    It all depends on if you want to use Payload's pino logger instance or not

  • default discord avatar
    58bits
    5 months ago

    Understood. Thanks @denolfe @jacobsfletch @thisisnotchris - as per Elliot's suggestions, this works...


    Middleware:


    const requestLoggerMiddleware = (req, res, next) => {
      req.payload.logger.info(`request: ${req.method} ${req.url}`)
      next()
    }

    And then in

    payload.config.ts
     express: {
        preMiddleware: [requestLoggerMiddleware],
      },
  • discord user avatar
    denolfe
    Payload Team
    5 months ago

    Beautiful! 🙌



    Also heads up on json logging w/ Pino (especially if querying in cloudwatch). If you want to have more json properties, you'll want to log an object w/

    msg

    property for the message. Multiple parameters didn't appear to pass to json properly despite the TS type saying so.



    req.payload.logger.info({ msg: 'my message', method: req.method, url: req.url})
  • default discord avatar
    58bits
    5 months ago

    @denolfe - thanks - and yup - that's how we're doing it in production with a topic top level property, followed by the payload / params.



    and so we would probably do something like this ( although it's late here, so ¯\_(ツ)_/¯)


    req.payload.logger.info({ payload_request: { method: req.method, url: req.url } })
  • discord user avatar
    denolfe
    Payload Team
    5 months ago

    For anyone stumbling upon this thread, the mentioned custom middleware above is now documented here:

    https://payloadcms.com/docs/configuration/express#custom-middleware
Open the post
Continue the discussion in Discord
Like what we're doing?
Star us on GitHub!

Star

Connect with the Payload Community on Discord

Discord

online

Can't find what you're looking for?

Get help straight from the Payload team with an Enterprise License.