Error when uploading video of 1.7 gb

haniel.u2 years ago

18

• 

  denolfe

  2 years ago

  Any terminal errors?

• 

  haniel.u2 years ago

  None

• 

  denolfe

  2 years ago

  Is Payload configured to write uploads to disk or are you using some sort of plugin? How much RAM does your dev machine have? Does the server at least stay running?

  
  
  

  You can use temp files for uploading by putting this in your config

  
  
  

    uploadOptions = {
      useTempFiles: true,
    }

  
  
  

  That is worth a try

• 

  haniel.u2 years ago

  Same thing

  
  
  

  16gb ram 10gb free / 256gb

  
  
  
• 

  denolfe

  2 years ago

  That JSON goes

  on the top level payload config
  
  

  not inside the upload collection

• 

  haniel.u2 years ago

  Fixed it, reloaded page, still same thing

  
  
  

  I think the issue is that when I select the file, it's being loaded into ram and after a certain threshold the browser tab just gives up

  
  
  

  How do I stop the file from being loaded into browser memory?

  
  
  

  I believe that's not the default browser behaviour of

  <input type="file"/>
• 

  janpeini2 years ago

  I guess the server and or browser times out. I would use asnc upload options. For example with axios, where you also get a nice progress event, with which you can track the progress.

  
  
  

  We build a video hosting and encoding solution with payload as a backend/UI and we handle the large uploads via signed URLs directly to AWS s3

  
  
  

  1. Select file via file input

  
  

  2. onChange event to create the signedURL which get's returned

  
  

  3. take this signedURL and upload to this URL via axios

  
  

  4. Render a a progress bar via the axios progress event

  
  

  5. OnComplete: use the Payload local API to create the collection document with the path to the just uploaded file

• 

  haniel.u2 years ago

  This would require a custom admin ui element right?

  
  
  

  How do I change the default Upload UI for the uploads collection?

• 

  janpeini2 years ago

  Yes

  
  
  

  We deactivated the create action by setting the access: create to false and than added a custom button component which handles all the upload stuff

  
  
  

  And than added a custom route which creates the document in the end

  
  
  

  Oh wait, we create the document first which than already contains the signedURL. The code for the route looks like this:

  
  

  import Videos from '../collections/Videos';
  import { CreateNewVideoData } from '../types/custom-types';
  import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
  import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
  
  export const createNewVideo = async (req, res, next) => {
  
      if (req.user && req.body) {
  
          const { filename, extension, slug } = req.body as CreateNewVideoData;
          if (!filename || !extension || !slug) {
              res.status(404).send({ error: 'data is missing' });
              return
          }
  
          const newVideo = await req.payload.create({
              collection: Videos.slug,
              data: {
                  title: filename,
                  slug: slug
              },
          })
  
          const responseObj = req.body
          responseObj.videoId = newVideo.id
  
          // Create signed URL
  
          const bucket = process.env.AWS_S3_BUCKET_IN
          const region = process.env.AWS_REGION
          const credentials = {
              accessKeyId: process.env.AWS_ACCESS_KEY_ID,
              secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
          }
  
          // Create final video file name
          const key = `videos/${responseObj.videoId}/${slug}.${extension}`
          // -------------------------------
  
          const clientParams = {
              region,
              credentials,
          };
          const putObjectParams = {
              Bucket: bucket,
              Key: key,
          }
  
          const client = new S3Client(clientParams);
          const command = new PutObjectCommand(putObjectParams);
          const expiresIn = 3600 // 1 hour
          responseObj.signedUrl = await getSignedUrl(client, command, { expiresIn });
  
          res.status(200).send(responseObj);
          return
  
      } else {
          res.status(403).send({ error: 'not allowed' });
          return
      }
  
  }

  
  
  

  this returns the document object containing the signedURL to our custom upload/create button

  
  
  

  We insert the custom create/upload button via a hook in the collection list view

  
  
  https://payloadcms.com/docs/admin/components#collections
  
  

  The

  BeforeList

  hook fits quite good

• 

  haniel.u2 years ago

  Thank you so much for your explanations/suggestions, I'm going to try them out and let you know how it goes

• 

  janpeini2 years ago

  I'm in the train currently, but trying to upload a video that shows the final solution, so you get a better understanding

• 

  haniel.u2 years ago

  Thank you so much, I would very much appreciate it

• 

  janpeini2 years ago

  
  
  

  here we go
• 

  haniel.u2 years ago

  Interesting

• 

  janpeini2 years ago

  The upload is only a small part of a complete video hosting/encodig solution in this case. So there is a lot more stuff happing after the upload. But the upload should do the trick for you I guess.

  
  
  

  The whole logic is within the button component and the create route

• 

  haniel.u2 years ago

  The documentation for creating the custom elements for the route is in here right, thanks a lot

• 

  janpeini2 years ago

  yes

  
  
  

  I can also share the button component with you if it helps

• 

  haniel.u2 years ago

  Please do 🙏