User CSV Import for a Collection

jj22wham

3 months ago

6 1

• Selected Answer
  

  DanRibbens

  Payload Team

  2 months ago

  I can think of two good ways to get this done. I typically write an import script that starts up Payload with local: true, reads the CSV file and loops over the rows, mapping data to a data object and calls save on your collection(s). The other option is more work, but you can create a custom endpoint that handles the import from an uploaded file along with adding a custom component in Payload to the UI for users to submit data to.

  Here is a limited example of what I've used for an import script:

  import dotenv from 'dotenv';
  import path from 'path';
  import csv from 'csvtojson';
  import payload from 'payload';
  import { User } from '../payload-types';
  import { UserSource } from './types'; // I like to make types as all strings of my CSV data so that I know I have the field names correct, especially from legacy systems with poor naming conventions.
  
  dotenv.config();
  payload.init({
    secret: process.env.PAYLOAD_SECRET,
    mongoURL: process.env.MONGO_URL,
    local: true,
  });
  
  (async () => {
    const usersSource: { [key: string]: UserSource } = {};
    const csvPath = path.resolve(__dirname, 'source/users.csv');
    csv()
      .fromFile(csvPath)
      .then(async (fromCSV) => {
        fromCSV.forEach((source: UserSource) => {
          usersSource[source.id] = source;
        });
        // eslint-disable-next-line no-restricted-syntax
        for (const key of Object.keys(usersSource)) {
          const source = usersSource[key];
  
          const data: Omit<User, 'id' | 'createdAt' | 'updatedAt'> = {
            id: source.ID,
            email: source.EMAIL,
            name: source.NAME,
            // other fields you might have
          };
          try {
            // eslint-disable-next-line no-await-in-loop
            await payload.create({
              collection: 'users',
              overrideAccess: true,
              data,
            });
            console.log('created: ', data.email);
          } catch (e) {
            console.log(e);
            // eslint-disable-next-line no-continue
            console.log('skip: ', data.name);
            // eslint-disable-next-line no-continue
            continue;
          }
          console.log('DONE');
        }
      })
      .then(() => process.exit(0));
  })();

  This is a ts file, which you can run with npx ts-node -T imports/users.ts modifying the path for the directory you want.

  These tend to get longer and more complex if you need to also save relationships, but it all be handled this way as well.

  For users specfically, if you're using the local authentication (default) auth, you'll want to have a workflow for users to reset their passwords with a token for first time on the new system.

  Hopefully this gives you what you need. If you need any help we do offer enterprise support agreements that cover this kind of work.