Simplify your stack and build anything. Or everything.
Build tomorrow’s web with a modern solution you truly own.
Code-based nature means you can build on top of it to power anything.
It’s time to take back your content infrastructure.

several webpack polyfill errors with node:fs, jsdom and fast-csv

default discord avatar
djkato6 months ago

Yup, I moved that part to the server.ts and it works.



on wait is everything inside the paylad.config considered browser side? Do I have to somehow do this part outside the app, in the express part and then pass only the data down to payload?



includes:

https://payloadcms.com/docs/rich-text/lexical#lexical-html
import { $generateNodesFromDOM } from "@lexical/html";
import { $getRoot, $getSelection, EditorConfig } from "lexical";
import { JSDOM } from "jsdom";
import { createHeadlessEditor } from "@lexical/headless"; // <= make sure this package is installed
import {
  getEnabledNodes,
  sanitizeEditorConfig,
  defaultEditorConfig,
} from "@payloadcms/richtext-lexical";
import { readFileSync } from "node:fs";
import { Payload } from "payload";
import { parse } from "fast-csv"

async function parse_csv(csv: string): Promise<CsvType> {
  return new Promise((resolve, reject) => {
    let cumulator = []
    const parser = parse({ headers: true })
    parser.on("data", (data) => { cumulator.push(data) })
    parser.write(csv)
    parser.end()
    parser.on("end", () => resolve(cumulator))
    parser.on("error", (reason) => reject(reason))
  })
}

export const migration = async (payload: Payload): Promise<void> => {
  const csv_text = readFileSync("./old-blogs.csv").toString();
  const csv_data = await parse_csv(csv_text)

  const headlessEditor = createHeadlessEditor({
    nodes: getEnabledNodes({
      editorConfig: sanitizeEditorConfig(defaultEditorConfig),
    }),
  });

  for (const row of csv_data) {
    headlessEditor.update(
      () => {
        const dom = new JSDOM(row.HTML);
        const nodes = $generateNodesFromDOM(headlessEditor, dom.window.document);
        $getRoot().select();
        const selection = $getSelection();
        selection.insertNodes(nodes);
      },
      { discrete: true }
    );
    const obsah = headlessEditor.getEditorState().toJSON();
    payload.create({
      collection: "clanok",
      data: {
        autor: {
          relationTo: "autor",
          value: editor.id.toString()
        },
        nadpis: row.title,
        createdAt: row.created_at,
        updatedAt: row.updated_at,
        obsah: { root: obsah.root }
      },
    });
  }

};


Hello, I'm trying to import a csv and convert it to collection data, but both "fast-csv" module and "node:fs" throw polyfill issues.


import { readFileSync } from "node:fs";
import { parse } from "fast-csv"

I'm trying to read a file, and then parse it.



Also, while doing lexicals headless html parsing to parse my old blog texts from html to lexical, webpack complains about it too.



ERROR in ./node_modules/jsdom/lib/jsdom/living/xhr/XMLHttpRequest-impl.js 4:22-46
Module not found: Error: Can't resolve 'child_process' in '/home/node/app/node_modules/jsdom/lib/jsdom/living/xhr'

ERROR in ./node_modules/jsdom/lib/jsdom/living/websockets/WebSocket-impl.js 3:16-30
Module not found: Error: Can't resolve 'url' in '/home/node/app/node_modules/jsdom/lib/jsdom/living/websockets'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
    - add a fallback 'resolve.fallback: { "url": require.resolve("url/") }'
    - install 'url'
If you don't want to include a polyfill, you can use an empty module like this:
    resolve.fallback: { "url": false }


ERROR in ./node_modules/@fast-csv/parse/build/src/CsvParserStream.js 5:17-34
Module not found: Error: Can't resolve 'stream' in '/home/node/app/node_modules/@fast-csv/parse/build/src'

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
    - add a fallback 'resolve.fallback: { "stream": require.resolve("stream-browserify") }'
    - install 'stream-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
    resolve.fallback: { "stream": false }
    Star on GitHub

    Star

    Chat on Discord

    Discord

    online

    Can't find what you're looking for?

    Get dedicated engineering support directly from the Payload team.