Build a CSV Duplicate Remover in JavaScript — No Upload

Try the working MVP

Describe the result, then build it

A paste-and-download CSV cleaner that understands commas, escaped double quotes and newlines inside quoted fields. It preserves the header and original order of retained records.

1. Export a copy of your source data as comma-separated UTF-8 CSV. Keep the original; this tool cannot recover rows you discard.
2. Paste the header and data. Leave the key blank for whole-row equality, or enter one unique exact header such as email.
3. Parse records first, then deduplicate arrays. Do not split the file on commas or newline characters because quoted fields may contain both.
4. Review counts and sample rows, download the CSV, and test an import into a staging destination before using the cleaned data in your workflow.

Build your version with AI

Copy this prompt into your preferred coding assistant. It is a starting specification, not an assurance that generated code is correct.

Review the first draft

Ask next: “Explain each function and its inputs. Identify assumptions that could change the result. Add tests for the edge cases below, then show me the exact files you changed.” Run the tests yourself before replacing the working version.

Understand the important code

parseCSV() tracks quoted fields and escaped quotes instead of using split(). dedupe() uses a Set of JSON-encoded keys to avoid delimiter collisions. serialize() quotes every cell and doubles internal quotes. The download handler creates a temporary Blob URL and revokes it after use.

The tools share one dependency-free script. Its pure functions also run in Node; browser setup only runs when a document exists. This keeps arithmetic and parsing testable without a server.

Full runnable source

This page is the runnable implementation. Reuse its demo markup, the shared script and stylesheet. No build step, API key or database migration is needed.

• Complete JavaScript source
• Tool and print styles

<div data-business-tool="csv"><p>Paste comma-separated text, including the header. Input stays in this page; it is not uploaded or saved.</p><form id="tool-form"><label>CSV input <textarea id="csv-input" maxlength="1000000" required>email,name
[email protected],Alex
[email protected],Alex
[email protected],Sam</textarea></label><label>Key column (optional, exact header name) <input id="key-column" maxlength="200" placeholder="Leave empty to compare whole rows"></label><button>Remove duplicates</button></form><output id="tool-output" aria-live="polite"></output><label>Cleaned CSV <textarea id="csv-output" readonly></textarea></label><button type="button" id="download-csv" hidden>Download CSV</button><p id="tool-status" role="status"></p></div>

For a standalone page, place the markup inside a normal HTML document, link business_tools.css in the head, and load business_tools.js with defer. Keep the IDs unchanged; include only one tool per page.

<link rel="stylesheet" href="business_tools.css">
<script src="business_tools.js" defer></script>

Test the result before publishing

The sample must remove one row. Test quoted commas, embedded newlines, escaped quotes, a missing key and mismatched row lengths. Download and parse the result again; it must contain the same retained values.

Check keyboard navigation, narrow screens and the browser console. In this repository run node tests/business_tools.test.cjs for the calculation/parser checks and python manage.py test for Django routes and rendering.

Publish it

For a standalone tool, upload your HTML, CSS and JavaScript to a static host, enable HTTPS and test the deployed URL. No backend is needed for these local demos.

For this Django site, merge the reviewed branch, install the pinned requirements in your existing virtual environment, run the commands below from the project root and restart the existing Gunicorn service using its configured service name. Nginx must serve STATIC_ROOT at /static/. No new secrets are required.

python manage.py check
python manage.py test
python manage.py collectstatic --noinput

After restarting, open /en/tools/csv-duplicate-remover/, confirm the script and stylesheet return 200, test the demo and check /sitemap.xml. Keep the previous release available for rollback. These instructions do not deploy the site automatically.

Limits and production decisions

Limited to comma-separated input up to 1 MB; no Excel files, semicolon auto-detection or huge-file streaming. Equality is exact and case-sensitive: whitespace is preserved, and no email normalization is attempted. First occurrence wins even if a later record is more complete. Spreadsheet apps may interpret cells starting with =, +, - or @ as formulas; inspect untrusted exports and import as text. The tool preserves values rather than silently changing them.

This page makes no network request with tool input and does not write it to local storage. Data disappears on refresh. Browser extensions and scripts added by your own deployment may change that boundary; review them before processing sensitive information.