Text lists often arrive with repeated values, inconsistent spacing, blank rows, or an inconvenient order. This tool applies a configurable cleanup pipeline to newline-separated input and updates the result as you type.
The defaults use natural A-to-Z ordering, remove duplicates, trim spaces, and remove empty lines. Change those controls when whitespace, casing, or input order carries meaning.
How to Use the Line Sort and Dedupe Tool
Paste a list, then select the cleanup and ordering rules appropriate to the data.
- Paste or type newline-separated values into the Lines field.
- Pick an ordering from the Order select: "A → Z, natural (2 before 10)", "A → Z, character by character", Z → A, By length, Reverse current order, Shuffle, or Keep order.
- Leave Remove duplicates selected to keep the first occurrence of each matching line, or clear it to retain repetitions.
- Select Ignore case when case differences should not affect duplicate matching or sort collation.
- Select or clear Trim spaces and Remove empty lines according to whether surrounding whitespace and blank rows matter.
- When using Shuffle, select Shuffle again to produce another order.
- Review Result and the line count, then select Copy.
Natural, Alphabetical, Length, and Manual Ordering
Natural ordering uses Intl.Collator with numeric comparison, so digit sequences are compared numerically: item2 sorts before item10. A → Z, character by character uses the same browser collation without numeric comparison. Z → A reverses that comparison rather than simply reversing a previous result.
Collation uses the browser's default locale. Consequently, punctuation, accents, capitalization, and non-Latin scripts may not sort like raw Unicode code points or like a server configured with another locale. Ignore case changes the collator sensitivity to base, which may also reduce distinctions involving accents during ordering, depending on the locale.
By length compares JavaScript string length, which counts UTF-16 code units rather than user-perceived characters. Lines of equal length are then ordered with the collator. Some emoji and characters outside the Basic Multilingual Plane occupy two code units, so visual length and measured length can differ.
Reverse current order reverses the input sequence after the selected trimming, blank-line removal, and deduplication steps. Keep order performs those cleanup steps without sorting. Neither option remembers an older Result value; the current Lines input is processed from the beginning each time.
How Duplicate Removal and Whitespace Cleanup Work
When Trim spaces is selected, leading and trailing whitespace is removed from each line before duplicate comparison. With it cleared, the original surrounding whitespace remains, so apple and apple are different duplicate keys. Remove empty lines checks the trimmed content, which means it removes lines containing only spaces even when Trim spaces itself is off.
Duplicate removal keeps the first matching occurrence before ordering. With Ignore case cleared, matching is exact. With Ignore case selected, each line is lowercased for the duplicate key, so Apple and apple collapse to the first version encountered.
Case-insensitive deduplication is simpler than locale-aware collation. It does not normalize Unicode or use locale-specific casing, so visually similar strings, composed and decomposed accents, and language-specific case rules may remain distinct. Normalize or apply domain-specific matching before using the tool when identity rules matter.
The status reports the number of output lines and, when applicable, the duplicate count removed. Blank lines removed by Remove empty lines are not included in that duplicate count. Output is joined with line-feed characters, so original Windows CRLF line endings are not preserved.
Shuffling Lists and Understanding Randomness
Shuffle applies a Fisher-Yates shuffle after cleanup and duplicate removal. The helper chooses indices with crypto.getRandomValues and rejection sampling, avoiding the modulo bias produced by a naive remainder operation. Shuffle again reruns that process against the same current input and settings.
Cryptographically sourced random indices do not turn the feature into an auditable drawing system. The page does not save a seed, an execution record, or a reproducible permutation, and the result can change on each run. Use a purpose-built, reviewed process for regulated selections, contests, or security decisions.
If reproducibility matters, Keep order and perform a seeded shuffle in code where the seed and algorithm can be recorded. If you only need distinct IDs rather than reordered text, use the random string generator or UUID v7 generator according to the identifier requirements.
Repeated shuffling also does not improve data quality. Clean and validate the list first, decide whether case and whitespace are significant, and keep an original copy when the initial sequence has business meaning.
Edge Cases, Performance, and Privacy
The input is split on line-feed characters with optional preceding carriage returns. A file that uses lone carriage returns as separators is not split into separate lines by this implementation. Embedded line breaks cannot remain inside one item because the newline is the record boundary.
Large lists occupy memory as the original text, an array of lines, duplicate keys, and the joined result. Sorting also takes more work than Keep order or Reverse current order. For very large files or repeatable pipelines, a local script with explicit encoding, locale, and resource limits is a more dependable choice.
Sorting, cleanup, and shuffling happen locally in the browser; the tool code does not upload the list. Nevertheless, copied results enter the system clipboard, and lists can contain emails, customer identifiers, internal hostnames, or secrets. Redact sensitive values and follow the data-handling rules that apply to the source.
If changes need to be reviewed instead of accepted immediately, compare the original and result with the text diff checker. That makes removed duplicates and reordered lines easier to inspect.
Frequently Asked Questions
What is the difference between natural and character-by-character sorting?
Natural sorting enables numeric comparison, so item2 precedes item10. Character-by-character sorting leaves numeric comparison off. Both modes use the browser's default-locale collator rather than a raw byte or Unicode code-point ordering.
Which duplicate is kept?
The first matching line in the current Lines input is kept. Duplicate removal happens before sorting or shuffling, so the selected order does not decide which spelling or capitalization survives. Enable Ignore case only when those case variants represent the same item.
Does Ignore case also ignore accents?
For duplicate removal, it only compares lowercased strings and does not normalize accents. For sorting, it uses base collation sensitivity, whose treatment of accents depends on the browser's locale. Do not use this option as a substitute for explicit Unicode normalization or domain-specific identity rules.
Why did a line containing spaces disappear?
Remove empty lines tests each line after conceptually trimming its whitespace, even if Trim spaces is cleared. A row containing only spaces is therefore treated as empty. Clear Remove empty lines if whitespace-only rows must remain.
Can I reproduce the same shuffled order later?
No seed or shuffle history is exposed, so the page does not provide reproducible shuffles. Shuffle again generates another permutation with browser cryptographic randomness. Record the output itself or use a seeded tool when exact reproduction is required.
References: ECMAScript Internationalization API: Collator · MDN: Crypto.getRandomValues().