How to clean and compare text lists

Define duplicate items and organize shared entries, differences and selected lines.

Practical guideUpdated Approx. reading time: 3 min
On this page

Define what a line represents

Prepare one item per line, such as a code, label or reference. A row with several columns does not automatically become a structured record. For CSV tables, use converters that understand cells and quoting. In a text list, spaces and letter case may form part of an identifier rather than irrelevant formatting.

Keep a copy before normalization. ANA, Ana and a line with surrounding spaces might represent one person or different codes. Establish that rule before removing duplicates. Retain repetitions when each appearance represents a unit or attendance you must count. A list of unique items answers a different question from a list of all appearances.

Clean whitespace before removing duplicates

Clean text trims line ends and optionally reduces internal spaces or removes blank lines. Enable only necessary changes, since reducing internal spaces alters aligned text. Remove duplicate lines retains the first appearance, and whitespace remains part of each entry unless you clean it first.

Test a sample containing a repeated entry, surrounding spaces and an uppercase variant. Inspect case-sensitive options before choosing the final rule. Copy cleaned output before opening duplicate removal because content does not transfer automatically. Retain those settings with your final version so later updates can use the same criteria rather than gradually changing what counts as a matching item.

Ask a precise comparison question

Compare lists provides common items, only in A, only in B and a union. To find missing delivery references, place expected entries in A and received entries in B, then choose Only in A. Swapping lists changes what the difference means. A union includes entries from both without counting repeated appearances.

Blank lines are ignored and the first appearance is retained. You can ignore case and trim surrounding spaces. For A containing Ana, Luis and Ana, and B containing Luis and Marta, the common item is Luis and the union is Ana, Luis, Marta. Check one item in each group before using a large comparison.

Filter, sort and prepare the final output

Filter lines retains or excludes literal matches: .pdf searches that string without treating the dot as a regular expression. It preserves order, spaces and repetitions. Sort lines changes positions and compares numbers naturally, so 2 precedes 10. Filter before numbering if numbers should describe the final list.

Prefixes and suffixes prepare labels; Find and replace changes literal matches. Inspect terms appearing several times and decide whether every occurrence should change. Count entries removed by each operation and download or copy the finished version before navigating elsewhere. When the source changes, repeat from that source with your recorded settings rather than an already transformed copy.

TRY IT YOURSELF

Common items, differences and union

Paste Ana, Luis and Ana on separate lines into A, and Luis and Marta into B. Choose Common items and compare, then try Only in A, Only in B and Union without changing inputs.

List A:
Ana
Luis
Ana

List B:
Luis
Marta

Common items gives Luis; Only in A gives Ana; Only in B gives Marta. Union is Ana, Luis, Marta, in that order, one item per line.

Add surrounding spaces to Ana and change the trimming setting. Observe how your equality definition changes the comparison before applying it to identifiers where spaces can matter.