It will take you about 10 minutes to read this article and in the next 5 minutes (or even faster if you choose the 2nd solution described in the article) you will easily compare two Excel columns for duplicates and remove or highlight the found dupes. Okay, the countdown is started!
Excel is a very powerful and really cool application for creating and processing large arrays of data. Now that you have lots of workbooks with a pool of data, or maybe just one huge table, you may want to compare 2 columns for duplicates and then do something with found entries, for example delete duplicate rows, color dupes or clear the contents of duplicated cells. These two columns may be located in one table, contiguously or non-contiguously, or they may reside in 2 different worksheets or even workbooks.
Say, you have 2 columns with people names - 5 names in column A and 3 names in column B, and you want to compare data between these two columns to find duplicates. As you understand, this is bogus data just for a quick example; in real worksheets you usually have thousands and tens of thousands of entries.
Variant A: Both columns are located on one sheet, in a single table: Column A and Column B
Variant B: Two columns are located on different sheets: Column A in Sheet2 and Column A in Sheet3
The built-in Remove Duplicate tool available in Excel 2016, Excel 2013 and 2010 cannot handle this scenario because it cannot compare data between 2 columns. Furthermore, it can only remove dupes, no other choice such as highlighting or coloring is available, alas :-(.
Further on, I am going to describe 2 possible ways of comparing two Excel columns that let you find and remove duplicate entries:
In our formula, A1 is the first cell of the first column that we want to use for comparison. $B$1 and $B$10000 are the addresses of the first and the last cell of the 2nd column that you want to compare against.Pay attention to the absolute cell reference - dollar signs ($) preceding the column letters and row numbers. I use the absolute reference on purpose, in order for the cell addresses to remain unchanged when copying the formula.
If you want to find dupes in Column B, swap the column names so that the formula looks like this:
Instead of "Unique"/"Duplicate" you can write your own labels, e.g. "Not found"/"Found", or leave only "Duplicate" and type "" instead of "Unique". In the latter case, you will have empty cells next to cells for which duplicates were not found, I believe such presentation is more convenient for data analysis.
Click the left mouse button and holding it down drag the border downward selecting all cells where you want to copy the formula. When all needed cells are selected, release the left mouse button:
Tip: In large tables, it is faster to copy the formula using shortcuts. Click on cell C1 to select it and press Ctrl + C (to copy the formula to clipboard), then press Ctrl + Shift + End (to select all non-empty cells in Column C), and finally hit Ctrl + V (to paste the formula into all selected cells).
Where Sheet3 is the name of the sheet on which the 2nd column is located, and $A$1:$A$10000 are the addresses of the first and last cells of that 2nd column.
Click to download the worksheet with the above examples and the formula to compare 2 columns to find duplicates.
Perfect, we have found the entries in the first column (Column A) that also exist in the second column (Column B). Now we need to do something with them :)
It would be rather ineffective and would take too much time to look through the entire table and review the duplicate entries manually. There are much better ways.
If your columns do not have headers, you need to add them. To do this, put the cursor on the number indicating the 1st row and it will change to a black arrow as shown in the screenshot:
Right click the selected row and choose "Insert" from the context menu:
Give names to your columns, e.g. "Name" and "Duplicate?". Then switch to the Data tab and click Filter:
After that click a tiny grey arrow next to "Duplicate?" to open a drop down list, uncheck all items other than Duplicate in that list, and click OK:
That's it, now you see only those cells of Column A that have duplicated values in Colum B. There are only three such cells in our test worksheet, as you understand in real sheets there are likely to be more, far more of them:
In order to display all rows of Column A again, click the filter symbol in Column B that now looks like a funnel with a tiny arrow and check "Select all". Alternatively, you can do the same via Data tab -> Select & Filter -> Clear, as shown in the screenshot:
If the "Duplicate" flag does not suffice for your purposes and you want to mark duplicated cells by font color or fill color or in some other way…
Then filter the duplicates as explained above, select all filtered cells and press Ctrl + F1 to open the Format Cells dialog box. As an example, let's change the background color of duplicated rows to bright yellow. Of course, you can change the background color of cells using the Fill color option on the Home tab, but the advantage of the Format Cells dialog box is that it lets you make all formatting changes at a time:
Now you definitely won't miss a single duplicated cell:
Filter your table so that only cells with duplicated values show up, and select all those cells.
If 2 columns you are comparing are located on different worksheets, i.e. in separate tables, right-click the selected range and choose "Delete Row" from the context menu:
Click OK when Excel will ask you to confirm that you really want to "Delete entire sheet row" and then clear the filter. As you can see, only the rows with unique values are left:
If 2 columns are located on one worksheet, next to each other (adjacent) or not touching each other (nonadjacent), the removing duplicates is a bit more complex. We cannot delete entire rows that contain duplicate values because this would delete corresponding cells in the 2nd column too. So, in order to leave only unique entries in Column A, you do the following:
As you see, it's not so difficult to remove duplicates between two Excel columns using formulas. Though it's very time-consuming and boring process to write and copy the formula, apply and clear the filter every time you need to compare 2 columns in your worksheets. The other solution I am going to bring to your attention is much simpler and will take just a fraction of time we've spent on the first method. I believe you will find more pleasant things to spend the saved time on ;)
And now let me show you how to compare two columns for duplicates by using our Dedupe tools for Excel.
Note. If you want to compare not just 2 columns, but 2 tables, you need to select the entire first table in this step.
Tip. If you are comparing tables, you can choose several column pairs for comparison, for example, first and last name. For more details, please see How to remove duplicates from two Excel spreadsheets.
Tip. Do not choose to delete duplicates, especially if you are using the tool for the first time. Instead, choose to move dupes to another worksheet. This will remove duplicates from the first table, but gives you a chance to review the list of entries recognized as duplicates. When comparing by several matching columns in large tables, it may happen that you accidentally forgot to select a key column with unique data, and moving duplicates will prevent an irretrievable loss of data.
Remember the previous solution and feel the difference :) It is fast and easy indeed to dedupe your worksheets with Compare Two Tables. In fact, it will take you less time than you've spent on reading this article.
If you have questions or something is left unclear, please drop me a comment and I'll happily elaborate further. Thank you for reading!
Table of contents