How to Easily Find Duplicate Files in Hard Drive on Windows

If you're like most computer users, your hard drive can quickly become a cluttered mess of files, and duplicate files are often a major contributor to this chaos. Not only do duplicate files waste valuable storage space, but they can also make it challenging to locate the files you actually need. In this article, we will explore effective methods to find and eliminate duplicate files on your Windows PC.

Where do duplicate files come from?

Duplicate files can accumulate on a Windows computer for various reasons, and they are typically stored in different locations on the hard drive.

▶ Why are there duplicate files?

File copying: When you copy files from one location to another, whether within the same drive or to a different drive, you create duplicate files. These duplicates are usually stored in the destination folder.

File downloads: Downloading files from the internet can result in duplicates. For example, if you download the same file multiple times or download updates for software, you may end up with multiple copies of the same file in your download folder.

Backup and restore: Creating backups of your data, especially if done manually, can lead to duplicate files. If you restore files from a backup, you may inadvertently create duplicates in the original location.

File synchronization: File synchronization tools and cloud storage services can sometimes create duplicates. For example, if you sync a folder to a cloud service and then sync it back to your computer, duplicates may occur.

File versioning: Some software applications automatically create versioned copies of files. For instance, when you save changes to a document in Microsoft Word, it may create a backup copy of the previous version.

Media management: Media players and photo management software often create thumbnails or catalog files that can result in duplicates of media files.

▶ Where are duplicate files stored on the hard drive?

As for where duplicate files are usually stored on the hard drive, they can be scattered across various folders and directories, depending on how they were created.

User documents: Duplicate documents, images, and media files may be found in user-specific folders such as "Documents," "Pictures," and "Music."

Downloads: Duplicate files downloaded from the internet can accumulate in the "Downloads" folder.

Desktop: Some users may save files directly to the desktop, leading to duplicate files on this location.

Program installations: Some software installations may include duplicate files or create backup copies during the installation process.

Backup folders: If you have a backup or synchronization software in use, you may find duplicates in backup or sync folders.

Temporary folders: Temporary folders used by software or the Windows operating system may also contain duplicates.

System folders: Duplicate system files can sometimes be found in Windows system folders, but this is less common.

2 ways to find duplicate files in hard drive and remove them

Finding and removing duplicate files might be difficult. It mostly necessitates a substantial amount of labor, and finding and comparing two files is not exactly simple. Furthermore, the two almost similar files may, in fact, be distinct. In this section, we'll teach you how to utilize PowerShell and a simple duplicate file finder to scan the whole hard disk and erase duplicate files.

Method 1. Use PowerShell to find duplicate files

Step 1. Open PowerShell on your Windows computer. You can search for “PowerShell" in the start menu and run it as administrator.

Step 2. Use the cd (change directory) command to change the current working directory to the folder that you want to search for duplicates. For example, if you want to find duplicate files in the "Downloads" folder, enter the following command:

cd C:\Users\YourUsername\ Downloads

Then press Enter on your keyboard.

Step 3. Run the following command to count the hash value of each files in this folder and find duplicate files:

Get-ChildItem -Recurse | Get-FileHash | Group-Object -Property Hash | Where-Object { $_.Count -gt 1 } | Select-Object -ExpandProperty Group | Select-Object -ExpandProperty Path

Press Enter. Allow the command execution to finish. Depending on the amount and size of files in the folder, this may take some time. The files in this folder will then be grouped based on their hash.

Step 4. Now you can run the following command to remove all duplicate files except one from each group:

Get-ChildItem -Recurse | Get-FileHash | Group-Object -Property Hash | Where-Object { $_.Count -gt 1 } | foreach {

$group = $_.Group | Select-Object -ExpandProperty Path

$group | Select-Object -Skip 1 | Remove-Item -Force

}

Press Enter. Return to the folder that previously contained duplicate files. All duplicate files with identical contents (independent of file name) will be eliminated.

Method 2. Use duplicate file finder to scan and remove duplicate files

As you see, although you can use PowerShell to find duplicate files on a hard drive, the commands you’ll use are very complex. And you can’t even preview those files to decide whether to delete them or not.

In this case, we suggest you choose a duplicate file finder. Here, we recommend you to AOMEI Partition Assistant for Cleanup. It is a disk and partition management tool that supports Windows 11/10/8/8.1/7.

This tool provides users with a fast and effective solution. It conducts quick scans to identify duplicate files, enabling users to set up automated cleaning routines for their devices. This software is versatile, supporting various file types like images, videos, audio, documents, and archives. Crucially, it intelligently excludes essential file types like .dll, .lib, .sys, and others, ensuring the integrity and performance of the system remain intact.

AOMEI Partition Assistant for Cleanup

Free Up Disk Space To Keep Your PC Running Smoothly and Efficiently

Free Download
Windows PCs and Servers

Step 1. Initiate Find Duplicate Files. Install and launch AOMEI Partition Assistant for Cleanup. Click the "Find Duplicate Files" tab.

Step 2. Select a path to scan duplicate files. You can either click "Scan Now" to start the scan of your system C: drive. Or, click "Custom Scan" to manually select paths to be scanned. (Here, we take the "Custom Scan" as an example.)

Step 3. After clicking "Custom Scan", all local and external drives on your PC will be displayed. You can also click the "Add Path" button, and then a browse folder window will pop up. So you can add more paths on this PC, like Desktop, Downloads, and other folder paths.

Step 4. Once necessary paths are added, you can click the "Scan Now" button to start the scan process.

Step 5. The scan process might take time. After the scan is complete, all duplicate files will be listed. You can click the button behind to locate them.

Step 6. Select duplicate files to start deletion. You can either manually select duplicate files, or click the "Smart Select" button to automatically select unnecessary duplicate files to delete.

Tips: If the "Smart Select" is ticked, only the oldest duplicate file will be unticked. It means that all duplicate files except the oldest one will be ticked to delete. (The oldest file means the file that was created first. This file will not be deleted. All other duplicate files are considered copies of the oldest file. )

If you manually select all copies of duplicate files, you will get a pop-up warning window to keep at least one copy of the duplicate files. If you indeed want to delete all copies of duplicate files, you can click "OK" so all copies of duplicate files will be selected to delete.  

After unnecessary duplicate files are selected, please click the "Clean Now" button to start the deletion.

Step 7. Delete successfully. The deletion might take time. After the process is finished, the selected files will be deleted, and you will get a cleaning completed window displaying how much data you deleted in total. 

To sum up

How to find duplicate files in hard drive? Now you have known 2 ways. Duplicate files slow down your computer and make it more difficult to find the files you need. AOMEI Partition Assistant for Cleanup can help you easily and safely remove these files to free up hard drive space.