One of my year 2008 resolution is to clean up and organize all my files. I have software, movies, mp3s, documents and notes everywhere in I think at least 5 separate external hard drives. All my external hard drive enclosure looks the same and I honestly tell you I can’t identify which is which unless I connect to my computer to see what’s in there. Both my laptop and desktop hard disk is running out of disk space. When I am in these conditions, don’t expect me to have a proper backup of my stuff when all my files are in such mess. I know it’s bad… really bad. Actually having less or no duplicate files is good because you can free up your hard disk space, have proper backup, reduce backup time and reduce file searching time.
So what I am going to do is I have to start organizing my files. I’ll put all software into one hard disk, all mp3s into another hard disk and so on… Then I’ll have an extra hard disk solely for backing up my important files. But before I do all that, I expect to have a lot of same and duplicate files which is taking up a lot of unnecessary hard drive space.
I’ve found a total of 6 freeware to help me find and remove duplicate files on my computer. It is interesting because I learn the methods used by each different software to scan for duplicate files.
If you do a search in google on “remove duplicate files”, you’ll easily find a lot of software that is able to do that. I’ve found 6 of them and I will try my best to find out which is the best to use. For me, most important thing in a software is how easy it is to use and also the method being used by the software do perform a task.
1. Duplicate File Finder
- This is the easiest software to find duplicate files and also with only a click of a button, it will remove all old duplicate files. Portable, simple graphical user interface and no configuration at all. The method being used to scan for duplicate file is raw binary comparison of the actual data inside the file. It is 100% accurate using this method. Original Files are determined by creation date, if it was created first, it is considered the original file. Took 2 minutes and 29 secs to scan 106GB (23950 files). Not able to select which folder to exclude scanning, so you have to avoid adding root of a drive to the path.
[ Download Duplicate File Finder ]
2. Heatsoft Clone Cleaner Lite
- Nothing much to configure. Just add the path that you want to scan and search. By default it protects Windows, Program Files and hidden folders. Took 2 minutes 5 seconds to scan 106GB (23950 files). One thing I don’t like is it promotes the PRO(shareware) version at a few places on the software. It uses TABS to show you the scan result in same name, same size, same name and size, same content, full clones and zero size. It has Smart Mark where you can automatically mark the duplicate files to be deleted.
[ Download Heatsoft Clone Cleaner Lite ]
3. DupKiller
- DupKiller has all types of scanning methods. Same name, same date and time, same size, same type(extension), similar name and same content. The same content is raw binary comparison which is 100% accurate. Has very good exclusion option but unfortunately very slow in binary comparison. Took 4 minutes 55 seconds to scan 0.3% of 106GB (23950 files). I also don’t like DupKiller automatically saving and loading of previous scanned results by default. You can turn it off at the options page.
[ Download DupKiller ]
4. Easy Duplicate Finder
- Able to exclude system folders (windows and program files). It says that it search for true duplicates (files with same contents) which I think it meant raw binary comparison. Took 1 minute 46 seconds to scan 106GB (23950 files). This can be a really good duplicate finder BUT there is no way I can easily select all duplicates and delete them with one click. It found 5809 duplicates on my computer and I am going to have a hard time selecting them one by one.
[ Download Easy Duplicate Finder ]
5. DoubleKiller
- Ability to scan for duplicates using names, sizes, dates and CRC32. All methods used here is not 100% accurate. CRC32 is not 100% because there is a very small probability of two files having the same checksum although they are not identical. If you want to use raw binary comparison, you need to PURCHASE the PRO version. I didn’t test this because it doesn’t have raw binary comparison.
[ Download DoubleKiller ]
6. CloneSpy
- CloneSpy is another one that doesn’t do raw binary comparison. It only compares checksums, same name and approximate size. I’d totally avoid using this to scan and delete duplicate files on my computer.
[ Download CloneSpy ]
There you have it. The best duplicate file scanner and remover is Heatsoft Clone Cleaner Lite and the easiest to use is Duplicate File Finder. I’d be using this two in future to remove all the duplicate files on my computer, laptop and external hard drives.
Related posts:
You should add Directory Report to your list
It can find duplicates by both CRC32 and comparing byte by byte
file-utilities.com
If you are looking for the solution to the duplicate files or image. You can use following utility (duplicatefilesdeleter.com/). its a good solution to the duplicate files.
thanks bro – was looking for somebody to “cut to the chase” and you did.
appreciated
“Duplicate File Finder” is absolutely roaring through duplicate photo archives here –
much appreciated
You can also take a look at Exact Duplicate Finder.
If you want things to be simple, I mean real simple.
If you also want to be in control, then give it a try.
Press F1 to read help, after the app starts. Enjoy ;)
Buongiorno, è possibile definire una directory come master e cercare e cancellare i files nelle directory slave comparati solo con la master e non anche tra le directory slave.
Grazie
You miss the best of all and it is FREE. I guess because it is pretty new, but was featured at ZDnet recently:
Find Duplicate Files Free – Fast Duplicate File Finder
Try Directory Report
file-utilities.com/downloads/wdir.zip
It can find dupes based on the same
name, size, CRC and/or comparing byte by byte
Plus it can replace dupes by hard links
The best out there. Supports also RAW
mindgems.com/products/VS-Duplicate-Image-Finder/VSDIF-About.htm
For those looking to compare two folders, try WinMerge:
winmerge.org/
Help For Windowsxp pro
This is the best free hassle free software i have ever used! Dont waste paying money for deleting files. This should be your one stop solution!
Thanks for sharing the information, i could get back almost 5gb of disk space from my mp3 collection!
So far the Duplicate Finder has worked great for me! It has been a terrific assistance for me in eliminating duplicate images and songs from my hard drives – saving me loads of time.
ashisoft.com
If you are looking for something working on folder level then try Clone Tools pilotman.com/
With that piece of software you can compare folder content, merge folders etc. it will never delete any file that does not have an identical clone.
The drawback is that it is not freeware :(
Also it is a bit more complicated to use since it is more advanced then the programs in this list.
Kevin and Alfonso – great idea and a big need for that one! I am currently sorting through software on an offshore drilling rig in the North Sea, and man do I need exactly what you have described in your request! Somebody through us a bone here.
Terry
I also have 6 externals and both the mac laptop and the windows vista desktop are near or at full capacity. Any recommendations on backing up the externals and checking for duplicate files on all of them. Do you prefer to partition the external and keep your backup of the external on the partitioned side or on an entirely different drive? Also get into trouble with compatibility issues with the externals being used on both the mac and windows even if they are formatted as Ms-Dos (FAT). Is it the file size? The files are mostly EPS, AI, PSD and Gifs that create the most trouble??? Is there any relation to the type of file? However, my biggest priority is cleaning up the unnecessary junk and finding and erasing duplicates. Its time to clean house! Any and all help is greatly appreciated. Have a wonderful day!!!
Hi!
I’m totally with Kevin here. I have exactly the same need. I’d like to set a master folder and check an other folder against it.
Over time I copied my pics here and there and now I have a “pics” folder and a “pics dups” folder with twice the size, but I want to be sure I don’t loose any data!
So, anybody heard about a software like that?
thanks,
alfonso
Thanks Raymond a good start.
I think the speed in the first 3 is because they don\’t do a full binary compare first up. Certainly HCCLite makes its lists based on name, size etc. Then it will do a Binary compare on any of the lists it found with the fast methods (using the toolbar button at the bottom). It takes much longer to do this. FileDup may be doing this on the first pass which would explain why it is so slow.
Ciao
I’ve tried a couple duplicate finders from the list. They don’t seem intuitive. I’d like to identify a master folder that all of the original or unique files will remain and check the other folders on the drive against the original, only deleting files outside of the master folder.
It would even be nice if it would transfer any remaining unique files outside of the master folder to inside that folder.
Perhaps I’ve overlooked some existing features.
Duplicate File Finder crashed 49% of the way through a very long scan (hours).
Thanks
David stop promoting your own product!!!
i downloaded duplicate finder today and it works great under Windows Vista.
Duplicate Finder works buggy on Vista.
I really like Duplicate Finder. An especially nice touch was when it told me that I was deleting all copies of a
file. I tried a few other duplicate file finders and hated them.
ashisoft.com
I’d like my software is Clone Remover (moleskinsoft.com). It can search for duplicate mp3 files by tag and similar images. And can restore files that moved into separate folder. But Clone Remover is shareware.
Had to undo it on my own–I will get up with you if I need help on something else. But thanks anyway.
Raymond–long complicated story but the end result is the I have duplicate files (many many) after doing a restore of an F partition–data files. I thought I could use your duplicate file finder program but when I went to the actual copies of the original they had an earlier time: 1:25 am versus actual file 9:25 am. When I tried to open the duplicate it could not find a program to open it with. That could be a problem. I don’t see an option to choose. I have lots of duplicates but it is only showing what looks like the good file name. Please help me.
Request article: File Compare softwares
Raymond,
for a couple of years I believed ‘Black Viper’ was the most respected Guru online …. I have to say …. you have snatched that crown, in my humble opinion.
Thank you for a very credible and useful site. My compliments – you are required reading in my arsenal now.
Merlin
PS – any chance of an answer to my emailed question about U3 USB Hack please?
Thanks … I was getting fed up myself with all the duplicates on my poor overused computer ;-)
LOL, looks like the direct link checks for referrer. I’ve fixed it once and for all :P
Oh, and Duplicate File Finder link is still wrong, it is pointing to Web Hosting Pro, or something like that. I google it and found this link:
ukms.tucows.com/files2/DupSetup.exe
It’s for a USA mirror, but that is more mirrors here:
tucows.com/download.html?software_id=373411&t=2
;)
Lol, same situation here. A lot of drives, small space left and no ideia of what is inside. Very nice tools, will try it soon…
[...] Check it out! While looking through the blogosphere we stumbled on an interesting post today.Here’s a quick excerpt One of my year 2008 resolution is to clean up and organize all my files. I have software, movies, mp3s, documents and notes everywhere in I think at least 5 separate external hard drives. All my external hard drive enclosure looks the same and I honestly tell you I can’t identify which is which unless I connect to my computer to see what’s in there. Both my laptop and desktop hard disk is running out of disk space. When I am in these conditions, don’t expect me to have a proper backup of [...]
thanx Rymond
Duplicate File Finder link updated. Sorry for any inconveniences.
Hi Raymond, Your link to Duplicate File Finder is a link to Smiley Central , It continually loads Pop Ups, (Despite my pop up blocker being set to it’s strongest level.
I used the “Red” link above No 1. That worked fine.
Regards
Colin