There are times when you will end up on a web page that doesn’t look like a webpage but instead looks like a folder and you can only find files being listed. This is because the webserver directory index file (index.html, default.asp and etc) is not present in the folder and the directory listings option in the webserver is turned on.
One of the reasons to offer directory listings is to provide a convenient way for the visitor to quickly browse the files in the folders and allow them to easily download the files to their computer. Sometimes directory listings are accidental due to webmasters who forget to include a .htaccess or a blank index file to prevent all the files from being listed.
To download a file, you can either left click on it or right click and select “Save link as” in your favorite browser like Firefox or Chrome. However, if you need to download multiple or even all of the files from the directory including the subfolders automatically, you will need third party tools to help you achieve that. Here are 5 different methods that you can use to download all files from a folder on a website.1. Download Managers
If you are a frequent downloader, you probably already have a download manager program installed. Some of the popular and feature rich download managers like JDownloader are even open source software. JDownloader is based on Java which means it’s cross platform and available for Windows, macOS and Linux.
While this program is able to download all files in a specific folder very easily it cannot recurse into sub folders. All you have to do is copy a URL to the clipboard while JDownloader is running and it will add a new package or set of packages to the Link Grabber with all the files. File types can be filtered out with checkboxes or deselected/removed manually. Note the JDownloader installer version contains adware.
This next download manager program is quite old but has a feature called Site Explorer which allows you to browse websites like in Windows Explorer. Flashget has more recent versions than the 1.96 we’re looking at here, but we found this version handles HTTP and FTP sites better.
After installing and running Flashget go to the Tools menu > Site Explorer. Enter the URL and then you can browse through the site and download the files in any folder. If the site is using FTP, folders can also be multi selected and the files inside those folders will be downloaded. Only the files inside the root folder will download if the site is HTTP. Make sure to avoid the Google Toolbar offer during install.
2. Browser Extensions
Popular browser extensions for downloading files in recent times have been DownThemAll! and Flashgot for Firefox. Sadly, they don’t work anymore since the change to the WebExtensions system in Firefox 57. However, there are still extensions available for both Chrome and Firefox that can download files from a website or FTP folder. Here’s a couple of selections for each.
NOTE: All the browser extensions below will only download the files from the root folder in the browser tab, they will not recurse into sub folders. If you select a folder from the download list it will simply download as an unknown file.
Extensions for Chrome (and Chromium Based Browsers)
Chrono Download Manager is one of the most popular extensions of its type for Chrome. There are some advanced options but it’s relatively straightforward to quickly download all the files in a folder. Click the Chrono toolbar button and switch to the sniffer mode with the top right button in the window.
Then, cycle through the tabs selecting all the files with the top checkbox, check files individually or use the file type filter boxes below. Click the “Download All!” button and the files will be added to the queue and downloaded.
Download Master is another Chrome extension that downloads a load of files in a folder pretty easily. It works in a similar way to Chrono but is a little more straightforward to use and what you see in the main window is it, there are no separate settings or options windows.
After you press the icon to open the download window all you have to do is check the file extension filter boxes, supply a custom filter or add files manually. Then press Download. As all the selection filters are in the same window it makes Download Master a bit faster to select multiple files or all files at once.
Extensions for Firefox
Simple Mass Downloader is a pretty good option for Firefox since the classic old extensions no longer work. It also has some useful features like being able to add downloads from multiple tabs at once and add automatic folders to directly download files of certain types into specific folders.
Files do not automatically populate the resource list and you have to click the “Load Page Links” button to start. The checkbox at the top will select all files at once while the extensions or text filter boxes at the bottom will filter the list by whatever characters are entered. Files can be added to a queue or downloaded directly with the buttons at the bottom right.
The final extension we’ll talk about is DownloadStar for Firefox. It’s not much different than the options we’ve mentioned above but to make all files visible you have to click “Show media from links” at the bottom left of the window or only images will be shown.
The filter checkboxes don’t have a select all files option but simply putting an asterisk “*” into the Fast Filter box will quickly do that. The button to download the files isn’t immediately obvious but it’s located at the top right with the number of files to download in brackets.
To download using something other than browser extensions or download managers, try the methods on Page 2.
Wget is a free and very powerful file downloader that comes with a lot of useful features including resume support, recursive download, FTP/HTTPS support, and etc. In “The Social Network” movie, Mark Zuckerberg is seen using the Wget tool to download all the student photos from his university to create Facemash. Wget is a command line tool which can be a bit difficult to use for some basic users.
Thankfully there are free front-end GUI’s for Wget such as VisualWget that make it so much easier to use Wget by simply clicking on the checkboxes rather than manually typing the command line arguments. Download VisualWget, extract and run VisualWget.exe. Click on the New icon to open a New Download window. Enter the URL that you want to download and select the location that you want to save the files to.
If you need to download multiple folders including subfolders, go to Advanced, click on Recursive Retrieval and tick on the first checkbox “--recursive“. Finally, click OK to start downloading.
Although there are a few GUIs for Wget, we recommend VisualWget because it is free, portable, comes together with Wget and there is no need to download separately and manually unpacking it to the program’s folder or even configuring the path. Note that if you are a Windows 10 user and don’t already have .NET Framework 2/3.5 installed, you will need to download it during the VisualWget installation.
4. Offline Browsers
Offline Browsers are actually tools that download the whole website for offline viewing. These pieces of software are capable of crawling into subfolders, downloading all or filtered files and then converting the live hyperlinks to an offline version. Some of the popular offline browsers are Offline Explorer and Teleport but they are shareware and come with powerful parsing capabilities.
HTTrack is an excellent free alternative to the paid offline browsers. You can download the portable version of HTTrack, extract and run WinHTTrack.exe. Click Next, give the project a name and click Next.
Now enter or paste the URL(s) that you want to download to the Web Addresses box and click Next. If the URL requires authentication, click the Add URL button and you get to enter the login and password for the URL. Click Next and Finish. HTTrack will now start crawling the given URL and download files that it finds.
5. Downloading from FTP
The best program to download all files and subfolders from an FTP server is of course going to be dedicated FTP client software like FileZilla. They obviously understand FTP commands and are able to crawl recursively into subfolders without problems. In FileZilla Client, all you need to do is enter the FTP address in the Host box, enter a username and password if it requires authentication, or leave it blank if not, and click the Quickconnect button.
Once you’re logged in, simply right click on a folder in the right pane and select Download which will start downloading all files and folders from the selected directory.
Please take note that FTP clients can only handle the FTP protocol and they cannot download files from HTTP web pages.