Tools to download multiple files

I suggest taking a look at gevent for this task. Then you can use urllib2 for most of the other requirements, such as handling HTTP status codes , and displaying download progress. I would suggest Twisted, although it is not a ready made solution, but provides the main building blocks to get every feature you listed in an easy way and it does not use threads.

Nowadays there are excellent Python libs you might want to use - urllib3 and requests. Try using aria2 through simple python subprocess module. It provide all requirements from your list, except 7, out of the box, and 7 is easy to write. If it doesn't, you could consider volunteering to help finish it. Contact the authors, Michael Stenner and Ryan Tomayko. I used the standard libs for that, urllib. I did about 10 simultanous connections, more should not be a problem.

Continue a interrupted download, maybe not. Ctrl-C could be handled, I guess. Worked on Windows, installed a handler for progress bars. All in all 2 screens of code, 2 screens for generating the URLs to retrieve.

Threading isn't "half-assed" unless you're a bad programmer. You have one dedicated URL producer, and N dedicated download threads or even processes if you use the multiprocessing model. Stack Overflow for Teams — Collaborate and share knowledge with a private group. Create a free Team What is Teams? Collectives on Stack Overflow.

Learn more. Library or tool to download multiple files in parallel [closed] Ask Question. Asked 10 years, 4 months ago. Active 9 years, 2 months ago. Viewed 7k times. Improve this question. Deanna This is what threading is for. And what problem specifically are you facing?

If you want a ready-made, battle-tested command-line tool Add a comment. Active Oldest Votes. No clicking next. Just pick your apps and go. Always Up-to-date You don't have to watch for updates. Our bots do that. Here's what's new: XnView updated to 2.

Tuesday at pm TeamViewer 15 updated to Tuesday at pm more news. Trusted by Millions We install and update about a million apps each day for our home users and Ninite Pro subscribers. You'll need to upgrade your Windows version to continue using Ninite.

NET 4. Cyotek Really works the Best and better fine. I first used htttrack and it would give me nothing better than this. After 30 days it only for for pages. Regarding where A1WD places files, it is among the first options always visible when you start the software. In addition when viewing the downloaded results, you can see the individual path of all files downloaded two places. Left sidebar and at top. Simply paste in a URL and click Download.

Site Snatcher will download the website as well as any resources it needs to function locally. It will recursively download any linked pages up to a specified depth, or until it sees every page. Your email address will not be published. David [ Reply ]. Steven Durham [ Reply ]. Smiles [ Reply ]. Thomas Schulz [ Reply ]. Peter [ Reply ].