pyget URL Downloader

Pyget

This is python-based URL Downlader like wget.

Usage

  • python pyget.py UrlToDownload
  • python pyget.py UrlToDownload FilePath

Script Download

HDFS Version

Typically, you have to save in local before pushing it into HDFS.
But when file size is too large to save in local, this will download a file into HDFS directly.

Please ensure that you can access and have permission to HDFS.

Usage

  • python hdf_pyget.py UrlToDownload
  • python hdfs_pyget.py UrlToDownload FilePath

Script Download

Written on March 14, 2017