Chewing_Bever
/
4kwallpapers-scraper
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Added category selection

master
Jef Roosens 2021-01-19 18:54:12 +01:00
parent 2518a0dfc2
commit 97778df3f8
2 changed files with 21 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
README.md
View File

@ -3,7 +3,12 @@
A Python script to scrape wallpapers from https://4kwallpapers.com/. A Python script to scrape wallpapers from https://4kwallpapers.com/.
## Usage ## Usage
The program takes a single command line argument, namely the root folder of The program takes one or more arguments. The first argument is the path to the
whre you want all the pictures to go. It'll create sub-folders for all the download directory; this is where the pictures will end up. Each category
categories, and download all pictures in one go. Yes, this can take a while creates a subdirectory for itself.
(15m in my case, with a fast internet connection).
Any arguments following the path correspond to category names. This allows you
to only download certain categories.
Do note that a full download can take a while: upwards of 15 minutes on a fast
internet connection.

15
scraper.py
View File

@ -68,9 +68,10 @@ class Scraper(PageHandler):
def images(self): def images(self):
return sum([cat.images for cat in self.categories], []) return sum([cat.images for cat in self.categories], [])
def download(self, dir_path): def download(self, dir_path, cats=None):
for cat in self.categories: for cat in self.categories:
cat.download(dir_path) if cats and cat.name in cats:
cat.download(dir_path)
class Category(PageHandler): class Category(PageHandler):
@ -155,4 +156,12 @@ class Image(PageHandler):
if __name__ == "__main__": if __name__ == "__main__":
scraper = Scraper() scraper = Scraper()
scraper.download(sys.argv[1])
if len(sys.argv) == 1:
print("No path provided.")
elif len(sys.argv) == 2:
scraper.download(sys.argv[1])
else:
scraper.download(sys.argv[1], sys.argv[2:])