Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Lab needs a crawler to download some assets, none of my business though
But why not
Haven't touched crawler for two years
Google for latest state of art
Found scrapy
I have to define a class for a crawling script?
Got scared
Went back to beautifulsoup and request
Got the job done in 20 mins
Fuck yeah
rant