Ranter
Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API
From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Comments
-
Tayo8965y"professional" scripts look like someone just shat in a pan and made a burger out of it
Looks a lot better than the python I'm used to seeing, though I must admit, I don't see a lot of python -
endor56705yRound 2: make it async with curio! (don't use asyncio) Download all the images in parallel instead of one by one to reduce the waiting times.
Some good reading material:
https://curio.readthedocs.io/en/...
https://vorpus.org/blog/...
Also, I'd probably use 'requests' istead of urllib: much easier to work with, handles stuff for you in the background, less bloat. -
VaderNT16185yI skimmed through it, you're doing well!
In main.py replace the long if-elif with a dict and catch its KeyErrors to improve readability:
pathByCategory = {'gp': 'events', 'best_of': 'best+of'} # Add more
try:
full_url = main_url + pathByCategory[category]
content = urlopen(full_url)
soup = ...
download_category(...)
except KeyError:
print('wrong category')
exit(1)
For short functions I prefer the whole happy path in the try block. It allows to read it in one go and error cases one by one. It's unwieldy for many exceptions and if the same can be thrown from multiple places. In that case better split it.
You can define dict&main_url outside the function so they aren't redefined on each call.
Use non-zero exit codes on errors. Your shell can then indicate your script has failed, and it helps with integrating with others.
downloader.py line 26 is missing the closing parens.
Hope that helps, keep having fun! -
@dontbeevil
I like almost all sports[coz of the competitive feel], but recently fallen more in love with motogp coz of growing love for bikes -
hey @endor.. tried implement curio by just reading the first 10 lines of the doc... and ofcourse couldn't get it to work..
have no prior experience in async/await... will have to dig deeper..
and also I couldnt find much reading material on curio on Google or SO -
endor56705y@d02d33pak LOL dude, it's pretty much impossible to learn anything just by reading the first 10 lines. Try going through the whole tutorial section (in order) and *then* start thinking how to apply that to speed things up by running tasks in parallel.
Also, read that blog post I liked as well (second link, after the docs one), it gives some nice detailed examples of why you wanna do things in a certain way.
Take your time to properly digest those new concepts, you can't just smash your head into the docs and learn everything at once. -
@endor brrooo 😵
haven't done async/await ever before.. and not easy to understand and implement..
also curio seems less popular than asyncIO coz i cudnt find many tuts or utube videos on it.. so.. thats how it stands right now... slow and sequencial -
can anyone take another look at it now..
made a few changes..
github.com/d02d33pak/motoscript
Related Rants
Learning Python.
Wrote a script to download images from the official MotoGP website.
https://github.com/d02d33pak/...
Was curious to know what "professional" scripts look like.
Anyone willing to take a look and go I've some advise?
rant
script
motogp
python