Saving bandwidth on Amazon #3227
toxyy
started this conversation in
Ideas and feature requests
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I did web scraping a few years back, and a few tricks that I used to save bandwidth were:
This saved huge amounts of bandwidth. For the sites I had to scrape, the optimal file bytes to retrieve were:
Amazon - 700KB
Ebay - 100KB
Aliexpress: 2.5MB
Alibaba - 70KB
I haven't looked in the source to see if all of this was being taken place. The code I wrote was 6 years ago, though, I don't know if there are any better techniques that could be done.
Beta Was this translation helpful? Give feedback.
All reactions