What is web scraping?

Web scraping is extracting data from websites. You can then take the data to make it suit your (business) purpose. There are many ways to do web scraping, but most of them require money, coding experience and upfront configurations.

When should I use a web scraping tool?

Couple main use cases: if you are a data scientist, you can use it for research; if you are a business owner, you can use it for market analysis/comparing prices; if you are a digital marketer, you can use it for lead generation.

Why another scraping tool?

I was trying to gather a specific set of data that was scattered among blog posts and websites. Most of the scraping tools out there focus on recurring scraping of multi-page mega sites like Yelp. Although they do provide analytic data around the scraped results, it was not what I needed. I need something that allows me to easily scrap multiple webpages as I browse (like websites I only visit once), 0 friction (no upfront configuration needed), and persists the data (so I can keep growing the list as I scrape), and then generates a CSV.

Why is it called Spider?

The fact that it traversed DOM tree reminds me of a spider. Hence the name :)

Subscribe to Our Newsletter

The latest Spider news, articles, and resources, sent straight to your inbox every month.

*Writing email is tiring. We'll never spam you nor send out a lot of emails.