Showcase / Scraped into a collection
Live data - ScraperTen pages of someone else's site. One table on ours.
The Scraper read ten pages of books.toscrape.com, a practice bookshop made for trying scrapers on, and kept 200 books in a collection here. Below is that collection, as it came in. Setting it up took a few clicks; the run took 19 seconds.
200 books, scraped (demo)
Search for a title, or click a column heading to sort by it. Right-click the table to filter, group or export it.
[collection slug="scraper-demo-books" display="table" sort="title" page-size="15" /]
A few clicks, then let it run.
A recipe says which page to start on, what counts as one item, and which parts of it to keep.
Item
Click one book. All 20 on the page are outlined, so each becomes a row.
Fields
Click its title, price, stock line and link. The title is taken from the link's title attribute, because the shop shortens long ones on the page. The price becomes a number.
More pages
Click Next page link, then the shop's "next" button. The recipe stops at 10 pages.
Run
Matched on each book's link, so running it again updates these rows instead of adding them twice.
An ordinary collection at the end.
What a recipe collects lands in a collection like any other, so everything a collection can do works on it: the eight displays, filters and saved searches, Views that group and total, export to CSV or JSON, and the collection's own permissions.
We copied this one across from our test site so the page stays the same. On your site, a recipe on a schedule keeps its collection up to date by itself.
- Runs in the background, with progress and a Stop button
- Every hour, day or week, or by hand
- A report for each run: pages, rows, what changed, what failed
- robots.txt honoured and a wait between pages
- Sites built with JavaScript need Scraper Pro
Stop copying and pasting.
The Scraper is free. Point it at the list you keep retyping and let it run.