Scraper Pro

Scraper Pro adds two things to the Scraper.

Pages as a browser shows them

Some sites send an almost empty page and build their content with JavaScript once it loads. The Scraper reads pages as they are sent, so on those sites it finds nothing. With Scraper Pro, choose As a browser shows it (Pro) beside the address in the picker: the page is opened in a browser on the server, its scripts run, and the Scraper reads what they built. Recipes made this way run the same way.

• It is slower than reading a page as sent - a few seconds a page - so use it only for the sites that need it.
• The browser runs on the server that hosts your site, never in your own browser, and it can only reach public websites.
• If the picker says rendering is not set up on this server, ask your host.

Keeping rows in MongoDB

If your site has a MongoDB connection (System > Connections), a recipe's menu on the Scraper screen offers Where the rows are kept…: files on the site (the usual) or one of your connections. The rows already collected move with their ids, and later runs write to the new place. You can move them back the same way.