How to scrape all links placed in a website category with pagination

  • by

This method is used in the “Links” tab (single thread). If you want to scrape links in multi-thread mode use Website crawler tool (ctrl+7).

Before you start scraping create new project (main menu – file – create new project…)

Then go to “URLs” tab and open URL generation tool (ctrl+g).

URL scraper tool

Then open website category page in a webbrowser, scroll down to category pagination and open 2nd page, then 3rd and look, how URL will be changed.

Scraping website category

…/b/wet-food_c389_p2
…/b/wet-food_c389_p3

As we see, the category page number is in the end of the link (2, 3, …).

Put 2nd category page URL to the URL generating template, select page number and press F2 to paste {mun} macro.

How to scrape links from a website category

Then make scrape preview with a selected link in main URLs list and set URLs filters to filter unwanted links.

Scrape results preview

After you have set the URLs filters you can run scraping procedure…

Links scraping procedure


1 Star2 Stars3 Stars4 Stars5 Stars (No Ratings Yet)

Loading...

Leave a Reply

Your email address will not be published. Required fields are marked *