Currently there's no easy way for one scraper to call another. But it is
an interesting idea. Let us know if you flesh out this idea...
David
Prokopp, Christian wrote:
>Is it possible to call scrapers in scrapers? For example I have a web
>site with search results of books - if someone already wrote a scraper
>for the books detail page I could just plug it in in a loop. Following
>that idea there could be comment/reviews which again could be scraped of
>the book detail page and so on.
>The first idea is to reuse the scrapers and maybe it may even be useful
>to allow an automated discovery and crawling of sub pages - e.g. if a
>scraper exists it could offer me not only to scrape the initial page but
>also the detail pages with out me actually writing it into the code.
>That would save a lot of coding because it often is 'scrape this page
>and follow all sub pages and then scrape them'. Not very flexible
>especially considering multiple and changing concatenations.
>
>I know it is not a fully developed idea but I just ran into it and
>thought it would be worth mentioning but I wouldn't be surprised if you
>guys already thought of it and have it on your to-do list.
>
>Cheers,
>
>Christian Prokopp
>SAP Research CEC Brisbane
>SAP Australia Pty. Ltd
>Level 12, 133 Mary Street
>Brisbane QLD 4000
>Australia
>T + 61 7 3259 9 0
>E christian.prokopp_at_sap.com
>http://www.sap.com/australia
>
>
>
>
Received on Thu Jan 19 2006 - 07:17:44 EST