Re: geo-coordinates in Solvent

From: David Huynh <dfhuynh_at_csail.mit.edu>
Date: Wed, 04 Jan 2006 20:12:28 -0500

Dean Allemang wrote:

> I want to help you - you are doing very important work.

Thanks very much, Dean!

> I routinely create scrapers with Solvent (and after learning about the
> "arrow", I'm going to do even more!). I never save them; after all,
> they are quite simple to create.
>
> Should I be saving them somewhere, if only to help you make your
> point? It will mean changing my personal discipline, to actually
> organize the work I do, but I can make a sacrifice . . .
>
> If the answer to this is "yes", point me to the pages that tell me
> what I have to do.

Didn't mean to suggest that you change your work habit... But if you do
find yourself scraping a particular site more than once (often because
it has a search/browse engine and a rich database in the backend), then
perhaps it's also useful to you if you save your scraping javascript
code together with some metadata about the code (that combination
constitutes a scraper), and then install it into Piggy Bank so you can
use it again.

Solvent already helps you with this. After you are satisfied with your
javascript code, switch over to the URLs tab in Solvent and specify the
URL pattern that matches all pages for which your code applies. Then
click on the floppy disk button on the upper left corner of Solvent.
Enter the metadata, specify where to save the code and the metadata (as
an n3 file). Remember to enter a publicly accessible URL where you will
put your code... Also put the n3 file on the Web and tell us where to
find it.

Thanks again!

David
Received on Thu Jan 05 2006 - 01:10:21 EST