RE: Piggy bank ports

From: Prokopp, Christian <christian.prokopp_at_sap.com>
Date: Tue, 20 Dec 2005 06:02:07 +0800

Hi David,

Thank you for your answer. As mentioned in a different post to my
question I already checked
http://simile.mit.edu/issues/browse/PIGGYBANK-51 and it did not resolve
my problem. I do not think it is a java/piggy-bank problem itself but
rather my restrictive firewall/proxy I have to deal with. My question is
to investigate the problem to maybe find a work around.

Example:
1.) After I installed the scraper for jobsearch.monster.com I open the
website:
http://jobsearch.monster.com/jobsearch.asp?&q=java&re=112&refine=1
2.) I then click on the piggy bank icon to scrape the page and get a
message like "Piggy bank will need to retrieve code from
http://people.csail.mit.edu/people/dfhuynh/research/download/screen-scra
pers/monster-com-search-scraper.js This might take a bit of time."
3.) It follows a redirect to my localhost piggy bank with a screen:
"Monster - Search Jobs
Collected Information

No typed data found."

This basically happens with every scraper I used from the SIMILE site.
My question therefore is how the piggy bank retrieves the js code and
any other possible data? I would have guessed plain http but maybe not?
I also add the relevant output from the java console at the end of this
email. Please excuse the long post.

Cheers,
Christian


System used:
Windows XP
Firefox 1.5
Latest piggy bank and solvent plug-in.

Java Plug-in 1.5.0_06
Using JRE version 1.5.0_06 Java HotSpot(TM) Client VM
[...]
07:55:51.993 [...RDFUtilities] Query took 10ms:
        SELECT DISTINCT o FROM
{<http://people.csail.mit.edu/people/dfhuynh/research/downloads/screen-s
crapers/screen-scraper#MonsterComSearchScraper>}
<http://simile.mit.edu/2005/04/piggy-bank#code> {o} (5201ms)
07:55:53.629 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT o FROM
{<http://people.csail.mit.edu/people/dfhuynh/research/downloads/screen-s
crapers/screen-scraper#MonsterComSearchScraper>}
<http://simile.mit.edu/2005/04/piggy-bank#code> {o} (1636ms)
07:55:53.629 [...k.GRDDLModel] java.net.UnknownHostException:
people.csail.mit.edu (0ms)
07:55:53.639 [...RDFUtilities] Query took 10ms:
        SELECT DISTINCT o FROM
{<http://people.csail.mit.edu/people/dfhuynh/research/downloads/screen-s
crapers/screen-scraper#MonsterComSearchScraper>}
<http://simile.mit.edu/2005/04/piggy-bank#code> {o} (10ms)
07:55:53.639 [...k.GRDDLModel] java.net.UnknownHostException:
people.csail.mit.edu (0ms)
07:55:53.700 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT o FROM
{<urn:edu.mit.simile.piggyBank:model1134966681380>}
<http://simile.mit.edu/2005/04/piggy-bank#dataLink> {o} (61ms)
07:55:53.770 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT o FROM {}
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> {o} (70ms)
07:55:53.770 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT p FROM {} p {} (0ms)
07:55:53.780 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT o FROM {}
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> {o} (10ms)
07:55:53.780 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT p FROM {} p {} (0ms)
07:55:53.780 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT s FROM {s}
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
{<http://simile.mit.edu/2005/04/flair#QueryBasedFacade>} (0ms)
07:55:53.780 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT s FROM {s}
<http://simile.mit.edu/2005/04/longwell#systemStatus>
{<http://simile.mit.edu/2005/04/longwell#Trusted>} (0ms)
07:55:53.780 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT s FROM {s}
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>
{<http://simile.mit.edu/2005/04/flair#QueryBasedFacade>} (0ms)
07:55:53.790 [...rvletHandler] servlet=/*=Flair (10ms)
07:55:53.790 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT s FROM {s}
<http://simile.mit.edu/2005/04/longwell#systemStatus>
{<http://simile.mit.edu/2005/04/longwell#Trusted>} (0ms)
07:55:53.800 [...rvletHandler] session=null (10ms)
07:55:53.800 [...FlairServlet] > doGet /model1134966681380 (0ms)
07:55:53.800 [...FlairServlet] > makeMessage (0ms)
07:55:53.800 [...FlairServlet] < makeMessage (0ms)
07:55:53.810 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT s FROM {s}
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> {} (10ms)
07:55:53.840 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT s FROM {s}
<http://www.w3.org/1999/02/22-rdf-syntax-ns#type> {} (30ms)
07:55:53.850 [...RDFUtilities] Count took 0ms:
        SELECT DISTINCT s, o FROM {s}
<http://www.w3.org/2000/01/rdf-schema#label> {o} (10ms)
07:55:53.850 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT o FROM {}
<http://simile.mit.edu/2005/04/ontologies/tags#tag> {o} (0ms)
07:55:53.850 [...RDFUtilities] Count took 0ms:
        SELECT DISTINCT s, o FROM {s}
<http://www.w3.org/2000/01/rdf-schema#label> {o} (0ms)
07:55:53.850 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT o FROM {}
<http://simile.mit.edu/2005/04/ontologies/tags#tag> {o} (0ms)
07:55:53.850 [...StartCommand] > start (0ms)
07:55:53.850 [...StartCommand] > getFacades (0ms)
07:55:53.850 [...StartCommand] < getFacades (0ms)
07:55:53.850 [...StartCommand] > getClasses (0ms)
07:55:53.850 [...StartCommand] < getClasses (0ms)
07:55:53.850 [...StartCommand] > mergeTemplate (0ms)
07:55:53.860 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT o FROM
{<urn:edu.mit.simile.piggyBank:model1134966681380>}
<http://simile.mit.edu/2005/04/piggy-bank#originTitle> {o} (10ms)
07:55:53.870 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT o FROM
{<urn:edu.mit.simile.piggyBank:model1134966681380>}
<http://simile.mit.edu/2005/04/piggy-bank#originTitle> {o} (10ms)
07:55:53.870 [...RDFUtilities] Query took 0ms:
        SELECT DISTINCT o FROM
{<urn:edu.mit.simile.piggyBank:model1134966681380>}
<http://simile.mit.edu/2005/04/piggy-bank#originURL> {o} (0ms)
07:55:53.870 [...StartCommand] < mergeTemplate (0ms)
07:55:53.870 [...StartCommand] < start (0ms)
07:55:53.870 [...FlairServlet] < doGet (0ms)
Received on Mon Dec 19 2005 - 21:55:16 EST