Re: Sesame Native Store OPS indexing (was Re: [update] piggybank performance profiling)

From: Stefano Mazzocchi <stefanom_at_mit.edu>
Date: Tue, 13 Sep 2005 12:20:00 -0400

Vineet Sinha wrote:
> The summary
> -----------
> OPS indexing for the Sesame Native Store has been implemented. And it
> works beautifully in Relo. Stefano should be looking at it in Piggy Bank.

You rock!!

I'm on it.

> Details
> -------
> Single Modified File:
> http://simile.mit.edu/repository/relo/branches/sesame/src/org/openrdf/sesame/sailimpl/nativerdf/TripleStore.java

NOTE: I have moved the modified sesame store in a more prominent
location (since both relo and longwell are going to depend on it), find
it at

http://simile.mit.edu/repository/sesame/branches/1.2.1-ops/src/org/openrdf/sesame/sailimpl/nativerdf/TripleStore.java

> The working jar file
> http://simile.mit.edu/repository/relo/trunk/edu.mit.csail.relo.store/lib/sesame-1.2.1-ops.jar

This one is still there.

> Implementation was as Arjohn had suggested. I did not see any bugs.
> Adding statements seems to have added a 25% cost.
>
> Beyond adding the second comparator, I also renamed previous
> btree/file/filename variables to include 'spo' before them and made a
> copy for 'ops'. triples.dat is now triples-1.dat and triples-2.dat.
>
> The other issue is a Jeen mentioned, the large file size for
> transmission. The best solution could be to not require it for
> transmission and build the second index automatically (in fact this
> should also increase the add performance, relying on the spo index until
> the ops index is ready).
>
> There were no unit tests, but tests on Relo should have been good.
>
> I can send patch, after we fix any issues. Let me know if you have
> questions.
>
> Arjohn, thanks for your help!

NOTE: To comply with licensing restrictions, the patch is released under
the LGPL.

-- 
Stefano Mazzocchi
Research Scientist                 Digital Libraries Research Group
Massachusetts Institute of Technology            location: E25-131C
77 Massachusetts Ave                   telephone: +1 (617) 253-1096
Cambridge, MA  02139-4307              email: stefanom at mit . edu
-------------------------------------------------------------------
Received on Tue Sep 13 2005 - 16:15:35 EDT

This archive was generated by hypermail 2.3.0 : Thu Aug 09 2012 - 16:39:18 EDT