Re: Querying and caching with large datasets

From: Jeen Broekstra <jeen_at_aduna.biz>
Date: Mon, 23 Jan 2006 14:15:46 +0100

Rickard Öberg wrote:

[snip]

> I looked through the Sesame 1 API though, and it doesn't seem like I
> can do this as a Sail feature, since the query handling is not done
> by each Sail, but rather by a separate query engine (if I understood
> things correctly). Looking at Sesame 2 it does indeed look like it's
> a chained architecture that would allow such caching, so I'm assuming
> you're talking about Sesame 2.

You are right that it is somewhat more difficult to do in Sesame 1, but
it is still *possible*.

In Sesame 1, the SAIL API has an 'optimizeQuery' method that can be used
as a kind of loophole. Basically, what goes in is a Query object, and
what comes out is an optimized Query object.

You could implement a cache as a stacked SAIL that uses the
'optimizeQuery' hook to catch queries that are cached. What you'd
essentially do is catch the query, look up its cached result, and return
a (customized) query object that contains a pointer to the result. The
Sesame query engine evaluates a query by executing the 'evaluate' method
on the query object, so in this customized object, all that the
'evaluate' would do would be return the cached result.

> My only problem with using Sesame 2 is that it's an alpha, and I
> don't know when I'll have to go into production with this stuff.
> Preferably within the next six months. Any thoughts on that? Sesame 1
> or 2?

Well, to be honest, we would of course encourage you to use Sesame 2,
since that is where your development can do the most good for us and for
other Sesame users. Then again, you are more familiar probably with how
Sesame 1 works, and we still do plan to do a few Sesame 1.x releases.
Also, keep in mind that Sesame 2 is Java 5 only, which may be a problem
for you (I understand that for Piggybank it is a problem as MacOSX still
does not ship with Java 5 per default).

As for stable releases: for Sesame 2, our plans are to be less
monolithic in our release strategy. There will be separate releases of
different components: a core Sesame module (containing APIs, query
engines, query models, and the main memory backend), and the different
persistent backends, web client and the parsers/writers etc. will be
distributed separately.

The way things currently are, we aim to have the Sesame 2 core stable in
about two months from now. The native store will be available separately
then as well (it is already released as part of alpha-2 and as far as we
have tested it is stable). The MySQL backend and web client will
probably be released a bit later, but those are not really an issue for
you, I guess.

Jeen
Received on Mon Jan 23 2006 - 13:14:00 EST