Re: Querying and caching with large datasets

From: Rickard Öberg <rickard.oberg_at_senselogic.se>
Date: Mon, 23 Jan 2006 14:30:58 +0100

Jeen Broekstra wrote:
> You are right that it is somewhat more difficult to do in Sesame 1, but
> it is still *possible*.
>
> In Sesame 1, the SAIL API has an 'optimizeQuery' method that can be used
> as a kind of loophole. Basically, what goes in is a Query object, and
> what comes out is an optimized Query object.
>
> You could implement a cache as a stacked SAIL that uses the
> 'optimizeQuery' hook to catch queries that are cached. What you'd
> essentially do is catch the query, look up its cached result, and return
> a (customized) query object that contains a pointer to the result. The
> Sesame query engine evaluates a query by executing the 'evaluate' method
> on the query object, so in this customized object, all that the
> 'evaluate' would do would be return the cached result.

Hacky, but works. Kewl.

> Well, to be honest, we would of course encourage you to use Sesame 2,
> since that is where your development can do the most good for us and for
> other Sesame users. Then again, you are more familiar probably with how
> Sesame 1 works, and we still do plan to do a few Sesame 1.x releases.

I've been working for a week with Sesame 1 so familiarity is not an
issue. Whatever we choose will be used by all of us for all eternity
hereafter, so we're more concerned with getting the right thing. BUT,
release dates is an issue, which is why we looked at Sesame 1 to begin
with. I personally would very much like to use Sesame 2 instead.

> Also, keep in mind that Sesame 2 is Java 5 only, which may be a problem
> for you (I understand that for Piggybank it is a problem as MacOSX still
> does not ship with Java 5 per default).

We have customers running on Windows, Linux, Netware (1.4 only) and
Solaris. We're trying to get all of our Netware customers to migrate to
SUSE though, and by next Christmas it should not be an issue. We want
1.5 for other reasons as well.

> As for stable releases: for Sesame 2, our plans are to be less
> monolithic in our release strategy. There will be separate releases of
> different components: a core Sesame module (containing APIs, query
> engines, query models, and the main memory backend), and the different
> persistent backends, web client and the parsers/writers etc. will be
> distributed separately.

Sounds good to me!

> The way things currently are, we aim to have the Sesame 2 core stable in
> about two months from now. The native store will be available separately
> then as well (it is already released as part of alpha-2 and as far as we
> have tested it is stable). The MySQL backend and web client will
> probably be released a bit later, but those are not really an issue for
> you, I guess.

Actually, some concerns have been voiced internally that we'd like to be
able to work with an RDBMS as backend, for two reasons: our customers
are currently quite scared of our Berkeley-DB-lookalike-database,
because they have no idea how it works, and such a custom native
database typically does not have good support for various things like
backup handling (we need to do daily backups of 10G+ databases in the
large deployments, and the system has to be up while the backup is in
progress) and transaction management. I.e. lots of non-developer issues
but which are crucial for deployment.

/Rickard

-- 
Rickard berg
rickard.oberg_at_senselogic.se
_at_work +46-(0)19-173036



Received on Mon Jan 23 2006 - 13:41:58 EST

This archive was generated by hypermail 2.3.0 : Thu Aug 09 2012 - 16:39:18 EDT