Saturday, August 9, 2008

Java Performance : Caching Clustering ... and "FlushCache"

Hi i came back after long time. This time again with java. Recenly playing with lot of java solutions relating to scaling and clustering. Lot of places java high performacne systems you see buzzwords like multi-threading, clustering, caching, map-reduce, partitioning, grid solutions. Lets discuss some of them.

I have been using hibernate for last 1.5 year in my personal experiments and at work place too. In one of project we had cache requirement for master data which was read-only. There we had implemented caching by hand as follows.

1. Wrapper around DAOs which first check in cache and then go to database.
2. Cache was implemented with Websphere provided object pooling API.
3. Websphere provided object pooling mechanism which even works in clustered set-up or network deployment with cache replication strategies.

Our requirement was well met and caching brought lot of improvement in response time as expected.

After caching, horizontal scaling or clustering comes into my mind when i think about performance.
Since then, I discovered lot of caching/clustering projects/apis, some of which are very much i feel worth to take a note.

1. Tangosol - recently acquired by Oracle, along with oracle, timesten ( oracle in-memory database, another acquisition :-) ) forms formidable data-tier. Tangasol is very proven prodcut which provides lot of features like distributed cache, data partitioning. Cameron Purdy on Theserverside claims to reach upto 0.5 milllion cache transaction per second.

2. Terracotta :- Terracotta is very revolutionary java product in fact listed in top 10 java things of year. Terracotta actually speaking is not caching solution but a clustering solution. Its basically JVM level clustering. They have provided cache solutions for lot of commons requirements : Hibernate, HTTP Session, Spring Beans etc. They provide good extension for lot of open-source projects. I had tested tomcat clustering and terracotta, similar to one on terracotta site and it gives good performance boost. Among all other clustering solutions terracotta has simplest programming model : NO PROGRAMMING MODEL. right : objects basically clustered at jvm level so only configuration change no code change. Practically you do require to change java code but that's minimal All java semantics work well in terracotta cluster. Terracotta is very useful in patterns like Master/Worker or bunch processes looking for some coordination or data sharing. Master Worker pattern term i guess was introduced much before Googles Map/Reduce and very much the same idea.

3. Memcached : this one is in c but has client libraries in almost all major programming languages. Idea of memcached is actually little bit different. You keep bunch of memcached process running objects are stored on these processes with hash key. Memcached works best when its processes run on web servers which are cpu-intensive with lot of memory available.

4. New bunch of grind frameworks : gigaspaces, hadoop - java map-reduce implementation

But now i had come across very different requirement where cache modification (writes) were significant in numbers and jdbc-batching with asynchronous operation was key to performance sometime this is called - write behind. When i looked upon only tangosol claims to have such facility so i decided to implement by hand. Here is an idea.

1. A background threads basically monitors queue.
2. All cache write operations append modified objects to this queue.
3. queue periodically flushes them to database and notifies successful operation to clients.

Here catch is how do you handle object graph updates
1. Delta Calculation
2. New Object insertion may require updates in one specific order where result of first inserts basically required for next one. (foreign key)

Initially i went with hibernate where i used StatelessSession as SQL generation engine hoping that hibernate will calculate delta properly. But lack of L1 cache means no life cycle hence no delta monitoring. So i had two choices.

1. Use merge : Fires extra selects
2. Generate sql by hand.

With help of cglib proxies i managed to get list of dirty columns but then how do you handle object relationships?

I soon realized whole ORM needs to be implemented... here list of requirements for basic ORM cum cache with write-behind

1. easy configuration : declarative orm mapping sufficient for most of scenarios
2. Should provide different execution strategies timer driven, threadpool driven, batch reached, batch and timer combined, entity level strategy for sync
3. in-memory multi-indexing based on columns or own implementation : cache object can be queried with different criteria
4. notification - listeners
5. target rate with % of write operation : 500 requests/second.
5. recovery test : check points : automated and manual
6. clustering with help of terracotta.

This is exhaustive list .. where first one itself is very big one..... my take on it is to restrict the scope and focus on caching instead of fancy orm stuff the one like hibernate.... it should be able to support only object graph ... all lifecycle is left to user..

I will add code snippets in coming posts ... now only idea has been finalized.


Nati Shalom said...

Hi Tushar

If what your looking for is:

". A background threads basically monitors queue.
2. All cache write operations append modified objects to this queue.
3. queue periodically flushes them to database and notifies successful operation to clients."

I'd recommend that you would read the following posts where i discuss a pattern for async-write to the database in a seamless way to the application and how you can still maintain reliability in-memory without relying on the database. We used this pattern in lots of projects to achieve highly scalable write/read caches:

Persistency as a Service (Using Hibernate)

Scaling out mySQL

If your looking for a more end to end solution for scaling out your web application I'd recommend that you would see how were able to scale-out jetty (and later tomcat) and inject the cache for session replication seamlessly as well as managing the deployment and SLA.

Scaling out web applications

Nati S

Talip Ozturk said...

Good stuff Tushar. Keep it up!


Geek said...

Hi Nati,

Thanks for links and guidance.

I looked upon Gigaspaces and found this interesting option :

Our main concern is ORM : Existing data model is fairly complex and hibernate based.
Certainly, Gigaspaces evaluation will give better insights to us.

One question, whats better for write driven systems as opposed to read-driven system where Hibernate L2 cache gives performance improvement? Option that Gigaspace provides looks good to me.
But how Gigaspace works with hibernate because my experiments with hibernate were not helpful
I tried with StatelessSesion for asynchronous updates but it fires updates for all columns. Best option for me right now is to modify merge code to issue update sql for delta-change only.
How gigaspace manages object life cycle when hibernate is used in backend as orm.

Thanks and Regards,

Uri Cohen said...

Hi Tushar

For write intensive scenarios we recommend using our asynchronous persistency mechanism (we call it mirror service). As far as your application is concerned it interacts with the data grid. The data grid is batching write operations and asynchronously propagates then to the persistency service and from there to the database. The propagation is done by default with Hibernate to map the objects stored in the data grid to the database.
This maintains good runtime performance since your transactions are not hitting the database, and it optimizes database write operations using batching. Nati's blog post (referenced by the above link) also describes how this is done reliably, even though the database writes are not part of the runtime transaction.

Uri Cohen

Cameron said...

Tushar, the write-behind capability that you describe is built in to Oracle (Tangosol ;-) Coherence, and queues up the changes, writing them asynchronously -- even if one of the servers dies.


Cameron Purdy
Oracle Coherence