Wednesday, April 8, 2009
Links : Distributed Hash Tables
Nice List of Distributed Key-Value Stores : http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/
Querying Java Objects stored in Terracotta's NAM
This post is inspired from : http://forums.terracotta.org/forums/posts/list/1965.page . Nothing new .. just another word in blogosphere.
Terracotta is gr8 clustering solution in-fact its platform-level service hence has large number of uses. One of the use is using it as database. Terracotta can never replace database but it can play role of data storage media very well. One of major disadvantage is lack of querying data. Only way you can query data is Map. Map is like single index so if you want to get list of object satisfying some criteria you are required to integrate through entire collection. There are already APIs written for querying java collections. So you can use them with Terracotta NAM.
When you think about querying there are lot of factors : Query Language, Its Performance - Optimizers, Operations Supported : Select, Update, Delete, Joins
JoSQL is good API for querying java collections with good documentation. I did small test with 1 million objects and random query took around 800ms which is way too much. Again its simple iteration through collection due to lack of indexes and query execution plan. Problem with Indexes is that Object graphs can change and at every change you are required to recompute the index which would be difficult to do : as complex as Terracotta's bytecode instrumentation. You can find test code here
Query Language : Moderately good, Performance : Not good for large collections, No update or delete only select projection queries. No joins
Quaere is a very flexible DSL that lets you perform a wide range of queries against any data structure that is an array, or implements the java.lang.Iterable. Its sort of port of LINQ of .Net world. I think linq is next generation data quering tool -cleaner. Quaere is not query language but query API just like Hiberate Criteria query but more elegant. I really liked quaere - its really powerful its support join operation. You can read this post for details : Solving Puzzles with Quaere Its still beta level and not released. One of Queare's another sister project is its JPA integration. Imagine you could write standard JPA application with Quaere as query language and Terracotta as persistent store. No need of database. But as with JoSQL Quaere is also slow. I mean slower than RDBMS. I did small test with 1 million object similar to JoSQL test and response time was similar to JoSQL. You can download test code here
This post also discusses jmap's OQL implementation. It uses rhino javascript engine behind with hashtables.I did consider it to port for Terracotta but its custom written for Object Heap Dumps. JxPath is another tool with which you can query java collections using XPath expressions. I did not evaluate JxPath since i felt it will be on similar lines of JoSQL and Quaere, only different flavor. If you have used XPath earlier then this is much easier to use.
GlazedList is event driven list API specially designed for Swing Applications displaying table and list data. But if you consider List of Objects as table (each object is row and its properties as columns) a proper in-memory index can be maintained for querying. But this applies to only root object level. What if inner object in object graph store in your container changes?. You may then need to update the container whenever object changes. So i guess maintaining in-memory index for java objects is pretty difficult thing to do.
With such tools i think you can easily query moderate size java collections stored in Terracotta's durable memory with acceptable response time.
Tushar
Terracotta is gr8 clustering solution in-fact its platform-level service hence has large number of uses. One of the use is using it as database. Terracotta can never replace database but it can play role of data storage media very well. One of major disadvantage is lack of querying data. Only way you can query data is Map. Map is like single index so if you want to get list of object satisfying some criteria you are required to integrate through entire collection. There are already APIs written for querying java collections. So you can use them with Terracotta NAM.
When you think about querying there are lot of factors : Query Language, Its Performance - Optimizers, Operations Supported : Select, Update, Delete, Joins
JoSQL
JoSQL is good API for querying java collections with good documentation. I did small test with 1 million objects and random query took around 800ms which is way too much. Again its simple iteration through collection due to lack of indexes and query execution plan. Problem with Indexes is that Object graphs can change and at every change you are required to recompute the index which would be difficult to do : as complex as Terracotta's bytecode instrumentation. You can find test code here
Query Language : Moderately good, Performance : Not good for large collections, No update or delete only select projection queries. No joins
Quaere
Quaere is a very flexible DSL that lets you perform a wide range of queries against any data structure that is an array, or implements the java.lang.Iterable. Its sort of port of LINQ of .Net world. I think linq is next generation data quering tool -cleaner. Quaere is not query language but query API just like Hiberate Criteria query but more elegant. I really liked quaere - its really powerful its support join operation. You can read this post for details : Solving Puzzles with Quaere Its still beta level and not released. One of Queare's another sister project is its JPA integration. Imagine you could write standard JPA application with Quaere as query language and Terracotta as persistent store. No need of database. But as with JoSQL Quaere is also slow. I mean slower than RDBMS. I did small test with 1 million object similar to JoSQL test and response time was similar to JoSQL. You can download test code here
This post also discusses jmap's OQL implementation. It uses rhino javascript engine behind with hashtables.I did consider it to port for Terracotta but its custom written for Object Heap Dumps. JxPath is another tool with which you can query java collections using XPath expressions. I did not evaluate JxPath since i felt it will be on similar lines of JoSQL and Quaere, only different flavor. If you have used XPath earlier then this is much easier to use.
GlazedList is event driven list API specially designed for Swing Applications displaying table and list data. But if you consider List of Objects as table (each object is row and its properties as columns) a proper in-memory index can be maintained for querying. But this applies to only root object level. What if inner object in object graph store in your container changes?. You may then need to update the container whenever object changes. So i guess maintaining in-memory index for java objects is pretty difficult thing to do.
With such tools i think you can easily query moderate size java collections stored in Terracotta's durable memory with acceptable response time.
Tushar
Tuesday, April 7, 2009
Maven : Java Profiling
At workplace I use maven extensively. In fact I like maven so much that I am slowly moving lot of java eclipse project into maven project. Maven has very good eclipse plugin which makes integration with Eclipse very easy. You can run maven from eclipse. Maven is really good tool. If you have any specific needs you can write your own plugin. At my previous workplace, senior engineer in my project automated everything (starting jboss servers on remote machine in various environments : testing, perf-testing, integration testing etc. database population, database schema drop/rebuild etc).
But now I wanted to profile java application which I used to run from maven. One of the best part of maven is dependency management and repository - it builds classpath automatically for you, but then its pain too. If you want to run java application you have to do through Maven. There is exec plugin with which you can run any Application or shell script and there is exec:java with which you can run java Main class. The problem is that exec:java is in same JVM. So you cant run any java agent(-agent) or other things : Specifically Java Profiling. You should make sure you run the application within the same environment/settings.
So my first task was to get the complete classpath and then launch java with profiler java options. I am using jprofiler which uses JVMTI agent hence you need to append "-agentlib:jprofilerti=port=31757 -Xbootclasspath/a:/Applications/jprofiler5/bin/agent.jar" to java command line.
Here is little shell script through which I managed to do java profiling for maven Project. This will work only for J2SE applications tough!. Frustrating part was variable DYLD_LIBRARY_PATH. I was new to MacOS and was trying with usual LD_LIBRARY_PATH and -agentpath jvm option. Surprisingly -agentpath option should work on MacOS but it didnt work i guess some problem with Jprofiler binary. But lastly i managed to profile my application properly.
This is the first time I had to do away with maven command. I wished somebody had written maven plugin for lauching jprofiler enabled apps. There is maven plugin for Yourkit Java Profiler : http://code.google.com/p/maven-yourkit-plugin/ but it did now work.
But now I wanted to profile java application which I used to run from maven. One of the best part of maven is dependency management and repository - it builds classpath automatically for you, but then its pain too. If you want to run java application you have to do through Maven. There is exec plugin with which you can run any Application or shell script and there is exec:java with which you can run java Main class. The problem is that exec:java is in same JVM. So you cant run any java agent(-agent) or other things : Specifically Java Profiling. You should make sure you run the application within the same environment/settings.
So my first task was to get the complete classpath and then launch java with profiler java options. I am using jprofiler which uses JVMTI agent hence you need to append "-agentlib:jprofilerti=port=31757 -Xbootclasspath/a:/Applications/jprofiler5/bin/agent.jar" to java command line.
Here is little shell script through which I managed to do java profiling for maven Project. This will work only for J2SE applications tough!. Frustrating part was variable DYLD_LIBRARY_PATH. I was new to MacOS and was trying with usual LD_LIBRARY_PATH and -agentpath jvm option. Surprisingly -agentpath option should work on MacOS but it didnt work i guess some problem with Jprofiler binary. But lastly i managed to profile my application properly.
mvn dependency:build-classpath -Dmdep.outputFile=mycp.txt
export CLASSP=`cat ./mycp.txt`
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/Applications/jprofiler5/bin/macos
$JAVA_HOME/bin/java -cp $CLASSP:./target/classes -agentlib:jprofilerti=port=31757 -Xbootclasspath/a:/Applications/jprofiler5/bin/agent.jar $*
This is the first time I had to do away with maven command. I wished somebody had written maven plugin for lauching jprofiler enabled apps. There is maven plugin for Yourkit Java Profiler : http://code.google.com/p/maven-yourkit-plugin/ but it did now work.
Monday, March 9, 2009
Links : What is REST?
I needed to learn about REST interface and wanted some good article for learning basics of REST : REpresentational State Transfer. This article : A Brief Introduction to REST is really the best one I found so thought of sharing it with you. REST is really nice idea of further simplifying the modeling of web-applications. I implemented by REST program with JSON and my client is in Javascript which is really cool (People are writing entire web based OS and here i am playing with simple java script stuff :-) ) With help of java-script libraries you can easily achieve data binding : like FLEX and other RIA. I really enjoyed working with this totally different stuff. I will add snippets in coming days.
..
Tushar
..
Tushar
Thursday, February 26, 2009
Links : Things I Wish I’d Been Told
When I look back in my engineering days and now, I certainly figured out one thing : Whatever they teach in class hardly matters - the syllabus is totally outdated. What matters is that how you learn new things quickly, be updated in this fast moving world of technology.
Here I am sharing a link I Wish I’d Been Told, when I graduated with computer science engineering degree. Tips For Students with a Bachelors in Computer Science
Here I am sharing a link I Wish I’d Been Told, when I graduated with computer science engineering degree. Tips For Students with a Bachelors in Computer Science
Saturday, February 14, 2009
Funny Tech Videos : Part 5
Geeky Valentine .. for bloggers and twiters.
Happy Valentine Days May all you get linklove that you deserve.
..
Tushar
Happy Valentine Days May all you get linklove that you deserve.
..
Tushar
Download Your Favorite Youtube Videos in Batch
Last week SRGMAP - LC ( Sa-Re-Ga-Ma-Pa Marathi Little Champs) singing contest for children finished. These children are really are talented and boy, crowd loved every performance from them. From long time this was the only show that I wanted to watch every episode of it and did not miss much initially. But when I moved out for my job I was missing it but then this guy "ahonkan" made sure i at least get to see performance of these god gifted children. Thanks a million ahonkan .. there will countless others like me. Now competition is over I thought of downloading all videos and keeping copy of it for as my personal collection. Initially I thought I will write my small program in java where I will download the user specific feeds, playlists, Chanel and the via youtube API i will download all FLV files. But then I did came across this great program. : 1-Click Youtube Downloaded. It allows you download and match bunch of videos It allows you donwload and match bunch of videos from youtube.
Below is the video from 1-Click which shows you how to download videos using this program.
great stuff. may be older but still good enough for another word in blogo-sphere.
Note : Videos uploaded on youtube could be subject to copyright and may be illegal. Here I am only writing about the useful computer program. Make sure you know what you are downloading. Thanks
Tushar
Below is the video from 1-Click which shows you how to download videos using this program.
great stuff. may be older but still good enough for another word in blogo-sphere.
Note : Videos uploaded on youtube could be subject to copyright and may be illegal. Here I am only writing about the useful computer program. Make sure you know what you are downloading. Thanks
Tushar
Subscribe to:
Posts (Atom)