You're the tertiary storage; I'm the L1 cache.
You're a dialup connection; I'm a gigabit LAN.
Wednesday, December 24, 2008
Funny Tech Video Part 3
Below is really gr8 video called kill dash nine. "Kill -9" is the command in unix to kill the process like Task Manager in Windows. You can find entire lyrics here. You can download mp3 version of it too from the same page. Some of the lines are really gr8 like
You're running csh and my shell is bash,
You're the tertiary storage; I'm the L1 cache.
You're a dialup connection; I'm a gigabit LAN.
You're the tertiary storage; I'm the L1 cache.
You're a dialup connection; I'm a gigabit LAN.
Sunday, December 21, 2008
An Insane idea to Solve Java's Memory Problem
This could be insane idea to solve java's memory problem.
Garbage collection overhead has been one of the major performance problem. Recent improvements in Garbage collector : ability to parallel collect garbage using more processors or cores available, concurrent garbage collection has drastically reduced this belief that GC is mostly cause of performance problems. Earlier this problem could be solved by running multiple instances of application on the same box to use its all CPU power and memory. But for some memory-intensive application GC overhead is still a problem to be solved. As JVM is being touted as more of Virtual Machine for many interpreted languages as opposed to only Java runtime memory problem is bound to occur. I have been working recently on Terracotta which is Network Attached Memory (NAM) for Java Applications. When you say NAM, you application now can access more number of objects that could fit in your local heap since objects which are not in your local heap are transparently loaded into local JVM when they are accessed .. sort of lazy loading in hibernate. Now this idea can also be applied to JVM .. offload some objects which are not accessed often down to local disk and load them when they are not accessed.Imagine a situation where you are running application on 16 core or 32 core processors ( intel has one prototype of 80 cores) ..data processing ability of such hugh machine. With 64-bit platform JVM size can grow beyond 2 GB limit but when Full GC happens on JVM sized more than 4 GB its really painful for applications. I have never worked on such hugh JVMs no really no idea how jdk 5 and 6 performs in such situations. Sure there will be some optimizations in JVM to operate at such scale.
A simple prototype of this is various cache solutions which offer cache eviction to local disk when number of objects cross defined cache size. But these implementations are targeted as "caching solutions" which know only how to "get" and "put" objects in a manner so as to make best use of available space.
At first this idea may be insane but various optimizations can be done to make it practical. This is what terracotta has done, to avoid object serialization and operate at field level. Consider a map of 10000 fat objects. When this map is offloaded to disk all value objects will also be written to disk. Now when some value object is looked upon only the object can be loaded while rest of the objects still being stored on disk. This basically means you have to implement some sort of virtual memory manager for virtual machine which will use application access pattern and some stored intelligence to minimize the total load delay and at the same time allow application to operate on large data set.
Any comments are welcome .. I am sure there will be a lot of comments on this insane idea.
..
Tushar
Garbage collection overhead has been one of the major performance problem. Recent improvements in Garbage collector : ability to parallel collect garbage using more processors or cores available, concurrent garbage collection has drastically reduced this belief that GC is mostly cause of performance problems. Earlier this problem could be solved by running multiple instances of application on the same box to use its all CPU power and memory. But for some memory-intensive application GC overhead is still a problem to be solved. As JVM is being touted as more of Virtual Machine for many interpreted languages as opposed to only Java runtime memory problem is bound to occur. I have been working recently on Terracotta which is Network Attached Memory (NAM) for Java Applications. When you say NAM, you application now can access more number of objects that could fit in your local heap since objects which are not in your local heap are transparently loaded into local JVM when they are accessed .. sort of lazy loading in hibernate. Now this idea can also be applied to JVM .. offload some objects which are not accessed often down to local disk and load them when they are not accessed.Imagine a situation where you are running application on 16 core or 32 core processors ( intel has one prototype of 80 cores) ..data processing ability of such hugh machine. With 64-bit platform JVM size can grow beyond 2 GB limit but when Full GC happens on JVM sized more than 4 GB its really painful for applications. I have never worked on such hugh JVMs no really no idea how jdk 5 and 6 performs in such situations. Sure there will be some optimizations in JVM to operate at such scale.
A simple prototype of this is various cache solutions which offer cache eviction to local disk when number of objects cross defined cache size. But these implementations are targeted as "caching solutions" which know only how to "get" and "put" objects in a manner so as to make best use of available space.
At first this idea may be insane but various optimizations can be done to make it practical. This is what terracotta has done, to avoid object serialization and operate at field level. Consider a map of 10000 fat objects. When this map is offloaded to disk all value objects will also be written to disk. Now when some value object is looked upon only the object can be loaded while rest of the objects still being stored on disk. This basically means you have to implement some sort of virtual memory manager for virtual machine which will use application access pattern and some stored intelligence to minimize the total load delay and at the same time allow application to operate on large data set.
Any comments are welcome .. I am sure there will be a lot of comments on this insane idea.
..
Tushar
Tuesday, December 16, 2008
Funny Tech Video Part 2
Hug a developer ....
Now this is one is reaaly funny and for the developers
What makes an engineer .. little dilbert video
Every bulid you break ...
Now this is one is reaaly funny and for the developers
What makes an engineer .. little dilbert video
Every bulid you break ...
Funny Tech Videos
Here are some really funny tech videos.
First one is my favorite : Matrix runs on windows. Listen carefully for some meaningful (real-world universal truth) lines like : "progress bar is moving but remaining time is going up". Its time to upgrade to Ubuntu.
Second one is music remix based on XP error sounds.
Third one is really creative geeky rap video from some "Sniper Twins".
Fourth one is from movie Office Space : Movie based on life of frustrated IT-Software Engineer, really describes life of some my friends : working on weekends, reporting to more than one boss. Somebody come up with that one, good, its old movie though 1996.
First one is my favorite : Matrix runs on windows. Listen carefully for some meaningful (real-world universal truth) lines like : "progress bar is moving but remaining time is going up". Its time to upgrade to Ubuntu.
Second one is music remix based on XP error sounds.
Third one is really creative geeky rap video from some "Sniper Twins".
Fourth one is from movie Office Space : Movie based on life of frustrated IT-Software Engineer, really describes life of some my friends : working on weekends, reporting to more than one boss. Somebody come up with that one, good, its old movie though 1996.
Saturday, December 13, 2008
Online TV : Reality for Indians
I recently moved to Noida for my new job and missing watching TV, especially Zee Marathi SA-RE-GA-MA-PA. And then very unthinkable happened : Shootout in the heart of mumbai - my beloved city. It was really hard to think whats going on in Mumbai, sitting in air-conditioned office in Noida. I could watch those boring and ever-repeating clips of news channel videos online but these are only dramatic and not informative at all. Then I thought what if I could watch TV online where I can get whole feel of whats happening, and yes by doing little bit of googling I managed to find out lot of channel which are available online. I could watch My favorite TV serial (one-and-only-one) online. Not only that I could see live news channel. Go to : http://www.ibollytv.com/iTV.php. Lot of regional channels are covered there. Internet connection speed required for these feed depends on the quality but 512 Kbps (that's around 64 KBps download speed) is sufficient for it. Sad thing is that, in India wireless internet is still hanging around 144 Kbps boundary so you can not avail this facility on move. On broadband front though thing look OK now, lot of ISP now provide 2 Mbps speed so things
One thing I want say is Thanks and I really appreciate to this to CNN-IBN. CNN-IBN (Leading news channel in India) has official free live feed. That's what you call free press. In events like Mumbai shootout this could really save your and your families day. Thanks once gain : Rajdeep Sardesai.
..
Tushar
One thing I want say is Thanks and I really appreciate to this to CNN-IBN. CNN-IBN (Leading news channel in India) has official free live feed. That's what you call free press. In events like Mumbai shootout this could really save your and your families day. Thanks once gain : Rajdeep Sardesai.
..
Tushar
Friday, December 12, 2008
Links : Kill Your Database with Terracotta
I am going to start a new series from this post. Many times you find great article on the internet that you would like to share with others. "Links" is one such series.
And very first link here is an Article written on Terracotta : An amazing technology for java application clustering. Its here : Kill Your Database. Terracotta is clustering technology with built-in support for HA. Clustering is all about shared data and when you talk about HA, you are actually caring for the shared data, its availability. Terracotta implements it by writing every change to shared data to disk in transactional manner. So if your data whose life is short or medium, (often you need to store derived data from the temporary data which is very large, and we use RDBMS for all of this) you can use terracotta to store it in very fast manner : since you don't have to convert data to relational model its all java. Pretty powerful!!. Other use case is Database off-loading which means you can use terracotta to store all data temporarily and then write to your database in manner which would not affect end-user response time. By doing this you remove database access and operation time from response time ( which mostly is the very significant part of the response time). Article mentioned above touches all these concepts. I will add one more link on terracotta org-site which describes this case ( and all other wonderful use-cases for terracotta). Here it is : Write Behind SOR. If you read my previous post, I had exactly discussed the same idea : Write Behind or Asynchronous writes. That time I knew only coherence supported this. Then I came to know that Gigaspaces also supports it. ( See comments below on the same post). But terracotta would be the most exciting among them all. Why? One reason is all existing application can easily integrate this pattern in there application with very small change in code-base. Second reason terracotta supports lot of java framework out of the box.
..
Tushar
And very first link here is an Article written on Terracotta : An amazing technology for java application clustering. Its here : Kill Your Database. Terracotta is clustering technology with built-in support for HA. Clustering is all about shared data and when you talk about HA, you are actually caring for the shared data, its availability. Terracotta implements it by writing every change to shared data to disk in transactional manner. So if your data whose life is short or medium, (often you need to store derived data from the temporary data which is very large, and we use RDBMS for all of this) you can use terracotta to store it in very fast manner : since you don't have to convert data to relational model its all java. Pretty powerful!!. Other use case is Database off-loading which means you can use terracotta to store all data temporarily and then write to your database in manner which would not affect end-user response time. By doing this you remove database access and operation time from response time ( which mostly is the very significant part of the response time). Article mentioned above touches all these concepts. I will add one more link on terracotta org-site which describes this case ( and all other wonderful use-cases for terracotta). Here it is : Write Behind SOR. If you read my previous post, I had exactly discussed the same idea : Write Behind or Asynchronous writes. That time I knew only coherence supported this. Then I came to know that Gigaspaces also supports it. ( See comments below on the same post). But terracotta would be the most exciting among them all. Why? One reason is all existing application can easily integrate this pattern in there application with very small change in code-base. Second reason terracotta supports lot of java framework out of the box.
..
Tushar
Subscribe to:
Posts (Atom)