So you probably know that I have been working with HP’s Vertica. I’m very glad that I had this opportunity to interact with something like this. The technology behind it is pretty new and immature and lacks the power of PL/SQL but the speed that it achieves through massive parallel execution of queries is simply amazing.
I don’t know if you have heard of the term K-Safe or K-Safety. When I first heard it, I had no idea what it was. Then after going through some of the documentation I kind of got the point.
K-safety is a way to measure fault tolerance. And K here is a number of duplicate copies of the data in the database cluster. These duplicated copies of data help nodes take over responsibilities of failed nodes, thereby allowing the database to maintain high availability, despite the loss of K nodes.
Quite interesting isn’t it?
More about it can be found in the Vertica documentation: https://my.vertica.com/docs/6.1.x/HTML/index.htm#11175.htm
K-Safety can be set at a table level and also set at a cluster level when you setup the database for the first time. So if you had your cluster wide K-Safety setting set to 2 (K=2, means 2 additional copies of each projection per table) then you can’t really set the K-Safety of a particular table in that database to something less than 2. Vertica would then give you an error like the following:
ERROR 4478: Projection KSAFE override 1 cannot be less than current system K-safe value 2
It makes sense, because overriding to a value less than 2, defeats the whole purpose of setting the system wide k-safe value to 2.
So keep that in mind when you decide to set a fault tolerance level. And Vertica these days, recommend a K-Safety of 1. Not because fault tolerance is bad, but they have developed more sophisticated management tools that will let you take down a node safely for maintenance and let other things carry on. And lesser number of projection copies, would improve write performance also. But I still think there are many more benefits that you get by reducing the k-safety apart from the disk space that you save and the write speed that you gain. But probably Vertica could give you a better explanation on that.