Custom blooms on index data to speed up index data look up

[anoopsjohn]

Will add the details later in description

[anoopsjohn]

Any one interested to work on this? Will need some enabling work in HBase as well. This should help in Scan performance.

[chrajeshbabu]

Anoop, can't we use existing ROW or ROWCOL bloom filters? How this custom bloom filters help to improve performance?

[ramkrish86]

Am just seeing these updates in the JIRA. If we are really aiming in making it more public and gain more visibility we could definitely spend some solid time in this. +1 for it. I need to refresh the code before I could comment on this but we could make this more visible. One thing I was seeing is that some concerns people raise is that about the data type supported and its format while using indices which Phoenix tries to handle. How big is that gap in this Hindex? We could also take those activities up so that this soln is also does not have such gaps, (if any)?

[anoopsjohn]

@chrajeshbabu
No existing row filter can not be used on index table HFiles. The rk of index data includes rk of the actual table also.
When we have a query like select * from table where c1 = ? and having 100 regions, we will do scan on index table on all 100 regions. Now if there was some blooms using which we can say clearly any data in the index region with c1=?, we can avoid those region's scan. So if out of 100 regions, only 10 regions we have c1=? data, we can save lot of time. The global index have this benefit and the issue with local index is we have to go to all index regions. Make sense?