diff --git a/LICENSE.txt b/LICENSE.txt new file mode 100644 index 00000000..d6456956 --- /dev/null +++ b/LICENSE.txt @@ -0,0 +1,202 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/NOTICE.txt b/NOTICE.txt new file mode 100644 index 00000000..08364793 --- /dev/null +++ b/NOTICE.txt @@ -0,0 +1,25 @@ +This product includes software developed by The Apache Software +Foundation (http://www.apache.org/). + +It includes software from other Apache Software Foundation projects, +including, but not limited to: + - Apache HBase + - Apache Hadoop + - Apache Commons + +In addition, this product includes/uses software developed by: + +JLine (http://jline.sourceforge.net/), +Copyright (c) 2002-2006, Marc Prud'hommeaux . + +SQLLine ((http://jline.sourceforge.net/), +Copyright (c) 2002-2006, Marc Prud'hommeaux . + +SLF4J (http://www.slf4j.org/), +Copyright (c) 2004-2008 QOS.ch + +ANTLR (http://www.antlr.org/), +Copyright (c) 2003-2008, Terrence Parr. + +JUnit (http://www.junit.org/) included under the Common Public License v1.0. +See the full text here: http://junit.sourceforge.net/cpl-v10.html diff --git a/README.md b/README.md deleted file mode 100644 index cb1ade73..00000000 --- a/README.md +++ /dev/null @@ -1,136 +0,0 @@ -![logo](http://forcedotcom.github.com/phoenix/images/logo.jpg) - -Phoenix is a SQL skin over HBase, delivered as a client-embedded JDBC driver, powering the HBase use cases at Salesforce.com. Phoenix targets low-latency queries (milliseconds), as opposed to batch operation via map/reduce. To see what's supported, go to our [language reference guide](http://forcedotcom.github.com/phoenix/), read more on our [wiki](https://github.com/forcedotcom/phoenix/wiki), and download it [here](https://github.com/forcedotcom/phoenix/wiki/Download). -## Mission -Become the standard means of accessing HBase data through a well-defined, industry standard API. - -## Quick Start -Tired of reading already and just want to get started? Listen to the Phoenix talks from [Hadoop Summit 2013](http://www.youtube.com/watch?v=YHsHdQ08trg) and [HBaseConn 2013](http://www.cloudera.com/content/cloudera/en/resources/library/hbasecon/hbasecon-2013--how-and-why-phoenix-puts-the-sql-back-into-nosql-video.html), check out our [FAQs](https://github.com/forcedotcom/phoenix/wiki/F.A.Q.), and jump over to our quick start guide [here](https://github.com/forcedotcom/phoenix/wiki/Phoenix-in-15-minutes-or-less) or map to your existing HBase tables as described [here](https://github.com/forcedotcom/phoenix/wiki/Phoenix-Introduction#mapping-to-an-existing-hbase-table) and start querying now. - -## How It Works ## - -The Phoenix query engine transforms your [SQL query](http://forcedotcom.github.com/phoenix/#select) into one or more HBase scans, and orchestrates their execution to produce standard JDBC result sets. Direct use of the HBase API, along with coprocessors and custom filters, results in [performance](https://github.com/forcedotcom/phoenix/wiki/Performance) on the order of milliseconds for small queries, or seconds for tens of millions of rows. - -Tables are created and altered through [DDL statements](http://forcedotcom.github.com/phoenix/#create), and their schema is stored and versioned on the server in an HBase table. Columns are defined as either being part of a multi-part row key, or as key/value cells. You can also map Phoenix on to existing tables (see the [wiki](https://github.com/forcedotcom/phoenix/wiki) for more details). - -Applications interact with Phoenix through a standard JDBC interface; all the usual interfaces are supported, including `Connection`, `Statement`, `PreparedStatement`, and `ResultSet`. The driver class is `com.salesforce.phoenix.jdbc.PhoenixDriver`, JDK 1.5+ automatically registers JDBC driver on classpath, and the [connection url](https://github.com/forcedotcom/phoenix/wiki/Phoenix-Introduction#wiki-connStr) is `jdbc:phoenix:` followed by the zookeeper quorum hostname specification plus optionally the port number and/or root node. For example: - - Connection conn = DriverManager.getConnection("jdbc:phoenix:localhost"); - -For detailed documentation on the current level of SQL support, see our [language reference guide](http://forcedotcom.github.com/phoenix/). For details about how Phoenix handles schema, transactions, and more, see the [wiki](https://github.com/forcedotcom/phoenix/wiki). - -## System Requirements ## -* HBase v0.94.4+ (0.96 is not yet supported) -* JDK 6 or higher - -## Build Requirements ## -* All the system requirements -* Maven 3.X (https://maven.apache.org/) - - -## Installation ## -To install a pre-built phoenix, use these directions: - -* Download and expand the latest phoenix-[version]-install.tar from [download page](https://github.com/forcedotcom/phoenix/wiki/Download) -* Remove prior phoenix-[version].jar from every HBase region server. -* Add the phoenix-[version].jar to the classpath of every HBase region server. An easy way to do this is to copy it into the HBase lib directory. -* Restart all region servers. -* Remove prior phoenix-[version]-client.jar from the classpath of any Phoenix client. -* Add the phoenix-[version]-client.jar to the classpath of any Phoenix client. - -Alternatively, you can build it yourself using maven by following these [build instructions](https://github.com/forcedotcom/phoenix/wiki/Building-Project). - - -## Getting Started ## -Wanted to get started quickly? Take a look at our [FAQs](https://github.com/forcedotcom/phoenix/wiki/F.A.Q.) and take our quick start guide [here](https://github.com/forcedotcom/phoenix/wiki/Phoenix-in-15-minutes-or-less). - -

Command Line

- -A terminal interface to execute SQL from the command line is now bundled with Phoenix v 1.2. To -start it, execute the following from the bin directory: - - $ sqlline.sh localhost - -To execute SQL scripts from the command line, you can include a SQL file argument like this: - - $ sqlline.sh localhost ../examples/stock_symbol.sql - -![sqlline](http://forcedotcom.github.com/phoenix/images/sqlline.png) - -For more information, see the [manual](http://www.hydromatic.net/sqlline/manual.html). - -
Loading Data
- -In addition, you can use the bin/psql.sh to load CSV data or execute SQL scripts. For example: - - $ psql.sh localhost ../examples/web_stat.sql ../examples/web_stat.csv ../examples/web_stat_queries.sql - -Other alternatives include: -* Using our [map-reduce based CSV loader](https://github.com/forcedotcom/phoenix/wiki/Bulk-CSV-loading-through-map-reduce) for bigger data sets -* [Mapping an existing HBase table to a Phoenix table](https://github.com/forcedotcom/phoenix/wiki/Phoenix-Introduction#mapping-to-an-existing-hbase-table) and using the [UPSERT SELECT](http://forcedotcom.github.io/phoenix/#upsert_select) command to populate a new table. -* Populating the table through our [UPSERT VALUES](http://forcedotcom.github.io/phoenix/#upsert_values) command. - -

SQL Client

- -If you'd rather use a client GUI to interact with Phoenix, download and install [SQuirrel](http://squirrel-sql.sourceforge.net/). Since Phoenix is a JDBC driver, integration with tools such as this are seamless. Here are the setup steps necessary: - -1. Remove prior phoenix-[version]-client.jar from the lib directory of SQuirrel -2. Copy the phoenix-[version]-client.jar into the lib directory of SQuirrel (Note that on a Mac, this is the *internal* lib directory). -3. Start SQuirrel and add new driver to SQuirrel (Drivers -> New Driver) -4. In Add Driver dialog box, set Name to Phoenix -5. Enter com.salesforce.phoenix.jdbc.PhoenixDriver in the Class Name textbox. Press OK to close this dialog. -6. Switch to Alias tab and create the new Alias (Aliases -> New Aliases) -7. In the dialog box, Name: _any name_, Driver: Phoenix, User Name: _anything_, Password: _anything_ -8. Construct URL as follows: jdbc:phoenix: _zookeeper quorum server_. For example, to connect to a local HBase use: jdbc:phoenix:localhost -9. Press Test (which should succeed if everything is setup correctly) and press OK to close. -10. Now double click on your newly created Phoenix alias and click Connect. -11. Navigate to File --> New Session Properties --> Tab SQL and uncheck the "Remove multi line comment (/.../) from SQL before sending it to database" so that hints you include in queries are not stripped. -11. Now you are ready to run SQL queries against Phoenix. - -Through SQuirrel, you can issue SQL statements in the SQL tab (create tables, insert data, run queries), and inspect table metadata in the Object tab (i.e. list tables, their columns, primary keys, and types). - -![squirrel](http://forcedotcom.github.com/phoenix/images/squirrel.png) - -## Maven ## - -Currently, Phoenix hosts its own maven repository in github. This is done for convience and will later be moved to a 'real' maven repository. You can add it to your mavenized project by adding the following to your pom: -``` - - ... - - phoenix-github - Phoenix Github Maven - https://raw.github.com/forcedotcom/phoenix/maven-artifacts/releases - - true - - - true - - - ... - - - - ... - - com.salesforce - phoenix - 2.2.1 - - ... - -``` -## Samples ## -The best place to see samples are in our unit tests under src/test/java. The ones in the endToEnd package are tests demonstrating how to use all aspects of the Phoenix JDBC driver. We also have some examples in the examples directory. - -##Mailing List## -Join one or both of our Google groups: - -* [Phoenix HBase User](https://groups.google.com/forum/#!forum/phoenix-hbase-user) for users of Phoenix. -* [Phoenix HBase Dev](https://groups.google.com/forum/#!forum/phoenix-hbase-dev) for developers of Phoenix. - -and follow the Phoenix blog [here](http://phoenix-hbase.blogspot.com/). - -[![githalytics.com alpha](https://cruel-carlota.pagodabox.com/8438f3b844208e09a811699265666a8d "githalytics.com")](http://githalytics.com/forcedotcom/phoenix.git) - diff --git a/bin/csv-bulk-loader.sh b/bin/csv-bulk-loader.sh index d19f6104..ded249e8 100755 --- a/bin/csv-bulk-loader.sh +++ b/bin/csv-bulk-loader.sh @@ -1,30 +1,24 @@ #!/bin/bash ############################################################################ -# Copyright (c) 2013, Salesforce.com, Inc. -# All rights reserved. # -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are met: +# Copyright 2010 The Apache Software Foundation # -# Redistributions of source code must retain the above copyright notice, -# this list of conditions and the following disclaimer. -# Redistributions in binary form must reproduce the above copyright notice, -# this list of conditions and the following disclaimer in the documentation -# and/or other materials provided with the distribution. -# Neither the name of Salesforce.com nor the names of its contributors may -# be used to endorse or promote products derived from this software without -# specific prior written permission. +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. # -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ############################################################################ # Phoenix client jar. To generate new jars: $ mvn package -DskipTests @@ -43,4 +37,4 @@ # -help Print all options (optional) phoenix_client_jar=$(find ../phoenix-assembly/target/phoenix-*-client.jar) -java -cp "$phoenix_client_jar" com.salesforce.phoenix.map.reduce.CSVBulkLoader "$@" +java -cp "$phoenix_client_jar" org.apache.phoenix.map.reduce.CSVBulkLoader "$@" diff --git a/bin/performance.sh b/bin/performance.sh index 05708656..43ebaaba 100755 --- a/bin/performance.sh +++ b/bin/performance.sh @@ -1,30 +1,24 @@ #!/bin/bash ############################################################################ -# Copyright (c) 2013, Salesforce.com, Inc. -# All rights reserved. # -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are met: +# Copyright 2010 The Apache Software Foundation # -# Redistributions of source code must retain the above copyright notice, -# this list of conditions and the following disclaimer. -# Redistributions in binary form must reproduce the above copyright notice, -# this list of conditions and the following disclaimer in the documentation -# and/or other materials provided with the distribution. -# Neither the name of Salesforce.com nor the names of its contributors may -# be used to endorse or promote products derived from this software without -# specific prior written permission. +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. # -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ############################################################################ # Note: This script is tested on Linux environment only. It should work on any Unix platform but is not tested. @@ -45,12 +39,12 @@ statements="" current_dir=$(cd $(dirname $0);pwd) phoenix_jar_path="$current_dir/../phoenix-assembly/target" phoenix_client_jar=$(find $phoenix_jar_path/phoenix-*-client.jar) -testjar="$phoenix_jar_path/phoenix-*-tests.jar" +testjar="$current_dir/../phoenix-core/target/phoenix-*-tests.jar" # HBase configuration folder path (where hbase-site.xml reside) for HBase/Phoenix client side property override hbase_config_path="$current_dir" -execute="java -cp "$hbase_config_path:$phoenix_client_jar" -Dlog4j.configuration=file:$current_dir/log4j.properties com.salesforce.phoenix.util.PhoenixRuntime -t $table $zookeeper " +execute="java -cp "$hbase_config_path:$phoenix_client_jar" -Dlog4j.configuration=file:$current_dir/log4j.properties org.apache.phoenix.util.PhoenixRuntime -t $table $zookeeper " function usage { echo "Performance script arguments not specified. Usage: performance.sh " echo "Example: performance.sh localhost 100000" diff --git a/bin/psql.sh b/bin/psql.sh index 24ac1937..c17f7c33 100755 --- a/bin/psql.sh +++ b/bin/psql.sh @@ -1,30 +1,24 @@ #!/bin/bash ############################################################################ -# Copyright (c) 2013, Salesforce.com, Inc. -# All rights reserved. # -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are met: +# Copyright 2010 The Apache Software Foundation # -# Redistributions of source code must retain the above copyright notice, -# this list of conditions and the following disclaimer. -# Redistributions in binary form must reproduce the above copyright notice, -# this list of conditions and the following disclaimer in the documentation -# and/or other materials provided with the distribution. -# Neither the name of Salesforce.com nor the names of its contributors may -# be used to endorse or promote products derived from this software without -# specific prior written permission. +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. # -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ############################################################################ # Phoenix client jar. To generate new jars: $ mvn package -DskipTests @@ -35,4 +29,4 @@ phoenix_client_jar=$(find $phoenix_jar_path/phoenix-*-client.jar) # HBase configuration folder path (where hbase-site.xml reside) for HBase/Phoenix client side property override hbase_config_path="$current_dir" -java -cp "$hbase_config_path:$phoenix_client_jar" -Dlog4j.configuration=file:$current_dir/log4j.properties com.salesforce.phoenix.util.PhoenixRuntime "$@" +java -cp "$hbase_config_path:$phoenix_client_jar" -Dlog4j.configuration=file:$current_dir/log4j.properties org.apache.phoenix.util.PhoenixRuntime "$@" diff --git a/bin/sqlline.sh b/bin/sqlline.sh index 77eba84e..066f3842 100755 --- a/bin/sqlline.sh +++ b/bin/sqlline.sh @@ -1,30 +1,24 @@ #!/bin/bash ############################################################################ -# Copyright (c) 2013, Salesforce.com, Inc. -# All rights reserved. # -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are met: +# Copyright 2010 The Apache Software Foundation # -# Redistributions of source code must retain the above copyright notice, -# this list of conditions and the following disclaimer. -# Redistributions in binary form must reproduce the above copyright notice, -# this list of conditions and the following disclaimer in the documentation -# and/or other materials provided with the distribution. -# Neither the name of Salesforce.com nor the names of its contributors may -# be used to endorse or promote products derived from this software without -# specific prior written permission. +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. # -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ############################################################################ # Phoenix client jar. To generate new jars: $ mvn package -DskipTests @@ -42,4 +36,4 @@ if [ "$2" ] then sqlfile="--run=$2"; fi -java -cp ".:$phoenix_client_jar" -Dlog4j.configuration=file:$current_dir/log4j.properties sqlline.SqlLine -d com.salesforce.phoenix.jdbc.PhoenixDriver -u jdbc:phoenix:$1 -n none -p none --color=true --fastConnect=false --verbose=true --isolation=TRANSACTION_READ_COMMITTED $sqlfile +java -cp ".:$phoenix_client_jar" -Dlog4j.configuration=file:$current_dir/log4j.properties sqlline.SqlLine -d org.apache.phoenix.jdbc.PhoenixDriver -u jdbc:phoenix:$1 -n none -p none --color=true --fastConnect=false --verbose=true --isolation=TRANSACTION_READ_COMMITTED $sqlfile diff --git a/bin/upgradeTo2.sh b/bin/upgradeTo2.sh index 0e3f9aaa..c2a4a0dd 100755 --- a/bin/upgradeTo2.sh +++ b/bin/upgradeTo2.sh @@ -1,30 +1,24 @@ #!/bin/bash ############################################################################ -# Copyright (c) 2013, Salesforce.com, Inc. -# All rights reserved. # -# Redistribution and use in source and binary forms, with or without -# modification, are permitted provided that the following conditions are met: +# Copyright 2010 The Apache Software Foundation # -# Redistributions of source code must retain the above copyright notice, -# this list of conditions and the following disclaimer. -# Redistributions in binary form must reproduce the above copyright notice, -# this list of conditions and the following disclaimer in the documentation -# and/or other materials provided with the distribution. -# Neither the name of Salesforce.com nor the names of its contributors may -# be used to endorse or promote products derived from this software without -# specific prior written permission. +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. # -# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ############################################################################ # Phoenix client jar. To generate new jars: $ mvn package -DskipTests @@ -35,4 +29,4 @@ phoenix_client_jar=$(find $phoenix_jar_path/phoenix-*-client.jar) # HBase configuration folder path (where hbase-site.xml reside) for HBase/Phoenix client side property override hbase_config_path="$current_dir" -java -cp "$hbase_config_path:$phoenix_client_jar" -Dlog4j.configuration=file:$current_dir/log4j.properties com.salesforce.phoenix.util.PhoenixRuntime -u "$@" +java -cp "$hbase_config_path:$phoenix_client_jar" -Dlog4j.configuration=file:$current_dir/log4j.properties org.apache.phoenix.util.PhoenixRuntime -u "$@" diff --git a/build.txt b/build.txt index 6dc841f4..ca3304dc 100644 --- a/build.txt +++ b/build.txt @@ -1,5 +1,28 @@ -# Building Phoenix -================ +############################################################################ +# +# Copyright 2010 The Apache Software Foundation +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +############################################################################ + + +# Building Apache Phoenix +========================= Phoenix uses Maven (3.X) to build all its necessary resources. @@ -36,3 +59,9 @@ Use the m2e eclipse plugin and do Import->Maven Project and just pick the root ' Findbugs report is generated in /target/site $ mvn site + +## Generate Apache Web Site +=========================== + $ mvn -pl phoenix-core site -Ddependency.locations.enabled=false + +Note: site is generated in phoenix-core/target/site diff --git a/config/apache-access-logs.properties b/config/apache-access-logs.properties index 78077cdf..2733cba6 100644 --- a/config/apache-access-logs.properties +++ b/config/apache-access-logs.properties @@ -21,7 +21,7 @@ agent.sources.spooling-source.interceptors=i1 agent.sources.spooling-source.interceptors.i1.type=host agent.sources.spooling-source.interceptors.i1.hostHeader=f_host # configurationforsink -agent.sinks.phoenix-sink.type=com.salesforce.phoenix.flume.sink.PhoenixSink +agent.sinks.phoenix-sink.type=org.apache.phoenix.flume.sink.PhoenixSink agent.sinks.phoenix-sink.channel=memoryChannel agent.sinks.phoenix-sink.batchSize=100 agent.sinks.phoenix-sink.table=APACHE_LOGS diff --git a/dev/eclipse_prefs_phoenix.epf b/dev/eclipse_prefs_phoenix.epf index b6e6f15c..fb8df40f 100644 --- a/dev/eclipse_prefs_phoenix.epf +++ b/dev/eclipse_prefs_phoenix.epf @@ -800,7 +800,7 @@ file_export_version=3.0 /instance/org.eclipse.jdt.ui/org.eclipse.jdt.ui.packages.linktoeditor=true /instance/org.eclipse.jdt.ui/org.eclipse.jdt.ui.staticondemandthreshold=99 /instance/org.eclipse.jdt.ui/org.eclipse.jdt.ui.text.code_templates_migrated=true -/instance/org.eclipse.jdt.ui/org.eclipse.jdt.ui.text.custom_code_templates= +/instance/org.eclipse.jdt.ui/org.eclipse.jdt.ui.text.custom_code_templates= /instance/org.eclipse.jdt.ui/org.eclipse.jdt.ui.text.custom_templates= /instance/org.eclipse.jdt.ui/org.eclipse.jdt.ui.text.templates_migrated=true /instance/org.eclipse.jdt.ui/proposalOrderMigrated=true diff --git a/examples/pig/test.pig b/examples/pig/test.pig index 670c3e3c..aba5b1ba 100644 --- a/examples/pig/test.pig +++ b/examples/pig/test.pig @@ -1,2 +1,2 @@ A = load 'examples/pig/testdata' as (a:chararray, b:chararray, c:int, d:chararray, e: datetime) ; -STORE A into 'hbase://TESTPHX' using com.salesforce.phoenix.pig.PhoenixHBaseStorage('localhost','-batchSize 1000'); +STORE A into 'hbase://TESTPHX' using org.apache.phoenix.pig.PhoenixHBaseStorage('localhost','-batchSize 1000'); diff --git a/license.txt b/license.txt deleted file mode 100644 index e21b6dc3..00000000 --- a/license.txt +++ /dev/null @@ -1,26 +0,0 @@ -Copyright (c) 2013, Salesforce.com, Inc. -All rights reserved. - -Redistribution and use in source and binary forms, with or without -modification, are permitted provided that the following conditions are met: - - Redistributions of source code must retain the above copyright notice, - this list of conditions and the following disclaimer. - Redistributions in binary form must reproduce the above copyright notice, - this list of conditions and the following disclaimer in the documentation - and/or other materials provided with the distribution. - Neither the name of Salesforce.com nor the names of its contributors may - be used to endorse or promote products derived from this software without - specific prior written permission. - -THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - diff --git a/phoenix-assembly/pom.xml b/phoenix-assembly/pom.xml index e6c32574..9a89206a 100644 --- a/phoenix-assembly/pom.xml +++ b/phoenix-assembly/pom.xml @@ -2,7 +2,7 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 4.0.0 - com.salesforce + org.apache phoenix 3.0.0-SNAPSHOT @@ -29,7 +29,7 @@ true true - com.salesforce.phoenix.util.PhoenixRuntime + org.apache.phoenix.util.PhoenixRuntime true true @@ -92,24 +92,24 @@ - com.salesforce + org.apache phoenix-core - com.salesforce + org.apache phoenix-hadoop-compat - com.salesforce + org.apache ${compat.module} - com.salesforce + org.apache phoenix-flume - com.salesforce + org.apache phoenix-pig - \ No newline at end of file + diff --git a/phoenix-core/pom.xml b/phoenix-core/pom.xml index 0be278c3..0e2b8e59 100644 --- a/phoenix-core/pom.xml +++ b/phoenix-core/pom.xml @@ -2,7 +2,7 @@ xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 4.0.0 - com.salesforce + org.apache phoenix 3.0.0-SNAPSHOT @@ -10,6 +10,20 @@ Phoenix Core Core Phoenix codebase + + + The Apache Software License, Version 2.0 + http://www.apache.org/licenses/LICENSE-2.0.txt + repo + + + + + + Apache Software Foundation + http://www.apache.org + + @@ -23,6 +37,54 @@ + + org.apache.maven.plugins + maven-site-plugin + 3.2 + + + org.apache.maven.doxia + doxia-module-markdown + 1.3 + + + lt.velykis.maven.skins + reflow-velocity-tools + 1.0.0 + + + org.apache.velocity + velocity + 1.7 + + + + + + org.codehaus.mojo + findbugs-maven-plugin + 2.5.2 + + + + + + exec-maven-plugin + org.codehaus.mojo + 1.2.1 + + + Merge Language Reference + site + + exec + + + ${basedir}/src/site/bin/merge.sh + + + + org.codehaus.mojo build-helper-maven-plugin @@ -56,7 +118,7 @@ - ${antlr-output.dir}/com/salesforce/phoenix/parse + ${antlr-output.dir}/org/apache/phoenix/parse - com.salesforce + org.apache phoenix-hadoop-compat - com.salesforce + org.apache phoenix-hadoop-compat tests test @@ -254,4 +316,4 @@ - \ No newline at end of file + diff --git a/phoenix-core/src/main/antlr3/PhoenixSQL.g b/phoenix-core/src/main/antlr3/PhoenixSQL.g index a22fc783..3311e5ad 100644 --- a/phoenix-core/src/main/antlr3/PhoenixSQL.g +++ b/phoenix-core/src/main/antlr3/PhoenixSQL.g @@ -1,30 +1,22 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ +/** + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ grammar PhoenixSQL; tokens @@ -111,34 +103,26 @@ tokens @parser::header { -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; +/** + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; ///CLOVER:OFF import com.google.common.collect.ImmutableMap; @@ -150,45 +134,37 @@ import java.util.Arrays; import java.util.Collections; import java.util.Stack; import java.sql.SQLException; -import com.salesforce.phoenix.expression.function.CountAggregateFunction; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.IllegalDataException; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PIndexState; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.phoenix.expression.function.CountAggregateFunction; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.IllegalDataException; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PIndexState; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.util.SchemaUtil; } @lexer::header { -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; +/** + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; ///CLOVER:OFF } @@ -545,11 +521,19 @@ dyn_column_def returns [ColumnDef ret] dyn_column_name_or_def returns [ColumnDef ret] : c=column_name (dt=identifier (LPAREN l=NUMBER (COMMA s=NUMBER)? RPAREN)? )? (lsq=LSQUARE (a=NUMBER)? RSQUARE)? - {$ret = factory.columnDef(c, dt, true, - l == null ? null : Integer.parseInt( l.getText() ), - s == null ? null : Integer.parseInt( s.getText() ), - false, - null); } + { + if(lsq != null) + { + throwRecognitionException(lsq); + } else + { + $ret = factory.columnDef(c, dt, true, + l == null ? null : Integer.parseInt( l.getText() ), + s == null ? null : Integer.parseInt( s.getText() ), + false, + null); + } + } ; select_expression returns [SelectStatement ret] diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/CapturingAbortable.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/CapturingAbortable.java deleted file mode 100644 index 5493d7fa..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/CapturingAbortable.java +++ /dev/null @@ -1,76 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index; - -import org.apache.hadoop.hbase.Abortable; - -/** - * {@link Abortable} that can rethrow the cause of the abort. - */ -public class CapturingAbortable implements Abortable { - - private Abortable delegate; - private Throwable cause; - private String why; - - public CapturingAbortable(Abortable delegate) { - this.delegate = delegate; - } - - @Override - public void abort(String why, Throwable e) { - if (delegate.isAborted()) { - return; - } - this.why = why; - this.cause = e; - delegate.abort(why, e); - - } - - @Override - public boolean isAborted() { - return delegate.isAborted(); - } - - /** - * Throw the cause of the abort, if this was aborted. If there was an exception causing - * the abort, re-throws that. Otherwise, just throws a generic {@link Exception} with the reason - * why the abort was caused. - * @throws Throwable the cause of the abort. - */ - public void throwCauseIfAborted() throws Throwable { - if (!this.isAborted()) { - return; - } - if (cause == null) { - throw new Exception(why); - } - throw cause; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/IndexLogRollSynchronizer.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/IndexLogRollSynchronizer.java deleted file mode 100644 index ae27cfa6..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/IndexLogRollSynchronizer.java +++ /dev/null @@ -1,133 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index; - -import java.io.IOException; -import java.util.concurrent.locks.ReentrantReadWriteLock.WriteLock; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.fs.Path; -import org.apache.hadoop.hbase.HRegionInfo; -import org.apache.hadoop.hbase.HTableDescriptor; -import org.apache.hadoop.hbase.regionserver.wal.HLogKey; -import org.apache.hadoop.hbase.regionserver.wal.WALActionsListener; -import org.apache.hadoop.hbase.regionserver.wal.WALEdit; - -/** - * Ensure that the log isn't rolled while we are the in middle of doing a pending index write. - *

- * The problem we are trying to solve is the following sequence: - *

    - *
  1. Write to the indexed table
    2. - *
  2. Write the index-containing WALEdit
    3. - *
  3. Start writing to the index tables in the postXXX hook
    4. - *
  4. WAL gets rolled and archived
    5. - *
  5. An index update fails, in which case we should kill ourselves to get WAL replay
    6. - *
  6. Since the WAL got archived, we won't get the replay of the index writes
    7. - *
- *

- * The usual course of events should be: - *

    - *
  1. In a preXXX hook, - *
      - *
    1. Build the {@link WALEdit} + index information
      2. - *
    2. Lock the {@link IndexLogRollSynchronizer#logArchiveLock}
      3. - *
        - *
      • This is a reentrant readlock on the WAL archiving, so we can make multiple WAL/index updates - * concurrently
        • - *
      - * - *
    - *
    2. - *
  2. Pass that {@link WALEdit} to the WAL, ensuring its durable and replayable
    3. - *
  3. In the corresponding postXXX, - *
      - *
    1. make the updates to the index tables
      2. - *
    2. Unlock {@link IndexLogRollSynchronizer#logArchiveLock}
      3. - *
    - *
- *

- * this should be added as a {@link WALActionsListener} by updating - */ -public class IndexLogRollSynchronizer implements WALActionsListener { - - private static final Log LOG = LogFactory.getLog(IndexLogRollSynchronizer.class); - private WriteLock logArchiveLock; - - public IndexLogRollSynchronizer(WriteLock logWriteLock){ - this.logArchiveLock = logWriteLock; - } - - - @Override - public void preLogArchive(Path oldPath, Path newPath) throws IOException { - //take a write lock on the index - any pending index updates will complete before we finish - LOG.debug("Taking INDEX_UPDATE writelock"); - logArchiveLock.lock(); - LOG.debug("Got the INDEX_UPDATE writelock"); - } - - @Override - public void postLogArchive(Path oldPath, Path newPath) throws IOException { - // done archiving the logs, any WAL updates will be replayed on failure - LOG.debug("Releasing INDEX_UPDATE writelock"); - logArchiveLock.unlock(); - } - - @Override - public void logCloseRequested() { - // don't care- before this is called, all the HRegions are closed, so we can't get any new - // requests and all pending request can finish before the WAL closes. - } - - @Override - public void preLogRoll(Path oldPath, Path newPath) throws IOException { - // noop - } - - @Override - public void postLogRoll(Path oldPath, Path newPath) throws IOException { - // noop - } - - @Override - public void logRollRequested() { - // noop - } - - @Override - public void visitLogEntryBeforeWrite(HRegionInfo info, HLogKey logKey, WALEdit logEdit) { - // noop - } - - @Override - public void visitLogEntryBeforeWrite(HTableDescriptor htd, HLogKey logKey, WALEdit logEdit) { - // noop - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/ValueGetter.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/ValueGetter.java deleted file mode 100644 index 0cf05886..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/ValueGetter.java +++ /dev/null @@ -1,46 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index; - -import java.io.IOException; - -import com.salesforce.hbase.index.covered.update.ColumnReference; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; - -public interface ValueGetter { - - /** - * Get the most recent (largest timestamp) for the given column reference - * @param ref to match against an underlying key value. Uses the passed object to match the - * keyValue via {@link ColumnReference#matches} - * @return the stored value for the given {@link ColumnReference}, or null if no value is - * present. - * @throws IOException if there is an error accessing the underlying data storage - */ - public ImmutableBytesPtr getLatestValue(ColumnReference ref) throws IOException; -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/builder/BaseIndexBuilder.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/builder/BaseIndexBuilder.java deleted file mode 100644 index 968ae780..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/builder/BaseIndexBuilder.java +++ /dev/null @@ -1,107 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.builder; - -import java.io.IOException; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; -import org.apache.hadoop.hbase.regionserver.MiniBatchOperationInProgress; -import org.apache.hadoop.hbase.util.Pair; - -import com.salesforce.hbase.index.covered.CoveredColumnsIndexBuilder; - -/** - * Basic implementation of the {@link IndexBuilder} that doesn't do any actual work of indexing. - *

- * You should extend this class, rather than implementing IndexBuilder directly to maintain - * compatability going forward. - *

- * Generally, you should consider using one of the implemented IndexBuilders (e.g - * {@link CoveredColumnsIndexBuilder}) as there is a lot of work required to keep an index table - * up-to-date. - */ -public abstract class BaseIndexBuilder implements IndexBuilder { - - private static final Log LOG = LogFactory.getLog(BaseIndexBuilder.class); - protected boolean stopped; - - @Override - public void extendBaseIndexBuilderInstead() { } - - @Override - public void setup(RegionCoprocessorEnvironment conf) throws IOException { - // noop - } - - @Override - public void batchStarted(MiniBatchOperationInProgress> miniBatchOp) throws IOException { - // noop - } - - @Override - public void batchCompleted(MiniBatchOperationInProgress> miniBatchOp) { - // noop - } - - /** - * By default, we always attempt to index the mutation. Commonly this can be slow (because the - * framework spends the time to do the indexing, only to realize that you don't need it) or not - * ideal (if you want to turn on/off indexing on a table without completely reloading it). - * @throws IOException - */ - @Override - public boolean isEnabled(Mutation m) throws IOException { - return true; - } - - /** - * {@inheritDoc} - *

- * By default, assumes that all mutations should not be batched. That is to say, each - * mutation always applies to different rows, even if they are in the same batch, or are - * independent updates. - */ - @Override - public byte[] getBatchId(Mutation m) { - return null; - } - - @Override - public void stop(String why) { - LOG.debug("Stopping because: " + why); - this.stopped = true; - } - - @Override - public boolean isStopped() { - return this.stopped; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/builder/IndexBuildingFailureException.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/builder/IndexBuildingFailureException.java deleted file mode 100644 index 92f58682..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/builder/IndexBuildingFailureException.java +++ /dev/null @@ -1,58 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.builder; - -import java.io.IOException; - -import org.apache.hadoop.hbase.DoNotRetryIOException; - -/** - * Unexpected failure while building index updates that wasn't caused by an {@link IOException}. - * This should be used if there is some basic issue with indexing - and no matter of retries will - * fix it. - */ -@SuppressWarnings("serial") -public class IndexBuildingFailureException extends DoNotRetryIOException { - - /** - * Constructor for over the wire propagation. Generally, shouldn't be used since index failure - * should have an underlying cause to propagate. - * @param msg reason for the failure - */ - public IndexBuildingFailureException(String msg) { - super(msg); - } - - /** - * @param msg reason - * @param cause underlying cause for the failure - */ - public IndexBuildingFailureException(String msg, Throwable cause) { - super(msg, cause); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/Batch.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/Batch.java deleted file mode 100644 index 14647e43..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/Batch.java +++ /dev/null @@ -1,70 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered; - -import java.util.ArrayList; -import java.util.List; - -import org.apache.hadoop.hbase.KeyValue; - -/** - * A collection of {@link KeyValue KeyValues} to the primary table - */ -public class Batch { - - private static final long pointDeleteCode = KeyValue.Type.Delete.getCode(); - private final long timestamp; - private List batch = new ArrayList(); - private boolean allPointDeletes = true; - - /** - * @param ts - */ - public Batch(long ts) { - this.timestamp = ts; - } - - public void add(KeyValue kv){ - if (pointDeleteCode != kv.getType()) { - allPointDeletes = false; - } - batch.add(kv); - } - - public boolean isAllPointDeletes() { - return allPointDeletes; - } - - public long getTimestamp() { - return this.timestamp; - } - - public List getKvs() { - return this.batch; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/CoveredColumns.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/CoveredColumns.java deleted file mode 100644 index 439c4aab..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/CoveredColumns.java +++ /dev/null @@ -1,59 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered; - -import java.util.ArrayList; -import java.util.Collection; -import java.util.HashSet; -import java.util.List; -import java.util.Set; - -import com.salesforce.hbase.index.covered.update.ColumnReference; - -/** - * Manage a set of {@link ColumnReference}s for the {@link LocalTableState}. - */ -public class CoveredColumns { - - Set columns = new HashSet(); - - public Collection findNonCoveredColumns( - Collection columns2) { - List uncovered = new ArrayList(); - for (ColumnReference column : columns2) { - if (!columns.contains(column)) { - uncovered.add(column); - } - } - return uncovered; - } - - public void addColumn(ColumnReference column) { - this.columns.add(column); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/IndexCodec.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/IndexCodec.java deleted file mode 100644 index 5de43e4d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/IndexCodec.java +++ /dev/null @@ -1,120 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered; - -import java.io.IOException; - -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.client.Put; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; - -import com.salesforce.phoenix.index.BaseIndexCodec; - - -/** - * Codec for creating index updates from the current state of a table. - *

- * Generally, you should extend {@link BaseIndexCodec} instead, so help maintain compatibility as - * features need to be added to the codec, as well as potentially not haivng to implement some - * methods. - */ -public interface IndexCodec { - - /** - * Do any code initialization necessary - * @param env environment in which the codec is operating - * @throws IOException if the codec cannot be initalized correctly - */ - public void initialize(RegionCoprocessorEnvironment env) throws IOException; - - /** - * Get the index cleanup entries. Currently, this must return just single row deletes (where just - * the row-key is specified and no columns are returned) mapped to the table name. For instance, - * to you have an index 'myIndex' with row : - * - * 

-   * v1,v2,v3 | CF:CQ0  | rowkey
-   *          | CF:CQ1  | rowkey
-   *
- * - * To then cleanup this entry, you would just return 'v1,v2,v3', 'myIndex'. - * @param state the current state of the table that needs to be cleaned up. Generally, you only - * care about the latest column values, for each column you are indexing for each index - * table. - * @return the pairs of (deletes, index table name) that should be applied. - * @throws IOException - */ - public Iterable getIndexDeletes(TableState state) throws IOException; - - // table state has the pending update already applied, before calling - // get the new index entries - /** - * Get the index updates for the primary table state, for each index table. The returned - * {@link Put}s need to be fully specified (including timestamp) to minimize passes over the same - * key-values multiple times. - *

- * You must specify the same timestamps on the Put as {@link TableState#getCurrentTimestamp()} so - * the index entries match the primary table row. This could be managed at a higher level, but - * would require iterating all the kvs in the Put again - very inefficient when compared to the - * current interface where you must provide a timestamp anyways (so you might as well provide the - * right one). - * @param state the current state of the table that needs to an index update Generally, you only - * care about the latest column values, for each column you are indexing for each index - * table. - * @return the pairs of (updates,index table name) that should be applied. - * @throws IOException - */ - public Iterable getIndexUpserts(TableState state) throws IOException; - - /** - * This allows the codec to dynamically change whether or not indexing should take place for a - * table. If it doesn't take place, we can save a lot of time on the regular Put patch. By making - * it dynamic, we can save offlining and then onlining a table just to turn indexing on. - *

- * We can also be smart about even indexing a given update here too - if the update doesn't - * contain any columns that we care about indexing, we can save the effort of analyzing the put - * and further. - * @param m mutation that should be indexed. - * @return true if indexing is enabled for the given table. This should be on a per-table - * basis, as each codec is instantiated per-region. - * @throws IOException - */ - public boolean isEnabled(Mutation m) throws IOException; - - /** - * Get the batch identifier of the given mutation. Generally, updates to the table will take place - * in a batch of updates; if we know that the mutation is part of a batch, we can build the state - * much more intelligently. - *

- * If you have batches that have multiple updates to the same row state, you must specify a - * batch id for each batch. Otherwise, we cannot guarantee index correctness - * @param m mutation that may or may not be part of the batch - * @return null if the mutation is not part of a batch or an id for the batch. - */ - public byte[] getBatchId(Mutation m); -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/IndexUpdate.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/IndexUpdate.java deleted file mode 100644 index c50c0932..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/IndexUpdate.java +++ /dev/null @@ -1,87 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered; - -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.client.Put; -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.hbase.index.covered.update.ColumnTracker; - -/** - * Update to make to the index table. - */ -public class IndexUpdate { - Mutation update; - byte[] tableName; - ColumnTracker columns; - - IndexUpdate(ColumnTracker tracker) { - this.columns = tracker; - } - - public void setUpdate(Mutation p) { - this.update = p; - } - - public void setTable(byte[] tableName) { - this.tableName = tableName; - } - - public Mutation getUpdate() { - return update; - } - - public byte[] getTableName() { - return tableName; - } - - public ColumnTracker getIndexedColumns() { - return columns; - } - - @Override - public String toString() { - return "IndexUpdate: \n\ttable - " + Bytes.toString(tableName) + "\n\tupdate: " + update - + "\n\tcolumns: " + columns; - } - - public static IndexUpdate createIndexUpdateForTesting(ColumnTracker tracker, byte[] table, Put p) { - IndexUpdate update = new IndexUpdate(tracker); - update.setTable(table); - update.setUpdate(p); - return update; - } - - /** - * @return true if the necessary state for a valid index update has been set. - */ - public boolean isValid() { - return this.tableName != null && this.update != null; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/KeyValueStore.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/KeyValueStore.java deleted file mode 100644 index 88a99cce..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/KeyValueStore.java +++ /dev/null @@ -1,43 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.regionserver.KeyValueScanner; - -/** - * Store a collection of KeyValues in memory. - */ -public interface KeyValueStore { - - public void add(KeyValue kv, boolean overwrite); - - public KeyValueScanner getScanner(); - - public void rollback(KeyValue kv); -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/LocalTableState.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/LocalTableState.java deleted file mode 100644 index 9c32ed47..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/LocalTableState.java +++ /dev/null @@ -1,252 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered; - -import java.io.IOException; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.Collection; -import java.util.HashSet; -import java.util.List; -import java.util.Map; -import java.util.Set; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.client.Result; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; -import org.apache.hadoop.hbase.regionserver.KeyValueScanner; -import org.apache.hadoop.hbase.util.Pair; - -import com.salesforce.hbase.index.covered.data.IndexMemStore; -import com.salesforce.hbase.index.covered.data.LocalHBaseState; -import com.salesforce.hbase.index.covered.update.ColumnReference; -import com.salesforce.hbase.index.covered.update.ColumnTracker; -import com.salesforce.hbase.index.covered.update.IndexedColumnGroup; -import com.salesforce.hbase.index.scanner.Scanner; -import com.salesforce.hbase.index.scanner.ScannerBuilder; - -/** - * Manage the state of the HRegion's view of the table, for the single row. - *

- * Currently, this is a single-use object - you need to create a new one for each row that you need - * to manage. In the future, we could make this object reusable, but for the moment its easier to - * manage as a throw-away object. - *

- * This class is not thread-safe - it requires external synchronization is access - * concurrently. - */ -public class LocalTableState implements TableState { - - private long ts; - private RegionCoprocessorEnvironment env; - private KeyValueStore memstore; - private LocalHBaseState table; - private Mutation update; - private Set trackedColumns = new HashSet(); - private ScannerBuilder scannerBuilder; - private List kvs = new ArrayList(); - private List hints; - private CoveredColumns columnSet; - - public LocalTableState(RegionCoprocessorEnvironment environment, LocalHBaseState table, Mutation update) { - this.env = environment; - this.table = table; - this.update = update; - this.memstore = new IndexMemStore(); - this.scannerBuilder = new ScannerBuilder(memstore, update); - this.columnSet = new CoveredColumns(); - } - - public void addPendingUpdates(KeyValue... kvs) { - if (kvs == null) return; - addPendingUpdates(Arrays.asList(kvs)); - } - - public void addPendingUpdates(List kvs) { - if(kvs == null) return; - setPendingUpdates(kvs); - addUpdate(kvs); - } - - private void addUpdate(List list) { - addUpdate(list, true); - } - - private void addUpdate(List list, boolean overwrite) { - if (list == null) return; - for (KeyValue kv : list) { - this.memstore.add(kv, overwrite); - } - } - - @Override - public RegionCoprocessorEnvironment getEnvironment() { - return this.env; - } - - @Override - public long getCurrentTimestamp() { - return this.ts; - } - - @Override - public void setCurrentTimestamp(long timestamp) { - this.ts = timestamp; - } - - public void resetTrackedColumns() { - this.trackedColumns.clear(); - } - - public Set getTrackedColumns() { - return this.trackedColumns; - } - - @Override - public Pair getIndexedColumnsTableState( - Collection indexedColumns) throws IOException { - ensureLocalStateInitialized(indexedColumns); - // filter out things with a newer timestamp and track the column references to which it applies - ColumnTracker tracker = new ColumnTracker(indexedColumns); - synchronized (this.trackedColumns) { - // we haven't seen this set of columns before, so we need to create a new tracker - if (!this.trackedColumns.contains(tracker)) { - this.trackedColumns.add(tracker); - } - } - - Scanner scanner = - this.scannerBuilder.buildIndexedColumnScanner(indexedColumns, tracker, ts); - - return new Pair(scanner, new IndexUpdate(tracker)); - } - - /** - * Initialize the managed local state. Generally, this will only be called by - * {@link #getNonIndexedColumnsTableState(List)}, which is unlikely to be called concurrently from the outside. - * Even then, there is still fairly low contention as each new Put/Delete will have its own table - * state. - */ - private synchronized void ensureLocalStateInitialized( - Collection columns) throws IOException { - // check to see if we haven't initialized any columns yet - Collection toCover = this.columnSet.findNonCoveredColumns(columns); - // we have all the columns loaded, so we are good to go. - if (toCover.isEmpty()) { - return; - } - - // add the current state of the row - this.addUpdate(this.table.getCurrentRowState(update, toCover).list(), false); - - // add the covered columns to the set - for (ColumnReference ref : toCover) { - this.columnSet.addColumn(ref); - } - } - - @Override - public Map getUpdateAttributes() { - return this.update.getAttributesMap(); - } - - @Override - public byte[] getCurrentRowKey() { - return this.update.getRow(); - } - - public Result getCurrentRowState() { - KeyValueScanner scanner = this.memstore.getScanner(); - List kvs = new ArrayList(); - while (scanner.peek() != null) { - try { - kvs.add(scanner.next()); - } catch (IOException e) { - // this should never happen - something has gone terribly arwy if it has - throw new RuntimeException("Local MemStore threw IOException!"); - } - } - return new Result(kvs); - } - - /** - * Helper to add a {@link Mutation} to the values stored for the current row - * @param pendingUpdate update to apply - */ - public void addUpdateForTesting(Mutation pendingUpdate) { - for (Map.Entry> e : pendingUpdate.getFamilyMap().entrySet()) { - List edits = e.getValue(); - addUpdate(edits); - } - } - - /** - * @param hints - */ - public void setHints(List hints) { - this.hints = hints; - } - - @Override - public List getIndexColumnHints() { - return this.hints; - } - - @Override - public Collection getPendingUpdate() { - return this.kvs; - } - - /** - * Set the {@link KeyValue}s in the update for which we are currently building an index update, - * but don't actually apply them. - * @param update pending {@link KeyValue}s - */ - public void setPendingUpdates(Collection update) { - this.kvs.clear(); - this.kvs.addAll(update); - } - - /** - * Apply the {@link KeyValue}s set in {@link #setPendingUpdates(Collection)}. - */ - public void applyPendingUpdates() { - this.addUpdate(kvs); - } - - /** - * Rollback all the given values from the underlying state. - * @param values - */ - public void rollback(Collection values) { - for (KeyValue kv : values) { - this.memstore.rollback(kv); - } - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/LazyValueGetter.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/LazyValueGetter.java deleted file mode 100644 index 2f380370..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/LazyValueGetter.java +++ /dev/null @@ -1,97 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.data; - -import java.io.IOException; -import java.util.Collections; -import java.util.HashMap; -import java.util.Map; - -import org.apache.hadoop.hbase.KeyValue; - -import com.salesforce.hbase.index.ValueGetter; -import com.salesforce.hbase.index.covered.update.ColumnReference; -import com.salesforce.hbase.index.scanner.Scanner; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; - -/** - * {@link ValueGetter} that uses lazy initialization to get the value for the given - * {@link ColumnReference}. Once stored, the mapping for that reference is retained. - */ -public class LazyValueGetter implements ValueGetter { - - private Scanner scan; - private volatile Map values; - private byte[] row; - - /** - * Back the getter with a {@link Scanner} to actually access the local data. - * @param scan backing scanner - * @param currentRow row key for the row to seek in the scanner - */ - public LazyValueGetter(Scanner scan, byte[] currentRow) { - this.scan = scan; - this.row = currentRow; - } - - @Override - public ImmutableBytesPtr getLatestValue(ColumnReference ref) throws IOException { - // ensure we have a backing map - if (values == null) { - synchronized (this) { - values = Collections.synchronizedMap(new HashMap()); - } - } - - // check the value in the map - ImmutableBytesPtr value = values.get(ref); - if (value == null) { - value = get(ref); - values.put(ref, value); - } - - return value; - } - - /** - * @param ref - * @return the first value on the scanner for the given column - */ - private ImmutableBytesPtr get(ColumnReference ref) throws IOException { - KeyValue first = ref.getFirstKeyValueForRow(row); - if (!scan.seek(first)) { - return null; - } - // there is a next value - we only care about the current value, so we can just snag that - KeyValue next = scan.next(); - if (ref.matches(next)) { - return new ImmutableBytesPtr(next.getBuffer(), next.getValueOffset(), next.getValueLength()); - } - return null; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/LocalHBaseState.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/LocalHBaseState.java deleted file mode 100644 index 4e4f8062..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/LocalHBaseState.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.data; - -import java.io.IOException; -import java.util.Collection; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.client.Result; - -import com.salesforce.hbase.index.covered.update.ColumnReference; - -/** - * Access the current state of the row in the local HBase table, given a mutation - */ -public interface LocalHBaseState { - - /** - * @param m mutation for which we should get the current table state - * @param toCover all the columns the current row state needs to cover; hint the underlying lookup - * to save getting all the columns for the row - * @return the full state of the given row. Includes all current versions (even if they are not - * usually visible to the client (unless they are also doing a raw scan)). Never returns a - * null {@link Result} - instead, when there is not data for the row, returns a - * {@link Result} with no stored {@link KeyValue}s. - * @throws IOException if there is an issue reading the row - */ - public Result getCurrentRowState(Mutation m, Collection toCover) - throws IOException; - -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/LocalTable.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/LocalTable.java deleted file mode 100644 index 26048b88..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/LocalTable.java +++ /dev/null @@ -1,82 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.data; - -import java.io.IOException; -import java.util.ArrayList; -import java.util.Collection; -import java.util.Collections; -import java.util.List; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.client.Result; -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; -import org.apache.hadoop.hbase.regionserver.HRegion; -import org.apache.hadoop.hbase.regionserver.RegionScanner; - -import com.salesforce.hbase.index.covered.update.ColumnReference; -import com.salesforce.hbase.index.util.IndexManagementUtil; - -/** - * Wrapper around a lazily instantiated, local HTable. - *

- * Previously, we had used various row and batch caches. However, this ends up being very - * complicated when attempting manage updating and invalidating the cache with no real gain as any - * row accessed multiple times will likely be in HBase's block cache, invalidating any extra caching - * we are doing here. In the end, its simpler and about as efficient to just get the current state - * of the row from HBase and let HBase manage caching the row from disk on its own. - */ -public class LocalTable implements LocalHBaseState { - - private RegionCoprocessorEnvironment env; - - public LocalTable(RegionCoprocessorEnvironment env) { - this.env = env; - } - - @Override - public Result getCurrentRowState(Mutation m, Collection columns) - throws IOException { - byte[] row = m.getRow(); - // need to use a scan here so we can get raw state, which Get doesn't provide. - Scan s = IndexManagementUtil.newLocalStateScan(Collections.singletonList(columns)); - s.setStartRow(row); - s.setStopRow(row); - HRegion region = this.env.getRegion(); - RegionScanner scanner = region.getScanner(s); - List kvs = new ArrayList(1); - boolean more = scanner.next(kvs); - assert !more : "Got more than one result when scanning" + " a single row in the primary table!"; - - Result r = new Result(kvs); - scanner.close(); - return r; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/ColumnTrackingNextLargestTimestampFilter.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/ColumnTrackingNextLargestTimestampFilter.java deleted file mode 100644 index 21abc63a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/ColumnTrackingNextLargestTimestampFilter.java +++ /dev/null @@ -1,82 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.filter; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.filter.FilterBase; - -import com.salesforce.hbase.index.covered.update.ColumnTracker; - -/** - * Similar to the {@link MaxTimestampFilter}, but also updates the 'next largest' timestamp seen - * that is not skipped by the below criteria. Note that it isn't as quick as the - * {@link MaxTimestampFilter} as we can't just seek ahead to a key with the matching timestamp, but - * have to iterate each kv until we find the right one with an allowed timestamp. - *

- * Inclusively filter on the maximum timestamp allowed. Excludes all elements greater than (but not - * equal to) the given timestamp, so given ts = 5, a {@link KeyValue} with ts 6 is excluded, but not - * one with ts = 5. - *

- * This filter generally doesn't make sense on its own - it should follow a per-column filter and - * possible a per-delete filter to only track the most recent (but not exposed to the user) - * timestamp. - */ -public class ColumnTrackingNextLargestTimestampFilter extends FilterBase { - - private long ts; - private ColumnTracker column; - - public ColumnTrackingNextLargestTimestampFilter(long maxTime, ColumnTracker toTrack) { - this.ts = maxTime; - this.column = toTrack; - } - - @Override - public ReturnCode filterKeyValue(KeyValue v) { - long timestamp = v.getTimestamp(); - if (timestamp > ts) { - this.column.setTs(timestamp); - return ReturnCode.SKIP; - } - return ReturnCode.INCLUDE; - } - - @Override - public void write(DataOutput out) throws IOException { - throw new UnsupportedOperationException("Server-side only filter, cannot be serialized!"); - } - - @Override - public void readFields(DataInput in) throws IOException { - throw new UnsupportedOperationException("Server-side only filter, cannot be deserialized!"); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/MaxTimestampFilter.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/MaxTimestampFilter.java deleted file mode 100644 index 89c74aeb..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/MaxTimestampFilter.java +++ /dev/null @@ -1,84 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.filter; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.filter.FilterBase; -import org.apache.hadoop.hbase.util.Bytes; - -/** - * Inclusive filter on the maximum timestamp allowed. Excludes all elements greater than (but not - * equal to) the given timestamp, so given ts = 5, a {@link KeyValue} with ts 6 is excluded, but not - * one with ts = 5. - */ -public class MaxTimestampFilter extends FilterBase { - - private long ts; - - public MaxTimestampFilter(long maxTime) { - this.ts = maxTime; - } - - @Override - public KeyValue getNextKeyHint(KeyValue currentKV) { - // this might be a little excessive right now - better safe than sorry though, so we don't mess - // with other filters too much. - KeyValue kv = currentKV.deepCopy(); - int offset =kv.getTimestampOffset(); - //set the timestamp in the buffer - byte[] buffer = kv.getBuffer(); - byte[] ts = Bytes.toBytes(this.ts); - System.arraycopy(ts, 0, buffer, offset, ts.length); - - return kv; - } - - @Override - public ReturnCode filterKeyValue(KeyValue v) { - long timestamp = v.getTimestamp(); - if (timestamp > ts) { - return ReturnCode.SEEK_NEXT_USING_HINT; - } - return ReturnCode.INCLUDE; - } - - @Override - public void write(DataOutput out) throws IOException { - throw new UnsupportedOperationException("Server-side only filter, cannot be serialized!"); - - } - - @Override - public void readFields(DataInput in) throws IOException { - throw new UnsupportedOperationException("Server-side only filter, cannot be deserialized!"); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/ColumnReference.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/ColumnReference.java deleted file mode 100644 index 3e609961..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/ColumnReference.java +++ /dev/null @@ -1,176 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.update; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; - -/** - * - */ -public class ColumnReference implements Comparable { - - public static final byte[] ALL_QUALIFIERS = new byte[0]; - - private static int calcHashCode(byte[] family, byte[] qualifier) { - final int prime = 31; - int result = 1; - result = prime * result + Bytes.hashCode(family); - result = prime * result + Bytes.hashCode(qualifier); - return result; - } - - private final int hashCode; - protected final byte[] family; - protected final byte[] qualifier; - private volatile ImmutableBytesWritable familyPtr; - private volatile ImmutableBytesWritable qualifierPtr; - - public ColumnReference(byte[] family, byte[] qualifier) { - this.family = family; - this.qualifier = qualifier; - this.hashCode = calcHashCode(family, qualifier); - } - - public byte[] getFamily() { - return this.family; - } - - public byte[] getQualifier() { - return this.qualifier; - } - - public ImmutableBytesWritable getFamilyWritable() { - if (this.familyPtr == null) { - synchronized (this.family) { - if (this.familyPtr == null) { - this.familyPtr = new ImmutableBytesPtr(this.family); - } - } - } - return this.familyPtr; - } - - public ImmutableBytesWritable getQualifierWritable() { - if (this.qualifierPtr == null) { - synchronized (this.qualifier) { - if (this.qualifierPtr == null) { - this.qualifierPtr = new ImmutableBytesPtr(this.qualifier); - } - } - } - return this.qualifierPtr; - } - - public boolean matches(KeyValue kv) { - if (matchesFamily(kv.getBuffer(), kv.getFamilyOffset(), kv.getFamilyLength())) { - return matchesQualifier(kv.getBuffer(), kv.getQualifierOffset(), kv.getQualifierLength()); - } - return false; - } - - /** - * @param qual to check against - * @return true if this column covers the given qualifier. - */ - public boolean matchesQualifier(byte[] qual) { - return matchesQualifier(qual, 0, qual.length); - } - - public boolean matchesQualifier(byte[] bytes, int offset, int length) { - return allColumns() ? true : match(bytes, offset, length, qualifier); - } - - /** - * @param family to check against - * @return true if this column covers the given family. - */ - public boolean matchesFamily(byte[] family) { - return matchesFamily(family, 0, family.length); - } - - public boolean matchesFamily(byte[] bytes, int offset, int length) { - return match(bytes, offset, length, family); - } - - /** - * @return true if this should include all column qualifiers, false otherwise - */ - public boolean allColumns() { - return this.qualifier == ALL_QUALIFIERS; - } - - /** - * Check to see if the passed bytes match the stored bytes - * @param first - * @param storedKey the stored byte[], should never be null - * @return true if they are byte-equal - */ - private boolean match(byte[] first, int offset, int length, byte[] storedKey) { - return first == null ? false : Bytes.equals(first, offset, length, storedKey, 0, - storedKey.length); - } - - public KeyValue getFirstKeyValueForRow(byte[] row) { - return KeyValue.createFirstOnRow(row, family, qualifier == ALL_QUALIFIERS ? null : qualifier); - } - - @Override - public int compareTo(ColumnReference o) { - int c = Bytes.compareTo(family, o.family); - if (c == 0) { - // matching families, compare qualifiers - c = Bytes.compareTo(qualifier, o.qualifier); - } - return c; - } - - @Override - public boolean equals(Object o) { - if (o instanceof ColumnReference) { - ColumnReference other = (ColumnReference) o; - if (hashCode == other.hashCode && Bytes.equals(family, other.family)) { - return Bytes.equals(qualifier, other.qualifier); - } - } - return false; - } - - @Override - public int hashCode() { - return hashCode; - } - - @Override - public String toString() { - return "ColumnReference - " + Bytes.toString(family) + ":" + Bytes.toString(qualifier); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/ColumnTracker.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/ColumnTracker.java deleted file mode 100644 index 6c2bf4ce..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/ColumnTracker.java +++ /dev/null @@ -1,125 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.update; - -import java.util.ArrayList; -import java.util.Collection; -import java.util.Collections; -import java.util.List; - - -/** - * Simple POJO for tracking a bunch of column references and the next-newest timestamp for those - * columns - *

- * Two {@link ColumnTracker}s are considered equal if they track the same columns, even if their - * timestamps are different. - */ -public class ColumnTracker implements IndexedColumnGroup { - - public static final long NO_NEWER_PRIMARY_TABLE_ENTRY_TIMESTAMP = Long.MAX_VALUE; - public static final long GUARANTEED_NEWER_UPDATES = Long.MIN_VALUE; - private final List columns; - private long ts = NO_NEWER_PRIMARY_TABLE_ENTRY_TIMESTAMP; - private final int hashCode; - - private static int calcHashCode(List columns) { - return columns.hashCode(); - } - - public ColumnTracker(Collection columns) { - this.columns = new ArrayList(columns); - // sort the columns - Collections.sort(this.columns); - this.hashCode = calcHashCode(this.columns); - } - - /** - * Set the current timestamp, only if the passed timestamp is strictly less than the currently - * stored timestamp - * @param ts the timestmap to potentially store. - * @return the currently stored timestamp. - */ - public long setTs(long ts) { - this.ts = this.ts > ts ? ts : this.ts; - return this.ts; - } - - public long getTS() { - return this.ts; - } - - @Override - public int hashCode() { - return hashCode; - } - - @Override - public boolean equals(Object o){ - if(!(o instanceof ColumnTracker)){ - return false; - } - ColumnTracker other = (ColumnTracker)o; - if (hashCode != other.hashCode) { - return false; - } - if (other.columns.size() != columns.size()) { - return false; - } - - // check each column to see if they match - for (int i = 0; i < columns.size(); i++) { - if (!columns.get(i).equals(other.columns.get(i))) { - return false; - } - } - - return true; - } - - @Override - public List getColumns() { - return this.columns; - } - - /** - * @return true if this set of columns has seen a column with a timestamp newer than the - * requested timestamp, false otherwise. - */ - public boolean hasNewerTimestamps() { - return !isNewestTime(this.ts); - } - - /** - * @param ts timestamp to check - * @return true if the timestamp is at the most recent timestamp for a column - */ - public static boolean isNewestTime(long ts) { - return ts == NO_NEWER_PRIMARY_TABLE_ENTRY_TIMESTAMP; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/IndexedColumnGroup.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/IndexedColumnGroup.java deleted file mode 100644 index 97c0d3fd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/IndexedColumnGroup.java +++ /dev/null @@ -1,38 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.update; - -import java.util.List; - -/** - * Group of columns that were requested to build an index - */ -public interface IndexedColumnGroup { - - public List getColumns(); -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/SortedCollection.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/SortedCollection.java deleted file mode 100644 index f25f73c8..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/SortedCollection.java +++ /dev/null @@ -1,138 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.update; - -import java.util.Arrays; -import java.util.Collection; -import java.util.Comparator; -import java.util.Iterator; -import java.util.PriorityQueue; - -import com.google.common.collect.Iterators; - -/** - * A collection whose elements are stored and returned sorted. - *

- * We can't just use something like a {@link PriorityQueue} because it doesn't return the - * underlying values in sorted order. - * @param - */ -class SortedCollection implements Collection, Iterable { - - private PriorityQueue queue; - private Comparator comparator; - - /** - * Use the given comparator to compare all keys for sorting - * @param comparator - */ - public SortedCollection(Comparator comparator) { - this.queue = new PriorityQueue(1, comparator); - this.comparator = comparator; - } - - /** - * All passed elements are expected to be {@link Comparable} - */ - public SortedCollection() { - this.queue = new PriorityQueue(); - } - - @Override - public int size() { - return this.queue.size(); - } - - @Override - public boolean isEmpty() { - return this.queue.isEmpty(); - } - - @Override - public boolean contains(Object o) { - return this.queue.contains(o); - } - - @Override - public Iterator iterator() { - @SuppressWarnings("unchecked") - T[] array = (T[]) this.queue.toArray(); - if (this.comparator == null) { - Arrays.sort(array); - } else { - Arrays.sort( - array, this.comparator);} - return Iterators.forArray(array); - } - - @Override - public Object[] toArray() { - return this.queue.toArray(); - } - - @SuppressWarnings("hiding") - @Override - public T[] toArray(T[] a) { - return this.queue.toArray(a); - } - - @Override - public boolean add(T e) { - return this.queue.add(e); - } - - @Override - public boolean remove(Object o) { - return this.queue.remove(o); - } - - @Override - public boolean containsAll(Collection c) { - return this.queue.containsAll(c); - } - - @Override - public boolean addAll(Collection c) { - return this.queue.addAll(c); - } - - @Override - public boolean removeAll(Collection c) { - return queue.removeAll(c); - } - - @Override - public boolean retainAll(Collection c) { - return this.queue.retainAll(c); - } - - @Override - public void clear() { - this.queue.clear(); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/exception/IndexWriteException.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/exception/IndexWriteException.java deleted file mode 100644 index 35f48b57..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/exception/IndexWriteException.java +++ /dev/null @@ -1,53 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.exception; - -import org.apache.hadoop.hbase.HBaseIOException; - -/** - * Generic {@link Exception} that an index write has failed - */ -@SuppressWarnings("serial") -public class IndexWriteException extends HBaseIOException { - - public IndexWriteException() { - super(); - } - - public IndexWriteException(String message, Throwable cause) { - super(message, cause); - } - - public IndexWriteException(String message) { - super(message); - } - - public IndexWriteException(Throwable cause) { - super(cause); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/exception/MultiIndexWriteFailureException.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/exception/MultiIndexWriteFailureException.java deleted file mode 100644 index 6c57750e..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/exception/MultiIndexWriteFailureException.java +++ /dev/null @@ -1,54 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.exception; - -import java.util.List; - -import com.salesforce.hbase.index.table.HTableInterfaceReference; - -/** - * Indicate a failure to write to multiple index tables. - */ -@SuppressWarnings("serial") -public class MultiIndexWriteFailureException extends IndexWriteException { - - private List failures; - - /** - * @param failures the tables to which the index write did not succeed - */ - public MultiIndexWriteFailureException(List failures) { - super("Failed to write to multiple index tables"); - this.failures = failures; - - } - - public List getFailedTables() { - return this.failures; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/exception/SingleIndexWriteFailureException.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/exception/SingleIndexWriteFailureException.java deleted file mode 100644 index 66aefe9f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/exception/SingleIndexWriteFailureException.java +++ /dev/null @@ -1,71 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.exception; - -import java.util.List; - -import org.apache.hadoop.hbase.client.Mutation; - -/** - * Exception thrown if we cannot successfully write to an index table. - */ -@SuppressWarnings("serial") -public class SingleIndexWriteFailureException extends IndexWriteException { - - private String table; - - /** - * Cannot reach the index, but not sure of the table or the mutations that caused the failure - * @param msg more description of what happened - * @param cause original cause - */ - public SingleIndexWriteFailureException(String msg, Throwable cause) { - super(msg, cause); - } - - /** - * Failed to write the passed mutations to an index table for some reason. - * @param targetTableName index table to which we attempted to write - * @param mutations mutations that were attempted - * @param cause underlying reason for the failure - */ - public SingleIndexWriteFailureException(String targetTableName, List mutations, - Exception cause) { - super("Failed to make index update:\n\t table: " + targetTableName + "\n\t edits: " + mutations - + "\n\tcause: " + cause == null ? "UNKNOWN" : cause.getMessage(), cause); - this.table = targetTableName; - } - - /** - * @return The table to which we failed to write the index updates. If unknown, returns - * null - */ - public String getTableName() { - return this.table; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/BaseTaskRunner.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/BaseTaskRunner.java deleted file mode 100644 index f95d5bbd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/BaseTaskRunner.java +++ /dev/null @@ -1,130 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source - * and binary forms, with or without modification, are permitted provided that the following - * conditions are met: Redistributions of source code must retain the above copyright notice, this - * list of conditions and the following disclaimer. Redistributions in binary form must reproduce - * the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of - * Salesforce.com nor the names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED - * BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, - * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, - * OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF - * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.parallel; - -import java.util.ArrayList; -import java.util.List; -import java.util.concurrent.CancellationException; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.ExecutorService; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.hbase.Abortable; - -import com.google.common.util.concurrent.ListenableFuture; -import com.google.common.util.concurrent.ListeningExecutorService; -import com.google.common.util.concurrent.MoreExecutors; - -/** - * {@link TaskRunner} that just manages the underlying thread pool. On called to - * {@link #stop(String)}, the thread pool is shutdown immediately - all pending tasks are cancelled - * and running tasks receive and interrupt. - *

- * If we find a failure the failure is propagated to the {@link TaskBatch} so any {@link Task} that - * is interested can kill itself as well. - */ -public abstract class BaseTaskRunner implements TaskRunner { - - private static final Log LOG = LogFactory.getLog(BaseTaskRunner.class); - protected ListeningExecutorService writerPool; - private boolean stopped; - - public BaseTaskRunner(ExecutorService service) { - this.writerPool = MoreExecutors.listeningDecorator(service); - } - - @Override - public List submit(TaskBatch tasks) throws CancellationException, ExecutionException, - InterruptedException { - // submit each task to the pool and queue it up to be watched - List> futures = new ArrayList>(tasks.size()); - for (Task task : tasks.getTasks()) { - futures.add(this.writerPool.submit(task)); - } - try { - // This logic is actually much more synchronized than the previous logic. Now we rely on a - // synchronization around the status to tell us when we are done. While this does have the - // advantage of being (1) less code, and (2) supported as part of a library, it is just that - // little bit slower. If push comes to shove, we can revert back to the previous - // implementation, but for right now, this works just fine. - return submitTasks(futures).get(); - } catch (CancellationException e) { - // propagate the failure back out - logAndNotifyAbort(e, tasks); - throw e; - } catch (ExecutionException e) { - // propagate the failure back out - logAndNotifyAbort(e, tasks); - throw e; - } - } - - private void logAndNotifyAbort(Exception e, Abortable abort) { - String msg = "Found a failed task because: " + e.getMessage(); - LOG.error(msg, e); - abort.abort(msg, e.getCause()); - } - - /** - * Build a ListenableFuture for the tasks. Implementing classes can determine return behaviors on - * the given tasks - * @param futures to wait on - * @return a single {@link ListenableFuture} that completes based on the passes tasks. - */ - protected abstract ListenableFuture> submitTasks(List> futures); - - @Override - public List submitUninterruptible(TaskBatch tasks) throws EarlyExitFailure, - ExecutionException { - boolean interrupted = false; - try { - while (!this.isStopped()) { - try { - return this.submit(tasks); - } catch (InterruptedException e) { - interrupted = true; - } - } - } finally { - // restore the interrupted status - if (interrupted) { - Thread.currentThread().interrupt(); - } - } - - // should only get here if we are interrupted while waiting for a result and have been told to - // shutdown by an external source - throw new EarlyExitFailure("Interrupted and stopped before computation was complete!"); - } - - @Override - public void stop(String why) { - if (this.stopped) { - return; - } - LOG.info("Shutting down task runner because " + why); - this.writerPool.shutdownNow(); - } - - @Override - public boolean isStopped() { - return this.stopped; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/EarlyExitFailure.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/EarlyExitFailure.java deleted file mode 100644 index 0f08393e..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/EarlyExitFailure.java +++ /dev/null @@ -1,44 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.parallel; - -import java.io.IOException; - -/** - * Exception denoting a need to early-exit a task (or group of tasks) due to external notification - */ -@SuppressWarnings("serial") -public class EarlyExitFailure extends IOException { - - /** - * @param msg reason for the early exit - */ - public EarlyExitFailure(String msg) { - super(msg); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/QuickFailingTaskRunner.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/QuickFailingTaskRunner.java deleted file mode 100644 index 25dbd343..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/QuickFailingTaskRunner.java +++ /dev/null @@ -1,59 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.parallel; - -import java.util.List; -import java.util.concurrent.ExecutorService; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; - -import com.google.common.util.concurrent.Futures; -import com.google.common.util.concurrent.ListenableFuture; - -/** - * {@link TaskRunner} that attempts to run all tasks passed, but quits early if any {@link Task} - * fails, not waiting for the remaining {@link Task}s to complete. - */ -public class QuickFailingTaskRunner extends BaseTaskRunner { - - static final Log LOG = LogFactory.getLog(QuickFailingTaskRunner.class); - - /** - * @param service thread pool to which {@link Task}s are submitted. This service is then 'owned' - * by this and will be shutdown on calls to {@link #stop(String)}. - */ - public QuickFailingTaskRunner(ExecutorService service) { - super(service); - } - - @Override - protected ListenableFuture> submitTasks(List> futures) { - return Futures.allAsList(futures); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/Task.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/Task.java deleted file mode 100644 index c86d2953..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/Task.java +++ /dev/null @@ -1,50 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.parallel; - -import java.util.concurrent.Callable; - -import org.apache.hadoop.hbase.Abortable; - -/** - * Like a {@link Callable}, but supports an internal {@link Abortable} that can be checked - * periodically to determine if the batch should abort - * @param expected result of the task - */ -public abstract class Task implements Callable { - - private Abortable batch; - - void setBatchMonitor(Abortable abort) { - this.batch = abort; - } - - protected boolean isBatchFailed() { - return this.batch.isAborted(); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/TaskBatch.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/TaskBatch.java deleted file mode 100644 index d732b15c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/TaskBatch.java +++ /dev/null @@ -1,84 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.parallel; - -import java.util.ArrayList; -import java.util.Collection; -import java.util.List; -import java.util.concurrent.atomic.AtomicBoolean; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.hbase.Abortable; - -/** - * A group of {@link Task}s. The tasks are all bound together using the same {@link Abortable} ( - * this) to ensure that all tasks are aware when any of the other tasks fails. - * @param expected result type from all the tasks - */ -public class TaskBatch implements Abortable { - private static final Log LOG = LogFactory.getLog(TaskBatch.class); - private AtomicBoolean aborted = new AtomicBoolean(); - private List> tasks; - - /** - * @param size expected number of tasks - */ - public TaskBatch(int size) { - this.tasks = new ArrayList>(size); - } - - public void add(Task task) { - this.tasks.add(task); - task.setBatchMonitor(this); - } - - public Collection> getTasks() { - return this.tasks; - } - - @Override - public void abort(String why, Throwable e) { - if (this.aborted.getAndSet(true)) { - return; - } - LOG.info("Aborting batch of tasks because " + why); - } - - @Override - public boolean isAborted() { - return this.aborted.get(); - } - - /** - * @return the number of tasks assigned to this batch - */ - public int size() { - return this.tasks.size(); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/TaskRunner.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/TaskRunner.java deleted file mode 100644 index 4ad8f905..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/TaskRunner.java +++ /dev/null @@ -1,61 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source - * and binary forms, with or without modification, are permitted provided that the following - * conditions are met: Redistributions of source code must retain the above copyright notice, this - * list of conditions and the following disclaimer. Redistributions in binary form must reproduce - * the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of - * Salesforce.com nor the names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED - * BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, - * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, - * OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF - * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.parallel; - -import java.util.List; -import java.util.concurrent.ExecutionException; - -import org.apache.hadoop.hbase.Stoppable; - -/** - * - */ -public interface TaskRunner extends Stoppable { - - /** - * Submit the given tasks to the pool and wait for them to complete. fail. - *

- * Non-interruptible method. To stop any running tasks call {@link #stop(String)} - this will - * shutdown the thread pool, causing any pending tasks to be failed early (whose failure will be - * ignored) and interrupt any running tasks. It is up to the passed tasks to respect the interrupt - * notification - * @param tasks to run - * @return the result from each task - * @throws ExecutionException if any of the tasks fails. Wraps the underyling failure, which can - * be retrieved via {@link ExecutionException#getCause()}. - * @throws InterruptedException if the current thread is interrupted while waiting for the batch - * to complete - */ - public List submit(TaskBatch tasks) throws - ExecutionException, InterruptedException; - - /** - * Similar to {@link #submit(TaskBatch)}, but is not interruptible. If an interrupt is found while - * waiting for results, we ignore it and only stop is {@link #stop(String)} has been called. On - * return from the method, the interrupt status of the thread is restored. - * @param tasks to run - * @return the result from each task - * @throws EarlyExitFailure if there are still tasks to submit to the pool, but there is a stop - * notification - * @throws ExecutionException if any of the tasks fails. Wraps the underyling failure, which can - * be retrieved via {@link ExecutionException#getCause()}. - */ - public List submitUninterruptible(TaskBatch tasks) throws EarlyExitFailure, - ExecutionException; -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/ThreadPoolBuilder.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/ThreadPoolBuilder.java deleted file mode 100644 index fd160a4a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/ThreadPoolBuilder.java +++ /dev/null @@ -1,100 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.parallel; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.util.Pair; - -/** - * Helper utility to make a thread pool from a configuration based on reasonable defaults and passed - * configuration keys. - */ -public class ThreadPoolBuilder { - - private static final Log LOG = LogFactory.getLog(ThreadPoolBuilder.class); - private static final long DEFAULT_TIMEOUT = 60; - private static final int DEFAULT_MAX_THREADS = 1;// is there a better default? - private Pair timeout; - private Pair maxThreads; - private String name; - private Configuration conf; - - public ThreadPoolBuilder(String poolName, Configuration conf) { - this.name = poolName; - this.conf = conf; - } - - public ThreadPoolBuilder setCoreTimeout(String confkey, long defaultTime) { - if (defaultTime <= 0) { - defaultTime = DEFAULT_TIMEOUT; - } - this.timeout = new Pair(confkey, defaultTime); - return this; - } - - public ThreadPoolBuilder setCoreTimeout(String confKey) { - return this.setCoreTimeout(confKey, DEFAULT_TIMEOUT); - } - - public ThreadPoolBuilder setMaxThread(String confkey, int defaultThreads) { - if (defaultThreads <= 0) { - defaultThreads = DEFAULT_MAX_THREADS; - } - this.maxThreads = new Pair(confkey, defaultThreads); - return this; - } - - String getName() { - return this.name; - } - - int getMaxThreads() { - int maxThreads = DEFAULT_MAX_THREADS; - if (this.maxThreads != null) { - String key = this.maxThreads.getFirst(); - maxThreads = - key == null ? this.maxThreads.getSecond() : conf.getInt(key, this.maxThreads.getSecond()); - } - LOG.trace("Creating pool builder with max " + maxThreads + " threads "); - return maxThreads; - } - - long getKeepAliveTime() { - long timeout =DEFAULT_TIMEOUT; - if (this.timeout != null) { - String key = this.timeout.getFirst(); - timeout = - key == null ? this.timeout.getSecond() : conf.getLong(key, this.timeout.getSecond()); - } - - LOG.trace("Creating pool builder with core thread timeout of " + timeout + " seconds "); - return timeout; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/ThreadPoolManager.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/ThreadPoolManager.java deleted file mode 100644 index 5d56f8ea..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/ThreadPoolManager.java +++ /dev/null @@ -1,156 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.parallel; - -import java.util.Collections; -import java.util.List; -import java.util.Map; -import java.util.concurrent.BlockingQueue; -import java.util.concurrent.LinkedBlockingQueue; -import java.util.concurrent.ThreadFactory; -import java.util.concurrent.ThreadPoolExecutor; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicInteger; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; -import org.apache.hadoop.hbase.util.Threads; - -/** - * Manage access to thread pools - */ -public class ThreadPoolManager { - - private static final Log LOG = LogFactory.getLog(ThreadPoolManager.class); - - /** - * Get an executor for the given name, based on the passed {@link Configuration}. If a thread pool - * already exists with that name, it will be returned. - * @param builder - * @param env - * @return a {@link ThreadPoolExecutor} for the given name. Thread pool that only shuts down when - * there are no more explicit references to it. You do not need to shutdown the threadpool - * on your own - it is managed for you. When you are done, you merely need to release your - * reference. If you do attempt to shutdown the pool, you should be careful to call - * {@link ThreadPoolExecutor#shutdown()} XOR {@link ThreadPoolExecutor#shutdownNow()} - extra calls to either can lead to - * early shutdown of the pool. - */ - public static synchronized ThreadPoolExecutor getExecutor(ThreadPoolBuilder builder, - RegionCoprocessorEnvironment env) { - return getExecutor(builder, env.getSharedData()); - } - - static synchronized ThreadPoolExecutor getExecutor(ThreadPoolBuilder builder, - Map poolCache) { - ThreadPoolExecutor pool = (ThreadPoolExecutor) poolCache.get(builder.getName()); - if (pool == null || pool.isTerminating() || pool.isShutdown()) { - pool = getDefaultExecutor(builder); - LOG.info("Creating new pool for " + builder.getName()); - poolCache.put(builder.getName(), pool); - } - ((ShutdownOnUnusedThreadPoolExecutor) pool).addReference(); - - return pool; - } - - /** - * @param conf - * @return - */ - private static ShutdownOnUnusedThreadPoolExecutor getDefaultExecutor(ThreadPoolBuilder builder) { - int maxThreads = builder.getMaxThreads(); - long keepAliveTime = builder.getKeepAliveTime(); - - // we prefer starting a new thread to queuing (the opposite of the usual ThreadPoolExecutor) - // since we are probably writing to a bunch of index tables in this case. Any pending requests - // are then queued up in an infinite (Integer.MAX_VALUE) queue. However, we allow core threads - // to timeout, to we tune up/down for bursty situations. We could be a bit smarter and more - // closely manage the core-thread pool size to handle the bursty traffic (so we can always keep - // some core threads on hand, rather than starting from scratch each time), but that would take - // even more time. If we shutdown the pool, but are still putting new tasks, we can just do the - // usual policy and throw a RejectedExecutionException because we are shutting down anyways and - // the worst thing is that this gets unloaded. - ShutdownOnUnusedThreadPoolExecutor pool = - new ShutdownOnUnusedThreadPoolExecutor(maxThreads, maxThreads, keepAliveTime, - TimeUnit.SECONDS, new LinkedBlockingQueue(), - Threads.newDaemonThreadFactory(builder.getName() + "-"), builder.getName()); - pool.allowCoreThreadTimeOut(true); - return pool; - } - - /** - * Thread pool that only shuts down when there are no more explicit references to it. A reference - * is when obtained and released on calls to {@link #shutdown()} or {@link #shutdownNow()}. - * Therefore, users should be careful to call {@link #shutdown()} XOR {@link #shutdownNow()} - - * extra calls to either can lead to early shutdown of the pool. - */ - private static class ShutdownOnUnusedThreadPoolExecutor extends ThreadPoolExecutor { - - private AtomicInteger references; - private String poolName; - - public ShutdownOnUnusedThreadPoolExecutor(int coreThreads, int maxThreads, long keepAliveTime, - TimeUnit timeUnit, BlockingQueue workQueue, ThreadFactory threadFactory, - String poolName) { - super(coreThreads, maxThreads, keepAliveTime, timeUnit, workQueue, threadFactory); - this.references = new AtomicInteger(); - this.poolName = poolName; - } - - public void addReference() { - this.references.incrementAndGet(); - } - - @Override - protected void finalize() { - // override references counter if we go out of scope - ensures the pool gets cleaned up - LOG.info("Shutting down pool '" + poolName + "' because no more references"); - super.finalize(); - } - - @Override - public void shutdown() { - if (references.decrementAndGet() <= 0) { - LOG.debug("Shutting down pool " + this.poolName); - super.shutdown(); - } - } - - @Override - public List shutdownNow() { - if (references.decrementAndGet() <= 0) { - LOG.debug("Shutting down pool " + this.poolName + " NOW!"); - return super.shutdownNow(); - } - return Collections.emptyList(); - } - - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/WaitForCompletionTaskRunner.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/WaitForCompletionTaskRunner.java deleted file mode 100644 index b3ffaa66..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/parallel/WaitForCompletionTaskRunner.java +++ /dev/null @@ -1,61 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.parallel; - -import java.util.List; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.ExecutorService; - -import com.google.common.util.concurrent.Futures; -import com.google.common.util.concurrent.ListenableFuture; - -/** - * A {@link TaskRunner} that ensures that all the tasks have been attempted before we return, even - * if some of the tasks cause failures. - *

- * Because we wait until the entire batch is complete to see the failure, checking for failure of - * the {@link TaskBatch} on the submitted tasks will not help - they will never see the failure of - * the other tasks. You will need to provide an external mechanism to propagate the error. - *

- * Does not throw an {@link ExecutionException} if any of the tasks fail. - */ -public class WaitForCompletionTaskRunner extends BaseTaskRunner { - - /** - * @param service thread pool to which {@link Task}s are submitted. This service is then 'owned' - * by this and will be shutdown on calls to {@link #stop(String)}. - */ - public WaitForCompletionTaskRunner(ExecutorService service) { - super(service); - } - - @Override - public ListenableFuture> submitTasks(List> futures) { - return Futures.successfulAsList(futures); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/FilteredKeyValueScanner.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/FilteredKeyValueScanner.java deleted file mode 100644 index c90319fd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/FilteredKeyValueScanner.java +++ /dev/null @@ -1,134 +0,0 @@ -package com.salesforce.hbase.index.scanner; - -import java.io.IOException; -import java.util.SortedSet; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.filter.Filter; -import org.apache.hadoop.hbase.filter.Filter.ReturnCode; -import org.apache.hadoop.hbase.regionserver.KeyValueScanner; - -import com.salesforce.hbase.index.covered.KeyValueStore; - -/** - * Combine a simplified version of the logic in the ScanQueryMatcher and the KeyValueScanner. We can - * get away with this here because we are only concerned with a single MemStore for the index; we - * don't need to worry about multiple column families or minimizing seeking through file - we just - * want to iterate the kvs quickly, in-memory. - */ -public class FilteredKeyValueScanner implements KeyValueScanner { - - private KeyValueScanner delegate; - private Filter filter; - - public FilteredKeyValueScanner(Filter filter, KeyValueStore store) { - this(filter, store.getScanner()); - } - - private FilteredKeyValueScanner(Filter filter, KeyValueScanner delegate) { - this.delegate = delegate; - this.filter = filter; - } - - @Override - public KeyValue peek() { - return delegate.peek(); - } - - /** - * Same a {@link KeyValueScanner#next()} except that we filter out the next {@link KeyValue} until - * we find one that passes the filter. - * @return the next {@link KeyValue} or null if no next {@link KeyValue} is present and - * passes all the filters. - */ - @Override - public KeyValue next() throws IOException { - seekToNextUnfilteredKeyValue(); - return delegate.next(); - } - - @Override - public boolean seek(KeyValue key) throws IOException { - if(filter.filterAllRemaining()){ - return false; - } - // see if we can seek to the next key - if (!delegate.seek(key)) { - return false; - } - - return seekToNextUnfilteredKeyValue(); - } - - private boolean seekToNextUnfilteredKeyValue() throws IOException { - while (true) { - KeyValue peeked = delegate.peek(); - // no more key values, so we are done - if (peeked == null) { - return false; - } - - // filter the peeked value to see if it should be served - ReturnCode code = filter.filterKeyValue(peeked); - switch (code) { - // included, so we are done - case INCLUDE: - case INCLUDE_AND_NEXT_COL: - return true; - // not included, so we need to go to the next row - case SKIP: - case NEXT_COL: - case NEXT_ROW: - delegate.next(); - break; - // use a seek hint to find out where we should go - case SEEK_NEXT_USING_HINT: - delegate.seek(filter.getNextKeyHint(peeked)); - } - } - } - - @Override - public boolean reseek(KeyValue key) throws IOException { - this.delegate.reseek(key); - return this.seekToNextUnfilteredKeyValue(); - } - - @Override - public boolean requestSeek(KeyValue kv, boolean forward, boolean useBloom) throws IOException { - return this.reseek(kv); - } - - @Override - public boolean isFileScanner() { - return false; - } - - @Override - public long getSequenceID() { - return this.delegate.getSequenceID(); - } - - @Override - public boolean shouldUseScanner(Scan scan, SortedSet columns, long oldestUnexpiredTS) { - throw new UnsupportedOperationException(this.getClass().getName() - + " doesn't support checking to see if it should use a scanner!"); - } - - - @Override - public boolean realSeekDone() { - return this.delegate.realSeekDone(); - } - - @Override - public void enforceSeek() throws IOException { - this.delegate.enforceSeek(); - } - - @Override - public void close() { - this.delegate.close(); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/table/CachingHTableFactory.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/table/CachingHTableFactory.java deleted file mode 100644 index 8d7eb730..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/table/CachingHTableFactory.java +++ /dev/null @@ -1,120 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.table; - -import java.io.IOException; -import java.util.Map; - -import org.apache.commons.collections.map.LRUMap; -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.client.HTableInterface; -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; - -/** - * A simple cache that just uses usual GC mechanisms to cleanup unused {@link HTableInterface}s. - * When requesting an {@link HTableInterface} via {@link #getTable}, you may get the same table as - * last time, or it may be a new table. - *

- * You should not call {@link HTableInterface#close()} that is handled when the table goes - * out of scope. Along the same lines, you must ensure to not keep a reference to the table for - * longer than necessary - this leak will ensure that the table never gets closed. - */ -public class CachingHTableFactory implements HTableFactory { - - /** - * LRUMap that closes the {@link HTableInterface} when the table is evicted - */ - @SuppressWarnings("serial") - public class HTableInterfaceLRUMap extends LRUMap { - - public HTableInterfaceLRUMap(int cacheSize) { - super(cacheSize); - } - - @Override - protected boolean removeLRU(LinkEntry entry) { - HTableInterface table = (HTableInterface) entry.getValue(); - if (LOG.isDebugEnabled()) { - LOG.debug("Closing connection to table: " + Bytes.toString(table.getTableName()) - + " because it was evicted from the cache."); - } - try { - table.close(); - } catch (IOException e) { - LOG.info("Failed to correctly close HTable: " + Bytes.toString(table.getTableName()) - + " ignoring since being removed from queue."); - } - return true; - } - } - - public static int getCacheSize(Configuration conf) { - return conf.getInt(CACHE_SIZE_KEY, DEFAULT_CACHE_SIZE); - } - - private static final Log LOG = LogFactory.getLog(CachingHTableFactory.class); - private static final String CACHE_SIZE_KEY = "index.tablefactory.cache.size"; - private static final int DEFAULT_CACHE_SIZE = 10; - - private HTableFactory delegate; - - @SuppressWarnings("rawtypes") - Map openTables; - - public CachingHTableFactory(HTableFactory tableFactory, Configuration conf) { - this(tableFactory, getCacheSize(conf)); - } - - public CachingHTableFactory(HTableFactory factory, int cacheSize) { - this.delegate = factory; - openTables = new HTableInterfaceLRUMap(cacheSize); - } - - @Override - @SuppressWarnings("unchecked") - public HTableInterface getTable(ImmutableBytesPtr tablename) throws IOException { - ImmutableBytesPtr tableBytes = new ImmutableBytesPtr(tablename); - synchronized (openTables) { - HTableInterface table = (HTableInterface) openTables.get(tableBytes); - if (table == null) { - table = delegate.getTable(tablename); - openTables.put(tableBytes, table); - } - return table; - } - } - - @Override - public void shutdown() { - this.delegate.shutdown(); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/table/HTableFactory.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/table/HTableFactory.java deleted file mode 100644 index 6dca1da0..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/table/HTableFactory.java +++ /dev/null @@ -1,14 +0,0 @@ -package com.salesforce.hbase.index.table; - -import java.io.IOException; - -import org.apache.hadoop.hbase.client.HTableInterface; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; - -public interface HTableFactory { - - public HTableInterface getTable(ImmutableBytesPtr tablename) throws IOException; - - public void shutdown(); -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/util/ImmutableBytesPtr.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/util/ImmutableBytesPtr.java deleted file mode 100644 index b053767c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/util/ImmutableBytesPtr.java +++ /dev/null @@ -1,121 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.util; - -import java.io.DataInput; -import java.io.IOException; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; - -public class ImmutableBytesPtr extends ImmutableBytesWritable { - private int hashCode; - - public ImmutableBytesPtr() { - } - - public ImmutableBytesPtr(byte[] bytes) { - super(bytes); - hashCode = super.hashCode(); - } - - public ImmutableBytesPtr(ImmutableBytesWritable ibw) { - super(ibw.get(), ibw.getOffset(), ibw.getLength()); - hashCode = super.hashCode(); - } - - public ImmutableBytesPtr(ImmutableBytesPtr ibp) { - super(ibp.get(), ibp.getOffset(), ibp.getLength()); - hashCode = ibp.hashCode; - } - - public ImmutableBytesPtr(byte[] bytes, int offset, int length) { - super(bytes, offset, length); - hashCode = super.hashCode(); - } - - @Override - public int hashCode() { - return hashCode; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - ImmutableBytesPtr that = (ImmutableBytesPtr)obj; - if (this.hashCode != that.hashCode) return false; - if (Bytes.compareTo(this.get(), this.getOffset(), this.getLength(), that.get(), that.getOffset(), that.getLength()) != 0) return false; - return true; - } - - public void set(ImmutableBytesWritable ptr) { - set(ptr.get(),ptr.getOffset(),ptr.getLength()); - } - - /** - * @param b Use passed bytes as backing array for this instance. - */ - @Override - public void set(final byte [] b) { - super.set(b); - hashCode = super.hashCode(); - } - - /** - * @param b Use passed bytes as backing array for this instance. - * @param offset - * @param length - */ - @Override - public void set(final byte [] b, final int offset, final int length) { - super.set(b,offset,length); - hashCode = super.hashCode(); - } - - @Override - public void readFields(final DataInput in) throws IOException { - super.readFields(in); - hashCode = super.hashCode(); - } - - /** - * @return the backing byte array, copying only if necessary - */ - public byte[] copyBytesIfNecessary() { - return copyBytesIfNecessary(this); - } - - public static byte[] copyBytesIfNecessary(ImmutableBytesWritable ptr) { - if (ptr.getOffset() == 0 && ptr.getLength() == ptr.get().length) { - return ptr.get(); - } - return ptr.copyBytes(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexCommitter.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexCommitter.java deleted file mode 100644 index 0f17b982..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexCommitter.java +++ /dev/null @@ -1,47 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.write; - -import org.apache.hadoop.hbase.Stoppable; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; - -import com.google.common.collect.Multimap; -import com.salesforce.hbase.index.exception.IndexWriteException; -import com.salesforce.hbase.index.table.HTableInterfaceReference; - -/** - * Write the index updates to the index tables - */ -public interface IndexCommitter extends Stoppable { - - void setup(IndexWriter parent, RegionCoprocessorEnvironment env, String name); - - public void write(Multimap toWrite) - throws IndexWriteException; -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexFailurePolicy.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexFailurePolicy.java deleted file mode 100644 index e539aa68..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexFailurePolicy.java +++ /dev/null @@ -1,54 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.write; - -import java.io.IOException; - -import org.apache.hadoop.hbase.Stoppable; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; - -import com.google.common.collect.Multimap; -import com.salesforce.hbase.index.table.HTableInterfaceReference; - -/** - * Handle failures to write to the index tables. - */ -public interface IndexFailurePolicy extends Stoppable { - - public void setup(Stoppable parent, RegionCoprocessorEnvironment env); - - /** - * Handle the failure of the attempted index updates - * @param attempted map of index table -> mutations to apply - * @param cause reason why there was a failure - * @throws IOException - */ - public void - handleFailure(Multimap attempted, Exception cause) throws IOException; -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexWriterUtils.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexWriterUtils.java deleted file mode 100644 index 177d1c2f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexWriterUtils.java +++ /dev/null @@ -1,82 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.write; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.CoprocessorEnvironment; - -import com.salesforce.hbase.index.table.CoprocessorHTableFactory; -import com.salesforce.hbase.index.table.HTableFactory; -import com.salesforce.hbase.index.util.IndexManagementUtil; - -public class IndexWriterUtils { - - private static final Log LOG = LogFactory.getLog(IndexWriterUtils.class); - - /** - * Maximum number of threads to allow per-table when writing. Each writer thread (from - * {@link IndexWriterUtils#NUM_CONCURRENT_INDEX_WRITER_THREADS_CONF_KEY}) has a single HTable. - * However, each table is backed by a threadpool to manage the updates to that table. this - * specifies the number of threads to allow in each of those tables. Generally, you shouldn't need - * to change this, unless you have a small number of indexes to which most of the writes go. - * Defaults to: {@value #DEFAULT_NUM_PER_TABLE_THREADS}. - *

- * For tables to which there are not a lot of writes, the thread pool automatically will decrease - * the number of threads to one (though it can burst up to the specified max for any given table), - * so increasing this to meet the max case is reasonable. - *

- * Setting this value too small can cause catastrophic cluster failure. The way HTable's - * underlying pool works is such that is does direct hand-off of tasks to threads. This works fine - * because HTables are assumed to work in a single-threaded context, so we never get more threads - * than regionservers. In a multi-threaded context, we can easily grow to more than that number of - * threads. Currently, HBase doesn't support a custom thread-pool to back the HTable via the - * coprocesor hooks, so we can't modify this behavior. - */ - private static final String INDEX_WRITER_PER_TABLE_THREADS_CONF_KEY = - "index.writer.threads.pertable.max"; - private static final int DEFAULT_NUM_PER_TABLE_THREADS = Integer.MAX_VALUE; - - /** Configuration key that HBase uses to set the max number of threads for an HTable */ - public static final String HTABLE_THREAD_KEY = "hbase.htable.threads.max"; - private IndexWriterUtils() { - // private ctor for utilites - } - - public static HTableFactory getDefaultDelegateHTableFactory(CoprocessorEnvironment env) { - // create a simple delegate factory, setup the way we need - Configuration conf = env.getConfiguration(); - // set the number of threads allowed per table. - int htableThreads = - conf.getInt(IndexWriterUtils.INDEX_WRITER_PER_TABLE_THREADS_CONF_KEY, IndexWriterUtils.DEFAULT_NUM_PER_TABLE_THREADS); - LOG.trace("Creating HTableFactory with " + htableThreads + " threads for each HTable."); - IndexManagementUtil.setIfNotSet(conf, HTABLE_THREAD_KEY, htableThreads); - return new CoprocessorHTableFactory(env); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/KillServerOnFailurePolicy.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/write/KillServerOnFailurePolicy.java deleted file mode 100644 index c54e96e7..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/KillServerOnFailurePolicy.java +++ /dev/null @@ -1,90 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.write; - -import java.io.IOException; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.hbase.Abortable; -import org.apache.hadoop.hbase.Stoppable; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; - -import com.google.common.collect.Multimap; -import com.salesforce.hbase.index.table.HTableInterfaceReference; - -/** - * Naive failure policy - kills the server on which it resides - */ -public class KillServerOnFailurePolicy implements IndexFailurePolicy { - - private static final Log LOG = LogFactory.getLog(KillServerOnFailurePolicy.class); - private Abortable abortable; - private Stoppable stoppable; - - @Override - public void setup(Stoppable parent, RegionCoprocessorEnvironment env) { - setup(parent, env.getRegionServerServices()); - } - - public void setup(Stoppable parent, Abortable abort) { - this.stoppable = parent; - this.abortable = abort; - } - - @Override - public void stop(String why) { - // noop - } - - @Override - public boolean isStopped() { - return this.stoppable.isStopped(); - } - - @Override - public void - handleFailure(Multimap attempted, Exception cause) throws IOException { - // cleanup resources - this.stop("Killing ourselves because of an error:" + cause); - // notify the regionserver of the failure - String msg = - "Could not update the index table, killing server region because couldn't write to an index table"; - LOG.error(msg, cause); - try { - this.abortable.abort(msg, cause); - } catch (Exception e) { - LOG.fatal("Couldn't abort this server to preserve index writes, " - + "attempting to hard kill the server"); - System.exit(1); - } - - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/recovery/PerRegionIndexWriteCache.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/write/recovery/PerRegionIndexWriteCache.java deleted file mode 100644 index 18b4c2f1..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/recovery/PerRegionIndexWriteCache.java +++ /dev/null @@ -1,73 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.write.recovery; - -import java.util.Collection; -import java.util.HashMap; -import java.util.Map; - -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.regionserver.HRegion; - -import com.google.common.collect.ArrayListMultimap; -import com.google.common.collect.Multimap; -import com.salesforce.hbase.index.table.HTableInterfaceReference; - - -public class PerRegionIndexWriteCache { - - private Map> cache = - new HashMap>(); - - - /** - * Get the edits for the current region. Removes the edits from the cache. To add them back, call - * {@link #addEdits(HRegion, HTableInterfaceReference, Collection)}. - * @param region - * @return Get the edits for the given region. Returns null if there are no pending edits - * for the region - */ - public Multimap getEdits(HRegion region) { - return cache.remove(region); - } - - /** - * @param region - * @param table - * @param collection - */ - public void addEdits(HRegion region, HTableInterfaceReference table, - Collection collection) { - Multimap edits = cache.get(region); - if (edits == null) { - edits = ArrayListMultimap. create(); - cache.put(region, edits); - } - edits.putAll(table, collection); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/recovery/TrackingParallelWriterIndexCommitter.java b/phoenix-core/src/main/java/com/salesforce/hbase/index/write/recovery/TrackingParallelWriterIndexCommitter.java deleted file mode 100644 index 25f302b7..00000000 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/recovery/TrackingParallelWriterIndexCommitter.java +++ /dev/null @@ -1,236 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.write.recovery; - -import java.util.ArrayList; -import java.util.Collection; -import java.util.Collections; -import java.util.List; -import java.util.Map.Entry; -import java.util.Set; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.ExecutorService; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.Abortable; -import org.apache.hadoop.hbase.Stoppable; -import org.apache.hadoop.hbase.client.HTableInterface; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; - -import com.google.common.collect.Multimap; -import com.salesforce.hbase.index.CapturingAbortable; -import com.salesforce.hbase.index.exception.MultiIndexWriteFailureException; -import com.salesforce.hbase.index.exception.SingleIndexWriteFailureException; -import com.salesforce.hbase.index.parallel.EarlyExitFailure; -import com.salesforce.hbase.index.parallel.Task; -import com.salesforce.hbase.index.parallel.TaskBatch; -import com.salesforce.hbase.index.parallel.TaskRunner; -import com.salesforce.hbase.index.parallel.ThreadPoolBuilder; -import com.salesforce.hbase.index.parallel.ThreadPoolManager; -import com.salesforce.hbase.index.parallel.WaitForCompletionTaskRunner; -import com.salesforce.hbase.index.table.CachingHTableFactory; -import com.salesforce.hbase.index.table.HTableFactory; -import com.salesforce.hbase.index.table.HTableInterfaceReference; -import com.salesforce.hbase.index.write.IndexCommitter; -import com.salesforce.hbase.index.write.IndexWriter; -import com.salesforce.hbase.index.write.IndexWriterUtils; -import com.salesforce.hbase.index.write.ParallelWriterIndexCommitter; - -/** - * Like the {@link ParallelWriterIndexCommitter}, but blocks until all writes have attempted to - * allow the caller to retrieve the failed and succeeded index updates. Therefore, this class will - * be a lot slower, in the face of failures, when compared to the - * {@link ParallelWriterIndexCommitter} (though as fast for writes), so it should be used only when - * you need to at least attempt all writes and know their result; for instance, this is fine for - * doing WAL recovery - it's not a performance intensive situation and we want to limit the the - * edits we need to retry. - *

- * On failure to {@link #write(Multimap)}, we return a {@link MultiIndexWriteFailureException} that - * contains the list of {@link HTableInterfaceReference} that didn't complete successfully. - *

- * Failures to write to the index can happen several different ways: - *

    - *
  1. this is {@link #stop(String) stopped} or aborted (via the passed {@link Abortable}. - * This causing any pending tasks to fail whatever they are doing as fast as possible. Any writes - * that have not begun are not even attempted and marked as failures.
    2. - *
  2. A batch write fails. This is the generic HBase write failure - it may occur because the index - * table is not available, .META. or -ROOT- is unavailable, or any other (of many) possible HBase - * exceptions.
    3. - *
- * Regardless of how the write fails, we still wait for all writes to complete before passing the - * failure back to the client. - */ -public class TrackingParallelWriterIndexCommitter implements IndexCommitter { - private static final Log LOG = LogFactory.getLog(TrackingParallelWriterIndexCommitter.class); - - public static final String NUM_CONCURRENT_INDEX_WRITER_THREADS_CONF_KEY = "index.trackingwriter.threads.max"; - private static final int DEFAULT_CONCURRENT_INDEX_WRITER_THREADS = 10; - private static final String INDEX_WRITER_KEEP_ALIVE_TIME_CONF_KEY = - "index.trackingwriter.threads.keepalivetime"; - - private TaskRunner pool; - private HTableFactory factory; - private CapturingAbortable abortable; - private Stoppable stopped; - - @Override - public void setup(IndexWriter parent, RegionCoprocessorEnvironment env, String name) { - Configuration conf = env.getConfiguration(); - setup(IndexWriterUtils.getDefaultDelegateHTableFactory(env), - ThreadPoolManager.getExecutor( - new ThreadPoolBuilder(name, conf). - setMaxThread(NUM_CONCURRENT_INDEX_WRITER_THREADS_CONF_KEY, - DEFAULT_CONCURRENT_INDEX_WRITER_THREADS). - setCoreTimeout(INDEX_WRITER_KEEP_ALIVE_TIME_CONF_KEY), env), - env.getRegionServerServices(), parent, CachingHTableFactory.getCacheSize(conf)); - } - - /** - * Setup this. - *

- * Exposed for TESTING - */ - void setup(HTableFactory factory, ExecutorService pool, Abortable abortable, Stoppable stop, - int cacheSize) { - this.pool = new WaitForCompletionTaskRunner(pool); - this.factory = new CachingHTableFactory(factory, cacheSize); - this.abortable = new CapturingAbortable(abortable); - this.stopped = stop; - } - - @Override - public void write(Multimap toWrite) - throws MultiIndexWriteFailureException { - Set>> entries = toWrite.asMap().entrySet(); - TaskBatch tasks = new TaskBatch(entries.size()); - List tables = new ArrayList(entries.size()); - for (Entry> entry : entries) { - // get the mutations for each table. We leak the implementation here a little bit to save - // doing a complete copy over of all the index update for each table. - final List mutations = (List) entry.getValue(); - // track each reference so we can get at it easily later, when determing failures - final HTableInterfaceReference tableReference = entry.getKey(); - tables.add(tableReference); - - /* - * Write a batch of index updates to an index table. This operation stops (is cancelable) via - * two mechanisms: (1) setting aborted or stopped on the IndexWriter or, (2) interrupting the - * running thread. The former will only work if we are not in the midst of writing the current - * batch to the table, though we do check these status variables before starting and before - * writing the batch. The latter usage, interrupting the thread, will work in the previous - * situations as was at some points while writing the batch, depending on the underlying - * writer implementation (HTableInterface#batch is blocking, but doesn't elaborate when is - * supports an interrupt). - */ - tasks.add(new Task() { - - /** - * Do the actual write to the primary table. We don't need to worry about closing the table - * because that is handled the {@link CachingHTableFactory}. - */ - @Override - public Boolean call() throws Exception { - try { - // this may have been queued, but there was an abort/stop so we try to early exit - throwFailureIfDone(); - - if (LOG.isDebugEnabled()) { - LOG.debug("Writing index update:" + mutations + " to table: " + tableReference); - } - HTableInterface table = factory.getTable(tableReference.get()); - throwFailureIfDone(); - table.batch(mutations); - } catch (InterruptedException e) { - // reset the interrupt status on the thread - Thread.currentThread().interrupt(); - throw e; - } catch (Exception e) { - throw e; - } - return Boolean.TRUE; - } - - private void throwFailureIfDone() throws SingleIndexWriteFailureException { - if (stopped.isStopped() || abortable.isAborted() - || Thread.currentThread().isInterrupted()) { - throw new SingleIndexWriteFailureException( - "Pool closed, not attempting to write to the index!", null); - } - - } - }); - } - - List results = null; - try { - LOG.debug("Waiting on index update tasks to complete..."); - results = this.pool.submitUninterruptible(tasks); - } catch (ExecutionException e) { - throw new RuntimeException( - "Should not fail on the results while using a WaitForCompletionTaskRunner", e); - } catch (EarlyExitFailure e) { - throw new RuntimeException("Stopped while waiting for batch, quiting!", e); - } - - // track the failures. We only ever access this on return from our calls, so no extra - // synchronization is needed. We could update all the failures as we find them, but that add a - // lot of locking overhead, and just doing the copy later is about as efficient. - List failures = new ArrayList(); - int index = 0; - for (Boolean result : results) { - // there was a failure - if (result == null) { - // we know which table failed by the index of the result - failures.add(tables.get(index)); - } - index++; - } - - // if any of the tasks failed, then we need to propagate the failure - if (failures.size() > 0) { - // make the list unmodifiable to avoid any more synchronization concerns - throw new MultiIndexWriteFailureException(Collections.unmodifiableList(failures)); - } - return; - } - - @Override - public void stop(String why) { - LOG.info("Shutting down " + this.getClass().getSimpleName()); - this.pool.stop(why); - this.factory.shutdown(); - } - - @Override - public boolean isStopped() { - return this.stopped.isStopped(); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/GlobalCache.java b/phoenix-core/src/main/java/com/salesforce/phoenix/cache/GlobalCache.java deleted file mode 100644 index 9b2b035e..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/GlobalCache.java +++ /dev/null @@ -1,122 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.cache; - -import static com.salesforce.phoenix.query.QueryServices.MAX_MEMORY_PERC_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MAX_MEMORY_WAIT_MS_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MAX_TENANT_MEMORY_PERC_ATTRIB; - -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ConcurrentMap; - -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.memory.ChildMemoryManager; -import com.salesforce.phoenix.memory.GlobalMemoryManager; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.PTable; - - -/** - * - * Global root cache for the server. Each tenant is managed as a child tenant cache of this one. Queries - * not associated with a particular tenant use this as their tenant cache. - * - * @author jtaylor - * @since 0.1 - */ -public class GlobalCache extends TenantCacheImpl { - private static GlobalCache INSTANCE; - - private final Configuration config; - // TODO: Use Guava cache with auto removal after lack of access - private final ConcurrentMap perTenantCacheMap = new ConcurrentHashMap(); - // Cache for lastest PTable for a given Phoenix table - private final ConcurrentHashMap metaDataCacheMap = new ConcurrentHashMap(); - - public static synchronized GlobalCache getInstance(RegionCoprocessorEnvironment env) { - // See http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html - // for explanation of why double locking doesn't work. - if (INSTANCE == null) { - INSTANCE = new GlobalCache(env.getConfiguration()); - } - return INSTANCE; - } - - public ConcurrentHashMap getMetaDataCache() { - return metaDataCacheMap; - } - - /** - * Get the tenant cache associated with the tenantId. If tenantId is not applicable, null may be - * used in which case a global tenant cache is returned. - * @param env the HBase configuration - * @param tenantId the tenant ID or null if not applicable. - * @return TenantCache - */ - public static TenantCache getTenantCache(RegionCoprocessorEnvironment env, ImmutableBytesWritable tenantId) { - GlobalCache globalCache = GlobalCache.getInstance(env); - TenantCache tenantCache = tenantId == null ? globalCache : globalCache.getChildTenantCache(tenantId); - return tenantCache; - } - - private GlobalCache(Configuration config) { - super(new GlobalMemoryManager(Runtime.getRuntime().totalMemory() * - config.getInt(MAX_MEMORY_PERC_ATTRIB, QueryServicesOptions.DEFAULT_MAX_MEMORY_PERC) / 100, - config.getInt(MAX_MEMORY_WAIT_MS_ATTRIB, QueryServicesOptions.DEFAULT_MAX_MEMORY_WAIT_MS)), - config.getInt(QueryServices.MAX_SERVER_CACHE_TIME_TO_LIVE_MS, QueryServicesOptions.DEFAULT_MAX_SERVER_CACHE_TIME_TO_LIVE_MS)); - this.config = config; - } - - public Configuration getConfig() { - return config; - } - - /** - * Retrieve the tenant cache given an tenantId. - * @param tenantId the ID that identifies the tenant - * @return the existing or newly created TenantCache - */ - public TenantCache getChildTenantCache(ImmutableBytesWritable tenantId) { - TenantCache tenantCache = perTenantCacheMap.get(tenantId); - if (tenantCache == null) { - int maxTenantMemoryPerc = config.getInt(MAX_TENANT_MEMORY_PERC_ATTRIB, QueryServicesOptions.DEFAULT_MAX_TENANT_MEMORY_PERC); - int maxServerCacheTimeToLive = config.getInt(QueryServices.MAX_SERVER_CACHE_TIME_TO_LIVE_MS, QueryServicesOptions.DEFAULT_MAX_SERVER_CACHE_TIME_TO_LIVE_MS); - TenantCacheImpl newTenantCache = new TenantCacheImpl(new ChildMemoryManager(getMemoryManager(), maxTenantMemoryPerc), maxServerCacheTimeToLive); - tenantCache = perTenantCacheMap.putIfAbsent(tenantId, newTenantCache); - if (tenantCache == null) { - tenantCache = newTenantCache; - } - } - return tenantCache; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/HashCache.java b/phoenix-core/src/main/java/com/salesforce/phoenix/cache/HashCache.java deleted file mode 100644 index 6b7ebe96..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/HashCache.java +++ /dev/null @@ -1,48 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.cache; - -import java.io.Closeable; -import java.util.List; - -import org.apache.http.annotation.Immutable; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * Encapsulate deserialized hash cache from bytes into Map. - * The Map uses the row key as the key and the row as the value. - * @author jtaylor - * @since 0.1 - */ -@Immutable -public interface HashCache extends Closeable { - public List get(ImmutableBytesPtr hashKey); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/IndexMetaDataCache.java b/phoenix-core/src/main/java/com/salesforce/phoenix/cache/IndexMetaDataCache.java deleted file mode 100644 index b3eb8b1d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/IndexMetaDataCache.java +++ /dev/null @@ -1,10 +0,0 @@ -package com.salesforce.phoenix.cache; - -import java.io.Closeable; -import java.util.List; - -import com.salesforce.phoenix.index.IndexMaintainer; - -public interface IndexMetaDataCache extends Closeable { - public List getIndexMaintainers(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/TenantCache.java b/phoenix-core/src/main/java/com/salesforce/phoenix/cache/TenantCache.java deleted file mode 100644 index 75c49e04..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/TenantCache.java +++ /dev/null @@ -1,52 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.cache; - -import java.io.Closeable; -import java.sql.SQLException; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.coprocessor.ServerCachingProtocol.ServerCacheFactory; -import com.salesforce.phoenix.memory.MemoryManager; - - -/** - * - * Inteface to set and set cached values for a tenant - * - * @author jtaylor - * @since 0.1 - */ -public interface TenantCache { - MemoryManager getMemoryManager(); - Closeable getServerCache(ImmutableBytesPtr cacheId); - Closeable addServerCache(ImmutableBytesPtr cacheId, ImmutableBytesWritable cachePtr, ServerCacheFactory cacheFactory) throws SQLException; - void removeServerCache(ImmutableBytesPtr cacheId) throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/TenantCacheImpl.java b/phoenix-core/src/main/java/com/salesforce/phoenix/cache/TenantCacheImpl.java deleted file mode 100644 index 39f7f5c4..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/TenantCacheImpl.java +++ /dev/null @@ -1,104 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.cache; - -import java.io.Closeable; -import java.sql.SQLException; -import java.util.Collections; -import java.util.concurrent.TimeUnit; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.google.common.cache.*; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.coprocessor.ServerCachingProtocol.ServerCacheFactory; -import com.salesforce.phoenix.memory.MemoryManager; -import com.salesforce.phoenix.memory.MemoryManager.MemoryChunk; -import com.salesforce.phoenix.util.Closeables; - -/** - * - * Cache per tenant on server side. Tracks memory usage for each - * tenat as well and rolling up usage to global memory manager. - * - * @author jtaylor - * @since 0.1 - */ -public class TenantCacheImpl implements TenantCache { - private final int maxTimeToLiveMs; - private final MemoryManager memoryManager; - private volatile Cache serverCaches; - - public TenantCacheImpl(MemoryManager memoryManager, int maxTimeToLiveMs) { - this.memoryManager = memoryManager; - this.maxTimeToLiveMs = maxTimeToLiveMs; - } - - @Override - public MemoryManager getMemoryManager() { - return memoryManager; - } - - private Cache getServerCaches() { - /* Delay creation of this map until it's needed */ - if (serverCaches == null) { - synchronized(this) { - if (serverCaches == null) { - serverCaches = CacheBuilder.newBuilder() - .expireAfterAccess(maxTimeToLiveMs, TimeUnit.MILLISECONDS) - .removalListener(new RemovalListener(){ - @Override - public void onRemoval(RemovalNotification notification) { - Closeables.closeAllQuietly(Collections.singletonList(notification.getValue())); - } - }) - .build(); - } - } - } - return serverCaches; - } - - @Override - public Closeable getServerCache(ImmutableBytesPtr cacheId) { - return getServerCaches().getIfPresent(cacheId); - } - - @Override - public Closeable addServerCache(ImmutableBytesPtr cacheId, ImmutableBytesWritable cachePtr, ServerCacheFactory cacheFactory) throws SQLException { - MemoryChunk chunk = this.getMemoryManager().allocate(cachePtr.getLength()); - Closeable element = cacheFactory.newCache(cachePtr, chunk); - getServerCaches().put(cacheId, element); - return element; - } - - @Override - public void removeServerCache(ImmutableBytesPtr cacheId) throws SQLException { - getServerCaches().invalidate(cacheId); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/client/ClientKeyValueBuilder.java b/phoenix-core/src/main/java/com/salesforce/phoenix/client/ClientKeyValueBuilder.java deleted file mode 100644 index 22b53e4c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/client/ClientKeyValueBuilder.java +++ /dev/null @@ -1,75 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.client; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.KeyValue.Type; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; - -/** - * A {@link KeyValueBuilder} that builds {@link ClientKeyValue}, eliminating the extra byte copies - * inherent in the standard {@link KeyValue} implementation. - *

- * This {@link KeyValueBuilder} is only supported in HBase 0.94.14+ ( - * {@link PhoenixDatabaseMetaData#CLIENT_KEY_VALUE_BUILDER_THRESHOLD}), with the addition of - * HBASE-9834. - */ -public class ClientKeyValueBuilder extends KeyValueBuilder { - - public static final KeyValueBuilder INSTANCE = new ClientKeyValueBuilder(); - - private ClientKeyValueBuilder() { - // private ctor for singleton - } - - @Override - public KeyValue buildPut(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier, long ts, ImmutableBytesWritable value) { - return new ClientKeyValue(row, family, qualifier, ts, Type.Put, value); - } - - @Override - public KeyValue buildDeleteFamily(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier, long ts) { - return new ClientKeyValue(row, family, qualifier, ts, Type.DeleteFamily, null); - } - - @Override - public KeyValue buildDeleteColumns(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier, long ts) { - return new ClientKeyValue(row, family, qualifier, ts, Type.DeleteColumn, null); - } - - @Override - public KeyValue buildDeleteColumn(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier, long ts) { - return new ClientKeyValue(row, family, qualifier, ts, Type.Delete, null); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/client/GenericKeyValueBuilder.java b/phoenix-core/src/main/java/com/salesforce/phoenix/client/GenericKeyValueBuilder.java deleted file mode 100644 index b5529759..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/client/GenericKeyValueBuilder.java +++ /dev/null @@ -1,79 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.client; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.KeyValue.Type; -import org.apache.hadoop.hbase.client.Delete; -import org.apache.hadoop.hbase.client.Put; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import static com.salesforce.hbase.index.util.ImmutableBytesPtr.copyBytesIfNecessary; - -/** - * {@link KeyValueBuilder} that does simple byte[] copies to build the underlying key-value. This is - * exactly the same behavior as currently used in {@link Delete} and {@link Put}. - */ -public class GenericKeyValueBuilder extends KeyValueBuilder { - - public static final KeyValueBuilder INSTANCE = new GenericKeyValueBuilder(); - - private GenericKeyValueBuilder() { - // private ctor for singleton - } - - @Override - public KeyValue buildPut(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier, long ts, ImmutableBytesWritable value) { - return build(row, family, qualifier, ts, Type.Put, value); - } - - @Override - public KeyValue buildDeleteFamily(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier, long ts) { - return build(row, family, qualifier, ts, Type.DeleteFamily, null); - } - - @Override - public KeyValue buildDeleteColumns(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier, long ts) { - return build(row, family, qualifier, ts, Type.DeleteColumn, null); - } - - @Override - public KeyValue buildDeleteColumn(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier, long ts) { - return build(row, family, qualifier, ts, Type.Delete, null); - } - - private KeyValue build(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier, long ts, KeyValue.Type type, ImmutableBytesWritable value) { - return new KeyValue(copyBytesIfNecessary(row), copyBytesIfNecessary(family), - copyBytesIfNecessary(qualifier), ts, type, value == null? null: copyBytesIfNecessary(value)); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/client/KeyValueBuilder.java b/phoenix-core/src/main/java/com/salesforce/phoenix/client/KeyValueBuilder.java deleted file mode 100644 index 6a57b295..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/client/KeyValueBuilder.java +++ /dev/null @@ -1,116 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.client; - -import java.io.IOException; - -import org.apache.hadoop.hbase.HConstants; -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.client.Delete; -import org.apache.hadoop.hbase.client.Put; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.util.MetaDataUtil; - -/** - * Build {@link KeyValue} in an efficient way - */ -public abstract class KeyValueBuilder { - - /** - * Helper method for a {@link KeyValueBuilder} that catches an IOException from a {@link Put} - * when adding a {@link KeyValue} generated by the KeyValueBuilder. - * @throws RuntimeException if there is an IOException thrown from the underlying {@link Put} - */ - @SuppressWarnings("javadoc") - public static void addQuietly(Put put, KeyValueBuilder builder, KeyValue kv) { - try { - put.add(kv); - } catch (IOException e) { - throw new RuntimeException("KeyValue Builder " + builder + " created an invalid kv: " - + kv + "!"); - } - } - - /** - * Helper method for a {@link KeyValueBuilder} that catches an IOException from a {@link Put} - * when adding a {@link KeyValue} generated by the KeyValueBuilder. - * @throws RuntimeException if there is an IOException thrown from the underlying {@link Put} - */ - @SuppressWarnings("javadoc") - public static void deleteQuietly(Delete delete, KeyValueBuilder builder, KeyValue kv) { - try { - delete.addDeleteMarker(kv); - } catch (IOException e) { - throw new RuntimeException("KeyValue Builder " + builder + " created an invalid kv: " - + kv + "!"); - } - } - - private static final int CUSTOM_KEY_VALUE_MIN_VERSION = MetaDataUtil.encodeVersion("0.94.14"); - - public static KeyValueBuilder get(String hbaseVersion) { - int version = MetaDataUtil.encodeVersion(hbaseVersion); - if (version >= CUSTOM_KEY_VALUE_MIN_VERSION) { - return ClientKeyValueBuilder.INSTANCE; - } - return GenericKeyValueBuilder.INSTANCE; - } - - public KeyValue buildPut(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier, ImmutableBytesWritable value) { - return buildPut(row, family, qualifier, HConstants.LATEST_TIMESTAMP, value); - } - - public abstract KeyValue buildPut(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier, long ts, ImmutableBytesWritable value); - - public KeyValue buildDeleteFamily(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier) { - return buildDeleteFamily(row, family, qualifier, HConstants.LATEST_TIMESTAMP); - } - - public abstract KeyValue buildDeleteFamily(ImmutableBytesWritable row, - ImmutableBytesWritable family, ImmutableBytesWritable qualifier, long ts); - - public KeyValue buildDeleteColumns(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier) { - return buildDeleteColumns(row, family, qualifier, HConstants.LATEST_TIMESTAMP); - } - - public abstract KeyValue buildDeleteColumns(ImmutableBytesWritable row, - ImmutableBytesWritable family, ImmutableBytesWritable qualifier, long ts); - - public KeyValue buildDeleteColumn(ImmutableBytesWritable row, ImmutableBytesWritable family, - ImmutableBytesWritable qualifier) { - return buildDeleteColumn(row, family, qualifier, HConstants.LATEST_TIMESTAMP); - } - - public abstract KeyValue buildDeleteColumn(ImmutableBytesWritable row, - ImmutableBytesWritable family, ImmutableBytesWritable qualifier, long ts); -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/AggregationManager.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/AggregationManager.java deleted file mode 100644 index 0149886b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/AggregationManager.java +++ /dev/null @@ -1,61 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import com.salesforce.phoenix.expression.aggregator.ClientAggregators; - -/** - * - * Class that manages aggregations during query compilation - * - * @author jtaylor - * @since 0.1 - */ -public class AggregationManager { - private ClientAggregators aggregators; - private int position = 0; - - public AggregationManager() { - } - - public ClientAggregators getAggregators() { - return aggregators; - } - - /** - * @return allocate the next available zero-based positional index - * for the client-side aggregate function. - */ - protected int nextPosition() { - return position++; - } - - public void setAggregators(ClientAggregators clientAggregator) { - this.aggregators = clientAggregator; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/BindManager.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/BindManager.java deleted file mode 100644 index 8206597d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/BindManager.java +++ /dev/null @@ -1,90 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.ParameterMetaData; -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.jdbc.PhoenixParameterMetaData; -import com.salesforce.phoenix.parse.BindParseNode; -import com.salesforce.phoenix.schema.PDatum; - - -/** - * - * Class that manages binding parameters and checking type matching. There are - * two main usages: - * - * 1) the standard query case where we have the values for the binds. - * 2) the retrieve param metadata case where we don't have the bind values. - * - * In both cases, during query compilation we figure out what type the bind variable - * "should" be, based on how it's used in the query. For example foo < ? would expect - * that the bind variable type matches or can be coerced to the type of foo. For (1), - * we check that the bind value has the correct type and for (2) we set the param - * metadata type. - * - * @author jtaylor - * @since 0.1 - */ -public class BindManager { - public static final Object UNBOUND_PARAMETER = new Object(); - - private final List binds; - private final PhoenixParameterMetaData bindMetaData; - - public BindManager(List binds) { - this.binds = binds; - this.bindMetaData = new PhoenixParameterMetaData(binds.size()); - } - - public ParameterMetaData getParameterMetaData() { - return bindMetaData; - } - - public Object getBindValue(BindParseNode node) throws SQLException { - int index = node.getIndex(); - if (index < 0 || index >= binds.size()) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.PARAM_INDEX_OUT_OF_BOUND) - .setMessage("binds size: " + binds.size() + "; index: " + index).build().buildException(); - } - Object value = binds.get(index); - if (value == UNBOUND_PARAMETER) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.PARAM_VALUE_UNBOUND) - .setMessage(node.toString()).build().buildException(); - } - return value; - } - - public void addParamMetaData(BindParseNode bind, PDatum column) throws SQLException { - bindMetaData.addParam(bind,column); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ColumnProjector.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ColumnProjector.java deleted file mode 100644 index 0c155fb6..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ColumnProjector.java +++ /dev/null @@ -1,78 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.SQLException; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - - -/** - * - * Interface used to access the value of a projected column. - * - * @author jtaylor - * @since 0.1 - */ -public interface ColumnProjector { - /** - * Get the column name as it was referenced in the query - * @return the database column name - */ - String getName(); - - /** - * Get the expression - * @return the expression for the column projector - */ - public Expression getExpression(); - - // TODO: An expression may contain references to multiple tables. - /** - * Get the name of the hbase table containing the column - * @return the hbase table name - */ - String getTableName(); - - /** - * Get the value of the column, coercing it if necessary to the specified type - * @param tuple the row containing the column - * @param type the type to which to coerce the binary value - * @param ptr used to retrieve the value - * @return the object representation of the column value. - * @throws SQLException - */ - Object getValue(Tuple tuple, PDataType type, ImmutableBytesWritable ptr) throws SQLException; - - boolean isCaseSensitive(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ColumnResolver.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ColumnResolver.java deleted file mode 100644 index 93ef389d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ColumnResolver.java +++ /dev/null @@ -1,63 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.schema.ColumnRef; -import com.salesforce.phoenix.schema.TableRef; - - - -/** - * - * Interface used to resolve column references occurring - * in the select statement. - * - * @author jtaylor - * @since 0.1 - */ -public interface ColumnResolver { - - /** - * Returns the collection of resolved tables in the FROM clause. - */ - public List getTables(); - - /** - * Resolves column using name and alias. - * @param schemaName TODO - * @param tableName TODO - * @param colName TODO - * @return the resolved ColumnRef - * @throws ColumnNotFoundException if the column could not be resolved - * @throws AmbiguousColumnException if the column name is ambiguous - */ - public ColumnRef resolveColumn(String schemaName, String tableName, String colName) throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/CreateIndexCompiler.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/CreateIndexCompiler.java deleted file mode 100644 index d5d5ceff..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/CreateIndexCompiler.java +++ /dev/null @@ -1,96 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.ParameterMetaData; -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -import org.apache.hadoop.hbase.client.Scan; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.parse.CreateIndexStatement; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.schema.MetaDataClient; - -public class CreateIndexCompiler { - private final PhoenixStatement statement; - - public CreateIndexCompiler(PhoenixStatement statement) { - this.statement = statement; - } - - public MutationPlan compile(final CreateIndexStatement create) throws SQLException { - final PhoenixConnection connection = statement.getConnection(); - final ColumnResolver resolver = FromCompiler.getResolver(create, connection); - Scan scan = new Scan(); - final StatementContext context = new StatementContext(statement, resolver, statement.getParameters(), scan); - ExpressionCompiler expressionCompiler = new ExpressionCompiler(context); - List splitNodes = create.getSplitNodes(); - final byte[][] splits = new byte[splitNodes.size()][]; - for (int i = 0; i < splits.length; i++) { - ParseNode node = splitNodes.get(i); - if (!node.isStateless()) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.SPLIT_POINT_NOT_CONSTANT) - .setMessage("Node: " + node).build().buildException(); - } - LiteralExpression expression = (LiteralExpression)node.accept(expressionCompiler); - splits[i] = expression.getBytes(); - } - final MetaDataClient client = new MetaDataClient(connection); - - return new MutationPlan() { - - @Override - public ParameterMetaData getParameterMetaData() { - return context.getBindManager().getParameterMetaData(); - } - - @Override - public PhoenixConnection getConnection() { - return connection; - } - - @Override - public MutationState execute() throws SQLException { - return client.createIndex(create, splits); - } - - @Override - public ExplainPlan getExplainPlan() throws SQLException { - return new ExplainPlan(Collections.singletonList("CREATE INDEX")); - } - }; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/CreateSequenceCompiler.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/CreateSequenceCompiler.java deleted file mode 100644 index 0b5899d3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/CreateSequenceCompiler.java +++ /dev/null @@ -1,221 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.ParameterMetaData; -import java.sql.SQLException; -import java.util.Collections; - -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.parse.BindParseNode; -import com.salesforce.phoenix.parse.CreateSequenceStatement; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.MetaDataClient; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDatum; - - -public class CreateSequenceCompiler { - private final PhoenixStatement statement; - - public CreateSequenceCompiler(PhoenixStatement statement) { - this.statement = statement; - } - - private static class LongDatum implements PDatum { - - @Override - public boolean isNullable() { - return false; - } - - @Override - public PDataType getDataType() { - return PDataType.LONG; - } - - @Override - public Integer getByteSize() { - return PDataType.LONG.getByteSize(); - } - - @Override - public Integer getMaxLength() { - return null; - } - - @Override - public Integer getScale() { - return null; - } - - @Override - public ColumnModifier getColumnModifier() { - return null; - } - - } - private static class IntegerDatum implements PDatum { - - @Override - public boolean isNullable() { - return false; - } - - @Override - public PDataType getDataType() { - return PDataType.INTEGER; - } - - @Override - public Integer getByteSize() { - return PDataType.INTEGER.getByteSize(); - } - - @Override - public Integer getMaxLength() { - return null; - } - - @Override - public Integer getScale() { - return null; - } - - @Override - public ColumnModifier getColumnModifier() { - return null; - } - - } - private static final PDatum LONG_DATUM = new LongDatum(); - private static final PDatum INTEGER_DATUM = new IntegerDatum(); - - public MutationPlan compile(final CreateSequenceStatement sequence) throws SQLException { - ParseNode startsWithNode = sequence.getStartWith(); - ParseNode incrementByNode = sequence.getIncrementBy(); - if (!startsWithNode.isStateless()) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.STARTS_WITH_MUST_BE_CONSTANT) - .setSchemaName(sequence.getSequenceName().getSchemaName()) - .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); - } - if (!incrementByNode.isStateless()) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.INCREMENT_BY_MUST_BE_CONSTANT) - .setSchemaName(sequence.getSequenceName().getSchemaName()) - .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); - } - ParseNode cacheNode = sequence.getCacheSize(); - if (cacheNode != null && !cacheNode.isStateless()) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.CACHE_MUST_BE_NON_NEGATIVE_CONSTANT) - .setSchemaName(sequence.getSequenceName().getSchemaName()) - .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); - } - - final PhoenixConnection connection = statement.getConnection(); - final ColumnResolver resolver = FromCompiler.EMPTY_TABLE_RESOLVER; - - final StatementContext context = new StatementContext(statement, resolver, statement.getParameters(), new Scan()); - if (startsWithNode instanceof BindParseNode) { - context.getBindManager().addParamMetaData((BindParseNode)startsWithNode, LONG_DATUM); - } - if (incrementByNode instanceof BindParseNode) { - context.getBindManager().addParamMetaData((BindParseNode)incrementByNode, LONG_DATUM); - } - if (cacheNode instanceof BindParseNode) { - context.getBindManager().addParamMetaData((BindParseNode)cacheNode, INTEGER_DATUM); - } - ExpressionCompiler expressionCompiler = new ExpressionCompiler(context); - Expression startsWithExpr = startsWithNode.accept(expressionCompiler); - ImmutableBytesWritable ptr = context.getTempPtr(); - startsWithExpr.evaluate(null, ptr); - if (ptr.getLength() == 0 || !startsWithExpr.getDataType().isCoercibleTo(PDataType.LONG)) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.STARTS_WITH_MUST_BE_CONSTANT) - .setSchemaName(sequence.getSequenceName().getSchemaName()) - .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); - } - final long startsWith = (Long)PDataType.LONG.toObject(ptr, startsWithExpr.getDataType()); - - Expression incrementByExpr = incrementByNode.accept(expressionCompiler); - incrementByExpr.evaluate(null, ptr); - if (ptr.getLength() == 0 || !incrementByExpr.getDataType().isCoercibleTo(PDataType.LONG)) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.INCREMENT_BY_MUST_BE_CONSTANT) - .setSchemaName(sequence.getSequenceName().getSchemaName()) - .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); - } - final long incrementBy = (Long)PDataType.LONG.toObject(ptr, incrementByExpr.getDataType()); - - int cacheSizeValue = connection.getQueryServices().getProps().getInt(QueryServices.SEQUENCE_CACHE_SIZE_ATTRIB,QueryServicesOptions.DEFAULT_SEQUENCE_CACHE_SIZE); - if (cacheNode != null) { - Expression cacheSizeExpr = cacheNode.accept(expressionCompiler); - cacheSizeExpr.evaluate(null, ptr); - if (ptr.getLength() != 0 && (!cacheSizeExpr.getDataType().isCoercibleTo(PDataType.INTEGER) || (cacheSizeValue = (Integer)PDataType.INTEGER.toObject(ptr)) < 0)) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.CACHE_MUST_BE_NON_NEGATIVE_CONSTANT) - .setSchemaName(sequence.getSequenceName().getSchemaName()) - .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); - } - } - final int cacheSize = Math.max(1, cacheSizeValue); - - - final MetaDataClient client = new MetaDataClient(connection); - return new MutationPlan() { - - @Override - public MutationState execute() throws SQLException { - return client.createSequence(sequence, startsWith, incrementBy, cacheSize); - } - - @Override - public ExplainPlan getExplainPlan() throws SQLException { - return new ExplainPlan(Collections.singletonList("CREATE SEQUENCE")); - } - - @Override - public PhoenixConnection getConnection() { - return connection; - } - - @Override - public ParameterMetaData getParameterMetaData() { - return context.getBindManager().getParameterMetaData(); - } - - }; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/CreateTableCompiler.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/CreateTableCompiler.java deleted file mode 100644 index 39e1b7db..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/CreateTableCompiler.java +++ /dev/null @@ -1,230 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.ParameterMetaData; -import java.sql.SQLException; -import java.util.Collections; -import java.util.Iterator; -import java.util.List; - -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.google.common.collect.Iterators; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.expression.AndExpression; -import com.salesforce.phoenix.expression.ComparisonExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.IsNullExpression; -import com.salesforce.phoenix.expression.KeyValueColumnExpression; -import com.salesforce.phoenix.expression.RowKeyColumnExpression; -import com.salesforce.phoenix.expression.visitor.TraverseNoExpressionVisitor; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.parse.CreateTableStatement; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.query.DelegateConnectionQueryServices; -import com.salesforce.phoenix.schema.MetaDataClient; -import com.salesforce.phoenix.schema.PMetaData; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTable.ViewType; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.ByteUtil; - - -public class CreateTableCompiler { - private final PhoenixStatement statement; - - public CreateTableCompiler(PhoenixStatement statement) { - this.statement = statement; - } - - public MutationPlan compile(final CreateTableStatement create) throws SQLException { - final PhoenixConnection connection = statement.getConnection(); - ColumnResolver resolver = FromCompiler.getResolver(create, connection); - PTableType type = create.getTableType(); - PhoenixConnection connectionToBe = connection; - PTable parentToBe = null; - ViewType viewTypeToBe = null; - Scan scan = new Scan(); - final StatementContext context = new StatementContext(statement, resolver, statement.getParameters(), scan); - ExpressionCompiler expressionCompiler = new ExpressionCompiler(context); - ParseNode whereNode = create.getWhereClause(); - Expression where = null; - if (type == PTableType.VIEW) { - TableRef tableRef = resolver.getTables().get(0); - parentToBe = tableRef.getTable(); - viewTypeToBe = parentToBe.getViewType() == ViewType.MAPPED ? ViewType.MAPPED : ViewType.UPDATABLE; - if (whereNode != null) { - if (whereNode.isStateless()) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.VIEW_WHERE_IS_CONSTANT) - .build().buildException(); - } - whereNode = StatementNormalizer.normalize(whereNode, resolver); - where = whereNode.accept(expressionCompiler); - if (expressionCompiler.isAggregate()) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.AGGREGATE_IN_WHERE).build().buildException(); - } - if (viewTypeToBe != ViewType.MAPPED) { - Long scn = connection.getSCN(); - connectionToBe = scn != null ? connection : - // If we haved no SCN on our connection, freeze the SCN at when - // the base table was resolved to prevent any race condition on - // the error checking we do for the base table. The only potential - // issue is if the base table lives on a different region server - // than the new table will, then we're relying here on the system - // clocks being in sync. - new PhoenixConnection( - // When the new table is created, we still want to cache it - // on our connection. - new DelegateConnectionQueryServices(connection.getQueryServices()) { - @Override - public PMetaData addTable(PTable table) throws SQLException { - return connection.addTable(table); - } - }, - connection, tableRef.getTimeStamp()); - ViewWhereExpressionVisitor visitor = new ViewWhereExpressionVisitor(); - where.accept(visitor); - viewTypeToBe = visitor.isUpdatable() ? ViewType.UPDATABLE : ViewType.READ_ONLY; - } - } - } - final Expression viewExpression = where; - final ViewType viewType = viewTypeToBe; - List splitNodes = create.getSplitNodes(); - final byte[][] splits = new byte[splitNodes.size()][]; - ImmutableBytesWritable ptr = context.getTempPtr(); - for (int i = 0; i < splits.length; i++) { - ParseNode node = splitNodes.get(i); - if (node.isStateless()) { - Expression expression = node.accept(expressionCompiler); - if (expression.evaluate(null, ptr)) {; - splits[i] = ByteUtil.copyKeyBytesIfNecessary(ptr); - continue; - } - } - throw new SQLExceptionInfo.Builder(SQLExceptionCode.SPLIT_POINT_NOT_CONSTANT) - .setMessage("Node: " + node).build().buildException(); - } - final MetaDataClient client = new MetaDataClient(connectionToBe); - final PTable parent = parentToBe; - - return new MutationPlan() { - - @Override - public ParameterMetaData getParameterMetaData() { - return context.getBindManager().getParameterMetaData(); - } - - @Override - public MutationState execute() throws SQLException { - try { - return client.createTable(create, splits, parent, viewExpression, viewType); - } finally { - if (client.getConnection() != connection) { - client.getConnection().close(); - } - } - } - - @Override - public ExplainPlan getExplainPlan() throws SQLException { - return new ExplainPlan(Collections.singletonList("CREATE TABLE")); - } - - @Override - public PhoenixConnection getConnection() { - return connection; - } - - }; - } - - private static class ViewWhereExpressionVisitor extends TraverseNoExpressionVisitor { - private boolean isUpdatable = true; - - public boolean isUpdatable() { - return isUpdatable; - } - - @Override - public Boolean defaultReturn(Expression node, List l) { - // We only hit this if we're trying to traverse somewhere - // in which we don't have a visitLeave that returns non null - isUpdatable = false; - return null; - } - - @Override - public Iterator visitEnter(AndExpression node) { - return node.getChildren().iterator(); - } - - @Override - public Boolean visitLeave(AndExpression node, List l) { - return l.isEmpty() ? null : Boolean.TRUE; - } - - @Override - public Iterator visitEnter(ComparisonExpression node) { - return node.getFilterOp() == CompareOp.EQUAL && node.getChildren().get(1).isStateless() && node.getChildren().get(1).isDeterministic() ? Iterators.singletonIterator(node.getChildren().get(0)) : super.visitEnter(node); - } - - @Override - public Boolean visitLeave(ComparisonExpression node, List l) { - return l.isEmpty() ? null : Boolean.TRUE; - } - - @Override - public Iterator visitEnter(IsNullExpression node) { - return node.isNegate() ? super.visitEnter(node) : node.getChildren().iterator(); - } - - @Override - public Boolean visitLeave(IsNullExpression node, List l) { - return l.isEmpty() ? null : Boolean.TRUE; - } - - @Override - public Boolean visit(RowKeyColumnExpression node) { - return Boolean.TRUE; - } - - @Override - public Boolean visit(KeyValueColumnExpression node) { - return Boolean.TRUE; - } - - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/DropSequenceCompiler.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/DropSequenceCompiler.java deleted file mode 100644 index 568712e0..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/DropSequenceCompiler.java +++ /dev/null @@ -1,77 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.ParameterMetaData; -import java.sql.SQLException; -import java.util.Collections; - -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixParameterMetaData; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.parse.DropSequenceStatement; -import com.salesforce.phoenix.schema.MetaDataClient; - - -public class DropSequenceCompiler { - private final PhoenixStatement statement; - - public DropSequenceCompiler(PhoenixStatement statement) { - this.statement = statement; - } - - - public MutationPlan compile(final DropSequenceStatement sequence) throws SQLException { - final PhoenixConnection connection = statement.getConnection(); - final MetaDataClient client = new MetaDataClient(connection); - return new MutationPlan() { - - @Override - public MutationState execute() throws SQLException { - return client.dropSequence(sequence); - } - - @Override - public ExplainPlan getExplainPlan() throws SQLException { - return new ExplainPlan(Collections.singletonList("DROP SEQUENCE")); - } - - @Override - public PhoenixConnection getConnection() { - return connection; - } - - @Override - public ParameterMetaData getParameterMetaData() { - return PhoenixParameterMetaData.EMPTY_PARAMETER_META_DATA; - } - - }; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExplainPlan.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExplainPlan.java deleted file mode 100644 index a9900d51..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExplainPlan.java +++ /dev/null @@ -1,47 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.util.Collections; -import java.util.List; - -import com.google.common.collect.ImmutableList; - -public class ExplainPlan { - public static final ExplainPlan EMPTY_PLAN = new ExplainPlan(Collections.emptyList()); - - private final List planSteps; - - public ExplainPlan(List planSteps) { - this.planSteps = ImmutableList.copyOf(planSteps); - } - - public List getPlanSteps() { - return planSteps; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExpressionManager.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExpressionManager.java deleted file mode 100644 index 4a923fbe..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExpressionManager.java +++ /dev/null @@ -1,78 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.util.Iterator; -import java.util.Map; - - -import com.google.common.collect.Maps; -import com.salesforce.phoenix.expression.Expression; - -/** - * - * Class to manage list of expressions inside of a select statement by - * deduping them. - * - * @author jtaylor - * @since 0.1 - */ -public class ExpressionManager { - // Use a Map instead of a Set because we need to get and return - // the existing Expression - private final Map expressionMap; - - public ExpressionManager() { - expressionMap = Maps.newHashMap(); - } - - /** - * Add the expression to the set of known expressions for the select - * clause. If the expression is already in the set, then the new one - * passed in is ignored. - * @param expression the new expression to add - * @return the new expression if not already present in the set and - * the existing one otherwise. - */ - public Expression addIfAbsent(Expression expression) { - Expression existingExpression = expressionMap.get(expression); - if (existingExpression == null) { - expressionMap.put(expression, expression); - return expression; - } - return existingExpression; - } - - public int getExpressionCount() { - return expressionMap.size(); - } - - public Iterator getExpressions() { - return expressionMap.keySet().iterator(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExpressionProjector.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExpressionProjector.java deleted file mode 100644 index 384787bb..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExpressionProjector.java +++ /dev/null @@ -1,101 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - - -import java.sql.SQLException; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - - -/** - * - * Projector for getting value from a select statement for an expression - * - * @author jtaylor - * @since 0.1 - */ -public class ExpressionProjector implements ColumnProjector { - private final String name; - private final Expression expression; - private final String tableName; - private final boolean isCaseSensitive; - - public ExpressionProjector(String name, String tableName, Expression expression, boolean isCaseSensitive) { - this.name = name; - this.expression = expression; - this.tableName = tableName; - this.isCaseSensitive = isCaseSensitive; - } - - @Override - public String getTableName() { - return tableName; - } - - @Override - public Expression getExpression() { - return expression; - } - - @Override - public String getName() { - return name; - } - - @Override - public final Object getValue(Tuple tuple, PDataType type, ImmutableBytesWritable ptr) throws SQLException { - try { - Expression expression = getExpression(); - if (!expression.evaluate(tuple, ptr)) { - return null; - } - if (ptr.getLength() == 0) { - return null; - } - return type.toObject(ptr, expression.getDataType(), expression.getColumnModifier()); - } catch (RuntimeException e) { - // FIXME: Expression.evaluate does not throw SQLException - // so this will unwrap throws from that. - if (e.getCause() instanceof SQLException) { - throw (SQLException) e.getCause(); - } - throw e; - } - } - - @Override - public boolean isCaseSensitive() { - return isCaseSensitive; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/HavingCompiler.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/HavingCompiler.java deleted file mode 100644 index 0d8a0003..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/HavingCompiler.java +++ /dev/null @@ -1,249 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.SQLException; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Set; - -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.parse.AddParseNode; -import com.salesforce.phoenix.parse.AndParseNode; -import com.salesforce.phoenix.parse.BetweenParseNode; -import com.salesforce.phoenix.parse.CaseParseNode; -import com.salesforce.phoenix.parse.ColumnParseNode; -import com.salesforce.phoenix.parse.ComparisonParseNode; -import com.salesforce.phoenix.parse.DivideParseNode; -import com.salesforce.phoenix.parse.FunctionParseNode; -import com.salesforce.phoenix.parse.IsNullParseNode; -import com.salesforce.phoenix.parse.MultiplyParseNode; -import com.salesforce.phoenix.parse.OrParseNode; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.parse.SelectStatementRewriter; -import com.salesforce.phoenix.parse.SubtractParseNode; -import com.salesforce.phoenix.parse.TraverseNoParseNodeVisitor; -import com.salesforce.phoenix.schema.ColumnRef; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.TypeMismatchException; - - -public class HavingCompiler { - - private HavingCompiler() { - } - - public static Expression compile(StatementContext context, SelectStatement statement, GroupBy groupBy) throws SQLException { - ParseNode having = statement.getHaving(); - if (having == null) { - return null; - } - ExpressionCompiler expressionBuilder = new ExpressionCompiler(context, groupBy); - Expression expression = having.accept(expressionBuilder); - if (expression.getDataType() != PDataType.BOOLEAN) { - throw TypeMismatchException.newException(PDataType.BOOLEAN, expression.getDataType(), expression.toString()); - } - if (LiteralExpression.isFalse(expression)) { - context.setScanRanges(ScanRanges.NOTHING); - return null; - } else if (LiteralExpression.isTrue(expression)) { - return null; - } - if (!expressionBuilder.isAggregate()) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.ONLY_AGGREGATE_IN_HAVING_CLAUSE).build().buildException(); - } - return expression; - } - - public static SelectStatement rewrite(StatementContext context, SelectStatement statement, GroupBy groupBy) throws SQLException { - ParseNode having = statement.getHaving(); - if (having == null) { - return statement; - } - HavingClauseVisitor visitor = new HavingClauseVisitor(context, groupBy); - having.accept(visitor); - statement = SelectStatementRewriter.moveFromHavingToWhereClause(statement, visitor.getMoveToWhereClauseExpressions()); - return statement; - } - - /** - * - * Visitor that figures out if an expression can be moved from the HAVING clause to - * the WHERE clause, since it's more optimal to pre-filter instead of post-filter. - * - * The visitor traverses through AND expressions only and into comparison expresssions. - * If a comparison expression uses a GROUP BY column and does not use any aggregate - * functions, then it's moved. For example, these HAVING expressions would be moved: - * - * select count(1) from atable group by a_string having a_string = 'foo' - * select count(1) from atable group by a_date having round(a_date,'hour') > ? - * select count(1) from atable group by a_date,a_string having a_date > ? or a_string = 'a' - * select count(1) from atable group by a_string,b_string having a_string = 'a' and b_string = 'b' - * - * while these would not be moved: - * - * select count(1) from atable having min(a_integer) < 5 - * select count(1) from atable group by a_string having count(a_string) >= 1 - * select count(1) from atable group by a_date,a_string having a_date > ? or min(a_string) = 'a' - * select count(1) from atable group by a_date having round(min(a_date),'hour') < ? - * - * @author jtaylor - * @since 0.1 - */ - private static class HavingClauseVisitor extends TraverseNoParseNodeVisitor { - private ParseNode topNode = null; - private boolean hasNoAggregateFunctions = true; - private Boolean hasOnlyAggregateColumns; - private final StatementContext context; - private final GroupBy groupBy; - private final Set moveToWhereClause = new LinkedHashSet(); - - HavingClauseVisitor(StatementContext context, GroupBy groupBy) { - this.context = context; - this.groupBy = groupBy; - } - - public Set getMoveToWhereClauseExpressions() { - return moveToWhereClause; - } - - @Override - public boolean visitEnter(AndParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(OrParseNode node) throws SQLException { - enterBooleanNode(node); - return true; - } - - @Override - public boolean visitEnter(ComparisonParseNode node) throws SQLException { - enterBooleanNode(node); - return true; - } - - @Override - public boolean visitEnter(IsNullParseNode node) throws SQLException { - enterBooleanNode(node); - return true; - } - - private void enterBooleanNode(ParseNode node) { - if (topNode == null) { - topNode = node; - } - } - - private void leaveBooleanNode(ParseNode node) { - if (topNode == node) { - if ( hasNoAggregateFunctions && !Boolean.FALSE.equals(hasOnlyAggregateColumns)) { - moveToWhereClause.add(node); - } - hasNoAggregateFunctions = true; - hasOnlyAggregateColumns = null; - topNode = null; - } - } - - @Override - public Void visitLeave(OrParseNode node, List l) throws SQLException { - leaveBooleanNode(node); - return null; - } - - @Override - public Void visitLeave(ComparisonParseNode node, List l) throws SQLException { - leaveBooleanNode(node); - return null; - } - - @Override - public Void visitLeave(IsNullParseNode node, List l) throws SQLException { - leaveBooleanNode(node); - return null; - } - - @Override - public boolean visitEnter(FunctionParseNode node) throws SQLException { - boolean isAggregate = node.isAggregate(); - this.hasNoAggregateFunctions = this.hasNoAggregateFunctions && !isAggregate; - return !isAggregate; - } - - @Override - public boolean visitEnter(CaseParseNode node) throws SQLException { - return true; - } - - @Override - public Void visit(ColumnParseNode node) throws SQLException { - ColumnRef ref = context.getResolver().resolveColumn(node.getSchemaName(), node.getTableName(), node.getName()); - boolean isAggregateColumn = groupBy.getExpressions().indexOf(ref.newColumnExpression()) >= 0; - if (hasOnlyAggregateColumns == null) { - hasOnlyAggregateColumns = isAggregateColumn; - } else { - hasOnlyAggregateColumns &= isAggregateColumn; - } - - return null; - } - - @Override - public boolean visitEnter(SubtractParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(AddParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(MultiplyParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(DivideParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(BetweenParseNode node) throws SQLException { - return true; - } - - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/KeyPart.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/KeyPart.java deleted file mode 100644 index d4622dc2..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/KeyPart.java +++ /dev/null @@ -1,87 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.util.List; - -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.schema.PColumn; - -/** - * - * Interface that determines how a key part contributes to - * the forming of the key (start/stop of scan and SkipScanFilter) - * for each part of a multi-part primary key. It acts as the glue - * between a built-in function and the setting of the scan key - * during query compilation. - * - * @author jtaylor - * @since 0.12 - */ -public interface KeyPart { - /** - * Calculate the key range given an operator and the key on - * the RHS of an expression. For example, given the expression - * SUBSTR(foo,1,3) = 'bar', the key range would be ['bar','bas'), - * and if foo was fixed length, the upper and lower key range - * bytes would be filled out to the fixed length. - * @param op comparison operator (=, <=, <, >=, >, !=) - * @param rhs the constant on the RHS of an expression. - * @return the key range that encompasses the range for the - * expression for which this keyPart is associated - * - * @see com.salesforce.phoenix.expression.function.ScalarFunction#newKeyPart(KeyPart) - */ - public KeyRange getKeyRange(CompareOp op, Expression rhs); - - /** - * Determines whether an expression gets extracted from the - * WHERE clause if it contributes toward the building of the - * scan key. For example, the SUBSTR built-in function may - * be extracted, since it may be completely represented - * through a key range. However, the REGEXP_SUBSTR must be - * left in the WHERE clause, since only the constant prefix - * part of the evaluation can be represented through a key - * range (i.e. rows may pass through that will fail when - * the REGEXP_SUBSTR is evaluated). - * - * @return an empty list if the expression should remain in - * the WHEERE clause for post filtering or a singleton list - * containing the expression if it should be removed. - */ - public List getExtractNodes(); - - /** - * Gets the primary key column associated with the start of this key part. - * @return the primary key column for this key part - */ - public PColumn getColumn(); -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/LimitCompiler.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/LimitCompiler.java deleted file mode 100644 index 538183c1..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/LimitCompiler.java +++ /dev/null @@ -1,118 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.SQLException; - -import com.salesforce.phoenix.parse.*; -import com.salesforce.phoenix.schema.*; - - -public class LimitCompiler { - private static final ParseNodeFactory NODE_FACTORY = new ParseNodeFactory(); - - public static final PDatum LIMIT_DATUM = new PDatum() { - @Override - public boolean isNullable() { - return false; - } - @Override - public PDataType getDataType() { - return PDataType.INTEGER; - } - @Override - public Integer getByteSize() { - return getDataType().getByteSize(); - } - @Override - public Integer getMaxLength() { - return null; - } - @Override - public Integer getScale() { - return null; - } - @Override - public ColumnModifier getColumnModifier() { - return null; - } - }; - - private LimitCompiler() { - } - - public static Integer compile(StatementContext context, FilterableStatement statement) throws SQLException { - LimitNode limitNode = statement.getLimit(); - if (limitNode == null) { - return null; - } - LimitParseNodeVisitor visitor = new LimitParseNodeVisitor(context); - limitNode.getLimitParseNode().accept(visitor); - return visitor.getLimit(); - } - - private static class LimitParseNodeVisitor extends TraverseNoParseNodeVisitor { - private final StatementContext context; - private Integer limit; - - private LimitParseNodeVisitor(StatementContext context) { - this.context = context; - } - - public Integer getLimit() { - return limit; - } - - @Override - public Void visit(LiteralParseNode node) throws SQLException { - Object limitValue = node.getValue(); - // If limit is null, leave this.limit set to zero - // This means that we've bound limit to null for the purpose of - // collecting parameter metadata. - if (limitValue != null) { - Integer limit = (Integer)LIMIT_DATUM.getDataType().toObject(limitValue, node.getType()); - if (limit.intValue() >= 0) { // TODO: handle LIMIT 0 - this.limit = limit; - } - } - return null; - } - - @Override - public Void visit(BindParseNode node) throws SQLException { - Object value = context.getBindManager().getBindValue(node); - context.getBindManager().addParamMetaData(node, LIMIT_DATUM); - // Resolve the bind value, create a LiteralParseNode, and call the visit method for it. - // In this way, we can deal with just having a literal on one side of the expression. - visit(NODE_FACTORY.literal(value, LIMIT_DATUM.getDataType())); - return null; - } - - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/MutatingParallelIteratorFactory.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/MutatingParallelIteratorFactory.java deleted file mode 100644 index a78aa525..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/MutatingParallelIteratorFactory.java +++ /dev/null @@ -1,123 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import static com.salesforce.phoenix.query.QueryConstants.AGG_TIMESTAMP; -import static com.salesforce.phoenix.query.QueryConstants.SINGLE_COLUMN; -import static com.salesforce.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY; -import static com.salesforce.phoenix.query.QueryConstants.UNGROUPED_AGG_ROW_KEY; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.KeyValue; - -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; -import com.salesforce.phoenix.iterate.PeekingResultIterator; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.query.ConnectionQueryServices; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.schema.tuple.SingleKeyValueTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.KeyValueUtil; - -/** - * Factory class used to instantiate an iterator to handle mutations made during a parallel scan. - */ -public abstract class MutatingParallelIteratorFactory implements ParallelIteratorFactory { - protected final PhoenixConnection connection; - protected final TableRef tableRef; - - protected MutatingParallelIteratorFactory(PhoenixConnection connection, TableRef tableRef) { - this.connection = connection; - this.tableRef = tableRef; - } - - /** - * Method that does the actual mutation work - */ - abstract protected MutationState mutate(PhoenixConnection connection, ResultIterator iterator) throws SQLException; - - @Override - public PeekingResultIterator newIterator(ResultIterator iterator) throws SQLException { - // Clone the connection as it's not thread safe and will be operated on in parallel - final PhoenixConnection connection = new PhoenixConnection(this.connection); - MutationState state = mutate(connection, iterator); - long totalRowCount = state.getUpdateCount(); - if (connection.getAutoCommit()) { - connection.getMutationState().join(state); - connection.commit(); - ConnectionQueryServices services = connection.getQueryServices(); - int maxSize = services.getProps().getInt(QueryServices.MAX_MUTATION_SIZE_ATTRIB,QueryServicesOptions.DEFAULT_MAX_MUTATION_SIZE); - state = new MutationState(maxSize, connection, totalRowCount); - } - final MutationState finalState = state; - byte[] value = PDataType.LONG.toBytes(totalRowCount); - KeyValue keyValue = KeyValueUtil.newKeyValue(UNGROUPED_AGG_ROW_KEY, SINGLE_COLUMN_FAMILY, SINGLE_COLUMN, AGG_TIMESTAMP, value, 0, value.length); - final Tuple tuple = new SingleKeyValueTuple(keyValue); - return new PeekingResultIterator() { - private boolean done = false; - - @Override - public Tuple next() throws SQLException { - if (done) { - return null; - } - done = true; - return tuple; - } - - @Override - public void explain(List planSteps) { - } - - @Override - public void close() throws SQLException { - try { - // Join the child mutation states in close, since this is called in a single threaded manner - // after the parallel results have been processed. - if (!connection.getAutoCommit()) { - MutatingParallelIteratorFactory.this.connection.getMutationState().join(finalState); - } - } finally { - connection.close(); - } - } - - @Override - public Tuple peek() throws SQLException { - return done ? null : tuple; - } - }; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/MutationPlan.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/MutationPlan.java deleted file mode 100644 index 36d9dc16..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/MutationPlan.java +++ /dev/null @@ -1,39 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.SQLException; - -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.jdbc.PhoenixConnection; - - -public interface MutationPlan extends StatementPlan { - public PhoenixConnection getConnection(); - public MutationState execute() throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/OrderByCompiler.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/OrderByCompiler.java deleted file mode 100644 index f7c85351..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/OrderByCompiler.java +++ /dev/null @@ -1,138 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - - -import java.sql.SQLException; -import java.util.Collections; -import java.util.LinkedHashSet; -import java.util.List; - -import com.google.common.collect.ImmutableList; -import com.google.common.collect.Lists; -import com.google.common.collect.Sets; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.TrackOrderPreservingExpressionCompiler.Ordering; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.OrderByExpression; -import com.salesforce.phoenix.parse.FilterableStatement; -import com.salesforce.phoenix.parse.OrderByNode; -import com.salesforce.phoenix.schema.ColumnModifier; - -/** - * Validates ORDER BY clause and builds up a list of referenced columns. - * - * @author syyang - * @since 0.1 - */ -public class OrderByCompiler { - public static class OrderBy { - public static final OrderBy EMPTY_ORDER_BY = new OrderBy(Collections.emptyList()); - /** - * Used to indicate that there was an ORDER BY, but it was optimized out because - * rows are already returned in this order. - */ - public static final OrderBy ROW_KEY_ORDER_BY = new OrderBy(Collections.emptyList()); - - private final List orderByExpressions; - - private OrderBy(List orderByExpressions) { - this.orderByExpressions = ImmutableList.copyOf(orderByExpressions); - } - - public List getOrderByExpressions() { - return orderByExpressions; - } - } - /** - * Gets a list of columns in the ORDER BY clause - * @param context the query context for tracking various states - * associated with the given select statement - * @param statement TODO - * @param groupBy the list of columns in the GROUP BY clause - * @param limit the row limit or null if no limit - * @return the compiled ORDER BY clause - * @throws SQLException - */ - public static OrderBy compile(StatementContext context, - FilterableStatement statement, - GroupBy groupBy, Integer limit) throws SQLException { - List orderByNodes = statement.getOrderBy(); - if (orderByNodes.isEmpty()) { - return OrderBy.EMPTY_ORDER_BY; - } - // accumulate columns in ORDER BY - TrackOrderPreservingExpressionCompiler visitor = - new TrackOrderPreservingExpressionCompiler(context, groupBy, - orderByNodes.size(), Ordering.ORDERED); - LinkedHashSet orderByExpressions = Sets.newLinkedHashSetWithExpectedSize(orderByNodes.size()); - for (OrderByNode node : orderByNodes) { - boolean isAscending = node.isAscending(); - Expression expression = node.getNode().accept(visitor); - if (!expression.isStateless() && visitor.addEntry(expression, isAscending ? null : ColumnModifier.SORT_DESC)) { - // Detect mix of aggregate and non aggregates (i.e. ORDER BY txns, SUM(txns) - if (!visitor.isAggregate()) { - if (statement.isAggregate() || statement.isDistinct()) { - // Detect ORDER BY not in SELECT DISTINCT: SELECT DISTINCT count(*) FROM t ORDER BY x - if (statement.isDistinct()) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.ORDER_BY_NOT_IN_SELECT_DISTINCT) - .setMessage(expression.toString()).build().buildException(); - } - ExpressionCompiler.throwNonAggExpressionInAggException(expression.toString()); - } - } - if (expression.getDataType().isArrayType()) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.ORDER_BY_ARRAY_NOT_SUPPORTED) - .setMessage(expression.toString()).build().buildException(); - } - if (expression.getColumnModifier() == ColumnModifier.SORT_DESC) { - isAscending = !isAscending; - } - OrderByExpression orderByExpression = new OrderByExpression(expression, node.isNullsLast(), isAscending); - orderByExpressions.add(orderByExpression); - } - visitor.reset(); - } - - if (orderByExpressions.isEmpty()) { - return OrderBy.EMPTY_ORDER_BY; - } - // If we're ordering by the order returned by the scan, we don't need an order by - if (visitor.isOrderPreserving()) { - return OrderBy.ROW_KEY_ORDER_BY; - } - - return new OrderBy(Lists.newArrayList(orderByExpressions.iterator())); - } - - - private OrderByCompiler() { - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/PostDDLCompiler.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/PostDDLCompiler.java deleted file mode 100644 index 9ac09855..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/PostDDLCompiler.java +++ /dev/null @@ -1,252 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.ParameterMetaData; -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.google.common.collect.Lists; -import com.salesforce.phoenix.cache.ServerCacheClient.ServerCache; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.OrderByCompiler.OrderBy; -import com.salesforce.phoenix.coprocessor.UngroupedAggregateRegionObserver; -import com.salesforce.phoenix.execute.AggregatePlan; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixParameterMetaData; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.SQLParser; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.AmbiguousColumnException; -import com.salesforce.phoenix.schema.ColumnFamilyNotFoundException; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.ColumnRef; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PColumnFamily; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ScanUtil; - - -/** - * - * Class that compiles plan to update data values after a DDL command - * executes. - * - * TODO: get rid of this ugly code and just go through the standard APIs. - * The only time we may still need this is to manage updating the empty - * key value, as we sometimes need to "go back through time" to adjust - * this. - * @author jtaylor - * @since 0.1 - */ -public class PostDDLCompiler { - private final PhoenixConnection connection; - - public PostDDLCompiler(PhoenixConnection connection) { - this.connection = connection; - } - - public MutationPlan compile(final List tableRefs, final byte[] emptyCF, final byte[] projectCF, final List deleteList, - final long timestamp) throws SQLException { - - return new MutationPlan() { - - @Override - public PhoenixConnection getConnection() { - return connection; - } - - @Override - public ParameterMetaData getParameterMetaData() { - return PhoenixParameterMetaData.EMPTY_PARAMETER_META_DATA; - } - - @Override - public ExplainPlan getExplainPlan() throws SQLException { - return ExplainPlan.EMPTY_PLAN; - } - - @Override - public MutationState execute() throws SQLException { - if (tableRefs.isEmpty()) { - return null; - } - boolean wasAutoCommit = connection.getAutoCommit(); - try { - connection.setAutoCommit(true); - SQLException sqlE = null; - if (deleteList == null && emptyCF == null) { - return new MutationState(0, connection); - } - /* - * Handles: - * 1) deletion of all rows for a DROP TABLE and subsequently deletion of all rows for a DROP INDEX; - * 2) deletion of all column values for a ALTER TABLE DROP COLUMN - * 3) updating the necessary rows to have an empty KV - */ - long totalMutationCount = 0; - for (final TableRef tableRef : tableRefs) { - Scan scan = new Scan(); - scan.setAttribute(UngroupedAggregateRegionObserver.UNGROUPED_AGG, QueryConstants.TRUE); - PTable table = tableRef.getTable(); - ParseNode viewNode = SQLParser.parseCondition(table.getViewExpression()); - SelectStatement select = SelectStatement.create(SelectStatement.COUNT_ONE, viewNode); - // We need to use this tableRef - ColumnResolver resolver = new ColumnResolver() { - @Override - public List getTables() { - return Collections.singletonList(tableRef); - } - @Override - public ColumnRef resolveColumn(String schemaName, String tableName, String colName) throws SQLException { - PColumn column = tableName != null - ? tableRef.getTable().getColumnFamily(tableName).getColumn(colName) - : tableRef.getTable().getColumn(colName); - return new ColumnRef(tableRef, column.getPosition()); - } - }; - StatementContext context = new StatementContext(new PhoenixStatement(connection), resolver, Collections.emptyList(), scan); - ScanUtil.setTimeRange(scan, timestamp); - if (emptyCF != null) { - scan.setAttribute(UngroupedAggregateRegionObserver.EMPTY_CF, emptyCF); - } - ServerCache cache = null; - try { - if (deleteList != null) { - if (deleteList.isEmpty()) { - scan.setAttribute(UngroupedAggregateRegionObserver.DELETE_AGG, QueryConstants.TRUE); - // In the case of a row deletion, add index metadata so mutable secondary indexing works - /* TODO - ImmutableBytesWritable ptr = context.getTempPtr(); - tableRef.getTable().getIndexMaintainers(ptr); - if (ptr.getLength() > 0) { - IndexMetaDataCacheClient client = new IndexMetaDataCacheClient(connection, tableRef); - cache = client.addIndexMetadataCache(context.getScanRanges(), ptr); - byte[] uuidValue = cache.getId(); - scan.setAttribute(PhoenixIndexCodec.INDEX_UUID, uuidValue); - } - */ - } else { - // In the case of the empty key value column family changing, do not send the index - // metadata, as we're currently managing this from the client. It's possible for the - // data empty column family to stay the same, while the index empty column family - // changes. - PColumn column = deleteList.get(0); - if (emptyCF == null) { - scan.addColumn(column.getFamilyName().getBytes(), column.getName().getBytes()); - } - scan.setAttribute(UngroupedAggregateRegionObserver.DELETE_CF, column.getFamilyName().getBytes()); - scan.setAttribute(UngroupedAggregateRegionObserver.DELETE_CQ, column.getName().getBytes()); - } - } - List columnFamilies = Lists.newArrayListWithExpectedSize(tableRef.getTable().getColumnFamilies().size()); - if (projectCF == null) { - for (PColumnFamily family : tableRef.getTable().getColumnFamilies()) { - columnFamilies.add(family.getName().getBytes()); - } - } else { - columnFamilies.add(projectCF); - } - // Need to project all column families into the scan, since we haven't yet created our empty key value - RowProjector projector = ProjectionCompiler.compile(context, SelectStatement.COUNT_ONE, GroupBy.EMPTY_GROUP_BY); - // Explicitly project these column families and don't project the empty key value, - // since at this point we haven't added the empty key value everywhere. - if (columnFamilies != null) { - scan.getFamilyMap().clear(); - for (byte[] family : columnFamilies) { - scan.addFamily(family); - } - projector = new RowProjector(projector,false); - } - // Ignore exceptions due to not being able to resolve any view columns, - // as this just means the view is invalid. Continue on and try to perform - // any other Post DDL operations. - try { - WhereCompiler.compile(context, select); // Push where clause into scan - } catch (ColumnFamilyNotFoundException e) { - continue; - } catch (ColumnNotFoundException e) { - continue; - } catch (AmbiguousColumnException e) { - continue; - } - QueryPlan plan = new AggregatePlan(context, select, tableRef, projector, null, OrderBy.EMPTY_ORDER_BY, null, GroupBy.EMPTY_GROUP_BY, null); - ResultIterator iterator = plan.iterator(); - try { - Tuple row = iterator.next(); - ImmutableBytesWritable ptr = context.getTempPtr(); - totalMutationCount += (Long)projector.getColumnProjector(0).getValue(row, PDataType.LONG, ptr); - } catch (SQLException e) { - sqlE = e; - } finally { - try { - iterator.close(); - } catch (SQLException e) { - if (sqlE == null) { - sqlE = e; - } else { - sqlE.setNextException(e); - } - } finally { - if (sqlE != null) { - throw sqlE; - } - } - } - } finally { - if (cache != null) { // Remove server cache if there is one - cache.close(); - } - } - - } - final long count = totalMutationCount; - return new MutationState(1, connection) { - @Override - public long getUpdateCount() { - return count; - } - }; - } finally { - if (!wasAutoCommit) connection.setAutoCommit(wasAutoCommit); - } - } - }; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/PostIndexDDLCompiler.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/PostIndexDDLCompiler.java deleted file mode 100644 index fe15d076..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/PostIndexDDLCompiler.java +++ /dev/null @@ -1,127 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.ParameterMetaData; -import java.sql.PreparedStatement; -import java.sql.SQLException; - -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixParameterMetaData; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.IndexUtil; - - -/** - * Class that compiles plan to generate initial data values after a DDL command for - * index table. - */ -public class PostIndexDDLCompiler { - private final PhoenixConnection connection; - private final TableRef dataTableRef; - - public PostIndexDDLCompiler(PhoenixConnection connection, TableRef dataTableRef) { - this.connection = connection; - this.dataTableRef = dataTableRef; - } - - public MutationPlan compile(final PTable indexTable) throws SQLException { - return new MutationPlan() { - - @Override - public PhoenixConnection getConnection() { - return connection; - } - - @Override - public ParameterMetaData getParameterMetaData() { - return PhoenixParameterMetaData.EMPTY_PARAMETER_META_DATA; - } - - @Override - public ExplainPlan getExplainPlan() throws SQLException { - return ExplainPlan.EMPTY_PLAN; - } - - @Override - public MutationState execute() throws SQLException { - boolean wasAutoCommit = connection.getAutoCommit(); - try { - connection.setAutoCommit(true); - /* - * Handles: - * 1) Populate a newly created table with contents. - * 2) Activate the index by setting the INDEX_STATE to - */ - // NOTE: For first version, we would use a upsert/select to populate the new index table and - // returns synchronously. Creating an index on an existing table with large amount of data - // will as a result take a very very long time. - // In the long term, we should change this to an asynchronous process to populate the index - // that would allow the user to easily monitor the process of index creation. - StringBuilder indexColumns = new StringBuilder(); - StringBuilder dataColumns = new StringBuilder(); - for (PColumn col: dataTableRef.getTable().getColumns()) { - String indexColName = IndexUtil.getIndexColumnName(col); - try { - indexTable.getColumn(indexColName); - if (col.getFamilyName() != null) { - dataColumns.append('"').append(col.getFamilyName()).append("\"."); - } - dataColumns.append('"').append(col.getName()).append("\","); - indexColumns.append('"').append(indexColName).append("\","); - } catch (ColumnNotFoundException e) { - // Catch and ignore - means that this data column is not in the index - } - } - dataColumns.setLength(dataColumns.length()-1); - indexColumns.setLength(indexColumns.length()-1); - String schemaName = dataTableRef.getTable().getSchemaName().getString(); - String tableName = indexTable.getTableName().getString(); - - StringBuilder updateStmtStr = new StringBuilder(); - updateStmtStr.append("UPSERT /*+ NO_INDEX */ INTO ").append(schemaName.length() == 0 ? "" : '"' + schemaName + "\".").append('"').append(tableName).append("\"(") - .append(indexColumns).append(") SELECT ").append(dataColumns).append(" FROM ") - .append(schemaName.length() == 0 ? "" : '"' + schemaName + "\".").append('"').append(dataTableRef.getTable().getTableName().getString()).append('"'); - PreparedStatement updateStmt = connection.prepareStatement(updateStmtStr.toString()); - int rowsUpdated = 0; - updateStmt.execute(); - rowsUpdated = updateStmt.getUpdateCount(); - // Return number of rows built for index - return new MutationState(rowsUpdated, connection); - } finally { - if (!wasAutoCommit) connection.setAutoCommit(false); - } - } - }; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/QueryPlan.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/QueryPlan.java deleted file mode 100644 index 11e0cf22..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/QueryPlan.java +++ /dev/null @@ -1,77 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.OrderByCompiler.OrderBy; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.parse.FilterableStatement; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.schema.TableRef; - - - -/** - * - * Interface for an executable query plan - * - * @author jtaylor - * @since 0.1 - */ -public interface QueryPlan extends StatementPlan { - /** - * Get a result iterator to iterate over the results - * @return result iterator for iterating over the results - * @throws SQLException - */ - public ResultIterator iterator() throws SQLException; - - public long getEstimatedSize(); - - // TODO: change once joins are supported - TableRef getTableRef(); - /** - * Returns projector used to formulate resultSet row - */ - RowProjector getProjector(); - - Integer getLimit(); - - OrderBy getOrderBy(); - - GroupBy getGroupBy(); - - List getSplits(); - - StatementContext getContext(); - - FilterableStatement getStatement(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/RowProjector.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/RowProjector.java deleted file mode 100644 index 6113c37a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/RowProjector.java +++ /dev/null @@ -1,132 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.SQLException; -import java.util.*; - -import com.google.common.collect.Maps; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.util.SchemaUtil; - - -/** - * - * Class that manages a set of projected columns accessed through the zero-based - * column index for a SELECT clause projection. The column index may be looked up - * via the name using {@link #getColumnIndex(String)}. - * - * @author jtaylor - * @since 0.1 - */ -public class RowProjector { - public static final RowProjector EMPTY_PROJECTOR = new RowProjector(Collections.emptyList(),0, true); - - private final List columnProjectors; - private final Map reverseIndex; - private final boolean allCaseSensitive; - private final boolean someCaseSensitive; - private final int estimatedSize; - private final boolean isProjectEmptyKeyValue; - - public RowProjector(RowProjector projector, boolean isProjectEmptyKeyValue) { - this(projector.getColumnProjectors(), projector.getEstimatedRowByteSize(), isProjectEmptyKeyValue); - } - /** - * Construct RowProjector based on a list of ColumnProjectors. - * @param columnProjectors ordered list of ColumnProjectors corresponding to projected columns in SELECT clause - * aggregating coprocessor. Only required in the case of an aggregate query with a limit clause and otherwise may - * be null. - * @param estimatedRowSize - */ - public RowProjector(List columnProjectors, int estimatedRowSize, boolean isProjectEmptyKeyValue) { - this.columnProjectors = Collections.unmodifiableList(columnProjectors); - int position = columnProjectors.size(); - reverseIndex = Maps.newHashMapWithExpectedSize(position); - boolean allCaseSensitive = true; - boolean someCaseSensitive = false; - for (--position; position >= 0; position--) { - ColumnProjector colProjector = columnProjectors.get(position); - allCaseSensitive &= colProjector.isCaseSensitive(); - someCaseSensitive |= colProjector.isCaseSensitive(); - reverseIndex.put(colProjector.getName(), position); - } - this.allCaseSensitive = allCaseSensitive; - this.someCaseSensitive = someCaseSensitive; - this.estimatedSize = estimatedRowSize; - this.isProjectEmptyKeyValue = isProjectEmptyKeyValue; - } - - public boolean isProjectEmptyKeyValue() { - return isProjectEmptyKeyValue; - } - - public List getColumnProjectors() { - return columnProjectors; - } - - public int getColumnIndex(String name) throws SQLException { - if (!someCaseSensitive) { - name = SchemaUtil.normalizeIdentifier(name); - } - Integer index = reverseIndex.get(name); - if (index == null) { - if (!allCaseSensitive && someCaseSensitive) { - name = SchemaUtil.normalizeIdentifier(name); - index = reverseIndex.get(name); - } - if (index == null) { - throw new ColumnNotFoundException(name); - } - } - return index; - } - - public ColumnProjector getColumnProjector(int index) { - return columnProjectors.get(index); - } - - public int getColumnCount() { - return columnProjectors.size(); - } - - @Override - public String toString() { - StringBuilder buf = new StringBuilder("["); - for (ColumnProjector projector : columnProjectors) { - buf.append(projector.getExpression()); - buf.append(','); - } - buf.setCharAt(buf.length()-1, ']'); - return buf.toString(); - } - - public int getEstimatedRowByteSize() { - return estimatedSize; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ScanRanges.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ScanRanges.java deleted file mode 100644 index f58beb77..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ScanRanges.java +++ /dev/null @@ -1,193 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.util.Collections; -import java.util.List; - -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.google.common.collect.ImmutableList; -import com.google.common.collect.Lists; -import com.salesforce.phoenix.filter.SkipScanFilter; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.schema.RowKeySchema; -import com.salesforce.phoenix.util.ScanUtil; - - -public class ScanRanges { - private static final List> EVERYTHING_RANGES = Collections.>emptyList(); - private static final List> NOTHING_RANGES = Collections.>singletonList(Collections.singletonList(KeyRange.EMPTY_RANGE)); - public static final ScanRanges EVERYTHING = new ScanRanges(EVERYTHING_RANGES,null,false); - public static final ScanRanges NOTHING = new ScanRanges(NOTHING_RANGES,null,false); - - public static ScanRanges create(List> ranges, RowKeySchema schema) { - return create(ranges, schema, false); - } - - public static ScanRanges create(List> ranges, RowKeySchema schema, boolean forceRangeScan) { - if (ranges.isEmpty()) { - return EVERYTHING; - } else if (ranges.size() == 1 && ranges.get(0).size() == 1 && ranges.get(0).get(0) == KeyRange.EMPTY_RANGE) { - return NOTHING; - } - return new ScanRanges(ranges, schema, forceRangeScan); - } - - private SkipScanFilter filter; - private final List> ranges; - private final RowKeySchema schema; - private final boolean forceRangeScan; - - private ScanRanges (List> ranges, RowKeySchema schema, boolean forceRangeScan) { - List> sortedRanges = Lists.newArrayListWithExpectedSize(ranges.size()); - for (int i = 0; i < ranges.size(); i++) { - List sorted = Lists.newArrayList(ranges.get(i)); - Collections.sort(sorted, KeyRange.COMPARATOR); - sortedRanges.add(ImmutableList.copyOf(sorted)); - } - this.ranges = ImmutableList.copyOf(sortedRanges); - this.schema = schema; - if (schema != null && !ranges.isEmpty()) { - this.filter = new SkipScanFilter(this.ranges, schema); - } - this.forceRangeScan = forceRangeScan; - } - - public SkipScanFilter getSkipScanFilter() { - return filter; - } - - public List> getRanges() { - return ranges; - } - - public RowKeySchema getSchema() { - return schema; - } - - public boolean isEverything() { - return this == EVERYTHING; - } - - public boolean isDegenerate() { - return this == NOTHING; - } - - /** - * Use SkipScanFilter under two circumstances: - * 1) If we have multiple ranges for a given key slot (use of IN) - * 2) If we have a range (i.e. not a single/point key) that is - * not the last key slot - */ - public boolean useSkipScanFilter() { - if (forceRangeScan) { - return false; - } - boolean hasRangeKey = false, useSkipScan = false; - for (List orRanges : ranges) { - useSkipScan |= orRanges.size() > 1 | hasRangeKey; - if (useSkipScan) { - return true; - } - for (KeyRange range : orRanges) { - hasRangeKey |= !range.isSingleKey(); - } - } - return false; - } - - /** - * @return true if this represents the full key to a single row - */ - public boolean isSingleRowScan() { - if (schema == null || ranges.size() < schema.getMaxFields()) { - return false; - } - boolean isSingleKey = true; - for (List orRanges : ranges) { - if (orRanges.size() > 1) { - return false; - } - isSingleKey &= orRanges.get(0).isSingleKey(); - } - return isSingleKey; - } - - public void setScanStartStopRow(Scan scan) { - if (isEverything()) { - return; - } - if (isDegenerate()) { - scan.setStartRow(KeyRange.EMPTY_RANGE.getLowerRange()); - scan.setStopRow(KeyRange.EMPTY_RANGE.getUpperRange()); - return; - } - - byte[] expectedKey; - expectedKey = ScanUtil.getMinKey(schema, ranges); - if (expectedKey != null) { - scan.setStartRow(expectedKey); - } - expectedKey = ScanUtil.getMaxKey(schema, ranges); - if (expectedKey != null) { - scan.setStopRow(expectedKey); - } - } - - public static final ImmutableBytesWritable UNBOUND = new ImmutableBytesWritable(KeyRange.UNBOUND); - - /** - * Return true if the range formed by the lowerInclusiveKey and upperExclusiveKey - * intersects with any of the scan ranges and false otherwise. We cannot pass in - * a KeyRange here, because the underlying compare functions expect lower inclusive - * and upper exclusive keys. We cannot get their next key because the key must - * conform to the row key schema and if a null byte is added to a lower inclusive - * key, it's no longer a valid, real key. - * @param lowerInclusiveKey lower inclusive key - * @param upperExclusiveKey upper exclusive key - * @return true if the scan range intersects with the specified lower/upper key - * range - */ - public boolean intersect(byte[] lowerInclusiveKey, byte[] upperExclusiveKey) { - if (isEverything()) { - return true; - } - if (isDegenerate()) { - return false; - } - return filter.hasIntersect(lowerInclusiveKey, upperExclusiveKey); - } - - @Override - public String toString() { - return "ScanRanges[" + ranges.toString() + "]"; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/SequenceManager.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/SequenceManager.java deleted file mode 100644 index 0fa4c180..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/SequenceManager.java +++ /dev/null @@ -1,202 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.SQLException; -import java.util.BitSet; -import java.util.Collections; -import java.util.List; -import java.util.Map; - -import org.apache.hadoop.hbase.HConstants; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.google.common.collect.Lists; -import com.google.common.collect.Maps; -import com.salesforce.phoenix.expression.BaseTerminalExpression; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.parse.SequenceValueParseNode; -import com.salesforce.phoenix.parse.TableName; -import com.salesforce.phoenix.query.ConnectionQueryServices; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PName; -import com.salesforce.phoenix.schema.SequenceKey; -import com.salesforce.phoenix.schema.tuple.Tuple; - -public class SequenceManager { - private final PhoenixStatement statement; - private int[] sequencePosition; - private long[] srcSequenceValues; - private long[] dstSequenceValues; - private SQLException[] sqlExceptions; - private List nextSequences; - private List currentSequences; - private Map sequenceMap; - private BitSet isNextSequence; - - public SequenceManager(PhoenixStatement statement) { - this.statement = statement; - } - - public int getSequenceCount() { - return sequenceMap == null ? 0 : sequenceMap.size(); - } - - private void setSequenceValues() throws SQLException { - SQLException eTop = null; - for (int i = 0; i < sqlExceptions.length; i++) { - SQLException e = sqlExceptions[i]; - if (e != null) { - if (eTop == null) { - eTop = e; - } else { - e.setNextException(eTop.getNextException()); - eTop.setNextException(e); - } - } else { - dstSequenceValues[sequencePosition[i]] = srcSequenceValues[i]; - } - } - if (eTop != null) { - throw eTop; - } - } - - public void incrementSequenceValues() throws SQLException { - if (sequenceMap == null) { - return; - } - Long scn = statement.getConnection().getSCN(); - long timestamp = scn == null ? HConstants.LATEST_TIMESTAMP : scn; - ConnectionQueryServices services = this.statement.getConnection().getQueryServices(); - services.incrementSequenceValues(nextSequences, timestamp, srcSequenceValues, sqlExceptions); - setSequenceValues(); - int offset = nextSequences.size(); - for (int i = 0; i < currentSequences.size(); i++) { - dstSequenceValues[sequencePosition[offset+i]] = services.getSequenceValue(currentSequences.get(i), timestamp); - } - } - - public SequenceValueExpression newSequenceReference(SequenceValueParseNode node) { - if (sequenceMap == null) { - sequenceMap = Maps.newHashMap(); - isNextSequence = new BitSet(); - } - PName tenantName = statement.getConnection().getTenantId(); - String tenantId = tenantName == null ? null : tenantName.getString(); - TableName tableName = node.getTableName(); - SequenceKey key = new SequenceKey(tenantId, tableName.getSchemaName(), tableName.getTableName()); - SequenceValueExpression expression = sequenceMap.get(key); - if (expression == null) { - int index = sequenceMap.size(); - expression = new SequenceValueExpression(index); - sequenceMap.put(key, expression); - } - // If we see a NEXT and a CURRENT, treat the CURRENT just like a NEXT - if (node.getOp() == SequenceValueParseNode.Op.NEXT_VALUE) { - isNextSequence.set(expression.getIndex()); - } - - return expression; - } - - public void initSequences() throws SQLException { - if (sequenceMap == null) { - return; - } - int maxSize = sequenceMap.size(); - dstSequenceValues = new long[maxSize]; - sequencePosition = new int[maxSize]; - nextSequences = Lists.newArrayListWithExpectedSize(maxSize); - currentSequences = Lists.newArrayListWithExpectedSize(maxSize); - for (Map.Entry entry : sequenceMap.entrySet()) { - if (isNextSequence.get(entry.getValue().getIndex())) { - nextSequences.add(entry.getKey()); - } else { - currentSequences.add(entry.getKey()); - } - } - srcSequenceValues = new long[nextSequences.size()]; - sqlExceptions = new SQLException[nextSequences.size()]; - Collections.sort(nextSequences); - // Create reverse indexes - for (int i = 0; i < nextSequences.size(); i++) { - sequencePosition[i] = sequenceMap.get(nextSequences.get(i)).getIndex(); - } - int offset = nextSequences.size(); - for (int i = 0; i < currentSequences.size(); i++) { - sequencePosition[i+offset] = sequenceMap.get(currentSequences.get(i)).getIndex(); - } - ConnectionQueryServices services = this.statement.getConnection().getQueryServices(); - Long scn = statement.getConnection().getSCN(); - long timestamp = scn == null ? HConstants.LATEST_TIMESTAMP : scn; - services.reserveSequenceValues(nextSequences, timestamp, srcSequenceValues, sqlExceptions); - setSequenceValues(); - } - - private class SequenceValueExpression extends BaseTerminalExpression { - private final int index; - private final byte[] valueBuffer = new byte[PDataType.LONG.getByteSize()]; - - private SequenceValueExpression(int index) { - this.index = index; - } - - public int getIndex() { - return index; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - PDataType.LONG.getCodec().encodeLong(dstSequenceValues[index], valueBuffer, 0); - ptr.set(valueBuffer); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.LONG; - } - - @Override - public boolean isNullable() { - return false; - } - - @Override - public boolean isDeterministic() { - return false; - } - - @Override - public boolean isStateless() { - return true; - } - - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/StatementContext.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/StatementContext.java deleted file mode 100644 index 0ae42e23..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/StatementContext.java +++ /dev/null @@ -1,235 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.SQLException; -import java.text.Format; -import java.util.List; - -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.filter.FilterList; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.schema.MetaDataClient; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.DateUtil; -import com.salesforce.phoenix.util.NumberUtil; -import com.salesforce.phoenix.util.ScanUtil; - - -/** - * - * Class that keeps common state used across processing the various clauses in a - * top level JDBC statement such as SELECT, UPSERT, DELETE, etc. - * - * @author jtaylor - * @since 0.1 - */ -public class StatementContext { - private ColumnResolver resolver; - private final BindManager binds; - private final Scan scan; - private final ExpressionManager expressions; - private final AggregationManager aggregates; - private final String dateFormat; - private final Format dateFormatter; - private final Format dateParser; - private final String numberFormat; - private final ImmutableBytesWritable tempPtr; - private final PhoenixStatement statement; - - private long currentTime = QueryConstants.UNSET_TIMESTAMP; - private ScanRanges scanRanges = ScanRanges.EVERYTHING; - private KeyRange minMaxRange = null; - private final SequenceManager sequences; - - private TableRef currentTable; - - public StatementContext(PhoenixStatement statement, ColumnResolver resolver, List binds, Scan scan) { - this.statement = statement; - this.resolver = resolver; - this.scan = scan; - this.binds = new BindManager(binds); - this.aggregates = new AggregationManager(); - this.expressions = new ExpressionManager(); - PhoenixConnection connection = statement.getConnection(); - this.dateFormat = connection.getQueryServices().getProps().get(QueryServices.DATE_FORMAT_ATTRIB, DateUtil.DEFAULT_DATE_FORMAT); - this.dateFormatter = DateUtil.getDateFormatter(dateFormat); - this.dateParser = DateUtil.getDateParser(dateFormat); - this.numberFormat = connection.getQueryServices().getProps().get(QueryServices.NUMBER_FORMAT_ATTRIB, NumberUtil.DEFAULT_NUMBER_FORMAT); - this.tempPtr = new ImmutableBytesWritable(); - this.currentTable = resolver != null && !resolver.getTables().isEmpty() ? resolver.getTables().get(0) : null; - this.sequences = new SequenceManager(statement); - } - - public String getDateFormat() { - return dateFormat; - } - - public Format getDateFormatter() { - return dateFormatter; - } - - public Format getDateParser() { - return dateParser; - } - - public String getNumberFormat() { - return numberFormat; - } - - public Scan getScan() { - return scan; - } - - public BindManager getBindManager() { - return binds; - } - - public TableRef getCurrentTable() { - return currentTable; - } - - public void setCurrentTable(TableRef table) { - this.currentTable = table; - } - - public AggregationManager getAggregationManager() { - return aggregates; - } - - public ColumnResolver getResolver() { - return resolver; - } - - public void setResolver(ColumnResolver resolver) { - this.resolver = resolver; - } - - public ExpressionManager getExpressionManager() { - return expressions; - } - - - public ImmutableBytesWritable getTempPtr() { - return tempPtr; - } - - public ScanRanges getScanRanges() { - return this.scanRanges; - } - - public void setScanRanges(ScanRanges scanRanges) { - setScanRanges(scanRanges, null); - } - - public void setScanRanges(ScanRanges scanRanges, KeyRange minMaxRange) { - this.scanRanges = scanRanges; - this.scanRanges.setScanStartStopRow(scan); - PTable table = this.getCurrentTable().getTable(); - if (minMaxRange != null) { - // Ensure minMaxRange is lower inclusive and upper exclusive, as that's - // what we need to intersect against for the HBase scan. - byte[] lowerRange = minMaxRange.getLowerRange(); - if (!minMaxRange.lowerUnbound()) { - if (!minMaxRange.isLowerInclusive()) { - lowerRange = ScanUtil.nextKey(lowerRange, table, tempPtr); - } - } - - byte[] upperRange = minMaxRange.getUpperRange(); - if (!minMaxRange.upperUnbound()) { - if (minMaxRange.isUpperInclusive()) { - upperRange = ScanUtil.nextKey(upperRange, table, tempPtr); - } - } - if (minMaxRange.getLowerRange() != lowerRange || minMaxRange.getUpperRange() != upperRange) { - minMaxRange = KeyRange.getKeyRange(lowerRange, true, upperRange, false); - } - // If we're not salting, we can intersect this now with the scan range. - // Otherwise, we have to wait to do this when we chunk up the scan. - if (table.getBucketNum() == null) { - minMaxRange = minMaxRange.intersect(KeyRange.getKeyRange(scan.getStartRow(), scan.getStopRow())); - scan.setStartRow(minMaxRange.getLowerRange()); - scan.setStopRow(minMaxRange.getUpperRange()); - } - this.minMaxRange = minMaxRange; - } - } - - public PhoenixConnection getConnection() { - return statement.getConnection(); - } - - public PhoenixStatement getStatement() { - return statement; - } - - public long getCurrentTime() throws SQLException { - long ts = this.getCurrentTable().getTimeStamp(); - if (ts != QueryConstants.UNSET_TIMESTAMP) { - return ts; - } - if (currentTime != QueryConstants.UNSET_TIMESTAMP) { - return currentTime; - } - /* - * For an UPSERT VALUES where autocommit off, we won't hit the server until the commit. - * However, if the statement has a CURRENT_DATE() call as a value, we need to know the - * current time at execution time. In that case, we'll call MetaDataClient.updateCache - * purely to bind the current time based on the server time. - */ - PTable table = this.getCurrentTable().getTable(); - PhoenixConnection connection = getConnection(); - MetaDataClient client = new MetaDataClient(connection); - currentTime = client.getCurrentTime(table.getSchemaName().getString(), table.getTableName().getString()); - return currentTime; - } - - /** - * Get the key range derived from row value constructor usage in where clause. These are orthogonal to the ScanRanges - * and form a range for which each scan is intersected against. - */ - public KeyRange getMinMaxRange () { - return minMaxRange; - } - - public boolean isSingleRowScan() { - return this.getScanRanges().isSingleRowScan() && ! (this.getScan().getFilter() instanceof FilterList); - } - - public SequenceManager getSequenceManager(){ - return sequences; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/StatementNormalizer.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/StatementNormalizer.java deleted file mode 100644 index a8e7af36..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/StatementNormalizer.java +++ /dev/null @@ -1,110 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.SQLException; -import java.util.List; - -import com.google.common.collect.Lists; -import com.salesforce.phoenix.parse.BetweenParseNode; -import com.salesforce.phoenix.parse.ColumnParseNode; -import com.salesforce.phoenix.parse.ComparisonParseNode; -import com.salesforce.phoenix.parse.LessThanOrEqualParseNode; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.ParseNodeRewriter; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.util.SchemaUtil; - - -/** - * - * Class that creates a new select statement ensuring that a literal always occurs - * on the RHS (i.e. if literal found on the LHS, then the operator is reversed and - * the literal is put on the RHS) - * - * @author jtaylor - * @since 0.1 - */ -public class StatementNormalizer extends ParseNodeRewriter { - private boolean useFullNameForAlias; - - public StatementNormalizer(ColumnResolver resolver, int expectedAliasCount, boolean useFullNameForAlias) { - super(resolver, expectedAliasCount); - this.useFullNameForAlias = useFullNameForAlias; - } - - public static ParseNode normalize(ParseNode where, ColumnResolver resolver) throws SQLException { - return rewrite(where, new StatementNormalizer(resolver, 0, false)); - } - - /** - * Rewrite the select statement by switching any constants to the right hand side - * of the expression. - * @param statement the select statement - * @param resolver - * @return new select statement - * @throws SQLException - */ - public static SelectStatement normalize(SelectStatement statement, ColumnResolver resolver) throws SQLException { - return rewrite(statement, new StatementNormalizer(resolver, statement.getSelect().size(), statement.getFrom().size() > 1)); - } - - @Override - public ParseNode visitLeave(ComparisonParseNode node, List nodes) throws SQLException { - if (nodes.get(0).isStateless() && !nodes.get(1).isStateless()) { - List normNodes = Lists.newArrayListWithExpectedSize(2); - normNodes.add(nodes.get(1)); - normNodes.add(nodes.get(0)); - nodes = normNodes; - node = NODE_FACTORY.comparison(node.getInvertFilterOp(), nodes.get(0), nodes.get(1)); - } - return super.visitLeave(node, nodes); - } - - @Override - public ParseNode visitLeave(final BetweenParseNode node, List nodes) throws SQLException { - - LessThanOrEqualParseNode lhsNode = NODE_FACTORY.lte(node.getChildren().get(1), node.getChildren().get(0)); - LessThanOrEqualParseNode rhsNode = NODE_FACTORY.lte(node.getChildren().get(0), node.getChildren().get(2)); - List parseNodes = Lists.newArrayListWithExpectedSize(2); - parseNodes.add(this.visitLeave(lhsNode, lhsNode.getChildren())); - parseNodes.add(this.visitLeave(rhsNode, rhsNode.getChildren())); - return super.visitLeave(node, parseNodes); - } - - @Override - public ParseNode visit(ColumnParseNode node) throws SQLException { - if (useFullNameForAlias - && node.getAlias() != null - && node.getTableName() != null - && SchemaUtil.normalizeIdentifier(node.getAlias()).equals(node.getName())) { - node = NODE_FACTORY.column(NODE_FACTORY.table(node.getSchemaName(), node.getTableName()), node.isCaseSensitive() ? '"' + node.getName() + '"' : node.getName(), node.getFullName()); - } - return super.visit(node); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/StatementPlan.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/StatementPlan.java deleted file mode 100644 index b04eeb77..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/StatementPlan.java +++ /dev/null @@ -1,55 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.ParameterMetaData; -import java.sql.SQLException; - -import com.salesforce.phoenix.jdbc.PhoenixParameterMetaData; - - -public interface StatementPlan { - public final StatementPlan EMPTY_PLAN = new StatementPlan() { - @Override - public ParameterMetaData getParameterMetaData() { - return new PhoenixParameterMetaData(0); - } - - @Override - public ExplainPlan getExplainPlan() throws SQLException { - return ExplainPlan.EMPTY_PLAN; - } - }; - - /** - * Returns the ParameterMetaData for the statement - */ - ParameterMetaData getParameterMetaData(); - - ExplainPlan getExplainPlan() throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/WhereCompiler.java b/phoenix-core/src/main/java/com/salesforce/phoenix/compile/WhereCompiler.java deleted file mode 100644 index 7a407077..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/WhereCompiler.java +++ /dev/null @@ -1,222 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; - -import java.sql.SQLException; -import java.sql.SQLFeatureNotSupportedException; -import java.util.Iterator; -import java.util.Set; - -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.filter.Filter; - -import com.google.common.collect.Iterators; -import com.google.common.collect.Sets; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.KeyValueColumnExpression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.visitor.KeyValueExpressionVisitor; -import com.salesforce.phoenix.filter.MultiCFCQKeyValueComparisonFilter; -import com.salesforce.phoenix.filter.MultiCQKeyValueComparisonFilter; -import com.salesforce.phoenix.filter.RowKeyComparisonFilter; -import com.salesforce.phoenix.filter.SingleCFCQKeyValueComparisonFilter; -import com.salesforce.phoenix.filter.SingleCQKeyValueComparisonFilter; -import com.salesforce.phoenix.parse.ColumnParseNode; -import com.salesforce.phoenix.parse.FilterableStatement; -import com.salesforce.phoenix.parse.HintNode.Hint; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.ParseNodeFactory; -import com.salesforce.phoenix.schema.AmbiguousColumnException; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.ColumnRef; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.TypeMismatchException; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.ScanUtil; -import com.salesforce.phoenix.util.SchemaUtil; - - -/** - * - * Class to build the filter of a scan - * - * @author jtaylor - * @since 0.1 - */ -public class WhereCompiler { - protected static final ParseNodeFactory NODE_FACTORY = new ParseNodeFactory(); - - private WhereCompiler() { - } - - /** - * Pushes where clause filter expressions into scan by building and setting a filter. - * @param context the shared context during query compilation - * @param statement TODO - * @throws SQLException if mismatched types are found, bind value do not match binds, - * or invalid function arguments are encountered. - * @throws SQLFeatureNotSupportedException if an unsupported expression is encountered. - * @throws ColumnNotFoundException if column name could not be resolved - * @throws AmbiguousColumnException if an unaliased column name is ambiguous across multiple tables - */ - public static Expression compile(StatementContext context, FilterableStatement statement) throws SQLException { - return compileWhereClause(context, statement, Sets.newHashSet()); - } - - /** - * Used for testing to get access to the expressions that were used to form the start/stop key of the scan - * @param statement TODO - */ - public static Expression compileWhereClause(StatementContext context, FilterableStatement statement, - Set extractedNodes) throws SQLException { - WhereExpressionCompiler whereCompiler = new WhereExpressionCompiler(context); - ParseNode where = statement.getWhere(); - Expression expression = where == null ? LiteralExpression.newConstant(true,PDataType.BOOLEAN,true) : where.accept(whereCompiler); - if (whereCompiler.isAggregate()) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.AGGREGATE_IN_WHERE).build().buildException(); - } - if (expression.getDataType() != PDataType.BOOLEAN) { - throw TypeMismatchException.newException(PDataType.BOOLEAN, expression.getDataType(), expression.toString()); - } - - expression = WhereOptimizer.pushKeyExpressionsToScan(context, statement, expression, extractedNodes); - setScanFilter(context, statement, expression, whereCompiler.disambiguateWithFamily); - - return expression; - } - - private static class WhereExpressionCompiler extends ExpressionCompiler { - private boolean disambiguateWithFamily; - - WhereExpressionCompiler(StatementContext context) { - super(context); - } - - @Override - protected ColumnRef resolveColumn(ColumnParseNode node) throws SQLException { - ColumnRef ref = super.resolveColumn(node); - PTable table = ref.getTable(); - // Track if we need to compare KeyValue during filter evaluation - // using column family. If the column qualifier is enough, we - // just use that. - try { - if (!SchemaUtil.isPKColumn(ref.getColumn())) { - table.getColumn(ref.getColumn().getName().getString()); - } - } catch (AmbiguousColumnException e) { - disambiguateWithFamily = true; - } - return ref; - } - } - - private static final class Counter { - public enum Count {NONE, SINGLE, MULTIPLE}; - private Count count = Count.NONE; - private KeyValueColumnExpression column; - - public void increment(KeyValueColumnExpression column) { - switch (count) { - case NONE: - count = Count.SINGLE; - this.column = column; - break; - case SINGLE: - count = column.equals(this.column) ? Count.SINGLE : Count.MULTIPLE; - break; - case MULTIPLE: - break; - - } - } - public Count getCount() { - return count; - } - } - - /** - * Sets the start/stop key range based on the whereClause expression. - * @param context the shared context during query compilation - * @param whereClause the final where clause expression. - */ - private static void setScanFilter(StatementContext context, FilterableStatement statement, Expression whereClause, boolean disambiguateWithFamily) { - Filter filter = null; - Scan scan = context.getScan(); - assert scan.getFilter() == null; - - if (LiteralExpression.isFalse(whereClause)) { - context.setScanRanges(ScanRanges.NOTHING); - } else if (whereClause != null && !LiteralExpression.isTrue(whereClause)) { - final Counter counter = new Counter(); - whereClause.accept(new KeyValueExpressionVisitor() { - - @Override - public Iterator defaultIterator(Expression node) { - // Stop traversal once we've found multiple KeyValue columns - if (counter.getCount() == Counter.Count.MULTIPLE) { - return Iterators.emptyIterator(); - } - return super.defaultIterator(node); - } - - @Override - public Void visit(KeyValueColumnExpression expression) { - counter.increment(expression); - return null; - } - }); - switch (counter.getCount()) { - case NONE: - PTable table = context.getResolver().getTables().get(0).getTable(); - byte[] essentialCF = table.getType() == PTableType.VIEW - ? ByteUtil.EMPTY_BYTE_ARRAY - : SchemaUtil.getEmptyColumnFamily(table.getColumnFamilies()); - filter = new RowKeyComparisonFilter(whereClause, essentialCF); - break; - case SINGLE: - filter = disambiguateWithFamily ? new SingleCFCQKeyValueComparisonFilter(whereClause) : new SingleCQKeyValueComparisonFilter(whereClause); - break; - case MULTIPLE: - filter = disambiguateWithFamily ? new MultiCFCQKeyValueComparisonFilter(whereClause) : new MultiCQKeyValueComparisonFilter(whereClause); - break; - } - } - - scan.setFilter(filter); - ScanRanges scanRanges = context.getScanRanges(); - boolean forcedSkipScan = statement.getHint().hasHint(Hint.SKIP_SCAN); - boolean forcedRangeScan = statement.getHint().hasHint(Hint.RANGE_SCAN); - if (forcedSkipScan || (scanRanges.useSkipScanFilter() && !forcedRangeScan)) { - ScanUtil.andFilterAtBeginning(scan, scanRanges.getSkipScanFilter()); - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/BaseRegionScanner.java b/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/BaseRegionScanner.java deleted file mode 100644 index ff9c1f25..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/BaseRegionScanner.java +++ /dev/null @@ -1,77 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; - -import java.io.IOException; -import java.util.List; - -import org.apache.hadoop.hbase.DoNotRetryIOException; -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.regionserver.RegionScanner; - -public abstract class BaseRegionScanner implements RegionScanner { - - @Override - public boolean isFilterDone() { - return false; - } - - @Override - public boolean next(List results, String metric) throws IOException { - return next(results); - } - - @Override - public boolean next(List result, int limit, String metric) throws IOException { - return next(result); - } - @Override - public boolean next(List result, int limit) throws IOException { - return next(result); - } - - @Override - public boolean reseek(byte[] row) throws IOException { - throw new DoNotRetryIOException("Unsupported"); - } - - @Override - public long getMvccReadPoint() { - return Long.MAX_VALUE; - } - - @Override - public boolean nextRaw(List result, String metric) throws IOException { - return next(result, metric); - } - - @Override - public boolean nextRaw(List result, int limit, String metric) throws IOException { - return next(result, limit, metric); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/BaseScannerRegionObserver.java b/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/BaseScannerRegionObserver.java deleted file mode 100644 index 8e718ba7..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/BaseScannerRegionObserver.java +++ /dev/null @@ -1,65 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; - -import java.io.IOException; - -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.coprocessor.*; -import org.apache.hadoop.hbase.regionserver.RegionScanner; - -import com.salesforce.phoenix.util.ServerUtil; - - -abstract public class BaseScannerRegionObserver extends BaseRegionObserver { - - /** - * Used by logger to identify coprocessor - */ - @Override - public String toString() { - return this.getClass().getName(); - } - - abstract protected RegionScanner doPostScannerOpen(final ObserverContext c, final Scan scan, final RegionScanner s) throws Throwable; - - /** - * Wrapper for {@link #postScannerOpen(ObserverContext, Scan, RegionScanner)} that ensures no non IOException is thrown, - * to prevent the coprocessor from becoming blacklisted. - * - */ - @Override - public final RegionScanner postScannerOpen(final ObserverContext c, final Scan scan, final RegionScanner s) throws IOException { - try { - return doPostScannerOpen(c, scan, s); - } catch (Throwable t) { - ServerUtil.throwIOException(c.getEnvironment().getRegion().getRegionNameAsString(), t); - return null; // impossible - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/GroupByCache.java b/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/GroupByCache.java deleted file mode 100644 index 7107f3bd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/GroupByCache.java +++ /dev/null @@ -1,40 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source - * and binary forms, with or without modification, are permitted provided that the following - * conditions are met: Redistributions of source code must retain the above copyright notice, this - * list of conditions and the following disclaimer. Redistributions in binary form must reproduce - * the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of - * Salesforce.com nor the names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED - * BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, - * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, - * OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF - * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; - -import java.io.Closeable; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.regionserver.RegionScanner; - -import com.salesforce.phoenix.expression.aggregator.Aggregator; - -/** - * - * Interface to abstract the way in which distinct group by - * elements are cached - * - * @author jtaylor - * @since 3.0.0 - */ -public interface GroupByCache extends Closeable { - int size(); - Aggregator[] cache(ImmutableBytesWritable key); - RegionScanner getScanner(RegionScanner s); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/MetaDataRegionObserver.java b/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/MetaDataRegionObserver.java deleted file mode 100644 index 23013385..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/MetaDataRegionObserver.java +++ /dev/null @@ -1,48 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; - -import org.apache.hadoop.hbase.coprocessor.BaseRegionObserver; -import org.apache.hadoop.hbase.coprocessor.ObserverContext; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; - -import com.salesforce.phoenix.cache.GlobalCache; - - -/** - * Coprocessor for metadata related operations. This coprocessor would only be registered - * to SYSTEM.TABLE. - */ -public class MetaDataRegionObserver extends BaseRegionObserver { - - @Override - public void preClose(final ObserverContext c, - boolean abortRequested) { - GlobalCache.getInstance(c.getEnvironment()).getMetaDataCache().clear(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ScanProjector.java b/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ScanProjector.java deleted file mode 100644 index 062b7cc1..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ScanProjector.java +++ /dev/null @@ -1,210 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; - -import java.io.ByteArrayInputStream; -import java.io.ByteArrayOutputStream; -import java.io.DataInputStream; -import java.io.DataOutputStream; -import java.io.IOException; -import java.util.HashMap; -import java.util.Map; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.util.Bytes; -import org.apache.hadoop.hbase.util.Pair; -import org.apache.hadoop.io.WritableUtils; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.KeyValueUtil; - -public class ScanProjector { - - public enum ProjectionType {TABLE, CF, CQ}; - - private static final String SCAN_PROJECTOR = "scanProjector"; - private static final byte[] SEPERATOR = Bytes.toBytes(":"); - - private final ProjectionType type; - private final byte[] tablePrefix; - private final Map cfProjectionMap; - private final Map>> cqProjectionMap; - - private ScanProjector(ProjectionType type, byte[] tablePrefix, - Map cfProjectionMap, Map>> cqProjectionMap) { - this.type = ProjectionType.TABLE; - this.tablePrefix = tablePrefix; - this.cfProjectionMap = cfProjectionMap; - this.cqProjectionMap = cqProjectionMap; - } - - public static void serializeProjectorIntoScan(Scan scan, ScanProjector projector) { - ByteArrayOutputStream stream = new ByteArrayOutputStream(); - try { - DataOutputStream output = new DataOutputStream(stream); - WritableUtils.writeVInt(output, projector.type.ordinal()); - switch (projector.type) { - case TABLE: - WritableUtils.writeCompressedByteArray(output, projector.tablePrefix); - break; - case CF: - WritableUtils.writeVInt(output, projector.cfProjectionMap.size()); - for (Map.Entry entry : projector.cfProjectionMap.entrySet()) { - WritableUtils.writeCompressedByteArray(output, entry.getKey().get()); - WritableUtils.writeCompressedByteArray(output, entry.getValue()); - } - break; - case CQ: - WritableUtils.writeVInt(output, projector.cqProjectionMap.size()); - for (Map.Entry>> entry : - projector.cqProjectionMap.entrySet()) { - WritableUtils.writeCompressedByteArray(output, entry.getKey().get()); - Map> map = entry.getValue(); - WritableUtils.writeVInt(output, map.size()); - for (Map.Entry> e : map.entrySet()) { - WritableUtils.writeCompressedByteArray(output, e.getKey().get()); - WritableUtils.writeCompressedByteArray(output, e.getValue().getFirst()); - WritableUtils.writeCompressedByteArray(output, e.getValue().getSecond()); - } - } - break; - default: - throw new IOException("Unrecognized projection type '" + projector.type + "'"); - } - scan.setAttribute(SCAN_PROJECTOR, stream.toByteArray()); - } catch (IOException e) { - throw new RuntimeException(e); - } finally { - try { - stream.close(); - } catch (IOException e) { - throw new RuntimeException(e); - } - } - - } - - public static ScanProjector deserializeProjectorFromScan(Scan scan) { - byte[] proj = scan.getAttribute(SCAN_PROJECTOR); - if (proj == null) { - return null; - } - ByteArrayInputStream stream = new ByteArrayInputStream(proj); - try { - DataInputStream input = new DataInputStream(stream); - int t = WritableUtils.readVInt(input); - ProjectionType type = ProjectionType.values()[t]; - if (type == ProjectionType.TABLE) { - byte[] tablePrefix = WritableUtils.readCompressedByteArray(input); - return new ScanProjector(type, tablePrefix, null, null); - } - if (type == ProjectionType.CF) { - int count = WritableUtils.readVInt(input); - Map cfMap = new HashMap(); - for (int i = 0; i < count; i++) { - byte[] cf = WritableUtils.readCompressedByteArray(input); - byte[] renamed = WritableUtils.readCompressedByteArray(input); - cfMap.put(new ImmutableBytesPtr(cf), renamed); - } - return new ScanProjector(type, null, cfMap, null); - } - - int count = WritableUtils.readVInt(input); - Map>> cqMap = - new HashMap>>(); - for (int i = 0; i < count; i++) { - byte[] cf = WritableUtils.readCompressedByteArray(input); - int nQuals = WritableUtils.readVInt(input); - Map> map = - new HashMap>(); - for (int j = 0; j < nQuals; j++) { - byte[] cq = WritableUtils.readCompressedByteArray(input); - byte[] renamedCf = WritableUtils.readCompressedByteArray(input); - byte[] renamedCq = WritableUtils.readCompressedByteArray(input); - map.put(new ImmutableBytesPtr(cq), new Pair(renamedCf, renamedCq)); - } - cqMap.put(new ImmutableBytesPtr(cf), map); - } - return new ScanProjector(type, null, null, cqMap); - } catch (IOException e) { - throw new RuntimeException(e); - } finally { - try { - stream.close(); - } catch (IOException e) { - throw new RuntimeException(e); - } - } - } - - public ProjectionType getType() { - return this.type; - } - - public byte[] getTablePrefix() { - return this.tablePrefix; - } - - public Map getCfProjectionMap() { - return this.cfProjectionMap; - } - - public Map>> getCqProjectionMap() { - return this.cqProjectionMap; - } - - public KeyValue getProjectedKeyValue(KeyValue kv) { - if (type == ProjectionType.TABLE) { - byte[] cf = ByteUtil.concat(tablePrefix, SEPERATOR, kv.getFamily()); - return KeyValueUtil.newKeyValue(kv.getBuffer(), kv.getKeyOffset(), kv.getKeyLength(), - cf, kv.getQualifier(), kv.getTimestamp(), kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); - } - - if (type == ProjectionType.CF) { - byte[] cf = cfProjectionMap.get(new ImmutableBytesPtr(kv.getFamily())); - if (cf == null) - return kv; - return KeyValueUtil.newKeyValue(kv.getBuffer(), kv.getKeyOffset(), kv.getKeyLength(), - cf, kv.getQualifier(), kv.getTimestamp(), kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); - } - - Map> map = cqProjectionMap.get(new ImmutableBytesPtr(kv.getFamily())); - if (map == null) - return kv; - - Pair col = map.get(new ImmutableBytesPtr(kv.getQualifier())); - if (col == null) - return kv; - - return KeyValueUtil.newKeyValue(kv.getBuffer(), kv.getKeyOffset(), kv.getKeyLength(), - col.getFirst(), col.getSecond(), kv.getTimestamp(), kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ServerCachingEndpointImpl.java b/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ServerCachingEndpointImpl.java deleted file mode 100644 index a000bae4..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ServerCachingEndpointImpl.java +++ /dev/null @@ -1,65 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; - -import java.sql.SQLException; - -import org.apache.hadoop.hbase.coprocessor.BaseEndpointCoprocessor; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.cache.GlobalCache; -import com.salesforce.phoenix.cache.TenantCache; - - - - -/** - * - * Server-side implementation of {@link ServerCachingProtocol} - * - * @author jtaylor - * @since 0.1 - */ -public class ServerCachingEndpointImpl extends BaseEndpointCoprocessor implements ServerCachingProtocol { - - @Override - public boolean addServerCache(byte[] tenantId, byte[] cacheId, ImmutableBytesWritable cachePtr, ServerCacheFactory cacheFactory) throws SQLException { - TenantCache tenantCache = GlobalCache.getTenantCache((RegionCoprocessorEnvironment)this.getEnvironment(), tenantId == null ? null : new ImmutableBytesPtr(tenantId)); - tenantCache.addServerCache(new ImmutableBytesPtr(cacheId), cachePtr, cacheFactory); - return true; - } - - @Override - public boolean removeServerCache(byte[] tenantId, byte[] cacheId) throws SQLException { - TenantCache tenantCache = GlobalCache.getTenantCache((RegionCoprocessorEnvironment)this.getEnvironment(), tenantId == null ? null : new ImmutableBytesPtr(tenantId)); - tenantCache.removeServerCache(new ImmutableBytesPtr(cacheId)); - return true; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ServerCachingProtocol.java b/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ServerCachingProtocol.java deleted file mode 100644 index ddcf31cd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ServerCachingProtocol.java +++ /dev/null @@ -1,71 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; - -import java.io.Closeable; -import java.sql.SQLException; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.ipc.CoprocessorProtocol; -import org.apache.hadoop.io.Writable; - -import com.salesforce.phoenix.memory.MemoryManager.MemoryChunk; - -/** - * - * EndPoint coprocessor to send a cache to a region server. - * Used for: - * a) hash joins, to send the smaller side of the join to each region server - * b) secondary indexes, to send the necessary meta data to each region server - * @author jtaylor - * @since 0.1 - */ -public interface ServerCachingProtocol extends CoprocessorProtocol { - public static interface ServerCacheFactory extends Writable { - public Closeable newCache(ImmutableBytesWritable cachePtr, MemoryChunk chunk) throws SQLException; - } - /** - * Add the cache to the region server cache. - * @param tenantId the tenantId or null if not applicable - * @param cacheId unique identifier of the cache - * @param cachePtr pointer to the byte array of the cache - * @param cacheFactory factory that converts from byte array to object representation on the server side - * @return true on success and otherwise throws - * @throws SQLException - */ - public boolean addServerCache(byte[] tenantId, byte[] cacheId, ImmutableBytesWritable cachePtr, ServerCacheFactory cacheFactory) throws SQLException; - /** - * Remove the cache from the region server cache. Called upon completion of - * the operation when cache is no longer needed. - * @param tenantId the tenantId or null if not applicable - * @param cacheId unique identifier of the cache - * @return true on success and otherwise throws - * @throws SQLException - */ - public boolean removeServerCache(byte[] tenantId, byte[] cacheId) throws SQLException; -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/PhoenixIOException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/exception/PhoenixIOException.java deleted file mode 100644 index 5fd9049a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/PhoenixIOException.java +++ /dev/null @@ -1,41 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.exception; - -import java.sql.SQLException; - - -public class PhoenixIOException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.IO_EXCEPTION; - - public PhoenixIOException(Throwable e) { - super(e.getMessage(), code.getSQLState(), code.getErrorCode(), e); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/PhoenixParserException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/exception/PhoenixParserException.java deleted file mode 100644 index ada58587..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/PhoenixParserException.java +++ /dev/null @@ -1,120 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.exception; - -import java.sql.SQLSyntaxErrorException; - -import org.antlr.runtime.MismatchedTokenException; -import org.antlr.runtime.MissingTokenException; -import org.antlr.runtime.RecognitionException; -import org.antlr.runtime.Token; -import org.antlr.runtime.UnwantedTokenException; - - -public class PhoenixParserException extends SQLSyntaxErrorException { - private static final long serialVersionUID = 1L; - - public static final PhoenixParserException newException(Throwable cause, String[] tokens) { - return new PhoenixParserException(getErrorMessage(cause, tokens), cause); - } - - public PhoenixParserException(String msg, Throwable throwable) { - super(new SQLExceptionInfo.Builder(getErrorCode(throwable)).setRootCause(throwable) - .setMessage(msg).build().toString(), - getErrorCode(throwable).getSQLState(), getErrorCode(throwable).getErrorCode(), throwable); - } - - public static String getLine(RecognitionException e) { - return Integer.toString(e.token.getLine()); - } - - public static String getColumn(RecognitionException e) { - return Integer.toString(e.token.getCharPositionInLine() + 1); - } - - public static String getTokenLocation(RecognitionException e) { - return "line " + getLine(e) + ", column " + getColumn(e) + "."; - } - - public static String getErrorMessage(Throwable e, String[] tokenNames) { - String msg; - if (e instanceof MissingTokenException) { - MissingTokenException mte = (MissingTokenException)e; - String tokenName; - if (mte.expecting== Token.EOF) { - tokenName = "EOF"; - } else { - tokenName = tokenNames[mte.expecting]; - } - msg = "Missing \""+ tokenName +"\" at "+ getTokenLocation(mte); - } else if (e instanceof UnwantedTokenException) { - UnwantedTokenException ute = (UnwantedTokenException)e; - String tokenName; - if (ute.expecting== Token.EOF) { - tokenName = "EOF"; - } else { - tokenName = tokenNames[ute.expecting]; - } - msg = "Unexpected input. Expecting \"" + tokenName + "\", got \"" + ute.getUnexpectedToken().getText() - + "\" at " + getTokenLocation(ute); - } else if (e instanceof MismatchedTokenException) { - MismatchedTokenException mte = (MismatchedTokenException)e; - String tokenName; - if (mte.expecting== Token.EOF) { - tokenName = "EOF"; - } else { - tokenName = tokenNames[mte.expecting]; - } - msg = "Mismatched input. Expecting \"" + tokenName + "\", got \"" + mte.token.getText() - + "\" at " + getTokenLocation(mte); - } else if (e instanceof RecognitionException){ - RecognitionException re = (RecognitionException) e; - msg = "Encountered \"" + re.token.getText() + "\" at " + getTokenLocation(re); - } else if (e instanceof UnknownFunctionException) { - UnknownFunctionException ufe = (UnknownFunctionException) e; - msg = "Unknown function: \"" + ufe.getFuncName() + "\"."; - } else { - msg = e.getMessage(); - } - return msg; - } - - public static SQLExceptionCode getErrorCode(Throwable e) { - if (e instanceof MissingTokenException) { - return SQLExceptionCode.MISSING_TOKEN; - } else if (e instanceof UnwantedTokenException) { - return SQLExceptionCode.UNWANTED_TOKEN; - } else if (e instanceof MismatchedTokenException) { - return SQLExceptionCode.MISMATCHED_TOKEN; - } else if (e instanceof UnknownFunctionException) { - return SQLExceptionCode.UNKNOWN_FUNCTION; - } else { - return SQLExceptionCode.PARSER_ERROR; - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/SQLExceptionInfo.java b/phoenix-core/src/main/java/com/salesforce/phoenix/exception/SQLExceptionInfo.java deleted file mode 100644 index 289a185c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/SQLExceptionInfo.java +++ /dev/null @@ -1,174 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.exception; - -import java.sql.SQLException; - -import com.salesforce.phoenix.util.SchemaUtil; - - -/** - * Object serves as a closure of all coordinate information for SQLException messages. - * - * @author zhuang - * @since 1.0 - */ -public class SQLExceptionInfo { - - /** - * Constants used in naming exception location. - */ - public static final String SCHEMA_NAME = "schemaName"; - public static final String TABLE_NAME = "tableName"; - public static final String FAMILY_NAME = "familyName"; - public static final String COLUMN_NAME = "columnName"; - - private final Throwable rootCause; - private final SQLExceptionCode code; // Should always have one. - private final String message; - private final String schemaName; - private final String tableName; - private final String familyName; - private final String columnName; - - public static class Builder { - - private Throwable rootCause; - private SQLExceptionCode code; // Should always have one. - private String message; - private String schemaName; - private String tableName; - private String familyName; - private String columnName; - - public Builder(SQLExceptionCode code) { - this.code = code; - } - - public Builder setRootCause(Throwable t) { - this.rootCause = t; - return this; - } - - public Builder setMessage(String message) { - this.message = message; - return this; - } - - public Builder setSchemaName(String schemaName) { - this.schemaName = schemaName; - return this; - } - - public Builder setTableName(String tableName) { - this.tableName = tableName; - return this; - } - - public Builder setFamilyName(String familyName) { - this.familyName = familyName; - return this; - } - - public Builder setColumnName(String columnName) { - this.columnName = columnName; - return this; - } - - public SQLExceptionInfo build() { - return new SQLExceptionInfo(this); - } - - @Override - public String toString() { - return code.toString(); - } - } - - private SQLExceptionInfo(Builder builder) { - code = builder.code; - rootCause = builder.rootCause; - message = builder.message; - schemaName = builder.schemaName; - tableName = builder.tableName; - familyName = builder.familyName; - columnName = builder.columnName; - } - - @Override - public String toString() { - StringBuilder builder = new StringBuilder(code.toString()); - if (message != null) { - builder.append(" ").append(message); - } - String columnDisplayName = SchemaUtil.getMetaDataEntityName(schemaName, tableName, familyName, columnName); - if (columnName != null) { - builder.append(" ").append(COLUMN_NAME).append("=").append(columnDisplayName); - } else if (familyName != null) { - builder.append(" ").append(FAMILY_NAME).append("=").append(columnDisplayName); - } else if (tableName != null) { - builder.append(" ").append(TABLE_NAME).append("=").append(columnDisplayName); - } else if (schemaName != null) { - builder.append(" ").append(SCHEMA_NAME).append("=").append(columnDisplayName); - } - return builder.toString(); - } - - public SQLException buildException() { - return code.getExceptionFactory().newException(this); - } - - public Throwable getRootCause() { - return rootCause; - } - - public String getSchemaName() { - return schemaName; - } - - public String getTableName() { - return tableName; - } - - public String getFamilyName() { - return familyName; - } - - public String getColumnName() { - return columnName; - } - - public SQLExceptionCode getCode() { - return code; - } - - public String getMessage() { - return message; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/UnknownFunctionException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/exception/UnknownFunctionException.java deleted file mode 100644 index 4bf57b8f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/UnknownFunctionException.java +++ /dev/null @@ -1,45 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.exception; - -/** - * Thrown by ParseNodeFactory when it could not identify a node as a valid function. - */ -public class UnknownFunctionException extends RuntimeException { - private static final long serialVersionUID = 1L; - private final String funcName; - - public UnknownFunctionException(String funcName) { - super(); - this.funcName = funcName; - } - - public String getFuncName() { - return funcName; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/ValueTypeIncompatibleException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/exception/ValueTypeIncompatibleException.java deleted file mode 100644 index 1412ce8d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/ValueTypeIncompatibleException.java +++ /dev/null @@ -1,46 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.exception; - -import com.salesforce.phoenix.schema.IllegalDataException; -import com.salesforce.phoenix.schema.PDataType; - - -public class ValueTypeIncompatibleException extends IllegalDataException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.DATA_INCOMPATIBLE_WITH_TYPE; - - public ValueTypeIncompatibleException(PDataType type, Integer precision, Integer scale) { - super(new SQLExceptionInfo.Builder(code).setMessage(getTypeDisplayString(type, precision, scale)) - .build().toString()); - } - - private static String getTypeDisplayString(PDataType type, Integer precision, Integer scale) { - return type.toString() + "(" + precision + "," + scale + ")"; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/AggregatePlan.java b/phoenix-core/src/main/java/com/salesforce/phoenix/execute/AggregatePlan.java deleted file mode 100644 index 4232c2fe..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/AggregatePlan.java +++ /dev/null @@ -1,179 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.execute; - - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.OrderByCompiler.OrderBy; -import com.salesforce.phoenix.compile.RowProjector; -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.coprocessor.UngroupedAggregateRegionObserver; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.OrderByExpression; -import com.salesforce.phoenix.expression.RowKeyExpression; -import com.salesforce.phoenix.expression.aggregator.Aggregators; -import com.salesforce.phoenix.iterate.AggregatingResultIterator; -import com.salesforce.phoenix.iterate.ConcatResultIterator; -import com.salesforce.phoenix.iterate.DistinctAggregatingResultIterator; -import com.salesforce.phoenix.iterate.FilterAggregatingResultIterator; -import com.salesforce.phoenix.iterate.GroupedAggregatingResultIterator; -import com.salesforce.phoenix.iterate.LimitingResultIterator; -import com.salesforce.phoenix.iterate.MergeSortRowKeyResultIterator; -import com.salesforce.phoenix.iterate.OrderedAggregatingResultIterator; -import com.salesforce.phoenix.iterate.OrderedResultIterator; -import com.salesforce.phoenix.iterate.ParallelIterators; -import com.salesforce.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; -import com.salesforce.phoenix.iterate.PeekingResultIterator; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.iterate.SequenceResultIterator; -import com.salesforce.phoenix.iterate.SpoolingResultIterator; -import com.salesforce.phoenix.iterate.UngroupedAggregatingResultIterator; -import com.salesforce.phoenix.parse.FilterableStatement; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.TableRef; - - - -/** - * - * Query plan for aggregating queries - * - * @author jtaylor - * @since 0.1 - */ -public class AggregatePlan extends BasicQueryPlan { - private final Aggregators aggregators; - private final Expression having; - private List splits; - - public AggregatePlan( - StatementContext context, FilterableStatement statement, TableRef table, RowProjector projector, - Integer limit, OrderBy orderBy, ParallelIteratorFactory parallelIteratorFactory, GroupBy groupBy, - Expression having) { - super(context, statement, table, projector, context.getBindManager().getParameterMetaData(), limit, orderBy, groupBy, parallelIteratorFactory); - this.having = having; - this.aggregators = context.getAggregationManager().getAggregators(); - } - - @Override - public List getSplits() { - return splits; - } - - private static class OrderingResultIteratorFactory implements ParallelIteratorFactory { - private final QueryServices services; - - public OrderingResultIteratorFactory(QueryServices services) { - this.services = services; - } - @Override - public PeekingResultIterator newIterator(ResultIterator scanner) throws SQLException { - Expression expression = RowKeyExpression.INSTANCE; - OrderByExpression orderByExpression = new OrderByExpression(expression, false, true); - int threshold = services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB, QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES); - return new OrderedResultIterator(scanner, Collections.singletonList(orderByExpression), threshold); - } - } - - private static class WrappingResultIteratorFactory implements ParallelIteratorFactory { - private final ParallelIteratorFactory innerFactory; - private final ParallelIteratorFactory outerFactory; - - public WrappingResultIteratorFactory(ParallelIteratorFactory innerFactory, ParallelIteratorFactory outerFactory) { - this.innerFactory = innerFactory; - this.outerFactory = outerFactory; - } - @Override - public PeekingResultIterator newIterator(ResultIterator scanner) throws SQLException { - PeekingResultIterator iterator = innerFactory.newIterator(scanner); - return outerFactory.newIterator(iterator); - } - } - - private ParallelIteratorFactory wrapParallelIteratorFactory () { - ParallelIteratorFactory innerFactory; - QueryServices services = context.getConnection().getQueryServices(); - if (groupBy.isEmpty() || groupBy.isOrderPreserving()) { - innerFactory = new SpoolingResultIterator.SpoolingResultIteratorFactory(services); - } else { - innerFactory = new OrderingResultIteratorFactory(services); - } - if (parallelIteratorFactory == null) { - return innerFactory; - } - // wrap any existing parallelIteratorFactory - return new WrappingResultIteratorFactory(innerFactory, parallelIteratorFactory); - } - - @Override - protected ResultIterator newIterator() throws SQLException { - if (groupBy.isEmpty()) { - UngroupedAggregateRegionObserver.serializeIntoScan(context.getScan()); - } - ParallelIterators parallelIterators = new ParallelIterators(context, tableRef, statement, projection, groupBy, null, wrapParallelIteratorFactory()); - splits = parallelIterators.getSplits(); - - AggregatingResultIterator aggResultIterator; - // No need to merge sort for ungrouped aggregation - if (groupBy.isEmpty()) { - aggResultIterator = new UngroupedAggregatingResultIterator(new ConcatResultIterator(parallelIterators), aggregators); - } else { - aggResultIterator = new GroupedAggregatingResultIterator(new MergeSortRowKeyResultIterator(parallelIterators), aggregators); - } - - if (having != null) { - aggResultIterator = new FilterAggregatingResultIterator(aggResultIterator, having); - } - - if (statement.isDistinct() && statement.isAggregate()) { // Dedup on client if select distinct and aggregation - aggResultIterator = new DistinctAggregatingResultIterator(aggResultIterator, getProjector()); - } - - ResultIterator resultScanner = aggResultIterator; - if (orderBy.getOrderByExpressions().isEmpty()) { - if (limit != null) { - resultScanner = new LimitingResultIterator(aggResultIterator, limit); - } - } else { - int thresholdBytes = getConnectionQueryServices(context.getConnection().getQueryServices()).getProps().getInt( - QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB, QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES); - resultScanner = new OrderedAggregatingResultIterator(aggResultIterator, orderBy.getOrderByExpressions(), thresholdBytes, limit); - } - - if (context.getSequenceManager().getSequenceCount() > 0) { - resultScanner = new SequenceResultIterator(resultScanner, context.getSequenceManager()); - } - return resultScanner; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/BasicQueryPlan.java b/phoenix-core/src/main/java/com/salesforce/phoenix/execute/BasicQueryPlan.java deleted file mode 100644 index ee64192c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/BasicQueryPlan.java +++ /dev/null @@ -1,205 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.execute; - -import java.sql.ParameterMetaData; -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.google.common.collect.Lists; -import com.salesforce.phoenix.compile.ExplainPlan; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.OrderByCompiler.OrderBy; -import com.salesforce.phoenix.compile.QueryPlan; -import com.salesforce.phoenix.compile.RowProjector; -import com.salesforce.phoenix.compile.ScanRanges; -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.iterate.DelegateResultIterator; -import com.salesforce.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.parse.FilterableStatement; -import com.salesforce.phoenix.query.ConnectionQueryServices; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.SQLCloseable; -import com.salesforce.phoenix.util.SQLCloseables; -import com.salesforce.phoenix.util.ScanUtil; - - - -/** - * - * Query plan that has no child plans - * - * @author jtaylor - * @since 0.1 - */ -public abstract class BasicQueryPlan implements QueryPlan { - protected static final long DEFAULT_ESTIMATED_SIZE = 10 * 1024; // 10 K - - protected final TableRef tableRef; - protected final StatementContext context; - protected final FilterableStatement statement; - protected final RowProjector projection; - protected final ParameterMetaData paramMetaData; - protected final Integer limit; - protected final OrderBy orderBy; - protected final GroupBy groupBy; - protected final ParallelIteratorFactory parallelIteratorFactory; - - protected BasicQueryPlan( - StatementContext context, FilterableStatement statement, TableRef table, - RowProjector projection, ParameterMetaData paramMetaData, Integer limit, OrderBy orderBy, - GroupBy groupBy, ParallelIteratorFactory parallelIteratorFactory) { - this.context = context; - this.statement = statement; - this.tableRef = table; - this.projection = projection; - this.paramMetaData = paramMetaData; - this.limit = limit; - this.orderBy = orderBy; - this.groupBy = groupBy; - this.parallelIteratorFactory = parallelIteratorFactory; - } - - @Override - public GroupBy getGroupBy() { - return groupBy; - } - - - @Override - public OrderBy getOrderBy() { - return orderBy; - } - - @Override - public TableRef getTableRef() { - return tableRef; - } - - @Override - public Integer getLimit() { - return limit; - } - - @Override - public RowProjector getProjector() { - return projection; - } - - protected ConnectionQueryServices getConnectionQueryServices(ConnectionQueryServices services) { - // Get child services associated with tenantId of query. - ConnectionQueryServices childServices = context.getConnection().getTenantId() == null ? - services : - services.getChildQueryServices(new ImmutableBytesWritable(context.getConnection().getTenantId().getBytes())); - return childServices; - } - -// /** -// * Sets up an id used to do round robin queue processing on the server -// * @param scan -// */ -// private void setProducer(Scan scan) { -// byte[] producer = Bytes.toBytes(UUID.randomUUID().toString()); -// scan.setAttribute(HBaseServer.CALL_QUEUE_PRODUCER_ATTRIB_NAME, producer); -// } - - @Override - public final ResultIterator iterator() throws SQLException { - return iterator(Collections.emptyList()); - } - - public final ResultIterator iterator(final List dependencies) throws SQLException { - if (context.getScanRanges() == ScanRanges.NOTHING) { - return ResultIterator.EMPTY_ITERATOR; - } - - Scan scan = context.getScan(); - // Set producer on scan so HBase server does round robin processing - //setProducer(scan); - // Set the time range on the scan so we don't get back rows newer than when the statement was compiled - // The time stamp comes from the server at compile time when the meta data - // is resolved. - // TODO: include time range in explain plan? - PhoenixConnection connection = context.getConnection(); - Long scn = connection.getSCN(); - ScanUtil.setTimeRange(scan, scn == null ? context.getCurrentTime() : scn); - ScanUtil.setTenantId(scan, connection.getTenantId() == null ? null : connection.getTenantId().getBytes()); - ResultIterator iterator = newIterator(); - return dependencies.isEmpty() ? - iterator : new DelegateResultIterator(iterator) { - @Override - public void close() throws SQLException { - try { - super.close(); - } finally { - SQLCloseables.closeAll(dependencies); - } - } - }; - } - - abstract protected ResultIterator newIterator() throws SQLException; - - @Override - public long getEstimatedSize() { - return DEFAULT_ESTIMATED_SIZE; - } - - @Override - public ParameterMetaData getParameterMetaData() { - return paramMetaData; - } - - @Override - public FilterableStatement getStatement() { - return statement; - } - - @Override - public StatementContext getContext() { - return context; - } - - @Override - public ExplainPlan getExplainPlan() throws SQLException { - if (context.getScanRanges() == ScanRanges.NOTHING) { - return new ExplainPlan(Collections.singletonList("DEGENERATE SCAN OVER " + tableRef.getTable().getName().getString())); - } - - ResultIterator iterator = iterator(); - List planSteps = Lists.newArrayListWithExpectedSize(5); - iterator.explain(planSteps); - return new ExplainPlan(planSteps); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/CommitException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/execute/CommitException.java deleted file mode 100644 index 2ade76d7..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/CommitException.java +++ /dev/null @@ -1,51 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.execute; - -import java.sql.SQLException; - -public class CommitException extends SQLException { - private static final long serialVersionUID = 1L; - private final MutationState uncommittedState; - private final MutationState committedState; - - public CommitException(Exception e, MutationState uncommittedState, MutationState committedState) { - super(e); - this.uncommittedState = uncommittedState; - this.committedState = committedState; - } - - public MutationState getUncommittedState() { - return uncommittedState; - } - - public MutationState getCommittedState() { - return committedState; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/DegenerateQueryPlan.java b/phoenix-core/src/main/java/com/salesforce/phoenix/execute/DegenerateQueryPlan.java deleted file mode 100644 index 8c6877e3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/DegenerateQueryPlan.java +++ /dev/null @@ -1,60 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.execute; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.OrderByCompiler.OrderBy; -import com.salesforce.phoenix.compile.*; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.jdbc.PhoenixParameterMetaData; -import com.salesforce.phoenix.parse.FilterableStatement; -import com.salesforce.phoenix.query.*; -import com.salesforce.phoenix.schema.TableRef; - -public class DegenerateQueryPlan extends BasicQueryPlan { - - public DegenerateQueryPlan(StatementContext context, FilterableStatement statement, TableRef table) { - super(context, statement, table, RowProjector.EMPTY_PROJECTOR, PhoenixParameterMetaData.EMPTY_PARAMETER_META_DATA, null, OrderBy.EMPTY_ORDER_BY, GroupBy.EMPTY_GROUP_BY, null); - context.setScanRanges(ScanRanges.NOTHING); - } - - @Override - public List getSplits() { - return Collections.emptyList(); - } - - @Override - protected ResultIterator newIterator() throws SQLException { - return null; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/HashJoinPlan.java b/phoenix-core/src/main/java/com/salesforce/phoenix/execute/HashJoinPlan.java deleted file mode 100644 index 884ecf90..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/HashJoinPlan.java +++ /dev/null @@ -1,194 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.execute; - -import java.sql.ParameterMetaData; -import java.sql.SQLException; -import java.util.ArrayList; -import java.util.List; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.ExecutorService; -import java.util.concurrent.Future; -import org.apache.hadoop.hbase.client.Scan; - -import com.google.common.collect.Lists; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.cache.ServerCacheClient.ServerCache; -import com.salesforce.phoenix.compile.ExplainPlan; -import com.salesforce.phoenix.compile.QueryPlan; -import com.salesforce.phoenix.compile.RowProjector; -import com.salesforce.phoenix.compile.ScanRanges; -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.OrderByCompiler.OrderBy; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.job.JobManager.JobCallable; -import com.salesforce.phoenix.join.HashCacheClient; -import com.salesforce.phoenix.join.HashJoinInfo; -import com.salesforce.phoenix.parse.FilterableStatement; -import com.salesforce.phoenix.query.ConnectionQueryServices; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.SQLCloseable; - -public class HashJoinPlan implements QueryPlan { - - private BasicQueryPlan plan; - private HashJoinInfo joinInfo; - private List[] hashExpressions; - private QueryPlan[] hashPlans; - - public HashJoinPlan(BasicQueryPlan plan, HashJoinInfo joinInfo, - List[] hashExpressions, QueryPlan[] hashPlans) { - this.plan = plan; - this.joinInfo = joinInfo; - this.hashExpressions = hashExpressions; - this.hashPlans = hashPlans; - } - - @Override - public Integer getLimit() { - return plan.getLimit(); - } - - @Override - public OrderBy getOrderBy() { - return plan.getOrderBy(); - } - - @Override - public RowProjector getProjector() { - return plan.getProjector(); - } - - @Override - public ResultIterator iterator() throws SQLException { - ImmutableBytesPtr[] joinIds = joinInfo.getJoinIds(); - assert (joinIds.length == hashExpressions.length && joinIds.length == hashPlans.length); - - final HashCacheClient hashClient = new HashCacheClient(plan.getContext().getConnection()); - Scan scan = plan.getContext().getScan(); - final ScanRanges ranges = plan.getContext().getScanRanges(); - - int count = joinIds.length; - ConnectionQueryServices services = getContext().getConnection().getQueryServices(); - ExecutorService executor = services.getExecutor(); - List> futures = new ArrayList>(count); - List dependencies = new ArrayList(count); - for (int i = 0; i < count; i++) { - final int index = i; - futures.add(executor.submit(new JobCallable() { - - @Override - public ServerCache call() throws Exception { - QueryPlan hashPlan = hashPlans[index]; - return hashClient.addHashCache(ranges, hashPlan.iterator(), - hashPlan.getEstimatedSize(), hashExpressions[index], plan.getTableRef()); - } - - @Override - public Object getJobId() { - return HashJoinPlan.this; - } - })); - } - for (int i = 0; i < count; i++) { - try { - ServerCache cache = futures.get(i).get(); - joinIds[i].set(cache.getId()); - dependencies.add(cache); - } catch (InterruptedException e) { - throw new SQLException("Hash join execution interrupted.", e); - } catch (ExecutionException e) { - throw new SQLException("Encountered exception in hash plan execution.", - e.getCause()); - } - } - HashJoinInfo.serializeHashJoinIntoScan(scan, joinInfo); - - return plan.iterator(dependencies); - } - - @Override - public long getEstimatedSize() { - return plan.getEstimatedSize(); - } - - @Override - public List getSplits() { - return plan.getSplits(); - } - - @Override - public ExplainPlan getExplainPlan() throws SQLException { - List mainQuerySteps = plan.getExplainPlan().getPlanSteps(); - List planSteps = Lists.newArrayList(mainQuerySteps); - int count = hashPlans.length; - planSteps.add(" PARALLEL EQUI-JOIN " + count + " HASH TABLES:"); - for (int i = 0; i < count; i++) { - boolean skipMerge = joinInfo.getSchemas()[i].getFieldCount() == 0; - planSteps.add(" BUILD HASH TABLE " + i + (skipMerge ? " (SKIP MERGE)" : "")); - List steps = hashPlans[i].getExplainPlan().getPlanSteps(); - for (String step : steps) { - planSteps.add(" " + step); - } - } - if (joinInfo.getPostJoinFilterExpression() != null) { - planSteps.add(" AFTER-JOIN SERVER FILTER BY " + joinInfo.getPostJoinFilterExpression().toString()); - } - - return new ExplainPlan(planSteps); - } - - @Override - public ParameterMetaData getParameterMetaData() { - return plan.getParameterMetaData(); - } - - @Override - public StatementContext getContext() { - return plan.getContext(); - } - - @Override - public GroupBy getGroupBy() { - return plan.getGroupBy(); - } - - @Override - public TableRef getTableRef() { - return plan.getTableRef(); - } - - @Override - public FilterableStatement getStatement() { - return plan.getStatement(); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/ScanPlan.java b/phoenix-core/src/main/java/com/salesforce/phoenix/execute/ScanPlan.java deleted file mode 100644 index 4a0fc1ad..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/ScanPlan.java +++ /dev/null @@ -1,119 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.execute; - - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.OrderByCompiler.OrderBy; -import com.salesforce.phoenix.compile.RowProjector; -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.coprocessor.ScanRegionObserver; -import com.salesforce.phoenix.iterate.ConcatResultIterator; -import com.salesforce.phoenix.iterate.LimitingResultIterator; -import com.salesforce.phoenix.iterate.MergeSortRowKeyResultIterator; -import com.salesforce.phoenix.iterate.MergeSortTopNResultIterator; -import com.salesforce.phoenix.iterate.ParallelIterators; -import com.salesforce.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.iterate.SequenceResultIterator; -import com.salesforce.phoenix.iterate.SpoolingResultIterator; -import com.salesforce.phoenix.parse.FilterableStatement; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.SaltingUtil; -import com.salesforce.phoenix.schema.TableRef; - - - -/** - * - * Query plan for a basic table scan - * - * @author jtaylor - * @since 0.1 - */ -public class ScanPlan extends BasicQueryPlan { - private List splits; - - public ScanPlan(StatementContext context, FilterableStatement statement, TableRef table, RowProjector projector, Integer limit, OrderBy orderBy, ParallelIteratorFactory parallelIteratorFactory) { - super(context, statement, table, projector, context.getBindManager().getParameterMetaData(), limit, orderBy, null, parallelIteratorFactory == null ? new SpoolingResultIterator.SpoolingResultIteratorFactory(context.getConnection().getQueryServices()) : parallelIteratorFactory); - if (!orderBy.getOrderByExpressions().isEmpty()) { // TopN - int thresholdBytes = context.getConnection().getQueryServices().getProps().getInt( - QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB, QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES); - ScanRegionObserver.serializeIntoScan(context.getScan(), thresholdBytes, limit == null ? -1 : limit, orderBy.getOrderByExpressions(), projector.getEstimatedRowByteSize()); - } - } - - @Override - public List getSplits() { - return splits; - } - - @Override - protected ResultIterator newIterator() throws SQLException { - // Set any scan attributes before creating the scanner, as it will be too late afterwards - context.getScan().setAttribute(ScanRegionObserver.NON_AGGREGATE_QUERY, QueryConstants.TRUE); - ResultIterator scanner; - TableRef tableRef = this.getTableRef(); - PTable table = tableRef.getTable(); - boolean isSalted = table.getBucketNum() != null; - /* If no limit or topN, use parallel iterator so that we get results faster. Otherwise, if - * limit is provided, run query serially. - */ - boolean isOrdered = !orderBy.getOrderByExpressions().isEmpty(); - ParallelIterators iterators = new ParallelIterators(context, tableRef, statement, projection, GroupBy.EMPTY_GROUP_BY, isOrdered ? null : limit, parallelIteratorFactory); - splits = iterators.getSplits(); - if (isOrdered) { - scanner = new MergeSortTopNResultIterator(iterators, limit, orderBy.getOrderByExpressions()); - } else { - if (isSalted && - (getConnectionQueryServices(context.getConnection().getQueryServices()).getProps().getBoolean( - QueryServices.ROW_KEY_ORDER_SALTED_TABLE_ATTRIB, - QueryServicesOptions.DEFAULT_ROW_KEY_ORDER_SALTED_TABLE) || - orderBy == OrderBy.ROW_KEY_ORDER_BY)) { // ORDER BY was optimized out b/c query is in row key order - scanner = new MergeSortRowKeyResultIterator(iterators, SaltingUtil.NUM_SALTING_BYTES); - } else { - scanner = new ConcatResultIterator(iterators); - } - if (limit != null) { - scanner = new LimitingResultIterator(scanner, limit); - } - } - - if (context.getSequenceManager().getSequenceCount() > 0) { - scanner = new SequenceResultIterator(scanner, context.getSequenceManager()); - } - return scanner; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/AddExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/AddExpression.java deleted file mode 100644 index 749f6588..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/AddExpression.java +++ /dev/null @@ -1,64 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; - - -/** - * - * Subtract expression implementation - * - * @author jtaylor - * @since 0.1 - */ -public abstract class AddExpression extends BaseAddSubtractExpression { - public AddExpression() { - } - - public AddExpression(List children) { - super(children); - } - - @Override - public final T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } - - @Override - public String getOperatorString() { - return " + "; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/AndExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/AndExpression.java deleted file mode 100644 index 47fea8d9..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/AndExpression.java +++ /dev/null @@ -1,87 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; - - -/** - * - * AND expression implementation - * - * @author jtaylor - * @since 0.1 - */ -public class AndExpression extends AndOrExpression { - private static final String AND = "AND"; - - public static String combine(String expression1, String expression2) { - if (expression1 == null) { - return expression2; - } - if (expression2 == null) { - return expression1; - } - return "(" + expression1 + ") " + AND + " (" + expression2 + ")"; - } - - public AndExpression() { - } - - public AndExpression(List children) { - super(children); - } - - @Override - protected boolean getStopValue() { - return Boolean.FALSE; - } - - @Override - public String toString() { - StringBuilder buf = new StringBuilder("("); - for (int i = 0; i < children.size() - 1; i++) { - buf.append(children.get(i) + " " + AND + " "); - } - buf.append(children.get(children.size()-1)); - buf.append(')'); - return buf.toString(); - } - - @Override - public final T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/AndOrExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/AndOrExpression.java deleted file mode 100644 index c45dd843..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/AndOrExpression.java +++ /dev/null @@ -1,107 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.BitSet; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Abstract expression implementation for compound AND and OR expressions - * - * @author jtaylor - * @since 0.1 - */ -public abstract class AndOrExpression extends BaseCompoundExpression { - // Remember evaluation of child expression for partial evaluation - private BitSet partialEvalState; - - public AndOrExpression() { - } - - public AndOrExpression(List children) { - super(children); - } - - @Override - public int hashCode() { - return 31 * super.hashCode() + Boolean.valueOf(this.getStopValue()).hashCode(); - } - - @Override - public PDataType getDataType() { - return PDataType.BOOLEAN; - } - - @Override - public void reset() { - if (partialEvalState == null) { - partialEvalState = new BitSet(children.size()); - } else { - partialEvalState.clear(); - } - super.reset(); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - boolean isNull = false; - boolean stopValue = getStopValue(); - for (int i = 0; i < children.size(); i++) { - Expression child = children.get(i); - // If partial state is available, then use that to know we've already evaluated this - // child expression and do not need to do so again. - if (partialEvalState == null || !partialEvalState.get(i)) { - // Call through to child evaluate method matching parent call to allow child to optimize - // evaluate versus getValue code path. - if (child.evaluate(tuple, ptr)) { - // Short circuit if we see our stop value - if (Boolean.valueOf(stopValue).equals(PDataType.BOOLEAN.toObject(ptr, child.getDataType()))) { - return true; - } else if (partialEvalState != null) { - partialEvalState.set(i); - } - } else { - isNull = true; - } - } - } - if (isNull) { - return false; - } - return true; - } - - protected abstract boolean getStopValue(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ArithmeticExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ArithmeticExpression.java deleted file mode 100644 index 1721c623..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ArithmeticExpression.java +++ /dev/null @@ -1,53 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -public abstract class ArithmeticExpression extends BaseCompoundExpression { - - public ArithmeticExpression() { - } - - public ArithmeticExpression(List children) { - super(children); - } - - @Override - public String toString() { - StringBuilder buf = new StringBuilder("("); - for (int i = 0; i < children.size() - 1; i++) { - buf.append(children.get(i) + getOperatorString()); - } - buf.append(children.get(children.size()-1)); - buf.append(')'); - return buf.toString(); - } - - abstract protected String getOperatorString(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ArrayConstructorExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ArrayConstructorExpression.java deleted file mode 100644 index 63dc5a49..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ArrayConstructorExpression.java +++ /dev/null @@ -1,97 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2014, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; -import java.sql.Types; -import java.util.Arrays; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.io.WritableUtils; - -import com.salesforce.phoenix.schema.PArrayDataType; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PhoenixArray; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * Creates an expression for Upsert with Values/Select using ARRAY - */ -public class ArrayConstructorExpression extends BaseCompoundExpression { - private PDataType baseType; - private int position = -1; - private Object[] elements; - - - public ArrayConstructorExpression(List children, PDataType baseType) { - super(children); - init(baseType); - } - - private void init(PDataType baseType) { - this.baseType = baseType; - elements = new Object[getChildren().size()]; - } - - @Override - public PDataType getDataType() { - return PDataType.fromTypeId(baseType.getSqlType() + Types.ARRAY); - } - - @Override - public void reset() { - super.reset(); - position = 0; - Arrays.fill(elements, null); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - for (int i = position >= 0 ? position : 0; i < elements.length; i++) { - Expression child = children.get(i); - if (!child.evaluate(tuple, ptr)) { - if (tuple != null && !tuple.isImmutable()) { - if (position >= 0) position = i; - return false; - } - } else { - elements[i] = baseType.toObject(ptr, child.getDataType(), child.getColumnModifier()); - } - } - if (position >= 0) position = elements.length; - PhoenixArray array = PArrayDataType.instantiatePhoenixArray(baseType, elements); - // FIXME: Need to see if this creation of an array and again back to byte[] can be avoided - ptr.set(getDataType().toBytes(array)); - return true; - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - int baseTypeOrdinal = WritableUtils.readVInt(input); - init(PDataType.values()[baseTypeOrdinal]); - } - - @Override - public void write(DataOutput output) throws IOException { - super.write(output); - WritableUtils.writeVInt(output, baseType.ordinal()); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseAddSubtractExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseAddSubtractExpression.java deleted file mode 100644 index 1bd0d8f0..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseAddSubtractExpression.java +++ /dev/null @@ -1,61 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import com.salesforce.phoenix.schema.PDataType; - - -abstract public class BaseAddSubtractExpression extends ArithmeticExpression { - public BaseAddSubtractExpression() { - } - - public BaseAddSubtractExpression(List children) { - super(children); - } - - protected static Integer getPrecision(Integer lp, Integer rp, Integer ls, Integer rs) { - if (ls == null || rs == null) { - return PDataType.MAX_PRECISION; - } - int val = getScale(lp, rp, ls, rs) + Math.max(lp - ls, rp - rs) + 1; - return Math.min(PDataType.MAX_PRECISION, val); - } - - protected static Integer getScale(Integer lp, Integer rp, Integer ls, Integer rs) { - // If we are adding a decimal with scale and precision to a decimal - // with no precision nor scale, the scale system does not apply. - if (ls == null || rs == null) { - return null; - } - int val = Math.max(ls, rs); - return Math.min(PDataType.MAX_PRECISION, val); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseCompoundExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseCompoundExpression.java deleted file mode 100644 index 22bee05f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseCompoundExpression.java +++ /dev/null @@ -1,153 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; -import java.util.ArrayList; -import java.util.List; - -import org.apache.hadoop.io.WritableUtils; - -import com.google.common.collect.ImmutableList; -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; - - -public abstract class BaseCompoundExpression extends BaseExpression { - protected List children; - private boolean isNullable; - private boolean isStateless; - private boolean isDeterministic; - - public BaseCompoundExpression() { - } - - public BaseCompoundExpression(List children) { - init(children); - } - - private void init(List children) { - this.children = ImmutableList.copyOf(children); - boolean isStateless = true; - boolean isDeterministic = true; - boolean isNullable = false; - for (int i = 0; i < children.size(); i++) { - Expression child = children.get(i); - isNullable |= child.isNullable(); - isStateless &= child.isStateless(); - isDeterministic &= child.isDeterministic(); - } - this.isStateless = isStateless; - this.isDeterministic = isDeterministic; - this.isNullable = isNullable; - } - - @Override - public List getChildren() { - return children; - } - - - @Override - public boolean isDeterministic() { - return isDeterministic; - } - - @Override - public boolean isStateless() { - return isStateless; - } - - @Override - public boolean isNullable() { - return isNullable; - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + children.hashCode(); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - BaseCompoundExpression other = (BaseCompoundExpression)obj; - if (!children.equals(other.children)) return false; - return true; - } - - @Override - public void readFields(DataInput input) throws IOException { - int len = WritableUtils.readVInt(input); - Listchildren = new ArrayList(len); - for (int i = 0; i < len; i++) { - Expression child = ExpressionType.values()[WritableUtils.readVInt(input)].newInstance(); - child.readFields(input); - children.add(child); - } - init(children); - } - - @Override - public void write(DataOutput output) throws IOException { - WritableUtils.writeVInt(output, children.size()); - for (int i = 0; i < children.size(); i++) { - Expression child = children.get(i); - WritableUtils.writeVInt(output, ExpressionType.valueOf(child).ordinal()); - child.write(output); - } - } - - @Override - public void reset() { - for (int i = 0; i < children.size(); i++) { - children.get(i).reset(); - } - } - - @Override - public T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } - - @Override - public String toString() { - return this.getClass().getName() + " [children=" + children + "]"; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseDecimalAddSubtractExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseDecimalAddSubtractExpression.java deleted file mode 100644 index 9f88e4d1..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseDecimalAddSubtractExpression.java +++ /dev/null @@ -1,5 +0,0 @@ -package com.salesforce.phoenix.expression; - -public class BaseDecimalAddSubtractExpression { - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseExpression.java deleted file mode 100644 index 5d3ff48c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseExpression.java +++ /dev/null @@ -1,116 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; -import java.util.ArrayList; -import java.util.Collections; -import java.util.Iterator; -import java.util.List; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.ColumnModifier; - - - -/** - * - * Base class for Expression hierarchy that provides common - * default implementations for most methods - * - * @author jtaylor - * @since 0.1 - */ -public abstract class BaseExpression implements Expression { - @Override - public boolean isNullable() { - return false; - } - - @Override - public Integer getByteSize() { - return getDataType().isFixedWidth() ? getDataType().getByteSize() : null; - } - - @Override - public Integer getMaxLength() { - return null; - } - - @Override - public Integer getScale() { - return null; - } - - @Override - public ColumnModifier getColumnModifier() { - return null; - } - - @Override - public void readFields(DataInput input) throws IOException { - } - - @Override - public void write(DataOutput output) throws IOException { - } - - @Override - public void reset() { - } - - protected final List acceptChildren(ExpressionVisitor visitor, Iterator iterator) { - if (iterator == null) { - iterator = visitor.defaultIterator(this); - } - List l = Collections.emptyList(); - while (iterator.hasNext()) { - Expression child = iterator.next(); - T t = child.accept(visitor); - if (t != null) { - if (l.isEmpty()) { - l = new ArrayList(getChildren().size()); - } - l.add(t); - } - } - return l; - } - - @Override - public boolean isDeterministic() { - return true; - } - - @Override - public boolean isStateless() { - return false; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseSingleExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseSingleExpression.java deleted file mode 100644 index 43dc8a38..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseSingleExpression.java +++ /dev/null @@ -1,99 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; -import java.util.List; - -import org.apache.hadoop.io.WritableUtils; - -import com.google.common.collect.ImmutableList; -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; - - -/** - * - * Base class for expressions which have a single child expression - * - * @author jtaylor - * @since 0.1 - */ -public abstract class BaseSingleExpression extends BaseExpression { - - protected List children; - - public BaseSingleExpression() { - } - - public BaseSingleExpression(Expression expression) { - this.children = ImmutableList.of(expression); - } - - @Override - public List getChildren() { - return children; - } - - @Override - public void readFields(DataInput input) throws IOException { - Expression expression = ExpressionType.values()[WritableUtils.readVInt(input)].newInstance(); - expression.readFields(input); - children = ImmutableList.of(expression); - } - - @Override - public void write(DataOutput output) throws IOException { - WritableUtils.writeVInt(output, ExpressionType.valueOf(children.get(0)).ordinal()); - children.get(0).write(output); - } - - @Override - public boolean isNullable() { - return children.get(0).isNullable(); - } - - @Override - public void reset() { - children.get(0).reset(); - } - - @Override - public T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, null); - if (l.isEmpty()) { - return visitor.defaultReturn(this, l); - } - return l.get(0); - } - - public Expression getChild() { - return children.get(0); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseTerminalExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseTerminalExpression.java deleted file mode 100644 index e19ea68d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/BaseTerminalExpression.java +++ /dev/null @@ -1,55 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.Collections; -import java.util.List; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; - - - -/** - * - * Grouping class for expression that have no expression children - * - * @author jtaylor - * @since 0.1 - */ -public abstract class BaseTerminalExpression extends BaseExpression { - @Override - public List getChildren() { - return Collections.emptyList(); - } - - @Override - public T accept(ExpressionVisitor visitor) { - return null; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/CoerceExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/CoerceExpression.java deleted file mode 100644 index 1e35fce8..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/CoerceExpression.java +++ /dev/null @@ -1,161 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.io.WritableUtils; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -public class CoerceExpression extends BaseSingleExpression { - private PDataType toType; - private ColumnModifier toMod; - private Integer byteSize; - - public CoerceExpression() { - } - - public static Expression create(Expression expression, PDataType toType) throws SQLException { - if (toType == expression.getDataType()) { - return expression; - } - return new CoerceExpression(expression, toType); - } - - //Package protected for tests - CoerceExpression(Expression expression, PDataType toType) { - this(expression, toType, null, null); - } - - CoerceExpression(Expression expression, PDataType toType, ColumnModifier toMod, Integer byteSize) { - super(expression); - this.toType = toType; - this.toMod = toMod; - this.byteSize = byteSize; - } - - @Override - public Integer getByteSize() { - return byteSize; - } - - @Override - public Integer getMaxLength() { - return byteSize; - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + ((byteSize == null) ? 0 : byteSize.hashCode()); - result = prime * result + ((toMod == null) ? 0 : toMod.hashCode()); - result = prime * result + ((toType == null) ? 0 : toType.hashCode()); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - CoerceExpression other = (CoerceExpression)obj; - if (byteSize == null) { - if (other.byteSize != null) return false; - } else if (!byteSize.equals(other.byteSize)) return false; - if (toMod != other.toMod) return false; - if (toType != other.toType) return false; - return true; - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - toType = PDataType.values()[WritableUtils.readVInt(input)]; - toMod = ColumnModifier.fromSystemValue(WritableUtils.readVInt(input)); - int byteSize = WritableUtils.readVInt(input); - this.byteSize = byteSize == -1 ? null : byteSize; - } - - @Override - public void write(DataOutput output) throws IOException { - super.write(output); - WritableUtils.writeVInt(output, toType.ordinal()); - WritableUtils.writeVInt(output, ColumnModifier.toSystemValue(toMod)); - WritableUtils.writeVInt(output, byteSize == null ? -1 : byteSize); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (getChild().evaluate(tuple, ptr)) { - getDataType().coerceBytes(ptr, getChild().getDataType(), getChild().getColumnModifier(), getColumnModifier()); - return true; - } - return false; - } - - @Override - public PDataType getDataType() { - return toType; - } - - @Override - public ColumnModifier getColumnModifier() { - return toMod; - } - - @Override - public T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } - - @Override - public String toString() { - StringBuilder buf = new StringBuilder("TO_" + toType.toString() + "("); - for (int i = 0; i < children.size() - 1; i++) { - buf.append(children.get(i) + ", "); - } - buf.append(children.get(children.size()-1) + ")"); - return buf.toString(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ColumnExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ColumnExpression.java deleted file mode 100644 index ba78d086..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ColumnExpression.java +++ /dev/null @@ -1,162 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; - -import org.apache.hadoop.io.WritableUtils; - -import com.google.common.base.Objects; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDatum; - -/** - * - * Common base class for column value accessors - * - * @author jtaylor - * @since 0.1 - */ -abstract public class ColumnExpression extends BaseTerminalExpression { - protected PDataType type; - private Integer byteSize; - private boolean isNullable; - private Integer maxLength; - private Integer scale; - private ColumnModifier columnModifier; - - public ColumnExpression() { - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + (isNullable() ? 1231 : 1237); - Integer maxLength = this.getByteSize(); - result = prime * result + ((maxLength == null) ? 0 : maxLength.hashCode()); - PDataType type = this.getDataType(); - result = prime * result + ((type == null) ? 0 : type.hashCode()); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - ColumnExpression other = (ColumnExpression)obj; - if (this.isNullable() != other.isNullable()) return false; - if (!Objects.equal(this.getByteSize(),other.getByteSize())) return false; - if (this.getDataType() != other.getDataType()) return false; - return true; - } - - public ColumnExpression(PDatum datum) { - this.type = datum.getDataType(); - this.isNullable = datum.isNullable(); - if (type.isFixedWidth() && type.getByteSize() == null) { - this.byteSize = datum.getByteSize(); - } - this.maxLength = datum.getMaxLength(); - this.scale = datum.getScale(); - this.columnModifier = datum.getColumnModifier(); - } - - @Override - public boolean isNullable() { - return isNullable; - } - - @Override - public PDataType getDataType() { - return type; - } - - @Override - public ColumnModifier getColumnModifier() { - return columnModifier; - } - - @Override - public Integer getByteSize() { - if (byteSize != null) { - return byteSize; - } - return super.getByteSize(); - } - - @Override - public Integer getMaxLength() { - return maxLength; - } - - @Override - public Integer getScale() { - return scale; - } - - @Override - public void readFields(DataInput input) throws IOException { - // read/write type ordinal, maxLength presence, scale presence and isNullable bit together to save space - int typeAndFlag = WritableUtils.readVInt(input); - isNullable = (typeAndFlag & 0x01) != 0; - if ((typeAndFlag & 0x02) != 0) { - scale = WritableUtils.readVInt(input); - } - if ((typeAndFlag & 0x04) != 0) { - maxLength = WritableUtils.readVInt(input); - } - type = PDataType.values()[typeAndFlag >>> 3]; - if (type.isFixedWidth() && type.getByteSize() == null) { - byteSize = WritableUtils.readVInt(input); - } - columnModifier = ColumnModifier.fromSystemValue(WritableUtils.readVInt(input)); - } - - @Override - public void write(DataOutput output) throws IOException { - // read/write type ordinal, maxLength presence, scale presence and isNullable bit together to save space - int typeAndFlag = (isNullable ? 1 : 0) | ((scale != null ? 1 : 0) << 1) | ((maxLength != null ? 1 : 0) << 2) - | (type.ordinal() << 3); - WritableUtils.writeVInt(output,typeAndFlag); - if (scale != null) { - WritableUtils.writeVInt(output, scale); - } - if (maxLength != null) { - WritableUtils.writeVInt(output, maxLength); - } - if (type.isFixedWidth() && type.getByteSize() == null) { - WritableUtils.writeVInt(output, byteSize); - } - WritableUtils.writeVInt(output, ColumnModifier.toSystemValue(columnModifier)); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ComparisonExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ComparisonExpression.java deleted file mode 100644 index 9a74539d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ComparisonExpression.java +++ /dev/null @@ -1,167 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; -import java.util.List; - -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.io.WritableUtils; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.StringUtil; - - -/** - * - * Implementation for <,<=,>,>=,=,!= comparison expressions - * @author jtaylor - * @since 0.1 - */ -public class ComparisonExpression extends BaseCompoundExpression { - private CompareOp op; - private static final String[] CompareOpString = new String[CompareOp.values().length]; - static { - CompareOpString[CompareOp.EQUAL.ordinal()] = " = "; - CompareOpString[CompareOp.NOT_EQUAL.ordinal()] = " != "; - CompareOpString[CompareOp.GREATER.ordinal()] = " > "; - CompareOpString[CompareOp.LESS.ordinal()] = " < "; - CompareOpString[CompareOp.GREATER_OR_EQUAL.ordinal()] = " >= "; - CompareOpString[CompareOp.LESS_OR_EQUAL.ordinal()] = " <= "; - } - - public ComparisonExpression() { - } - - public ComparisonExpression(CompareOp op, List children) { - super(children); - if (op == null) { - throw new NullPointerException(); - } - this.op = op; - } - - @Override - public int hashCode() { - final int prime = 31; - int result = super.hashCode(); - result = prime * result + op.hashCode(); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (!super.equals(obj)) return false; - if (getClass() != obj.getClass()) return false; - ComparisonExpression other = (ComparisonExpression)obj; - if (op != other.op) return false; - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.BOOLEAN; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (!children.get(0).evaluate(tuple, ptr)) { - return false; - } - byte[] lhsBytes = ptr.get(); - int lhsOffset = ptr.getOffset(); - int lhsLength = ptr.getLength(); - PDataType lhsDataType = children.get(0).getDataType(); - ColumnModifier lhsColumnModifier = children.get(0).getColumnModifier(); - - if (!children.get(1).evaluate(tuple, ptr)) { - return false; - } - - byte[] rhsBytes = ptr.get(); - int rhsOffset = ptr.getOffset(); - int rhsLength = ptr.getLength(); - PDataType rhsDataType = children.get(1).getDataType(); - ColumnModifier rhsColumnModifier = children.get(1).getColumnModifier(); - if (rhsDataType == PDataType.CHAR) { - rhsLength = StringUtil.getUnpaddedCharLength(rhsBytes, rhsOffset, rhsLength, rhsColumnModifier); - } - if (lhsDataType == PDataType.CHAR) { - lhsLength = StringUtil.getUnpaddedCharLength(lhsBytes, lhsOffset, lhsLength, lhsColumnModifier); - } - - - int comparisonResult = lhsDataType.compareTo(lhsBytes, lhsOffset, lhsLength, lhsColumnModifier, - rhsBytes, rhsOffset, rhsLength, rhsColumnModifier, rhsDataType); - ptr.set(ByteUtil.compare(op, comparisonResult) ? PDataType.TRUE_BYTES : PDataType.FALSE_BYTES); - return true; - } - - @Override - public void readFields(DataInput input) throws IOException { - op = CompareOp.values()[WritableUtils.readVInt(input)]; - super.readFields(input); - } - - @Override - public void write(DataOutput output) throws IOException { - WritableUtils.writeVInt(output, op.ordinal()); - super.write(output); - } - - @Override - public final T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } - - public CompareOp getFilterOp() { - return op; - } - - public static String toString(CompareOp op, List children) { - return (children.get(0) + CompareOpString[op.ordinal()] + children.get(1)); - } - - @Override - public String toString() { - return toString(getFilterOp(), children); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/CurrentDateTimeFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/CurrentDateTimeFunction.java deleted file mode 100644 index f363af71..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/CurrentDateTimeFunction.java +++ /dev/null @@ -1,52 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import com.salesforce.phoenix.expression.function.ScalarFunction; - -public abstract class CurrentDateTimeFunction extends ScalarFunction { - - public CurrentDateTimeFunction() { - } - - public CurrentDateTimeFunction(List children) { - super(children); - } - - @Override - public boolean isStateless() { - return true; - } - - @Override - public boolean isDeterministic() { - return false; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DateAddExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DateAddExpression.java deleted file mode 100644 index 9b8827f8..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DateAddExpression.java +++ /dev/null @@ -1,88 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.math.BigDecimal; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -public class DateAddExpression extends AddExpression { - static private final BigDecimal BD_MILLIS_IN_DAY = BigDecimal.valueOf(QueryConstants.MILLIS_IN_DAY); - - public DateAddExpression() { - } - - public DateAddExpression(List children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - long finalResult=0; - - for(int i=0;i children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - long finalResult=0; - - for(int i=0;i children) { - super(children); - for (int i=0; i children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - BigDecimal result = null; - for (int i=0; i children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - BigDecimal result = null; - for (int i=0; i children) { - super(children); - for (int i=0; i children) { - super(children); - for (int i=0; i T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } - - @Override - public String getOperatorString() { - return " / "; - } - - private static Integer getPrecision(Integer lp, Integer rp, Integer ls, Integer rs) { - if (ls == null || rs == null) { - return PDataType.MAX_PRECISION; - } - int val = getScale(lp, rp, ls, rs) + lp - ls + rp; - return Math.min(PDataType.MAX_PRECISION, val); - } - - private static Integer getScale(Integer lp, Integer rp, Integer ls, Integer rs) { - // If we are adding a decimal with scale and precision to a decimal - // with no precision nor scale, the scale system does not apply. - if (ls == null || rs == null) { - return null; - } - int val = Math.max(PDataType.MAX_PRECISION - lp + ls - rs, 0); - return Math.min(PDataType.MAX_PRECISION, val); - } - - @Override - public Integer getScale() { - return scale; - } - - @Override - public Integer getMaxLength() { - return maxLength; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleAddExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleAddExpression.java deleted file mode 100644 index d17231e2..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleAddExpression.java +++ /dev/null @@ -1,78 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -public class DoubleAddExpression extends AddExpression { - - public DoubleAddExpression() { - } - - public DoubleAddExpression(List children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - double result = 0.0; - for (int i = 0; i < children.size(); i++) { - Expression child = children.get(i); - if (!child.evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - return true; - } - double childvalue = child.getDataType().getCodec() - .decodeDouble(ptr, child.getColumnModifier()); - if (!Double.isNaN(childvalue) - && childvalue != Double.NEGATIVE_INFINITY - && childvalue != Double.POSITIVE_INFINITY) { - result += childvalue; - } else { - return false; - } - } - byte[] resultPtr = new byte[getDataType().getByteSize()]; - ptr.set(resultPtr); - getDataType().getCodec().encodeDouble(result, ptr); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.DOUBLE; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleDivideExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleDivideExpression.java deleted file mode 100644 index 668e48c5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleDivideExpression.java +++ /dev/null @@ -1,82 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -public class DoubleDivideExpression extends DivideExpression { - - public DoubleDivideExpression() { - } - - public DoubleDivideExpression(List children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - double result = 0.0; - for (int i = 0; i < children.size(); i++) { - Expression child = children.get(i); - if (!child.evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - return true; - } - double childvalue = child.getDataType().getCodec() - .decodeDouble(ptr, child.getColumnModifier()); - if (!Double.isNaN(childvalue) - && childvalue != Double.NEGATIVE_INFINITY - && childvalue != Double.POSITIVE_INFINITY) { - if (i == 0) { - result = childvalue; - } else { - result /= childvalue; - } - } else { - return false; - } - } - byte[] resultPtr = new byte[getDataType().getByteSize()]; - ptr.set(resultPtr); - getDataType().getCodec().encodeDouble(result, ptr); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.DOUBLE; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleMultiplyExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleMultiplyExpression.java deleted file mode 100644 index 0a9f9468..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleMultiplyExpression.java +++ /dev/null @@ -1,78 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -public class DoubleMultiplyExpression extends MultiplyExpression { - - public DoubleMultiplyExpression() { - } - - public DoubleMultiplyExpression(List children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - double result = 1.0; - for (int i = 0; i < children.size(); i++) { - Expression child = children.get(i); - if (!child.evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - return true; - } - double childvalue = child.getDataType().getCodec() - .decodeDouble(ptr, child.getColumnModifier()); - if (!Double.isNaN(childvalue) - && childvalue != Double.NEGATIVE_INFINITY - && childvalue != Double.POSITIVE_INFINITY) { - result *= childvalue; - } else { - return false; - } - } - byte[] resultPtr = new byte[getDataType().getByteSize()]; - ptr.set(resultPtr); - getDataType().getCodec().encodeDouble(result, ptr); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.DOUBLE; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleSubtractExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleSubtractExpression.java deleted file mode 100644 index e25b3f74..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/DoubleSubtractExpression.java +++ /dev/null @@ -1,82 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -public class DoubleSubtractExpression extends SubtractExpression { - - public DoubleSubtractExpression() { - } - - public DoubleSubtractExpression(List children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - double result = 0.0; - for (int i = 0; i < children.size(); i++) { - Expression child = children.get(i); - if (!child.evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - return true; - } - double childvalue = child.getDataType().getCodec() - .decodeDouble(ptr, child.getColumnModifier()); - if (!Double.isNaN(childvalue) - && childvalue != Double.NEGATIVE_INFINITY - && childvalue != Double.POSITIVE_INFINITY) { - if (i == 0) { - result = childvalue; - } else { - result -= childvalue; - } - } else { - return false; - } - } - byte[] resultPtr = new byte[getDataType().getByteSize()]; - ptr.set(resultPtr); - getDataType().getCodec().encodeDouble(result, ptr); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.DOUBLE; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/Expression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/Expression.java deleted file mode 100644 index 1138ecfd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/Expression.java +++ /dev/null @@ -1,90 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.io.Writable; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.PDatum; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Interface for general expression evaluation - * - * @author jtaylor - * @since 0.1 - */ -public interface Expression extends PDatum, Writable { - /** - * Access the value by setting a pointer to it (as opposed to making - * a copy of it which can be expensive) - * @param tuple Single row result during scan iteration - * @param ptr Pointer to byte value being accessed - * @return true if the expression could be evaluated (i.e. ptr was set) - * and false otherwise - */ - boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr); - - /** - * Means of traversing expression tree through visitor. - * @param visitor - */ - T accept(ExpressionVisitor visitor); - - /** - * @return the child expressions - */ - List getChildren(); - - /** - * Resets the state of a expression back to its initial state and - * enables the expession to be evaluated incrementally (which - * occurs during filter evaluation where we see one key value at - * a time; it's possible to evaluate immediately rather than - * wait until all key values have been seen). Note that when - * evaluating incrementally, you must call this method before - * processing a new row. - */ - void reset(); - - /** - * @return true if the expression can be evaluated without - * requiring a row Tuple and false otherwise. - */ - boolean isStateless(); - /** - * @return true if the expression returns the same output every - * time given the same input. - */ - boolean isDeterministic(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ExpressionType.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ExpressionType.java deleted file mode 100644 index 5fb3dbd3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ExpressionType.java +++ /dev/null @@ -1,205 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.Map; - -import com.google.common.collect.Maps; -import com.salesforce.phoenix.expression.function.ArrayIndexFunction; -import com.salesforce.phoenix.expression.function.ArrayLengthFunction; -import com.salesforce.phoenix.expression.function.CeilDateExpression; -import com.salesforce.phoenix.expression.function.CeilDecimalExpression; -import com.salesforce.phoenix.expression.function.CeilFunction; -import com.salesforce.phoenix.expression.function.CeilTimestampExpression; -import com.salesforce.phoenix.expression.function.CoalesceFunction; -import com.salesforce.phoenix.expression.function.CountAggregateFunction; -import com.salesforce.phoenix.expression.function.DistinctCountAggregateFunction; -import com.salesforce.phoenix.expression.function.FloorDateExpression; -import com.salesforce.phoenix.expression.function.FloorDecimalExpression; -import com.salesforce.phoenix.expression.function.FloorFunction; -import com.salesforce.phoenix.expression.function.IndexStateNameFunction; -import com.salesforce.phoenix.expression.function.InvertFunction; -import com.salesforce.phoenix.expression.function.LTrimFunction; -import com.salesforce.phoenix.expression.function.LengthFunction; -import com.salesforce.phoenix.expression.function.LowerFunction; -import com.salesforce.phoenix.expression.function.MD5Function; -import com.salesforce.phoenix.expression.function.MaxAggregateFunction; -import com.salesforce.phoenix.expression.function.MinAggregateFunction; -import com.salesforce.phoenix.expression.function.PercentRankAggregateFunction; -import com.salesforce.phoenix.expression.function.PercentileContAggregateFunction; -import com.salesforce.phoenix.expression.function.PercentileDiscAggregateFunction; -import com.salesforce.phoenix.expression.function.RTrimFunction; -import com.salesforce.phoenix.expression.function.RegexpReplaceFunction; -import com.salesforce.phoenix.expression.function.RegexpSubstrFunction; -import com.salesforce.phoenix.expression.function.ReverseFunction; -import com.salesforce.phoenix.expression.function.RoundDateExpression; -import com.salesforce.phoenix.expression.function.RoundDecimalExpression; -import com.salesforce.phoenix.expression.function.RoundFunction; -import com.salesforce.phoenix.expression.function.RoundTimestampExpression; -import com.salesforce.phoenix.expression.function.SQLTableTypeFunction; -import com.salesforce.phoenix.expression.function.SQLViewTypeFunction; -import com.salesforce.phoenix.expression.function.SqlTypeNameFunction; -import com.salesforce.phoenix.expression.function.StddevPopFunction; -import com.salesforce.phoenix.expression.function.StddevSampFunction; -import com.salesforce.phoenix.expression.function.SubstrFunction; -import com.salesforce.phoenix.expression.function.SumAggregateFunction; -import com.salesforce.phoenix.expression.function.ToCharFunction; -import com.salesforce.phoenix.expression.function.ToDateFunction; -import com.salesforce.phoenix.expression.function.ToNumberFunction; -import com.salesforce.phoenix.expression.function.TrimFunction; -import com.salesforce.phoenix.expression.function.TruncFunction; -import com.salesforce.phoenix.expression.function.UpperFunction; - -/** - * - * Enumeration of all Expression types that may be evaluated on the server-side. - * Used during serialization and deserialization to pass Expression between client - * and server. - * - * @author jtaylor - * @since 0.1 - */ -public enum ExpressionType { - ReverseFunction(ReverseFunction.class), - RowKey(RowKeyColumnExpression.class), - KeyValue(KeyValueColumnExpression.class), - LiteralValue(LiteralExpression.class), - RoundFunction(RoundFunction.class), - FloorFunction(FloorFunction.class), - CeilFunction(CeilFunction.class), - RoundDateExpression(RoundDateExpression.class), - FloorDateExpression(FloorDateExpression.class), - CeilDateExpression(CeilDateExpression.class), - RoundTimestampExpression(RoundTimestampExpression.class), - CeilTimestampExpression(CeilTimestampExpression.class), - RoundDecimalExpression(RoundDecimalExpression.class), - FloorDecimalExpression(FloorDecimalExpression.class), - CeilDecimalExpression(CeilDecimalExpression.class), - TruncFunction(TruncFunction.class), - ToDateFunction(ToDateFunction.class), - ToCharFunction(ToCharFunction.class), - ToNumberFunction(ToNumberFunction.class), - CoerceFunction(CoerceExpression.class), - SubstrFunction(SubstrFunction.class), - AndExpression(AndExpression.class), - OrExpression(OrExpression.class), - ComparisonExpression(ComparisonExpression.class), - CountAggregateFunction(CountAggregateFunction.class), - SumAggregateFunction(SumAggregateFunction.class), - MinAggregateFunction(MinAggregateFunction.class), - MaxAggregateFunction(MaxAggregateFunction.class), - LikeExpression(LikeExpression.class), - NotExpression(NotExpression.class), - CaseExpression(CaseExpression.class), - InListExpression(InListExpression.class), - IsNullExpression(IsNullExpression.class), - LongSubtractExpression(LongSubtractExpression.class), - DateSubtractExpression(DateSubtractExpression.class), - DecimalSubtractExpression(DecimalSubtractExpression.class), - LongAddExpression(LongAddExpression.class), - DecimalAddExpression(DecimalAddExpression.class), - DateAddExpression(DateAddExpression.class), - LongMultiplyExpression(LongMultiplyExpression.class), - DecimalMultiplyExpression(DecimalMultiplyExpression.class), - LongDivideExpression(LongDivideExpression.class), - DecimalDivideExpression(DecimalDivideExpression.class), - CoalesceFunction(CoalesceFunction.class), - RegexpReplaceFunction(RegexpReplaceFunction.class), - SQLTypeNameFunction(SqlTypeNameFunction.class), - RegexpSubstrFunction(RegexpSubstrFunction.class), - StringConcatExpression(StringConcatExpression.class), - LengthFunction(LengthFunction.class), - LTrimFunction(LTrimFunction.class), - RTrimFunction(RTrimFunction.class), - UpperFunction(UpperFunction.class), - LowerFunction(LowerFunction.class), - TrimFunction(TrimFunction.class), - DistinctCountAggregateFunction(DistinctCountAggregateFunction.class), - PercentileContAggregateFunction(PercentileContAggregateFunction.class), - PercentRankAggregateFunction(PercentRankAggregateFunction.class), - StddevPopFunction(StddevPopFunction.class), - StddevSampFunction(StddevSampFunction.class), - PercentileDiscAggregateFunction(PercentileDiscAggregateFunction.class), - DoubleAddExpression(DoubleAddExpression.class), - DoubleSubtractExpression(DoubleSubtractExpression.class), - DoubleMultiplyExpression(DoubleMultiplyExpression.class), - DoubleDivideExpression(DoubleDivideExpression.class), - RowValueConstructorExpression(RowValueConstructorExpression.class), - MD5Function(MD5Function.class), - SQLTableTypeFunction(SQLTableTypeFunction.class), - IndexKeyValue(IndexKeyValueColumnExpression.class), - IndexStateName(IndexStateNameFunction.class), - InvertFunction(InvertFunction.class), - ProjectedColumnExpression(ProjectedColumnExpression.class), - TimestampAddExpression(TimestampAddExpression.class), - TimestampSubtractExpression(TimestampSubtractExpression.class), - ArrayIndexFunction(ArrayIndexFunction.class), - ArrayLengthFunction(ArrayLengthFunction.class), - ArrayConstructorExpression(ArrayConstructorExpression.class), - SQLViewTypeFunction(SQLViewTypeFunction.class); - ExpressionType(Class clazz) { - this.clazz = clazz; - } - - public Class getExpressionClass() { - return clazz; - } - - private final Class clazz; - - private static final Map,ExpressionType> classToEnumMap = Maps.newHashMapWithExpectedSize(3); - static { - for (ExpressionType type : ExpressionType.values()) { - classToEnumMap.put(type.clazz, type); - } - } - - /** - * Return the ExpressionType for a given Expression instance - */ - public static ExpressionType valueOf(Expression expression) { - ExpressionType type = classToEnumMap.get(expression.getClass()); - if (type == null) { // FIXME: this exception gets swallowed and retries happen - throw new IllegalArgumentException("No ExpressionType for " + expression.getClass()); - } - return type; - } - - /** - * Instantiates a DataAccessor based on its DataAccessorType - */ - public Expression newInstance() { - try { - return clazz.newInstance(); - } catch (InstantiationException e) { - throw new RuntimeException(e); - } catch (IllegalAccessException e) { - throw new RuntimeException(e); - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/IsNullExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/IsNullExpression.java deleted file mode 100644 index 0e26781b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/IsNullExpression.java +++ /dev/null @@ -1,114 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.io.*; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Implementation of IS NULL and IS NOT NULL expression - * - * @author jtaylor - * @since 0.1 - */ -public class IsNullExpression extends BaseSingleExpression { - private boolean isNegate; - - public IsNullExpression() { - } - - public IsNullExpression(Expression expression, boolean negate) { - super(expression); - this.isNegate = negate; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - boolean evaluated = getChild().evaluate(tuple, ptr); - if (evaluated) { - ptr.set(isNegate ^ ptr.getLength() == 0 ? PDataType.TRUE_BYTES : PDataType.FALSE_BYTES); - return true; - } - if (tuple.isImmutable()) { - ptr.set(isNegate ? PDataType.FALSE_BYTES : PDataType.TRUE_BYTES); - return true; - } - - return false; - } - - public boolean isNegate() { - return isNegate; - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - isNegate = input.readBoolean(); - } - - @Override - public void write(DataOutput output) throws IOException { - super.write(output); - output.writeBoolean(isNegate); - } - - @Override - public PDataType getDataType() { - return PDataType.BOOLEAN; - } - - @Override - public final T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } - - @Override - public String toString() { - StringBuilder buf = new StringBuilder(children.get(0).toString()); - if (isNegate) { - buf.append(" IS NOT NULL"); - } else { - buf.append(" IS NULL"); - } - return buf.toString(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/KeyValueColumnExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/KeyValueColumnExpression.java deleted file mode 100644 index 8a155389..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/KeyValueColumnExpression.java +++ /dev/null @@ -1,121 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; -import java.util.Arrays; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.SchemaUtil; - - -/** - * - * Class to access a column value stored in a KeyValue - * - * @author jtaylor - * @since 0.1 - */ -public class KeyValueColumnExpression extends ColumnExpression { - private byte[] cf; - private byte[] cq; - - public KeyValueColumnExpression() { - } - - public KeyValueColumnExpression(PColumn column) { - super(column); - this.cf = column.getFamilyName().getBytes(); - this.cq = column.getName().getBytes(); - } - - public byte[] getColumnFamily() { - return cf; - } - - public byte[] getColumnName() { - return cq; - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + Arrays.hashCode(cf); - result = prime * result + Arrays.hashCode(cq); - return result; - } - - // TODO: assumes single table - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - KeyValueColumnExpression other = (KeyValueColumnExpression)obj; - if (!Arrays.equals(cf, other.cf)) return false; - if (!Arrays.equals(cq, other.cq)) return false; - return true; - } - - @Override - public String toString() { - return SchemaUtil.getColumnDisplayName(cf, cq); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - return tuple.getValue(cf, cq, ptr); - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - cf = Bytes.readByteArray(input); - cq = Bytes.readByteArray(input); - } - - @Override - public void write(DataOutput output) throws IOException { - super.write(output); - Bytes.writeByteArray(output, cf); - Bytes.writeByteArray(output, cq); - } - - @Override - public final T accept(ExpressionVisitor visitor) { - return visitor.visit(this); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/LongAddExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/LongAddExpression.java deleted file mode 100644 index 2e2fe9bd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/LongAddExpression.java +++ /dev/null @@ -1,70 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -public class LongAddExpression extends AddExpression { - - public LongAddExpression() { - } - - public LongAddExpression(List children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - long finalResult=0; - - for(int i=0;i children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - long finalResult=0; - - for(int i=0;i children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - long finalResult=1; - - for(int i=0;i children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - long finalResult=0; - - for(int i=0;i children) { - super(children); - for (int i=0; i T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } - - @Override - public String getOperatorString() { - return " * "; - } - - private static Integer getPrecision(Integer lp, Integer rp, Integer ls, Integer rs) { - if (ls == null || rs == null) { - return PDataType.MAX_PRECISION; - } - int val = lp + rp; - return Math.min(PDataType.MAX_PRECISION, val); - } - - private static Integer getScale(Integer lp, Integer rp, Integer ls, Integer rs) { - // If we are adding a decimal with scale and precision to a decimal - // with no precision nor scale, the scale system does not apply. - if (ls == null || rs == null) { - return null; - } - int val = ls + rs; - return Math.min(PDataType.MAX_PRECISION, val); - } - - @Override - public Integer getScale() { - return scale; - } - - @Override - public Integer getMaxLength() { - return maxLength; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/NotExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/NotExpression.java deleted file mode 100644 index 0c59fd37..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/NotExpression.java +++ /dev/null @@ -1,94 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Implementation of the NOT operator that negates it's - * single boolean child expression. - * - * @author jtaylor - * @since 0.1 - */ -public class NotExpression extends BaseSingleExpression { - - public NotExpression() { - } - - public NotExpression(Expression expression) { - super(expression); - if (expression == null || expression.getDataType() != PDataType.BOOLEAN) { - throw new IllegalArgumentException("NOT expectes a single BOOLEAN expression, but got " + expression); - } - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (!getChild().evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - return true; - } - - ptr.set(Boolean.TRUE.equals(PDataType.BOOLEAN.toObject(ptr)) ? PDataType.FALSE_BYTES : PDataType.TRUE_BYTES); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.BOOLEAN; - } - - @Override - public final T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } - - @Override - public String toString() { - StringBuilder buf = new StringBuilder("NOT ("); - buf.append(children.get(0).toString()); - buf.append(")"); - return buf.toString(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/OrExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/OrExpression.java deleted file mode 100644 index ed2faf80..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/OrExpression.java +++ /dev/null @@ -1,75 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.util.List; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; - - -/** - * - * OR expression implementation - * - * @author jtaylor - * @since 0.1 - */ -public class OrExpression extends AndOrExpression { - public OrExpression() { - } - - public OrExpression(List children) { - super(children); - } - - @Override - protected boolean getStopValue() { - return Boolean.TRUE; - } - - @Override - public String toString() { - StringBuilder buf = new StringBuilder("("); - for (int i = 0; i < children.size() - 1; i++) { - buf.append(children.get(i) + " OR "); - } - buf.append(children.get(children.size()-1)); - buf.append(')'); - return buf.toString(); - } - - @Override - public final T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ProjectedColumnExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ProjectedColumnExpression.java deleted file mode 100644 index f0a95292..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/ProjectedColumnExpression.java +++ /dev/null @@ -1,152 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.join.ScanProjector; -import com.salesforce.phoenix.schema.KeyValueSchema; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.ValueBitSet; -import com.salesforce.phoenix.schema.KeyValueSchema.KeyValueSchemaBuilder; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.SchemaUtil; - -public class ProjectedColumnExpression extends ColumnExpression { - private KeyValueSchema schema; - ValueBitSet bitSet; - private int position; - private byte[] name; // for display purpose only - - public ProjectedColumnExpression() { - } - - public ProjectedColumnExpression(PColumn column, PTable table) { - super(column); - this.schema = buildSchema(table); - this.bitSet = ValueBitSet.newInstance(schema); - this.position = column.getPosition() - table.getPKColumns().size(); - this.name = column.getName().getBytes(); - } - - private static KeyValueSchema buildSchema(PTable table) { - KeyValueSchemaBuilder builder = new KeyValueSchemaBuilder(0); - for (PColumn column : table.getColumns()) { - if (!SchemaUtil.isPKColumn(column)) { - builder.addField(column); - } - } - return builder.build(); - } - - public KeyValueSchema getSchema() { - return schema; - } - - public int getPosition() { - return position; - } - - public byte[] getName() { - return name; - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + schema.hashCode(); - result = prime * result + position; - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - ProjectedColumnExpression other = (ProjectedColumnExpression)obj; - if (!schema.equals(other.schema)) return false; - if (position != other.position) return false; - return true; - } - - @Override - public String toString() { - return Bytes.toString(name); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - try { - ScanProjector.decodeProjectedValue(tuple, ptr); - int maxOffset = ptr.getOffset() + ptr.getLength(); - bitSet.clear(); - bitSet.or(ptr); - schema.iterator(ptr, position, bitSet); - Boolean hasValue = schema.next(ptr, position, maxOffset, bitSet); - if (hasValue == null || !hasValue.booleanValue()) - return false; - } catch (IOException e) { - return false; - } - - return true; - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - schema = new KeyValueSchema(); - schema.readFields(input); - bitSet = ValueBitSet.newInstance(schema); - position = input.readInt(); - name = Bytes.readByteArray(input); - } - - @Override - public void write(DataOutput output) throws IOException { - super.write(output); - schema.write(output); - output.writeInt(position); - Bytes.writeByteArray(output, name); - } - - @Override - public final T accept(ExpressionVisitor visitor) { - return visitor.visit(this); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/RowKeyColumnExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/RowKeyColumnExpression.java deleted file mode 100644 index 31107020..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/RowKeyColumnExpression.java +++ /dev/null @@ -1,158 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDatum; -import com.salesforce.phoenix.schema.RowKeyValueAccessor; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; - - -/** - * - * Class to access a value stored in the row key - * - * @author jtaylor - * @since 0.1 - */ -public class RowKeyColumnExpression extends ColumnExpression { - private PDataType fromType; - private RowKeyValueAccessor accessor; - protected final String name; - - public RowKeyColumnExpression() { - name = null; // Only on client - } - - private RowKeyColumnExpression(PDatum datum, RowKeyValueAccessor accessor, PDataType fromType, String name) { - super(datum); - this.accessor = accessor; - this.fromType = fromType; - this.name = name; - } - - public RowKeyColumnExpression(PDatum datum, RowKeyValueAccessor accessor) { - this(datum, accessor, datum.getDataType(), datum.toString()); - } - - public RowKeyColumnExpression(PDatum datum, RowKeyValueAccessor accessor, String name) { - this(datum, accessor, datum.getDataType(), name); - } - - public RowKeyColumnExpression(PDatum datum, RowKeyValueAccessor accessor, PDataType fromType) { - this(datum, accessor, fromType, datum.toString()); - } - - public int getPosition() { - return accessor.getIndex(); - } - - @Override - public int hashCode() { - final int prime = 31; - int result = super.hashCode(); - result = prime * result + ((accessor == null) ? 0 : accessor.hashCode()); - return result; - } - - @Override - public String toString() { - return name == null ? "PK[" + accessor.getIndex() + "]" : name; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (!super.equals(obj)) return false; - if (getClass() != obj.getClass()) return false; - RowKeyColumnExpression other = (RowKeyColumnExpression)obj; - return accessor.equals(other.accessor); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - tuple.getKey(ptr); - int offset = accessor.getOffset(ptr.get(), ptr.getOffset()); - // Null is represented in the last expression of a multi-part key - // by the bytes not being present. - int maxOffset = ptr.getOffset() + ptr.getLength(); - if (offset < maxOffset) { - byte[] buffer = ptr.get(); - int fixedByteSize = -1; - // FIXME: fixedByteSize <= maxByteSize ? fixedByteSize : 0 required because HBase passes bogus keys to filter to position scan (HBASE-6562) - if (fromType.isFixedWidth()) { - fixedByteSize = getByteSize(); - fixedByteSize = fixedByteSize <= maxOffset ? fixedByteSize : 0; - } - int length = fixedByteSize >= 0 ? fixedByteSize : accessor.getLength(buffer, offset, maxOffset); - // In the middle of the key, an empty variable length byte array represents null - if (length > 0) { - /* - if (type == fromType) { - ptr.set(buffer,offset,length); - } else { - ptr.set(type.toBytes(type.toObject(buffer, offset, length, fromType))); - } - */ - ptr.set(buffer,offset,length); - type.coerceBytes(ptr, fromType, getColumnModifier(), getColumnModifier()); - } else { - ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); - } - return true; - } - return false; - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - accessor = new RowKeyValueAccessor(); - accessor.readFields(input); - fromType = type; // fromType only needed on client side - } - - @Override - public void write(DataOutput output) throws IOException { - super.write(output); - accessor.write(output); - } - - @Override - public final T accept(ExpressionVisitor visitor) { - return visitor.visit(this); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/StringConcatExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/StringConcatExpression.java deleted file mode 100644 index c5a86927..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/StringConcatExpression.java +++ /dev/null @@ -1,103 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - - -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; - - -/** - * - * Implementation for || string concatenation expression. - * @author kmahadik - * @since 0.1 - */ - -public class StringConcatExpression extends BaseCompoundExpression { - public StringConcatExpression() { - } - - public StringConcatExpression(List children) { - super(children); - } - - @Override - public String toString() { - StringBuilder buf = new StringBuilder("("); - for (int i = 0; i < children.size() - 1; i++) { - buf.append(children.get(i) + " || "); - } - buf.append(children.get(children.size()-1)); - buf.append(')'); - return buf.toString(); - } - - @Override - public final T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - byte[] result = ByteUtil.EMPTY_BYTE_ARRAY; - for (int i=0; i children) { - super(children); - } - - @Override - public final T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } - - @Override - public String getOperatorString() { - return " - "; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/TimestampAddExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/TimestampAddExpression.java deleted file mode 100644 index 6ff37b48..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/TimestampAddExpression.java +++ /dev/null @@ -1,97 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; - -import java.math.BigDecimal; -import java.math.RoundingMode; -import java.sql.Timestamp; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.DateUtil; - -/** - * - * Class to encapsulate addition arithmetic for {@link PDataType#TIMESTAMP}. - * - * @author samarth.jain - * @since 2.1.3 - */ - -public class TimestampAddExpression extends AddExpression { - - public TimestampAddExpression() { - } - - public TimestampAddExpression(List children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - BigDecimal finalResult = BigDecimal.ZERO; - - for(int i=0; i children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - BigDecimal finalResult = BigDecimal.ZERO; - - for(int i=0; i exps, ColumnModifier columnModifier) { - super(columnModifier); - ColumnExpression stdDevColExp = (ColumnExpression)exps.get(0); - this.colPrecision = stdDevColExp.getMaxLength(); - this.colScale = stdDevColExp.getScale(); - } - - @Override - protected int getBufferLength() { - return PDataType.DECIMAL.getByteSize(); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (cachedResult == null) { - BigDecimal ssd = sumSquaredDeviation(); - ssd = ssd.divide(new BigDecimal(getDataPointsCount()), PDataType.DEFAULT_MATH_CONTEXT); - // Calculate the precision for the stddev result. - // There are totalCount #Decimal values for which we are calculating the stddev - // The resultant precision depends on precision and scale of all these values. (See - // BigDecimalUtil.getResultPrecisionScale) - // As of now we are not using the actual precision and scale of individual values but just using the table - // column's max length(precision) and scale for each of the values. - int resultPrecision = colPrecision; - for (int i = 1; i < this.totalCount; i++) { - // Max precision that we can support is 38 See PDataType.MAX_PRECISION - if (resultPrecision >= PDataType.MAX_PRECISION) break; - Pair precisionScale = BigDecimalUtil.getResultPrecisionScale(this.colPrecision, - this.colScale, this.colPrecision, this.colScale, Operation.OTHERS); - resultPrecision = precisionScale.getFirst(); - } - cachedResult = new BigDecimal(Math.sqrt(ssd.doubleValue()), new MathContext(resultPrecision, - RoundingMode.HALF_UP)); - cachedResult.setScale(this.colScale, RoundingMode.HALF_UP); - } - if (buffer == null) { - initBuffer(); - } - buffer = PDataType.DECIMAL.toBytes(cachedResult); - ptr.set(buffer); - return true; - } - - protected abstract long getDataPointsCount(); - - private BigDecimal sumSquaredDeviation() { - BigDecimal m = mean(); - BigDecimal result = BigDecimal.ZERO; - for (Entry entry : valueVsCount.entrySet()) { - BigDecimal colValue = (BigDecimal)PDataType.DECIMAL.toObject(entry.getKey()); - BigDecimal delta = colValue.subtract(m); - result = result.add(delta.multiply(delta).multiply(new BigDecimal(entry.getValue()))); - } - return result; - } - - private BigDecimal mean() { - BigDecimal sum = BigDecimal.ZERO; - for (Entry entry : valueVsCount.entrySet()) { - BigDecimal colValue = (BigDecimal)PDataType.DECIMAL.toObject(entry.getKey()); - sum = sum.add(colValue.multiply(new BigDecimal(entry.getValue()))); - } - return sum.divide(new BigDecimal(totalCount), PDataType.DEFAULT_MATH_CONTEXT); - } - - @Override - public void reset() { - super.reset(); - this.cachedResult = null; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/BaseStddevAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/BaseStddevAggregator.java deleted file mode 100644 index 52e6840a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/BaseStddevAggregator.java +++ /dev/null @@ -1,104 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.math.BigDecimal; -import java.util.List; -import java.util.Map.Entry; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * - * @author anoopsjohn - * @since 1.2.1 - */ -public abstract class BaseStddevAggregator extends DistinctValueWithCountClientAggregator { - - protected Expression stdDevColExp; - private BigDecimal cachedResult = null; - - public BaseStddevAggregator(List exps, ColumnModifier columnModifier) { - super(columnModifier); - this.stdDevColExp = exps.get(0); - } - - @Override - protected int getBufferLength() { - return PDataType.DECIMAL.getByteSize(); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (cachedResult == null) { - double ssd = sumSquaredDeviation(); - double result = Math.sqrt(ssd / getDataPointsCount()); - cachedResult = new BigDecimal(result); - } - if (buffer == null) { - initBuffer(); - } - buffer = PDataType.DECIMAL.toBytes(cachedResult); - ptr.set(buffer); - return true; - } - - protected abstract long getDataPointsCount(); - - private double sumSquaredDeviation() { - double m = mean(); - double result = 0.0; - for (Entry entry : valueVsCount.entrySet()) { - double colValue = (Double)PDataType.DOUBLE.toObject(entry.getKey(), this.stdDevColExp.getDataType()); - double delta = colValue - m; - result += (delta * delta) * entry.getValue(); - } - return result; - } - - private double mean() { - double sum = 0.0; - for (Entry entry : valueVsCount.entrySet()) { - double colValue = (Double)PDataType.DOUBLE.toObject(entry.getKey(), this.stdDevColExp.getDataType()); - sum += colValue * entry.getValue(); - } - return sum / totalCount; - } - - @Override - public void reset() { - super.reset(); - this.cachedResult = null; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/ClientAggregators.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/ClientAggregators.java deleted file mode 100644 index 01fc9768..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/ClientAggregators.java +++ /dev/null @@ -1,88 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.util.List; - -import com.salesforce.phoenix.expression.function.SingleAggregateFunction; -import com.salesforce.phoenix.schema.ValueBitSet; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.TupleUtil; - - - -/** - * - * Aggregators that execute on the client-side - * - * @author jtaylor - * @since 0.1 - */ -public class ClientAggregators extends Aggregators { - private final ValueBitSet tempValueSet; - - private static Aggregator[] getAggregators(List aggFuncs) { - Aggregator[] aggregators = new Aggregator[aggFuncs.size()]; - for (int i = 0; i < aggregators.length; i++) { - aggregators[i] = aggFuncs.get(i).getAggregator(); - } - return aggregators; - } - - public ClientAggregators(List functions, int minNullableIndex) { - super(functions.toArray(new SingleAggregateFunction[functions.size()]), getAggregators(functions), minNullableIndex); - this.tempValueSet = ValueBitSet.newInstance(schema); - } - - @Override - public void aggregate(Aggregator[] aggregators, Tuple result) { - TupleUtil.getAggregateValue(result, ptr); - tempValueSet.clear(); - tempValueSet.or(ptr); - - int i = 0, maxOffset = ptr.getOffset() + ptr.getLength(); - Boolean hasValue; - schema.iterator(ptr); - while ((hasValue=schema.next(ptr, i, maxOffset, tempValueSet)) != null) { - if (hasValue) { - aggregators[i].aggregate(result, ptr); - } - i++; - } - } - - @Override - public Aggregator[] newAggregators() { - Aggregator[] aggregators = new Aggregator[functions.length]; - for (int i = 0; i < functions.length; i++) { - aggregators[i] = functions[i].newClientAggregator(); - } - return aggregators; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/CountAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/CountAggregator.java deleted file mode 100644 index 849944d4..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/CountAggregator.java +++ /dev/null @@ -1,100 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.SizedUtil; - -/** - * - * Aggregator for COUNT aggregations - * - * @author jtaylor - * @since 0.1 - */ -public class CountAggregator extends BaseAggregator { - - private long count = 0; - private byte[] buffer = null; - - public CountAggregator() { - super(null); - } - - public CountAggregator(LongSumAggregator clientAgg) { - this(); - count = clientAgg.getSum(); - } - - @Override - public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { - count++; - } - - @Override - public boolean isNullable() { - return false; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (buffer == null) { - buffer = new byte[getDataType().getByteSize()]; - } - getDataType().getCodec().encodeLong(count, buffer, 0); - ptr.set(buffer); - return true; - } - - @Override - public final PDataType getDataType() { - return PDataType.LONG; - } - - @Override - public void reset() { - count = 0; - buffer = null; - super.reset(); - } - - @Override - public String toString() { - return "COUNT [count=" + count + "]"; - } - - @Override - public int getSize() { - return super.getSize() + SizedUtil.LONG_SIZE + SizedUtil.ARRAY_SIZE - + getDataType().getByteSize(); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DecimalStddevPopAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DecimalStddevPopAggregator.java deleted file mode 100644 index 65ee2d36..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DecimalStddevPopAggregator.java +++ /dev/null @@ -1,51 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.util.List; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.ColumnModifier; - -/** - * Client side Aggregator for STDDEV_POP aggregations for DECIMAL data type. - * - * @author anoopsjohn - * @since 1.2.1 - */ -public class DecimalStddevPopAggregator extends BaseDecimalStddevAggregator { - - public DecimalStddevPopAggregator(List exps, ColumnModifier columnModifier) { - super(exps, columnModifier); - } - - @Override - protected long getDataPointsCount() { - return totalCount; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DecimalStddevSampAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DecimalStddevSampAggregator.java deleted file mode 100644 index ddedc5c3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DecimalStddevSampAggregator.java +++ /dev/null @@ -1,51 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.util.List; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.ColumnModifier; - -/** - * Client side Aggregator for STDDEV_SAMP aggregations for DECIMAL data type. - * - * @author anoopsjohn - * @since 1.2.1 - */ -public class DecimalStddevSampAggregator extends BaseDecimalStddevAggregator { - - public DecimalStddevSampAggregator(List exps, ColumnModifier columnModifier) { - super(exps, columnModifier); - } - - @Override - protected long getDataPointsCount() { - return totalCount - 1; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DecimalSumAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DecimalSumAggregator.java deleted file mode 100644 index 1b4786b1..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DecimalSumAggregator.java +++ /dev/null @@ -1,111 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.math.BigDecimal; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.SizedUtil; - - -/** - * - * Aggregator that sums BigDecimal values - * - * @author jtaylor - * @since 0.1 - */ -public class DecimalSumAggregator extends BaseAggregator { - private BigDecimal sum = BigDecimal.ZERO; - private byte[] sumBuffer; - - public DecimalSumAggregator(ColumnModifier columnModifier, ImmutableBytesWritable ptr) { - super(columnModifier); - if (ptr != null) { - initBuffer(); - sum = (BigDecimal)PDataType.DECIMAL.toObject(ptr); - } - } - - private PDataType getInputDataType() { - return PDataType.DECIMAL; - } - - private int getBufferLength() { - return getDataType().getByteSize(); - } - - private void initBuffer() { - sumBuffer = new byte[getBufferLength()]; - } - - @Override - public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { - BigDecimal value = (BigDecimal)getDataType().toObject(ptr, getInputDataType(), columnModifier); - sum = sum.add(value); - if (sumBuffer == null) { - sumBuffer = new byte[getDataType().getByteSize()]; - } - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (sumBuffer == null) { - return false; - } - int len = getDataType().toBytes(sum, sumBuffer, 0); - ptr.set(sumBuffer, 0, len); - return true; - } - - @Override - public final PDataType getDataType() { - return PDataType.DECIMAL; - } - - @Override - public void reset() { - sum = BigDecimal.ZERO; - sumBuffer = null; - super.reset(); - } - - @Override - public String toString() { - return "DECIMAL SUM [sum=" + sum + "]"; - } - - @Override - public int getSize() { - return super.getSize() + SizedUtil.BIG_DECIMAL_SIZE + SizedUtil.ARRAY_SIZE + getDataType().getByteSize(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DistinctCountClientAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DistinctCountClientAggregator.java deleted file mode 100644 index e2142454..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DistinctCountClientAggregator.java +++ /dev/null @@ -1,63 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * Client side Aggregator for DISTINCT COUNT aggregations - * - * @author anoopsjohn - * @since 1.2.1 - */ -public class DistinctCountClientAggregator extends DistinctValueWithCountClientAggregator { - - public DistinctCountClientAggregator(ColumnModifier columnModifier) { - super(columnModifier); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (buffer == null) { - initBuffer(); - } - long value = this.valueVsCount.size(); - buffer = PDataType.LONG.toBytes(value); - ptr.set(buffer); - return true; - } - - @Override - protected int getBufferLength() { - return PDataType.LONG.getByteSize(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DistinctValueWithCountClientAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DistinctValueWithCountClientAggregator.java deleted file mode 100644 index 59e063af..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DistinctValueWithCountClientAggregator.java +++ /dev/null @@ -1,140 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.io.ByteArrayInputStream; -import java.io.DataInputStream; -import java.io.IOException; -import java.io.InputStream; -import java.util.Comparator; -import java.util.HashMap; -import java.util.Map; -import java.util.Map.Entry; -import java.util.TreeMap; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; -import org.apache.hadoop.io.WritableUtils; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * Client side Aggregator which will aggregate data and find distinct values with number of occurrences for each. - * - * @author anoopsjohn - * @since 1.2.1 - */ -public abstract class DistinctValueWithCountClientAggregator extends BaseAggregator { - protected Map valueVsCount = new HashMap(); - protected byte[] buffer; - protected long totalCount = 0L; - - public DistinctValueWithCountClientAggregator(ColumnModifier columnModifier) { - super(columnModifier); - } - - @Override - public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { - InputStream is = new ByteArrayInputStream(ptr.get(), ptr.getOffset() + 1, ptr.getLength() - 1); - try { - if (Bytes.equals(ptr.get(), ptr.getOffset(), 1, DistinctValueWithCountServerAggregator.COMPRESS_MARKER, 0, - 1)) { - InputStream decompressionStream = DistinctValueWithCountServerAggregator.COMPRESS_ALGO - .createDecompressionStream(is, - DistinctValueWithCountServerAggregator.COMPRESS_ALGO.getDecompressor(), 0); - is = decompressionStream; - } - DataInputStream in = new DataInputStream(is); - int mapSize = WritableUtils.readVInt(in); - for (int i = 0; i < mapSize; i++) { - int keyLen = WritableUtils.readVInt(in); - byte[] keyBytes = new byte[keyLen]; - in.read(keyBytes, 0, keyLen); - ImmutableBytesPtr key = new ImmutableBytesPtr(keyBytes); - int value = WritableUtils.readVInt(in); - Integer curCount = valueVsCount.get(key); - if (curCount == null) { - valueVsCount.put(key, value); - } else { - valueVsCount.put(key, curCount + value); - } - totalCount += value; - } - } catch (IOException ioe) { - throw new RuntimeException(ioe); // Impossible as we're using a ByteArrayInputStream - } - if (buffer == null) { - initBuffer(); - } - } - - protected abstract int getBufferLength(); - - protected void initBuffer() { - buffer = new byte[getBufferLength()]; - } - - @Override - public boolean isNullable() { - return false; - } - - @Override - public PDataType getDataType() { - return PDataType.VARBINARY; - } - - @Override - public void reset() { - valueVsCount = new HashMap(); - buffer = null; - totalCount = 0L; - super.reset(); - } - - protected Map getSortedValueVsCount(final boolean ascending, final PDataType type) { - // To sort the valueVsCount - Comparator comparator = new Comparator() { - @Override - public int compare(Object o1, Object o2) { - if (ascending) { - return type.compareTo(o1, o2); - } - return type.compareTo(o2, o1); - } - }; - Map sorted = new TreeMap(comparator); - for (Entry entry : valueVsCount.entrySet()) { - sorted.put(type.toObject(entry.getKey(), columnModifier), entry.getValue()); - } - return sorted; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DoubleSumAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DoubleSumAggregator.java deleted file mode 100644 index b1cf70e0..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DoubleSumAggregator.java +++ /dev/null @@ -1,102 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.SizedUtil; - -public class DoubleSumAggregator extends BaseAggregator { - - private double sum = 0; - private byte[] buffer; - - public DoubleSumAggregator(ColumnModifier columnModifier, ImmutableBytesWritable ptr) { - super(columnModifier); - if (ptr != null) { - initBuffer(); - sum = PDataType.DOUBLE.getCodec().decodeDouble(ptr, columnModifier); - } - } - - protected PDataType getInputDataType() { - return PDataType.DOUBLE; - } - - private void initBuffer() { - buffer = new byte[getDataType().getByteSize()]; - } - - @Override - public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { - double value = getInputDataType().getCodec().decodeDouble(ptr, columnModifier); - sum += value; - if (buffer == null) { - initBuffer(); - } - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (buffer == null) { - if (isNullable()) { - return false; - } - initBuffer(); - } - ptr.set(buffer); - getDataType().getCodec().encodeDouble(sum, ptr); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.DOUBLE; - } - - @Override - public String toString() { - return "SUM [sum=" + sum + "]"; - } - - @Override - public void reset() { - sum = 0; - buffer = null; - super.reset(); - } - - @Override - public int getSize() { - return super.getSize() + SizedUtil.LONG_SIZE + SizedUtil.ARRAY_SIZE + getDataType().getByteSize(); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/IntSumAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/IntSumAggregator.java deleted file mode 100644 index 41292958..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/IntSumAggregator.java +++ /dev/null @@ -1,51 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; - -/** - * - * Aggregator that sums integer values - * - * @author jtaylor - * @since 0.1 - */ -public class IntSumAggregator extends NumberSumAggregator { - - public IntSumAggregator(ColumnModifier columnModifier) { - super(columnModifier); - } - - @Override - protected PDataType getInputDataType() { - return PDataType.INTEGER; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/LongSumAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/LongSumAggregator.java deleted file mode 100644 index deb1837d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/LongSumAggregator.java +++ /dev/null @@ -1,50 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; - -/** - * - * Aggregator that sums long values - * - * @author jtaylor - * @since 0.1 - */ -public class LongSumAggregator extends NumberSumAggregator { - - public LongSumAggregator(ColumnModifier columnModifier) { - super(columnModifier); - } - - @Override - protected PDataType getInputDataType() { - return PDataType.LONG; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/MaxAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/MaxAggregator.java deleted file mode 100644 index 04c05b95..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/MaxAggregator.java +++ /dev/null @@ -1,58 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.phoenix.schema.ColumnModifier; - - -/** - * Aggregator that finds the max of values. Inverse of {@link MinAggregator}. - * - * @author syyang - * @since 0.1 - */ -abstract public class MaxAggregator extends MinAggregator { - - public MaxAggregator(ColumnModifier columnModifier) { - super(columnModifier); - } - - @Override - protected boolean keepFirst(ImmutableBytesWritable ibw1, ImmutableBytesWritable ibw2) { - return !super.keepFirst(ibw1, ibw2); - } - - @Override - public String toString() { - return "MAX [value=" + Bytes.toStringBinary(value.get(),value.getOffset(),value.getLength()) + "]"; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/MinAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/MinAggregator.java deleted file mode 100644 index d1b636d3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/MinAggregator.java +++ /dev/null @@ -1,106 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.SizedUtil; - - -/** - * Aggregator that finds the min of values. Inverse of {@link MaxAggregator}. - * - * @author syyang - * @since 0.1 - */ -abstract public class MinAggregator extends BaseAggregator { - /** Used to store the accumulate the results of the MIN function */ - protected final ImmutableBytesWritable value = new ImmutableBytesWritable(ByteUtil.EMPTY_BYTE_ARRAY); - - public MinAggregator(ColumnModifier columnModifier) { - super(columnModifier); - } - - @Override - public void reset() { - value.set(ByteUtil.EMPTY_BYTE_ARRAY); - super.reset(); - } - - @Override - public int getSize() { - return super.getSize() + /*value*/ SizedUtil.IMMUTABLE_BYTES_WRITABLE_SIZE; - } - - /** - * Compares two bytes writables, and returns true if the first one should be - * kept, and false otherwise. For the MIN function, this method will return - * true if the first bytes writable is less than the second. - * - * @param ibw1 the first bytes writable - * @param ibw2 the second bytes writable - * @return true if the first bytes writable should be kept - */ - protected boolean keepFirst(ImmutableBytesWritable ibw1, ImmutableBytesWritable ibw2) { - return 0 >= getDataType().compareTo(ibw1, columnModifier, ibw2, columnModifier, getDataType()); - } - - private boolean isNull() { - return value.get() == ByteUtil.EMPTY_BYTE_ARRAY; - } - - @Override - public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { - if (isNull()) { - value.set(ptr.get(), ptr.getOffset(), ptr.getLength()); - } else { - if (!keepFirst(value, ptr)) { - // replace the value with the new value - value.set(ptr.get(), ptr.getOffset(), ptr.getLength()); - } - } - } - - @Override - public String toString() { - return "MIN [value=" + Bytes.toStringBinary(value.get(),value.getOffset(),value.getLength()) + "]"; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (isNull()) { - return false; - } - ptr.set(value.get(), value.getOffset(), value.getLength()); - return true; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/NumberSumAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/NumberSumAggregator.java deleted file mode 100644 index 0ad4ae23..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/NumberSumAggregator.java +++ /dev/null @@ -1,122 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.SizedUtil; - -/** - * - * Aggregator that sums integral number values - * - * @author jtaylor - * @since 0.1 - */ -abstract public class NumberSumAggregator extends BaseAggregator { - private long sum = 0; - private byte[] buffer; - - public NumberSumAggregator(ColumnModifier columnModifier) { - super(columnModifier); - } - - public NumberSumAggregator(ColumnModifier columnModifier, - ImmutableBytesWritable ptr) { - this(columnModifier); - if (ptr != null) { - initBuffer(); - sum = PDataType.LONG.getCodec().decodeLong(ptr, columnModifier); - } - } - - public long getSum() { - return sum; - } - - abstract protected PDataType getInputDataType(); - - private int getBufferLength() { - return getDataType().getByteSize(); - } - - private void initBuffer() { - buffer = new byte[getBufferLength()]; - } - - @Override - public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { - // Get either IntNative or LongNative depending on input type - long value = getInputDataType().getCodec().decodeLong(ptr, - columnModifier); - sum += value; - if (buffer == null) { - initBuffer(); - } - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (buffer == null) { - if (isNullable()) { - return false; - } - initBuffer(); - } - ptr.set(buffer); - getDataType().getCodec().encodeLong(sum, ptr); - return true; - } - - @Override - public final PDataType getDataType() { - return PDataType.LONG; - } - - @Override - public void reset() { - sum = 0; - buffer = null; - super.reset(); - } - - @Override - public String toString() { - return "SUM [sum=" + sum + "]"; - } - - @Override - public int getSize() { - return super.getSize() + SizedUtil.LONG_SIZE + SizedUtil.ARRAY_SIZE - + getBufferLength(); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/PercentRankClientAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/PercentRankClientAggregator.java deleted file mode 100644 index 333431df..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/PercentRankClientAggregator.java +++ /dev/null @@ -1,100 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.math.BigDecimal; -import java.util.*; -import java.util.Map.Entry; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.*; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * Client side Aggregator for PERCENT_RANK aggregations - * - * @author anoopsjohn - * @since 1.2.1 - */ -public class PercentRankClientAggregator extends DistinctValueWithCountClientAggregator { - - private final List exps; - private BigDecimal cachedResult = null; - - public PercentRankClientAggregator(List exps, ColumnModifier columnModifier) { - super(columnModifier); - this.exps = exps; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (cachedResult == null) { - ColumnExpression columnExp = (ColumnExpression)exps.get(0); - // Second exp will be a LiteralExpression of Boolean type indicating whether the ordering to - // be ASC/DESC - LiteralExpression isAscendingExpression = (LiteralExpression)exps.get(1); - boolean isAscending = (Boolean)isAscendingExpression.getValue(); - - // Third expression will be LiteralExpression - LiteralExpression valueExp = (LiteralExpression)exps.get(2); - Map sorted = getSortedValueVsCount(isAscending, columnExp.getDataType()); - long distinctCountsSum = 0; - Object value = valueExp.getValue(); - for (Entry entry : sorted.entrySet()) { - Object colValue = entry.getKey(); - int compareResult = columnExp.getDataType().compareTo(colValue, value, valueExp.getDataType()); - boolean done = isAscending ? compareResult > 0 : compareResult <= 0; - if (done) break; - distinctCountsSum += entry.getValue(); - } - - float result = (float)distinctCountsSum / totalCount; - this.cachedResult = new BigDecimal(result); - } - if (buffer == null) { - initBuffer(); - } - buffer = PDataType.DECIMAL.toBytes(this.cachedResult); - ptr.set(buffer); - return true; - } - - @Override - protected int getBufferLength() { - return PDataType.DECIMAL.getByteSize(); - } - - @Override - public void reset() { - super.reset(); - this.cachedResult = null; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/PercentileClientAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/PercentileClientAggregator.java deleted file mode 100644 index cbfb5bb6..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/PercentileClientAggregator.java +++ /dev/null @@ -1,118 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.math.BigDecimal; -import java.util.*; -import java.util.Map.Entry; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.*; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * Client side Aggregator for PERCENTILE_CONT aggregations - * - * @author anoopsjohn - * @since 1.2.1 - */ -public class PercentileClientAggregator extends DistinctValueWithCountClientAggregator { - - private final List exps; - private BigDecimal cachedResult = null; - - public PercentileClientAggregator(List exps, ColumnModifier columnModifier) { - super(columnModifier); - this.exps = exps; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (cachedResult == null) { - ColumnExpression columnExp = (ColumnExpression)exps.get(0); - // Second exp will be a LiteralExpression of Boolean type indicating whether the ordering to - // be ASC/DESC - LiteralExpression isAscendingExpression = (LiteralExpression)exps.get(1); - boolean isAscending = (Boolean)isAscendingExpression.getValue(); - - // Third expression will be LiteralExpression - LiteralExpression percentileExp = (LiteralExpression)exps.get(2); - float p = ((Number)percentileExp.getValue()).floatValue(); - Map sorted = getSortedValueVsCount(isAscending, columnExp.getDataType()); - float i = (p * this.totalCount) + 0.5F; - long k = (long)i; - float f = i - k; - Object o1 = null; - Object o2 = null; - long distinctCountsSum = 0; - for (Entry entry : sorted.entrySet()) { - if (o1 != null) { - o2 = entry.getKey(); - break; - } - distinctCountsSum += entry.getValue(); - if (distinctCountsSum == k) { - o1 = entry.getKey(); - } else if (distinctCountsSum > k) { - o1 = o2 = entry.getKey(); - break; - } - } - - double result = 0.0; - Number n1 = (Number)o1; - if (o2 == null || o1 == o2) { - result = n1.doubleValue(); - } else { - Number n2 = (Number)o2; - result = (n1.doubleValue() * (1.0F - f)) + (n2.doubleValue() * f); - } - this.cachedResult = new BigDecimal(result); - } - if (buffer == null) { - initBuffer(); - } - buffer = PDataType.DECIMAL.toBytes(this.cachedResult); - ptr.set(buffer); - return true; - } - - @Override - protected int getBufferLength() { - return PDataType.DECIMAL.getByteSize(); - } - - @Override - public void reset() { - super.reset(); - this.cachedResult = null; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/PercentileDiscClientAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/PercentileDiscClientAggregator.java deleted file mode 100644 index 98efb0d3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/PercentileDiscClientAggregator.java +++ /dev/null @@ -1,114 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.util.*; -import java.util.Map.Entry; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.*; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * - * Built-in function for PERCENTILE_DISC() WITHIN GROUP (ORDER BY ASC/DESC) aggregate function - * - * @author ramkrishna - * @since 1.2.1 - */ -public class PercentileDiscClientAggregator extends - DistinctValueWithCountClientAggregator { - - private final List exps; - private Object cachedResult = null; - ColumnExpression columnExp = null; - - public PercentileDiscClientAggregator(List exps, ColumnModifier columnModifier) { - super(columnModifier); - this.exps = exps; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - // Reset buffer so that it gets initialized with the current datatype of the column - buffer = null; - if (cachedResult == null) { - columnExp = (ColumnExpression)exps.get(0); - // Second exp will be a LiteralExpression of Boolean type indicating - // whether the ordering to be ASC/DESC - LiteralExpression isAscendingExpression = (LiteralExpression) exps - .get(1); - boolean isAscending = (Boolean) isAscendingExpression.getValue(); - - // Third expression will be LiteralExpression - LiteralExpression percentileExp = (LiteralExpression) exps.get(2); - float p = ((Number) percentileExp.getValue()).floatValue(); - Map sorted = getSortedValueVsCount(isAscending, columnExp.getDataType()); - int currValue = 0; - Object result = null; - // Here the Percentile_disc returns the cum_dist() that is greater or equal to the - // Percentile (p) specified in the query. So the result set will be of that of the - // datatype of the column being selected - for (Entry entry : sorted.entrySet()) { - result = entry.getKey(); - Integer value = entry.getValue(); - currValue += value; - float cum_dist = (float) currValue / (float) totalCount; - if (cum_dist >= p) { - break; - } - } - this.cachedResult = result; - } - if (buffer == null) { - // Initialize based on the datatype - // columnExp cannot be null - buffer = new byte[columnExp.getDataType().getByteSize()]; - } - // Copy the result to the buffer. - System.arraycopy(columnExp.getDataType().toBytes(this.cachedResult), 0, buffer, 0, buffer.length); - ptr.set(buffer); - return true; - } - - @Override - public void reset() { - super.reset(); - this.cachedResult = null; - } - - @Override - protected int getBufferLength() { - // Will be used in the aggregate() call - return PDataType.DECIMAL.getByteSize(); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/ServerAggregators.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/ServerAggregators.java deleted file mode 100644 index de6ad316..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/ServerAggregators.java +++ /dev/null @@ -1,153 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.io.ByteArrayInputStream; -import java.io.ByteArrayOutputStream; -import java.io.DataInputStream; -import java.io.DataOutputStream; -import java.io.IOException; -import java.util.Arrays; -import java.util.List; - -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.io.WritableUtils; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.ExpressionType; -import com.salesforce.phoenix.expression.function.SingleAggregateFunction; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Aggregators that execute on the server-side - * - * @author jtaylor - * @since 0.1 - */ -public class ServerAggregators extends Aggregators { - public static final ServerAggregators EMPTY_AGGREGATORS = new ServerAggregators(new SingleAggregateFunction[0], new Aggregator[0], new Expression[0], 0); - private final Expression[] expressions; - - private ServerAggregators(SingleAggregateFunction[] functions, Aggregator[] aggregators, Expression[] expressions, int minNullableIndex) { - super(functions, aggregators, minNullableIndex); - if (aggregators.length != expressions.length) { - throw new IllegalArgumentException("Number of aggregators (" + aggregators.length - + ") must match the number of expressions (" + Arrays.toString(expressions) + ")"); - } - this.expressions = expressions; - } - - @Override - public void aggregate(Aggregator[] aggregators, Tuple result) { - for (int i = 0; i < expressions.length; i++) { - if (expressions[i].evaluate(result, ptr)) { - aggregators[i].aggregate(result, ptr); - } - } - } - - /** - * Serialize an Aggregator into a byte array - * @param aggFuncs list of aggregator to serialize - * @return serialized byte array respresentation of aggregator - */ - public static byte[] serialize(List aggFuncs, int minNullableIndex) { - ByteArrayOutputStream stream = new ByteArrayOutputStream(); - try { - DataOutputStream output = new DataOutputStream(stream); - WritableUtils.writeVInt(output, minNullableIndex); - WritableUtils.writeVInt(output, aggFuncs.size()); - for (int i = 0; i < aggFuncs.size(); i++) { - SingleAggregateFunction aggFunc = aggFuncs.get(i); - WritableUtils.writeVInt(output, ExpressionType.valueOf(aggFunc).ordinal()); - aggFunc.write(output); - } - return stream.toByteArray(); - } catch (IOException e) { - throw new RuntimeException(e); - } finally { - try { - stream.close(); - } catch (IOException e) { - throw new RuntimeException(e); - } - } - } - - @Override - public Aggregator[] newAggregators() { - return newAggregators(null); - } - - public Aggregator[] newAggregators(Configuration conf) { - Aggregator[] aggregators = new Aggregator[functions.length]; - for (int i = 0; i < functions.length; i++) { - aggregators[i] = functions[i].newServerAggregator(conf); - } - return aggregators; - } - - /** - * Deserialize aggregators from the serialized byte array representation - * @param b byte array representation of a list of Aggregators - * @param conf Server side configuration used by HBase - * @return newly instantiated Aggregators instance - */ - public static ServerAggregators deserialize(byte[] b, Configuration conf) { - if (b == null) { - return ServerAggregators.EMPTY_AGGREGATORS; - } - ByteArrayInputStream stream = new ByteArrayInputStream(b); - try { - DataInputStream input = new DataInputStream(stream); - int minNullableIndex = WritableUtils.readVInt(input); - int len = WritableUtils.readVInt(input); - Aggregator[] aggregators = new Aggregator[len]; - Expression[] expressions = new Expression[len]; - SingleAggregateFunction[] functions = new SingleAggregateFunction[len]; - for (int i = 0; i < aggregators.length; i++) { - SingleAggregateFunction aggFunc = (SingleAggregateFunction)ExpressionType.values()[WritableUtils.readVInt(input)].newInstance(); - aggFunc.readFields(input, conf); - functions[i] = aggFunc; - aggregators[i] = aggFunc.getAggregator(); - expressions[i] = aggFunc.getAggregatorExpression(); - } - return new ServerAggregators(functions, aggregators,expressions, minNullableIndex); - } catch (IOException e) { - throw new RuntimeException(e); - } finally { - try { - stream.close(); - } catch (IOException e) { - throw new RuntimeException(e); - } - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/StddevPopAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/StddevPopAggregator.java deleted file mode 100644 index 7c232501..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/StddevPopAggregator.java +++ /dev/null @@ -1,51 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.util.List; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.ColumnModifier; - -/** - * Client side Aggregator for STDDEV_POP aggregations - * - * @author anoopsjohn - * @since 1.2.1 - */ -public class StddevPopAggregator extends BaseStddevAggregator { - - public StddevPopAggregator(List exps, ColumnModifier columnModifier) { - super(exps, columnModifier); - } - - @Override - protected long getDataPointsCount() { - return totalCount; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/StddevSampAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/StddevSampAggregator.java deleted file mode 100644 index a4af51d5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/StddevSampAggregator.java +++ /dev/null @@ -1,51 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import java.util.List; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.ColumnModifier; - -/** - * Client side Aggregator for STDDEV_SAMP aggregations - * - * @author anoopsjohn - * @since 1.2.1 - */ -public class StddevSampAggregator extends BaseStddevAggregator { - - public StddevSampAggregator(List exps, ColumnModifier columnModifier) { - super(exps, columnModifier); - } - - @Override - protected long getDataPointsCount() { - return totalCount - 1; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/UnsignedIntSumAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/UnsignedIntSumAggregator.java deleted file mode 100644 index c0ddb92d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/UnsignedIntSumAggregator.java +++ /dev/null @@ -1,51 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; - -/** - * - * Aggregator that sums unsigned integer values - * - * @author jtaylor - * @since 0.12 - */ -public class UnsignedIntSumAggregator extends NumberSumAggregator { - - public UnsignedIntSumAggregator(ColumnModifier columnModifier) { - super(columnModifier); - } - - @Override - protected PDataType getInputDataType() { - return PDataType.UNSIGNED_INT; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/UnsignedLongSumAggregator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/UnsignedLongSumAggregator.java deleted file mode 100644 index 55e161f5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/UnsignedLongSumAggregator.java +++ /dev/null @@ -1,52 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; - -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; - -/** - * - * Aggregator that sums unsigned long values - * TODO: create these classes dynamically based on the type passed through - * - * @author jtaylor - * @since 0.12 - */ -public class UnsignedLongSumAggregator extends NumberSumAggregator { - - public UnsignedLongSumAggregator(ColumnModifier columnModifier) { - super(columnModifier); - } - - @Override - protected PDataType getInputDataType() { - return PDataType.UNSIGNED_LONG; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/AggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/AggregateFunction.java deleted file mode 100644 index 5d1c8493..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/AggregateFunction.java +++ /dev/null @@ -1,62 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import com.salesforce.phoenix.expression.Expression; - - - - -/** - * - * Compiled representation of a built-in aggregate function - * - * @author jtaylor - * @since 0.1 - */ -abstract public class AggregateFunction extends FunctionExpression { - - public AggregateFunction() { - } - - public AggregateFunction(List children) { - super(children); - } - - @Override - public boolean isStateless() { - return false; - } - - @Override - public boolean isDeterministic() { - return false; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ArrayIndexFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ArrayIndexFunction.java deleted file mode 100644 index a3c082c3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ArrayIndexFunction.java +++ /dev/null @@ -1,96 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.Types; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.parse.ParseException; -import com.salesforce.phoenix.schema.PArrayDataType; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -@BuiltInFunction(name = ArrayIndexFunction.NAME, args = { - @Argument(allowedTypes = { PDataType.BINARY_ARRAY, - PDataType.VARBINARY_ARRAY }), - @Argument(allowedTypes = { PDataType.INTEGER }) }) -public class ArrayIndexFunction extends ScalarFunction { - - public static final String NAME = "ARRAY_ELEM"; - - public ArrayIndexFunction() { - } - - public ArrayIndexFunction(List children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - Expression indexExpr = children.get(1); - if (!indexExpr.evaluate(tuple, ptr)) { - return false; - } else if (ptr.getLength() == 0) { - return true; - } - // Use Codec to prevent Integer object allocation - int index = PDataType.INTEGER.getCodec().decodeInt(ptr, indexExpr.getColumnModifier()); - if(index < 0) { - throw new ParseException("Index cannot be negative :" + index); - } - Expression arrayExpr = children.get(0); - if (!arrayExpr.evaluate(tuple, ptr)) { - return false; - } else if (ptr.getLength() == 0) { - return true; - } - - // Given a ptr to the entire array, set ptr to point to a particular element within that array - // given the type of an array element (see comments in PDataTypeForArray) - PArrayDataType.positionAtArrayElement(ptr, index, getDataType()); - return true; - - } - - @Override - public PDataType getDataType() { - return PDataType.fromTypeId(children.get(0).getDataType().getSqlType() - - Types.ARRAY); - } - - @Override - public String getName() { - return NAME; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ArrayLengthFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ArrayLengthFunction.java deleted file mode 100644 index 530468c8..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ArrayLengthFunction.java +++ /dev/null @@ -1,83 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.Types; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PArrayDataType; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -@BuiltInFunction(name = ArrayLengthFunction.NAME, args = { @Argument(allowedTypes = { - PDataType.BINARY_ARRAY, PDataType.VARBINARY_ARRAY }) }) -public class ArrayLengthFunction extends ScalarFunction { - public static final String NAME = "ARRAY_LENGTH"; - - public ArrayLengthFunction() { - } - - public ArrayLengthFunction(List children) { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - Expression arrayExpr = children.get(0); - if (!arrayExpr.evaluate(tuple, ptr)) { - return false; - } else if (ptr.getLength() == 0) { - return true; - } - PDataType baseType = PDataType.fromTypeId(children.get(0).getDataType() - .getSqlType() - - Types.ARRAY); - int length = PArrayDataType.getArrayLength(ptr, baseType); - byte[] lengthBuf = new byte[PDataType.INTEGER.getByteSize()]; - PDataType.INTEGER.getCodec().encodeInt(length, lengthBuf, 0); - ptr.set(lengthBuf); - return true; - } - - @Override - public PDataType getDataType() { - // Array length will return an Integer - return PDataType.INTEGER; - } - - @Override - public String getName() { - return NAME; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/AvgAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/AvgAggregateFunction.java deleted file mode 100644 index 15c2e7ae..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/AvgAggregateFunction.java +++ /dev/null @@ -1,121 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.math.BigDecimal; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.parse.*; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -@BuiltInFunction(name=AvgAggregateFunction.NAME, nodeClass=AvgAggregateParseNode.class, args= {@Argument(allowedTypes={PDataType.DECIMAL})} ) -public class AvgAggregateFunction extends CompositeAggregateFunction { - public static final String NAME = "AVG"; - private final CountAggregateFunction countFunc; - private final SumAggregateFunction sumFunc; - private Integer scale; - - // TODO: remove when not required at built-in func register time - public AvgAggregateFunction(List children) { - super(children); - this.countFunc = null; - this.sumFunc = null; - setScale(children); - } - - public AvgAggregateFunction(List children, CountAggregateFunction countFunc, SumAggregateFunction sumFunc) { - super(children); - this.countFunc = countFunc; - this.sumFunc = sumFunc; - setScale(children); - } - - private void setScale(List children) { - scale = PDataType.MIN_DECIMAL_AVG_SCALE; // At least 4; - for (Expression child: children) { - if (child.getScale() != null) { - scale = Math.max(scale, child.getScale()); - } - } - } - - @Override - public PDataType getDataType() { - return PDataType.DECIMAL; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (!countFunc.evaluate(tuple, ptr)) { - return false; - } - long count = countFunc.getDataType().getCodec().decodeLong(ptr, null); - if (count == 0) { - return false; - } - - // Normal case where a column reference was used as the argument to AVG - if (!countFunc.isConstantExpression()) { - sumFunc.evaluate(tuple, ptr); - BigDecimal sum = (BigDecimal)PDataType.DECIMAL.toObject(ptr, sumFunc.getDataType()); - // For the final column projection, we divide the sum by the count, both coerced to BigDecimal. - // TODO: base the precision on column metadata instead of constant - BigDecimal avg = sum.divide(BigDecimal.valueOf(count), PDataType.DEFAULT_MATH_CONTEXT); - avg = avg.setScale(scale, BigDecimal.ROUND_DOWN); - ptr.set(PDataType.DECIMAL.toBytes(avg)); - return true; - } - BigDecimal value = (BigDecimal) ((LiteralExpression)countFunc.getChildren().get(0)).getValue(); - value = value.setScale(scale, BigDecimal.ROUND_DOWN); - ptr.set(PDataType.DECIMAL.toBytes(value)); - return true; - } - - @Override - public boolean isNullable() { - return sumFunc != null && sumFunc.isNullable(); - } - - @Override - public String getName() { - return NAME; - } - - @Override - public Integer getScale() { - return scale; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilDateExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilDateExpression.java deleted file mode 100644 index 11d6a4e3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilDateExpression.java +++ /dev/null @@ -1,85 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.List; - -import com.google.common.collect.Lists; -import com.salesforce.phoenix.expression.Expression; - -/** - * - * Class encapsulating ceil operation on {@link com.salesforce.phoenix.schema.PDataType#DATE}. - * - * @author samarth.jain - * @since 3.0.0 - */ -public class CeilDateExpression extends RoundDateExpression { - - public CeilDateExpression() {} - - /** - * @param timeUnit - unit of time to round up to. - * Creates a {@link CeilDateExpression} with default multiplier of 1. - */ - public static Expression create(Expression expr, TimeUnit timeUnit) throws SQLException { - return create(expr, timeUnit, 1); - } - - /** - * @param timeUnit - unit of time to round up to - * @param multiplier - determines the roll up window size. - * Create a {@link CeilDateExpression}. - */ - public static Expression create(Expression expr, TimeUnit timeUnit, int multiplier) throws SQLException { - Expression timeUnitExpr = getTimeUnitExpr(timeUnit); - Expression defaultMultiplierExpr = getMultiplierExpr(multiplier); - List expressions = Lists.newArrayList(expr, timeUnitExpr, defaultMultiplierExpr); - return CeilDateExpression.create(expressions); - } - - public static Expression create(List children) throws SQLException { - return new CeilDateExpression(children); - } - - CeilDateExpression(List children) { - super(children); - } - - @Override - protected long getRoundUpAmount() { - return divBy - 1; - } - - @Override - public String getName() { - return CeilFunction.NAME; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilDecimalExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilDecimalExpression.java deleted file mode 100644 index 9bf0c05e..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilDecimalExpression.java +++ /dev/null @@ -1,85 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.math.RoundingMode; -import java.sql.SQLException; -import java.util.List; - -import com.google.common.collect.Lists; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.schema.PDataType; - -/** - * - * Class encapsulating the CEIL operation on a {@link com.salesforce.phoenix.schema.PDataType#DECIMAL} - * - * @author samarth.jain - * @since 3.0.0 - */ -public class CeilDecimalExpression extends RoundDecimalExpression { - - public CeilDecimalExpression() {} - - public CeilDecimalExpression(List children) { - super(children); - } - - /** - * Creates a {@link CeilDecimalExpression} with rounding scale given by @param scale. - * - */ - public static Expression create(Expression expr, int scale) throws SQLException { - if (expr.getDataType().isCoercibleTo(PDataType.LONG)) { - return expr; - } - Expression scaleExpr = LiteralExpression.newConstant(scale, PDataType.INTEGER, true); - List expressions = Lists.newArrayList(expr, scaleExpr); - return new CeilDecimalExpression(expressions); - } - - /** - * Creates a {@link CeilDecimalExpression} with a default scale of 0 used for rounding. - * - */ - public static Expression create(Expression expr) throws SQLException { - return create(expr, 0); - } - - @Override - protected RoundingMode getRoundingMode() { - return RoundingMode.CEILING; - } - - @Override - public String getName() { - return CeilFunction.NAME; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilFunction.java deleted file mode 100644 index 4678ca54..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilFunction.java +++ /dev/null @@ -1,65 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.CeilParseNode; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -/** - * - * Base class for built-in CEIL function. - * - * @author samarth.jain - * @since 3.0.0 - */ -@BuiltInFunction(name = CeilFunction.NAME, - nodeClass = CeilParseNode.class, - args = { - @Argument(allowedTypes={PDataType.TIMESTAMP, PDataType.DECIMAL}), - @Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValue = "null", isConstant=true), - @Argument(allowedTypes={PDataType.INTEGER}, defaultValue="1", isConstant=true) - } - ) -public abstract class CeilFunction extends ScalarFunction { - - public static final String NAME = "CEIL"; - - public CeilFunction(List children) { - super(children); - } - - @Override - public String getName() { - return NAME; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilTimestampExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilTimestampExpression.java deleted file mode 100644 index bd387eae..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CeilTimestampExpression.java +++ /dev/null @@ -1,119 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.sql.Timestamp; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.google.common.collect.Lists; -import com.salesforce.phoenix.expression.CoerceExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDataType.PDataCodec; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * - * Class encapsulating the CEIL operation on {@link com.salesforce.phoenix.schema.PDataType#TIMESTAMP} - * This class only supports CEIL {@link TimeUnit#MILLISECOND}. If you want more options of CEIL like - * using {@link TimeUnit#HOUR} use {@link CeilDateExpression} - * - * @author samarth.jain - * @since 3.0.0 - */ -public class CeilTimestampExpression extends CeilDateExpression { - - public CeilTimestampExpression() {} - - private CeilTimestampExpression(List children) { - super(children); - } - - /** - * Creates a {@link CeilTimestampExpression} that uses {@link TimeUnit#MILLISECOND} - * as the time unit for rounding. - */ - public static CeilTimestampExpression create(Expression expr, int multiplier) throws SQLException { - List childExprs = Lists.newArrayList(expr, getTimeUnitExpr(TimeUnit.MILLISECOND), getMultiplierExpr(multiplier)); - return new CeilTimestampExpression(childExprs); - } - - public static Expression create(List children) throws SQLException { - Expression firstChild = children.get(0); - PDataType firstChildDataType = firstChild.getDataType(); - String timeUnit = (String)((LiteralExpression)children.get(1)).getValue(); - if(TimeUnit.MILLISECOND.toString().equalsIgnoreCase(timeUnit)) { - return new CeilTimestampExpression(children); - } - // Coerce TIMESTAMP to DATE, as the nanos has no affect - List newChildren = Lists.newArrayListWithExpectedSize(children.size()); - newChildren.add(CoerceExpression.create(firstChild, firstChildDataType == PDataType.TIMESTAMP ? PDataType.DATE : PDataType.UNSIGNED_DATE)); - newChildren.addAll(children.subList(1, children.size())); - return CeilDateExpression.create(newChildren); - } - - /** - * Creates a {@link CeilTimestampExpression} that uses {@link TimeUnit#MILLISECOND} - * as the time unit for rounding. - */ - public static CeilTimestampExpression create (Expression expr) throws SQLException { - return create(expr, 1); - } - - @Override - protected PDataCodec getKeyRangeCodec(PDataType columnDataType) { - return columnDataType == PDataType.TIMESTAMP - ? PDataType.DATE.getCodec() - : columnDataType == PDataType.UNSIGNED_TIMESTAMP - ? PDataType.UNSIGNED_DATE.getCodec() - : super.getKeyRangeCodec(columnDataType); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (children.get(0).evaluate(tuple, ptr)) { - ColumnModifier columnModifier = children.get(0).getColumnModifier(); - PDataType dataType = getDataType(); - int nanos = dataType.getNanos(ptr, columnModifier); - if (nanos > 0) { - long millis = dataType.getMillis(ptr, columnModifier); - Timestamp roundedTs = new Timestamp(millis + 1); - byte[] byteValue = dataType.toBytes(roundedTs); - ptr.set(byteValue); - } - return true; // for timestamp we only support rounding up the milliseconds. - } - return false; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CoalesceFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CoalesceFunction.java deleted file mode 100644 index 7846a0aa..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CoalesceFunction.java +++ /dev/null @@ -1,108 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Function used to provide an alternative value when the first argument is null. - * Usage: - * COALESCE(expr1,expr2) - * If expr1 is not null, then it is returned, otherwise expr2 is returned. - * - * TODO: better bind parameter type matching, since arg2 must be coercible - * to arg1. consider allowing a common base type? - * @author jtaylor - * @since 0.1 - */ -@BuiltInFunction(name=CoalesceFunction.NAME, args= { - @Argument(), - @Argument()} ) -public class CoalesceFunction extends ScalarFunction { - public static final String NAME = "COALESCE"; - - public CoalesceFunction() { - } - - public CoalesceFunction(List children) throws SQLException { - super(children); - if (!children.get(1).getDataType().isCoercibleTo(children.get(0).getDataType())) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.CANNOT_CONVERT_TYPE) - .setMessage(getName() + " expected " + children.get(0).getDataType() + ", but got " + children.get(1).getDataType()) - .build().buildException(); - } - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (children.get(0).evaluate(tuple, ptr)) { - return true; - } - return children.get(1).evaluate(tuple, ptr); - } - - @Override - public PDataType getDataType() { - return children.get(0).getDataType(); - } - - @Override - public Integer getByteSize() { - Integer maxByteSize1 = children.get(0).getByteSize(); - if (maxByteSize1 != null) { - Integer maxByteSize2 = children.get(1).getByteSize(); - if (maxByteSize2 != null) { - return maxByteSize1 > maxByteSize2 ? maxByteSize1 : maxByteSize2; - } - } - return null; - } - - @Override - public boolean isNullable() { - return children.get(0).isNullable() && children.get(1).isNullable(); - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CompositeAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CompositeAggregateFunction.java deleted file mode 100644 index 86248bc1..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CompositeAggregateFunction.java +++ /dev/null @@ -1,55 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; - - -/** - * - * Base class for aggregation functions which are composed of other - * aggregation functions (for example, AVG is modeled as a SUM aggregate - * function and a COUNT aggregate function). - * - * @author jtaylor - * @since 0.1 - */ -abstract public class CompositeAggregateFunction extends AggregateFunction { - - public CompositeAggregateFunction(List children) { - super(children); - } - - @Override - public final T accept(ExpressionVisitor visitor) { - return null; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CountAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CountAggregateFunction.java deleted file mode 100644 index 0ed9b05d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CountAggregateFunction.java +++ /dev/null @@ -1,124 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.Arrays; -import java.util.List; - -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.CountAggregator; -import com.salesforce.phoenix.expression.aggregator.LongSumAggregator; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.util.SchemaUtil; - - -/** - * - * Built-in function for COUNT() aggregate function, - * for example COUNT(foo), COUNT(1), COUNT(*) - * - * @author jtaylor - * @since 0.1 - */ -@BuiltInFunction(name=CountAggregateFunction.NAME, args= {@Argument()} ) -public class CountAggregateFunction extends SingleAggregateFunction { - public static final String NAME = "COUNT"; - public static final List STAR = Arrays.asList(LiteralExpression.newConstant(1, true)); - public static final String NORMALIZED_NAME = SchemaUtil.normalizeIdentifier(NAME); - - public CountAggregateFunction() { - } - - public CountAggregateFunction(List childExpressions) { - super(childExpressions); - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - CountAggregateFunction other = (CountAggregateFunction)obj; - return (isConstantExpression() && other.isConstantExpression()) || children.equals(other.getChildren()); - } - - @Override - public int hashCode() { - return isConstantExpression() ? 0 : super.hashCode(); - } - - /** - * The COUNT function never returns null - */ - @Override - public boolean isNullable() { - return false; - } - - @Override - public PDataType getDataType() { - return PDataType.LONG; - } - - @Override - public LongSumAggregator newClientAggregator() { - // Since COUNT can never be null, ensure the aggregator is not nullable. - // This allows COUNT(*) to return 0 with the initial state of ClientAggregators - // when no rows are returned. - return new LongSumAggregator(null) { - @Override - public boolean isNullable() { - return false; - } - }; - } - - @Override - public Aggregator newServerAggregator(Configuration conf) { - return new CountAggregator(); - } - - @Override - public String getName() { - return NAME; - } - - @Override - public Aggregator newServerAggregator(Configuration config, ImmutableBytesWritable ptr) { - LongSumAggregator sumAgg = newClientAggregator(); - sumAgg.aggregate(null, ptr); - return new CountAggregator(sumAgg); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CurrentDateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CurrentDateFunction.java deleted file mode 100644 index 22140455..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CurrentDateFunction.java +++ /dev/null @@ -1,77 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.CurrentDateTimeFunction; -import com.salesforce.phoenix.parse.CurrentDateParseNode; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Function that returns the current date accurate to the millisecond. Note that this - * function is never evaluated on the server-side, instead the server side date is - * retrieved (piggy-backed on the call to check that the metadata is up-to-date) and - * passed into this function at create time. - * - * @author jtaylor - * @since 0.1 - */ -@BuiltInFunction(name=CurrentDateFunction.NAME, nodeClass=CurrentDateParseNode.class, args= {} ) -public class CurrentDateFunction extends CurrentDateTimeFunction { - public static final String NAME = "CURRENT_DATE"; - private final ImmutableBytesWritable currentDate = new ImmutableBytesWritable(new byte[PDataType.DATE.getByteSize()]); - - public CurrentDateFunction() { - this(System.currentTimeMillis()); - } - - public CurrentDateFunction(long timeStamp) { - getDataType().getCodec().encodeLong(timeStamp, currentDate); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - ptr.set(currentDate.get(), 0, PDataType.DATE.getByteSize()); - return true; - } - - @Override - public final PDataType getDataType() { - return PDataType.DATE; - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CurrentTimeFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CurrentTimeFunction.java deleted file mode 100644 index e23bf86c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/CurrentTimeFunction.java +++ /dev/null @@ -1,77 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.CurrentDateTimeFunction; -import com.salesforce.phoenix.parse.CurrentTimeParseNode; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Function that returns the current date accurate to the millisecond. Note that this - * function is never evaluated on the server-side, instead the server side date is - * retrieved (piggy-backed on the call to check that the metadata is up-to-date) and - * passed into this function at create time. - * - * @author jtaylor - * @since 0.1 - */ -@BuiltInFunction(name=CurrentTimeFunction.NAME, nodeClass=CurrentTimeParseNode.class, args={} ) -public class CurrentTimeFunction extends CurrentDateTimeFunction { - public static final String NAME = "CURRENT_TIME"; - private final ImmutableBytesWritable currentDate = new ImmutableBytesWritable(new byte[PDataType.TIME.getByteSize()]); - - public CurrentTimeFunction() { - this(System.currentTimeMillis()); - } - - public CurrentTimeFunction(long timeStamp) { - getDataType().getCodec().encodeLong(timeStamp, currentDate); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - ptr.set(currentDate.get(), 0, PDataType.TIME.getByteSize()); - return true; - } - - @Override - public final PDataType getDataType() { - return PDataType.TIME; - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/DelegateConstantToCountAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/DelegateConstantToCountAggregateFunction.java deleted file mode 100644 index e61a851e..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/DelegateConstantToCountAggregateFunction.java +++ /dev/null @@ -1,83 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Base class for non composite aggregation functions that optimize aggregation by - * delegating to {@link CountAggregateFunction} when the child expression is a - * constant. - * - * @author jtaylor - * @since 0.1 - */ -abstract public class DelegateConstantToCountAggregateFunction extends SingleAggregateFunction { - private static final ImmutableBytesWritable ZERO = new ImmutableBytesWritable(PDataType.LONG.toBytes(0L)); - private CountAggregateFunction delegate; - - public DelegateConstantToCountAggregateFunction() { - } - - public DelegateConstantToCountAggregateFunction(List childExpressions, CountAggregateFunction delegate) { - super(childExpressions); - // Using a delegate here causes us to optimize the number of aggregators - // by sharing the CountAggregator across functions. On the server side, - // this will always be null, since if it had not been null on the client, - // the function would not have been transfered over (the delegate would - // have instead). - this.delegate = delegate; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (delegate == null) { - return super.evaluate(tuple, ptr); - } - delegate.evaluate(tuple, ptr); - if (PDataType.LONG.compareTo(ptr,ZERO) == 0) { - return false; - } - return true; - } - - - @Override - protected SingleAggregateFunction getDelegate() { - return delegate != null ? delegate : super.getDelegate(); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/DistinctCountAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/DistinctCountAggregateFunction.java deleted file mode 100644 index 0f74d5a6..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/DistinctCountAggregateFunction.java +++ /dev/null @@ -1,132 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctCountClientAggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.SchemaUtil; - - -/** - * - * Built-in function for COUNT(distinct ) aggregate function, - * - * @author anoopsjohn - * @since 1.2.1 - */ -@BuiltInFunction(name=DistinctCountAggregateFunction.NAME, args= {@Argument()} ) -public class DistinctCountAggregateFunction extends DelegateConstantToCountAggregateFunction { - public static final String NAME = "DISTINCT_COUNT"; - public static final String NORMALIZED_NAME = SchemaUtil.normalizeIdentifier(NAME); - public final static byte[] ZERO = PDataType.LONG.toBytes(0L); - public final static byte[] ONE = PDataType.LONG.toBytes(1L); - - public DistinctCountAggregateFunction() { - } - - public DistinctCountAggregateFunction(List childExpressions) { - this(childExpressions, null); - } - - public DistinctCountAggregateFunction(List childExpressions, - CountAggregateFunction delegate) { - super(childExpressions, delegate); - assert childExpressions.size() == 1; - } - - @Override - public int hashCode() { - return isConstantExpression() ? 0 : super.hashCode(); - } - - /** - * The COUNT function never returns null - */ - @Override - public boolean isNullable() { - return false; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - DistinctCountAggregateFunction other = (DistinctCountAggregateFunction)obj; - return (isConstantExpression() && other.isConstantExpression()) || children.equals(other.getChildren()); - } - - @Override - public PDataType getDataType() { - return PDataType.LONG; - } - - @Override - public DistinctCountClientAggregator newClientAggregator() { - return new DistinctCountClientAggregator(getAggregatorExpression().getColumnModifier()); - } - - @Override - public Aggregator newServerAggregator(Configuration conf) { - return new DistinctValueWithCountServerAggregator(conf); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - // TODO: optimize query plan of this to run scan serially for a limit of one row - if (!super.evaluate(tuple, ptr)) { - ptr.set(ZERO); // If evaluate returns false, then no rows were found, so result is 0 - } else if (isConstantExpression()) { - ptr.set(ONE); // Otherwise, we found one or more rows, so a distinct on a constant is 1 - } - return true; // Always evaluates to a LONG value - } - - @Override - public String getName() { - return NAME; - } - - @Override - public Aggregator newServerAggregator(Configuration config, ImmutableBytesWritable ptr) { - DistinctCountClientAggregator clientAgg = newClientAggregator(); - clientAgg.aggregate(null, ptr); - return new DistinctValueWithCountServerAggregator(config, clientAgg); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/DistinctValueWithCountAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/DistinctValueWithCountAggregateFunction.java deleted file mode 100644 index 958431ee..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/DistinctValueWithCountAggregateFunction.java +++ /dev/null @@ -1,58 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; - -public abstract class DistinctValueWithCountAggregateFunction extends SingleAggregateFunction { - - public DistinctValueWithCountAggregateFunction() { - } - - public DistinctValueWithCountAggregateFunction(List children) { - super(children); - } - - @Override - abstract public DistinctValueWithCountClientAggregator newClientAggregator(); - - @Override - public Aggregator newServerAggregator(Configuration config, ImmutableBytesWritable ptr) { - DistinctValueWithCountClientAggregator clientAgg = newClientAggregator(); - clientAgg.aggregate(null, ptr); - return new DistinctValueWithCountServerAggregator(config, clientAgg); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FloorDateExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FloorDateExpression.java deleted file mode 100644 index 7adb4bd5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FloorDateExpression.java +++ /dev/null @@ -1,96 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.List; - -import com.google.common.collect.Lists; -import com.salesforce.phoenix.expression.CoerceExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.PDataType; - -/** - * - * Class encapsulating the FLOOR operation on - * a column/literal of type {@link com.salesforce.phoenix.schema.PDataType#DATE}. - * - * @author samarth.jain - * @since 3.0.0 - */ -public class FloorDateExpression extends RoundDateExpression { - - public FloorDateExpression() {} - - private FloorDateExpression(List children) { - super(children); - } - - public static Expression create(List children) throws SQLException { - Expression firstChild = children.get(0); - PDataType firstChildDataType = firstChild.getDataType(); - if (firstChildDataType == PDataType.TIMESTAMP || firstChildDataType == PDataType.UNSIGNED_TIMESTAMP){ - // Coerce TIMESTAMP to DATE, as the nanos has no affect - List newChildren = Lists.newArrayListWithExpectedSize(children.size()); - newChildren.add(CoerceExpression.create(firstChild, firstChildDataType == PDataType.TIMESTAMP ? PDataType.DATE : PDataType.UNSIGNED_DATE)); - newChildren.addAll(children.subList(1, children.size())); - children = newChildren; - } - return new FloorDateExpression(children); - } - - /** - * @param timeUnit - unit of time to round up to. - * Creates a {@link FloorDateExpression} with default multiplier of 1. - */ - public static Expression create(Expression expr, TimeUnit timeUnit) throws SQLException { - return create(expr, timeUnit, 1); - } - - /** - * @param timeUnit - unit of time to round up to - * @param multiplier - determines the roll up window size. - * Create a {@link FloorDateExpression}. - */ - public static Expression create(Expression expr, TimeUnit timeUnit, int multiplier) throws SQLException { - Expression timeUnitExpr = getTimeUnitExpr(timeUnit); - Expression defaultMultiplierExpr = getMultiplierExpr(multiplier); - List expressions = Lists.newArrayList(expr, timeUnitExpr, defaultMultiplierExpr); - return create(expressions); - } - - @Override - protected long getRoundUpAmount() { - return 0; - } - - @Override - public String getName() { - return FloorFunction.NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FloorDecimalExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FloorDecimalExpression.java deleted file mode 100644 index 75c12bce..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FloorDecimalExpression.java +++ /dev/null @@ -1,85 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.math.RoundingMode; -import java.sql.SQLException; -import java.util.List; - -import com.google.common.collect.Lists; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.schema.PDataType; - -/** - * - * Class encapsulating the FLOOR operation on - * a column/literal of type {@link com.salesforce.phoenix.schema.PDataType#DECIMAL}. - * - * @author samarth.jain - * @since 3.0.0 - */ -public class FloorDecimalExpression extends RoundDecimalExpression { - - public FloorDecimalExpression() {} - - public FloorDecimalExpression(List children) { - super(children); - } - - /** - * Creates a {@link FloorDecimalExpression} with rounding scale given by @param scale. - * - */ - public static Expression create(Expression expr, int scale) throws SQLException { - if (expr.getDataType().isCoercibleTo(PDataType.LONG)) { - return expr; - } - Expression scaleExpr = LiteralExpression.newConstant(scale, PDataType.INTEGER, true); - List expressions = Lists.newArrayList(expr, scaleExpr); - return new FloorDecimalExpression(expressions); - } - - /** - * Creates a {@link FloorDecimalExpression} with a default scale of 0 used for rounding. - * - */ - public static Expression create(Expression expr) throws SQLException { - return create(expr, 0); - } - - @Override - protected RoundingMode getRoundingMode() { - return RoundingMode.FLOOR; - } - - @Override - public String getName() { - return FloorFunction.NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FloorFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FloorFunction.java deleted file mode 100644 index 2b2309ed..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FloorFunction.java +++ /dev/null @@ -1,64 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FloorParseNode; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -/** - * - * Base class for built-in FLOOR function. - * - * @author samarth.jain - * @since 3.0.0 - */ -@BuiltInFunction(name = FloorFunction.NAME, - nodeClass = FloorParseNode.class, - args = { - @Argument(allowedTypes={PDataType.TIMESTAMP, PDataType.DECIMAL}), - @Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValue = "null", isConstant=true), - @Argument(allowedTypes={PDataType.INTEGER}, defaultValue="1", isConstant=true) - } - ) -public abstract class FloorFunction extends ScalarFunction { - - public static final String NAME = "FLOOR"; - - public FloorFunction(List children) { - super(children); - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FunctionExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FunctionExpression.java deleted file mode 100644 index e315fad5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FunctionExpression.java +++ /dev/null @@ -1,79 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import com.salesforce.phoenix.expression.BaseCompoundExpression; -import com.salesforce.phoenix.expression.Expression; - -/** - * - * Compiled representation of a built-in function - * - * @author jtaylor - * @since 0.1 - */ -public abstract class FunctionExpression extends BaseCompoundExpression { - public enum OrderPreserving {NO, YES_IF_LAST, YES}; - public FunctionExpression() { - } - - public FunctionExpression(List children) { - super(children); - } - - /** - * Determines whether or not the result of the function invocation - * will be ordered in the same way as the input to the function. - * Returning YES enables an optimization to occur when a - * GROUP BY contains function invocations using the leading PK - * column(s). - * @return YES if the function invocation will always preserve order for - * the inputs versus the outputs and false otherwise, YES_IF_LAST if the - * function preserves order, but any further column reference would not - * continue to preserve order, and NO if the function does not preserve - * order. - */ - public OrderPreserving preservesOrder() { - return OrderPreserving.NO; - } - - abstract public String getName(); - - @Override - public final String toString() { - StringBuilder buf = new StringBuilder(getName() + "("); - for (int i = 0; i < children.size() - 1; i++) { - buf.append(children.get(i) + ", "); - } - buf.append(children.get(children.size()-1) + ")"); - return buf.toString(); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/IndexStateNameFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/IndexStateNameFunction.java deleted file mode 100644 index 1d819600..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/IndexStateNameFunction.java +++ /dev/null @@ -1,89 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PIndexState; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Function used to get the index state name from the serialized byte value - * Usage: - * IndexStateName('a') - * will return 'ACTIVE' - * - * @author jtaylor - * @since 2.1 - */ -@BuiltInFunction(name=IndexStateNameFunction.NAME, args= { - @Argument(allowedTypes=PDataType.CHAR)} ) -public class IndexStateNameFunction extends ScalarFunction { - public static final String NAME = "IndexStateName"; - - public IndexStateNameFunction() { - } - - public IndexStateNameFunction(List children) throws SQLException { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - Expression child = children.get(0); - if (!child.evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - return true; - } - byte serializedByte = ptr.get()[ptr.getOffset()]; - PIndexState indexState = PIndexState.fromSerializedValue(serializedByte); - ptr.set(indexState.toBytes()); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.VARCHAR; - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/InvertFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/InvertFunction.java deleted file mode 100644 index a4d70c22..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/InvertFunction.java +++ /dev/null @@ -1,125 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.compile.KeyPart; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -@BuiltInFunction(name = InvertFunction.NAME, args = { @Argument() }) -public class InvertFunction extends ScalarFunction { - public static final String NAME = "INVERT"; - - public InvertFunction() throws SQLException {} - - public InvertFunction(List children) throws SQLException { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (!getChildExpression().evaluate(tuple, ptr)) { return false; } - if (ptr.getLength() == 0) { return true; } - byte[] buf = new byte[ptr.getLength()]; - ColumnModifier.SORT_DESC.apply(ptr.get(), ptr.getOffset(), buf, 0, ptr.getLength()); - ptr.set(buf); - return true; - } - - @Override - public ColumnModifier getColumnModifier() { - return getChildExpression().getColumnModifier() == null ? ColumnModifier.SORT_DESC : null; - } - - @Override - public PDataType getDataType() { - return getChildExpression().getDataType(); - } - - @Override - public Integer getMaxLength() { - return getChildExpression().getMaxLength(); - } - - @Override - public Integer getByteSize() { - return getChildExpression().getByteSize(); - } - - @Override - public boolean isNullable() { - return getChildExpression().isNullable(); - } - - @Override - public String getName() { - return NAME; - } - - /** - * INVERT may be optimized through - */ - @Override - public int getKeyFormationTraversalIndex() { - return 0; - } - - /** - * Invert the childPart key range - */ - @Override - public KeyPart newKeyPart(final KeyPart childPart) { - return new KeyPart() { - - @Override - public KeyRange getKeyRange(CompareOp op, Expression rhs) { - KeyRange range = childPart.getKeyRange(op, rhs); - return range.invert(); - } - - @Override - public List getExtractNodes() { - return childPart.getExtractNodes(); - } - - @Override - public PColumn getColumn() { - return childPart.getColumn(); - } - }; - } - - @Override - public OrderPreserving preservesOrder() { - return OrderPreserving.YES; - } - - private Expression getChildExpression() { - return children.get(0); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/LTrimFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/LTrimFunction.java deleted file mode 100644 index 42216e18..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/LTrimFunction.java +++ /dev/null @@ -1,123 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.StringUtil; - - -/** - * - * Implementation of the LTrim() build-in function. It removes from the left end of - * space character and other function bytes in single byte utf8 characters - * set. - * - * @author zhuang - * @since 0.1 - */ -@BuiltInFunction(name=LTrimFunction.NAME, args={ - @Argument(allowedTypes={PDataType.VARCHAR})}) -public class LTrimFunction extends ScalarFunction { - public static final String NAME = "LTRIM"; - - private Integer byteSize; - - public LTrimFunction() { } - - public LTrimFunction(List children) throws SQLException { - super(children); - if (getStringExpression().getDataType().isFixedWidth()) { - byteSize = getStringExpression().getByteSize(); - } - } - - private Expression getStringExpression() { - return children.get(0); - } - - @Override - public ColumnModifier getColumnModifier() { - return children.get(0).getColumnModifier(); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - // Starting from the front of the byte, look for all single bytes at the end of the string - // that is below SPACE_UTF8 (space and control characters) or 0x7f (control chars). - if (!getStringExpression().evaluate(tuple, ptr)) { - return false; - } - - if (ptr.getLength() == 0) { - ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); - return true; - } - byte[] string = ptr.get(); - int offset = ptr.getOffset(); - int length = ptr.getLength(); - - ColumnModifier columnModifier = getStringExpression().getColumnModifier(); - // TODO: when we have ColumnModifier.REVERSE, we'll need to trim from the end instead of - // the beginning (just delegate to RTrimFunction or replace from ExpressionCompiler instead?) - int i = StringUtil.getFirstNonBlankCharIdxFromStart(string, offset, length, columnModifier); - if (i == offset + length) { - ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); - return true; - } - - ptr.set(string, i, offset + length - i); - return true; - } - - @Override - public Integer getByteSize() { - return byteSize; - } - - @Override - public PDataType getDataType() { - return PDataType.VARCHAR; - } - - @Override - public String getName() { - return NAME; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/LengthFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/LengthFunction.java deleted file mode 100644 index 21dc6f5f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/LengthFunction.java +++ /dev/null @@ -1,104 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.io.UnsupportedEncodingException; -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.StringUtil; - - -/** - * - * Implementation of the LENGTH() build-in function. is the string - * of characters we want to find the length of. If is NULL or empty, null - * is returned. - * - * @author zhuang - * @since 0.1 - */ -@BuiltInFunction(name=LengthFunction.NAME, args={ - @Argument(allowedTypes={PDataType.VARCHAR})} ) -public class LengthFunction extends ScalarFunction { - public static final String NAME = "LENGTH"; - - public LengthFunction() { } - - public LengthFunction(List children) throws SQLException { - super(children); - } - - private Expression getStringExpression() { - return children.get(0); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - Expression child = getStringExpression(); - if (!child.evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); - return true; - } - int len; - if (child.getDataType() == PDataType.CHAR) { - // Only single-byte characters allowed in CHAR - len = ptr.getLength(); - } else { - try { - len = StringUtil.calculateUTF8Length(ptr.get(), ptr.getOffset(), ptr.getLength(), child.getColumnModifier()); - } catch (UnsupportedEncodingException e) { - return false; - } - } - ptr.set(PDataType.INTEGER.toBytes(len)); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.INTEGER; - } - - @Override - public String getName() { - return NAME; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/MD5Function.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/MD5Function.java deleted file mode 100644 index 9688ed85..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/MD5Function.java +++ /dev/null @@ -1,108 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -@BuiltInFunction(name = MD5Function.NAME, args={@Argument()}) -public class MD5Function extends ScalarFunction { - public static final String NAME = "MD5"; - public static final Integer LENGTH = 16; - - private final MessageDigest messageDigest; - - public MD5Function() throws SQLException { - try { - messageDigest = MessageDigest.getInstance("MD5"); - } catch (NoSuchAlgorithmException e) { - throw new SQLException(e); - } - } - - public MD5Function(List children) throws SQLException { - super(children); - try { - messageDigest = MessageDigest.getInstance("MD5"); - } catch (NoSuchAlgorithmException e) { - throw new RuntimeException(e); - } - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (!getChildExpression().evaluate(tuple, ptr)) { - return false; - } - - // Update the digest value - messageDigest.update(ptr.get(), ptr.getOffset(), ptr.getLength()); - // Get the digest bytes (note this resets the messageDigest as well) - ptr.set(messageDigest.digest()); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.BINARY; - } - - @Override - public Integer getMaxLength() { - return LENGTH; - } - - @Override - public Integer getByteSize() { - return LENGTH; - } - - @Override - public boolean isNullable() { - return getChildExpression().isNullable(); - } - - @Override - public String getName() { - return NAME; - } - - private Expression getChildExpression() { - return children.get(0); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/MaxAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/MaxAggregateFunction.java deleted file mode 100644 index b55301ba..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/MaxAggregateFunction.java +++ /dev/null @@ -1,83 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import org.apache.hadoop.conf.Configuration; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.MaxAggregator; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.parse.MaxAggregateParseNode; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; - - - -/** - * Built-in function for finding MAX. - * - * @author syyang - * @since 0.1 - */ -@BuiltInFunction(name=MaxAggregateFunction.NAME, nodeClass=MaxAggregateParseNode.class, args= {@Argument()} ) -public class MaxAggregateFunction extends MinAggregateFunction { - public static final String NAME = "MAX"; - - public MaxAggregateFunction() { - } - - public MaxAggregateFunction(List childExpressions, CountAggregateFunction delegate) { - super(childExpressions, delegate); - } - - @Override - public Aggregator newServerAggregator(Configuration conf) { - final PDataType type = getAggregatorExpression().getDataType(); - ColumnModifier columnModifier = getAggregatorExpression().getColumnModifier(); - return new MaxAggregator(columnModifier) { - @Override - public PDataType getDataType() { - return type; - } - }; - } - - @Override - public String getName() { - return NAME; - } - - @Override - public ColumnModifier getColumnModifier() { - return getAggregatorExpression().getColumnModifier(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/MinAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/MinAggregateFunction.java deleted file mode 100644 index a4acee60..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/MinAggregateFunction.java +++ /dev/null @@ -1,97 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.MinAggregator; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.parse.MinAggregateParseNode; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - - -/** - * Built-in function for finding MIN. - * - * @author syyang - * @since 0.1 - */ -@BuiltInFunction(name=MinAggregateFunction.NAME, nodeClass=MinAggregateParseNode.class, args= {@Argument()} ) -public class MinAggregateFunction extends DelegateConstantToCountAggregateFunction { - public static final String NAME = "MIN"; - - public MinAggregateFunction() { - } - - public MinAggregateFunction(List childExpressions, CountAggregateFunction delegate) { - super(childExpressions, delegate); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - boolean wasEvaluated = super.evaluate(tuple, ptr); - if (!wasEvaluated) { - return false; - } - if (isConstantExpression()) { - getAggregatorExpression().evaluate(tuple, ptr); - } - return true; - } - - @Override - public Aggregator newServerAggregator(Configuration conf) { - final PDataType type = getAggregatorExpression().getDataType(); - ColumnModifier columnModifier = getAggregatorExpression().getColumnModifier(); - return new MinAggregator(columnModifier) { - @Override - public PDataType getDataType() { - return type; - } - }; - } - - @Override - public ColumnModifier getColumnModifier() { - return getAggregatorExpression().getColumnModifier(); - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PercentRankAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PercentRankAggregateFunction.java deleted file mode 100644 index 384f85f1..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PercentRankAggregateFunction.java +++ /dev/null @@ -1,82 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import org.apache.hadoop.conf.Configuration; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; -import com.salesforce.phoenix.expression.aggregator.PercentRankClientAggregator; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; - -/** - * - * PERCENT_RANK([,]) WITHIN GROUP (ORDER BY [,] ASC/DESC) aggregate function - * - * @author anoopsjohn - * @since 1.2.1 - */ -@BuiltInFunction(name = PercentRankAggregateFunction.NAME, args = { @Argument(), - @Argument(allowedTypes = { PDataType.BOOLEAN }, isConstant = true), @Argument(isConstant = true) }) -public class PercentRankAggregateFunction extends DistinctValueWithCountAggregateFunction { - public static final String NAME = "PERCENT_RANK"; - - public PercentRankAggregateFunction() { - - } - - public PercentRankAggregateFunction(List childern) { - super(childern); - } - - @Override - public Aggregator newServerAggregator(Configuration conf) { - return new DistinctValueWithCountServerAggregator(conf); - } - - @Override - public DistinctValueWithCountClientAggregator newClientAggregator() { - return new PercentRankClientAggregator(children, getAggregatorExpression().getColumnModifier()); - } - - @Override - public String getName() { - return NAME; - } - - @Override - public PDataType getDataType() { - return PDataType.DECIMAL; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PercentileContAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PercentileContAggregateFunction.java deleted file mode 100644 index ff50c66a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PercentileContAggregateFunction.java +++ /dev/null @@ -1,83 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import org.apache.hadoop.conf.Configuration; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; -import com.salesforce.phoenix.expression.aggregator.PercentileClientAggregator; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; - -/** - * - * Built-in function for PERCENTILE_CONT() WITHIN GROUP (ORDER BY ASC/DESC) aggregate function - * - * @author anoopsjohn - * @since 1.2.1 - */ -@BuiltInFunction(name = PercentileContAggregateFunction.NAME, args = { @Argument(allowedTypes = { PDataType.DECIMAL }), - @Argument(allowedTypes = { PDataType.BOOLEAN }, isConstant = true), - @Argument(allowedTypes = { PDataType.DECIMAL }, isConstant = true, minValue = "0", maxValue = "1") }) -public class PercentileContAggregateFunction extends DistinctValueWithCountAggregateFunction { - public static final String NAME = "PERCENTILE_CONT"; - - public PercentileContAggregateFunction() { - - } - - public PercentileContAggregateFunction(List childern) { - super(childern); - } - - @Override - public Aggregator newServerAggregator(Configuration conf) { - return new DistinctValueWithCountServerAggregator(conf); - } - - @Override - public DistinctValueWithCountClientAggregator newClientAggregator() { - return new PercentileClientAggregator(children, getAggregatorExpression().getColumnModifier()); - } - - @Override - public String getName() { - return NAME; - } - - @Override - public PDataType getDataType() { - return PDataType.DECIMAL; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PercentileDiscAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PercentileDiscAggregateFunction.java deleted file mode 100644 index 1fdec4dc..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PercentileDiscAggregateFunction.java +++ /dev/null @@ -1,80 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import org.apache.hadoop.conf.Configuration; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; -import com.salesforce.phoenix.expression.aggregator.PercentileDiscClientAggregator; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; - - -/** - * - * Built-in function for PERCENTILE_DISC() WITHIN GROUP (ORDER BY ASC/DESC) aggregate function - * - * @author ramkrishna - * @since 1.2.1 - */ -@BuiltInFunction(name = PercentileDiscAggregateFunction.NAME, args = { @Argument(allowedTypes = { PDataType.DECIMAL }), - @Argument(allowedTypes = { PDataType.BOOLEAN }, isConstant = true), - @Argument(allowedTypes = { PDataType.DECIMAL }, isConstant = true, minValue = "0", maxValue = "1") }) -public class PercentileDiscAggregateFunction extends DistinctValueWithCountAggregateFunction { - - public static final String NAME = "PERCENTILE_DISC"; - - public PercentileDiscAggregateFunction() { - } - - public PercentileDiscAggregateFunction(List childern) { - super(childern); - } - - @Override - public Aggregator newServerAggregator(Configuration conf) { - return new DistinctValueWithCountServerAggregator(conf); - } - - @Override - public DistinctValueWithCountClientAggregator newClientAggregator() { - return new PercentileDiscClientAggregator(children, getAggregatorExpression().getColumnModifier()); - } - - @Override - public String getName() { - return NAME; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RTrimFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RTrimFunction.java deleted file mode 100644 index 841bc50a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RTrimFunction.java +++ /dev/null @@ -1,169 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.compile.KeyPart; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.StringUtil; - - -/** - * - * Implementation of the RTrim() build-in function. It removes from the right end of - * space character and other function bytes in single byte utf8 characters set - * - * @author zhuang - * @since 0.1 - */ -@BuiltInFunction(name=RTrimFunction.NAME, args={ - @Argument(allowedTypes={PDataType.VARCHAR})}) -public class RTrimFunction extends ScalarFunction { - public static final String NAME = "RTRIM"; - - private Integer byteSize; - - public RTrimFunction() { } - - public RTrimFunction(List children) throws SQLException { - super(children); - if (getStringExpression().getDataType().isFixedWidth()) { - byteSize = getStringExpression().getByteSize(); - } - } - - private Expression getStringExpression() { - return children.get(0); - } - - @Override - public ColumnModifier getColumnModifier() { - return children.get(0).getColumnModifier(); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - // Starting from the end of the byte, look for all single bytes at the end of the string - // that is below SPACE_UTF8 (space and control characters) or above (control chars). - if (!getStringExpression().evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); - return true; - } - byte[] string = ptr.get(); - int offset = ptr.getOffset(); - int length = ptr.getLength(); - - ColumnModifier columnModifier = getStringExpression().getColumnModifier(); - int i = StringUtil.getFirstNonBlankCharIdxFromEnd(string, offset, length, columnModifier); - if (i == offset - 1) { - ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); - return true; - } - ptr.set(string, offset, i - offset + 1); - return true; - } - - @Override - public OrderPreserving preservesOrder() { - return OrderPreserving.YES_IF_LAST; - } - - @Override - public int getKeyFormationTraversalIndex() { - return 0; - } - - @Override - public KeyPart newKeyPart(final KeyPart childPart) { - return new KeyPart() { - @Override - public KeyRange getKeyRange(CompareOp op, Expression rhs) { - ImmutableBytesWritable ptr = new ImmutableBytesWritable(); - rhs.evaluate(null, ptr); - byte[] key = ByteUtil.copyKeyBytesIfNecessary(ptr); - PDataType type = getColumn().getDataType(); - KeyRange range; - switch (op) { - case EQUAL: - range = type.getKeyRange(key, true, ByteUtil.nextKey(ByteUtil.concat(key, new byte[] {StringUtil.SPACE_UTF8})), false); - break; - case LESS_OR_EQUAL: - range = type.getKeyRange(KeyRange.UNBOUND, false, ByteUtil.nextKey(ByteUtil.concat(key, new byte[] {StringUtil.SPACE_UTF8})), false); - break; - default: - range = childPart.getKeyRange(op, rhs); - break; - } - Integer length = getColumn().getByteSize(); - return length == null ? range : range.fill(length); - } - - @Override - public List getExtractNodes() { - return Collections.emptyList(); - } - - @Override - public PColumn getColumn() { - return childPart.getColumn(); - } - }; - } - - @Override - public Integer getByteSize() { - return byteSize; - } - - @Override - public PDataType getDataType() { - return PDataType.VARCHAR; - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RegexpReplaceFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RegexpReplaceFunction.java deleted file mode 100644 index 0c551806..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RegexpReplaceFunction.java +++ /dev/null @@ -1,138 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.io.DataInput; -import java.io.IOException; -import java.util.List; -import java.util.regex.Pattern; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Function similar to the regexp_replace function in Postgres, which is used to pattern - * match a segment of the string. Usage: - * REGEXP_REPLACE(,,) - * source_char is the string in which we want to perform string replacement. pattern is a - * Java compatible regular expression string, and we replace all the matching part with - * replace_string. The first 2 arguments are required and are {@link com.salesforce.phoenix.schema.PDataType#VARCHAR}, - * the replace_string is default to empty string. - * - * The function returns a {@link com.salesforce.phoenix.schema.PDataType#VARCHAR} - * - * @author zhuang - * @since 0.1 - */ -@BuiltInFunction(name=RegexpReplaceFunction.NAME, args= { - @Argument(allowedTypes={PDataType.VARCHAR}), - @Argument(allowedTypes={PDataType.VARCHAR}), - @Argument(allowedTypes={PDataType.VARCHAR},defaultValue="null")} ) -public class RegexpReplaceFunction extends ScalarFunction { - public static final String NAME = "REGEXP_REPLACE"; - - private boolean hasReplaceStr; - private Pattern pattern; - - public RegexpReplaceFunction() { } - - // Expect 1 arguments, the pattern. - public RegexpReplaceFunction(List children) { - super(children); - init(); - } - - private void init() { - hasReplaceStr = ((LiteralExpression)getReplaceStrExpression()).getValue() != null; - Object patternString = ((LiteralExpression)children.get(1)).getValue(); - if (patternString != null) { - pattern = Pattern.compile((String)patternString); - } - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - // Can't parse if there is no replacement pattern. - if (pattern == null) { - return false; - } - Expression sourceStrExpression = getSourceStrExpression(); - if (!sourceStrExpression.evaluate(tuple, ptr)) { - return false; - } - String sourceStr = (String)PDataType.VARCHAR.toObject(ptr, sourceStrExpression.getColumnModifier()); - if (sourceStr == null) { - return false; - } - String replaceStr; - if (hasReplaceStr) { - Expression replaceStrExpression = this.getReplaceStrExpression(); - if (!replaceStrExpression.evaluate(tuple, ptr)) { - return false; - } - replaceStr = (String)PDataType.VARCHAR.toObject(ptr, replaceStrExpression.getColumnModifier()); - } else { - replaceStr = ""; - } - String replacedStr = pattern.matcher(sourceStr).replaceAll(replaceStr); - ptr.set(PDataType.VARCHAR.toBytes(replacedStr)); - return true; - } - - private Expression getSourceStrExpression() { - return children.get(0); - } - - private Expression getReplaceStrExpression() { - return children.get(2); - } - - @Override - public PDataType getDataType() { - return PDataType.VARCHAR; - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - init(); - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RegexpSubstrFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RegexpSubstrFunction.java deleted file mode 100644 index 2948591d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RegexpSubstrFunction.java +++ /dev/null @@ -1,183 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; -import java.util.regex.Matcher; -import java.util.regex.Pattern; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; - - -/** - * - * Implementation of REGEXP_SUBSTR(, , ) built-in function, - * where is the offset from the start of . Positive offset is treated as 1-based, - * a zero offset is treated as 0-based, and a negative offset starts from the end of the string - * working backwards. The is the pattern we would like to search for in the string. - * The function returns the first occurrence of any substring in the string that matches - * the input as a VARCHAR. - * - * @author zhuang - * @since 0.1 - */ -@BuiltInFunction(name=RegexpSubstrFunction.NAME, args={ - @Argument(allowedTypes={PDataType.VARCHAR}), - @Argument(allowedTypes={PDataType.VARCHAR}), - @Argument(allowedTypes={PDataType.LONG}, defaultValue="1")} ) -public class RegexpSubstrFunction extends PrefixFunction { - public static final String NAME = "REGEXP_SUBSTR"; - - private Pattern pattern; - private boolean isOffsetConstant; - private Integer byteSize; - - public RegexpSubstrFunction() { } - - public RegexpSubstrFunction(List children) { - super(children); - init(); - } - - private void init() { - Object patternString = ((LiteralExpression)children.get(1)).getValue(); - if (patternString != null) { - pattern = Pattern.compile((String)patternString); - } - // If the source string has a fixed width, then the max length would be the length - // of the source string minus the offset, or the absolute value of the offset if - // it's negative. Offset number is a required argument. However, if the source string - // is not fixed width, the maxLength would be null. - isOffsetConstant = getOffsetExpression() instanceof LiteralExpression; - Number offsetNumber = (Number)((LiteralExpression)getOffsetExpression()).getValue(); - if (offsetNumber != null) { - int offset = offsetNumber.intValue(); - if (getSourceStrExpression().getDataType().isFixedWidth()) { - if (offset >= 0) { - byteSize = getSourceStrExpression().getByteSize() - offset - (offset == 0 ? 0 : 1); - } else { - byteSize = -offset; - } - } - } - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (pattern == null) { - return false; - } - if (!getSourceStrExpression().evaluate(tuple, ptr)) { - return false; - } - String sourceStr = (String)PDataType.VARCHAR.toObject(ptr, getSourceStrExpression().getColumnModifier()); - if (sourceStr == null) { - return false; - } - - Expression offsetExpression = getOffsetExpression(); - if (!offsetExpression.evaluate(tuple, ptr)) { - return false; - } - int offset = offsetExpression.getDataType().getCodec().decodeInt(ptr, offsetExpression.getColumnModifier()); - - int strlen = sourceStr.length(); - // Account for 1 versus 0-based offset - offset = offset - (offset <= 0 ? 0 : 1); - if (offset < 0) { // Offset < 0 means get from end - offset = strlen + offset; - } - if (offset < 0 || offset >= strlen) { - return false; - } - - Matcher matcher = pattern.matcher(sourceStr); - boolean hasSubString = matcher.find(offset); - if (!hasSubString) { - ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); - return true; - } - String subString = matcher.group(); - ptr.set(PDataType.VARCHAR.toBytes(subString)); - return true; - } - - @Override - public Integer getByteSize() { - return byteSize; - } - - @Override - public OrderPreserving preservesOrder() { - if (isOffsetConstant) { - LiteralExpression literal = (LiteralExpression) getOffsetExpression(); - Number offsetNumber = (Number) literal.getValue(); - if (offsetNumber != null) { - int offset = offsetNumber.intValue(); - if (offset == 0 || offset == 1) { - return OrderPreserving.YES_IF_LAST; - } - } - } - return OrderPreserving.NO; - } - - @Override - public int getKeyFormationTraversalIndex() { - return preservesOrder() == OrderPreserving.NO ? NO_TRAVERSAL : 0; - } - - private Expression getOffsetExpression() { - return children.get(2); - } - - private Expression getSourceStrExpression() { - return children.get(0); - } - - @Override - public PDataType getDataType() { - // ALways VARCHAR since we do not know in advanced how long the - // matched string will be. - return PDataType.VARCHAR; - } - - @Override - public String getName() { - return NAME; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundDecimalExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundDecimalExpression.java deleted file mode 100644 index 4da2badb..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundDecimalExpression.java +++ /dev/null @@ -1,144 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; -import java.math.BigDecimal; -import java.math.RoundingMode; -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.io.WritableUtils; - -import com.google.common.collect.Lists; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.schema.IllegalDataException; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * - * Class encapsulating the process for rounding off a column/literal of - * type {@link com.salesforce.phoenix.schema.PDataType#DECIMAL} - * - * @author samarth.jain - * @since 3.0.0 - */ - -public class RoundDecimalExpression extends ScalarFunction { - - private int scale; - - /** - * Creates a {@link RoundDecimalExpression} with rounding scale given by @param scale. - * - */ - public static Expression create(Expression expr, int scale) throws SQLException { - if (expr.getDataType().isCoercibleTo(PDataType.LONG)) { - return expr; - } - Expression scaleExpr = LiteralExpression.newConstant(scale, PDataType.INTEGER, true); - List expressions = Lists.newArrayList(expr, scaleExpr); - return new RoundDecimalExpression(expressions); - } - - /** - * Creates a {@link RoundDecimalExpression} with a default scale of 0 used for rounding. - * - */ - public static Expression create(Expression expr) throws SQLException { - return create(expr, 0); - } - - public RoundDecimalExpression() {} - - public RoundDecimalExpression(List children) { - super(children); - LiteralExpression scaleChild = (LiteralExpression)children.get(1); - PDataType scaleType = scaleChild.getDataType(); - Object scaleValue = scaleChild.getValue(); - if(scaleValue != null) { - if (scaleType.isCoercibleTo(PDataType.INTEGER, scaleValue)) { - int scale = (Integer)PDataType.INTEGER.toObject(scaleValue, scaleType); - if (scale >=0 && scale <= PDataType.MAX_PRECISION) { - this.scale = scale; - return; - } - } - throw new IllegalDataException("Invalid second argument for scale: " + scaleValue + ". The scale must be between 0 and " + PDataType.MAX_PRECISION + " inclusive."); - } - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - Expression childExpr = children.get(0); - if(childExpr.evaluate(tuple, ptr)) { - BigDecimal value = (BigDecimal)PDataType.DECIMAL.toObject(ptr, childExpr.getColumnModifier()); - BigDecimal scaledValue = value.setScale(scale, getRoundingMode()); - ptr.set(getDataType().toBytes(scaledValue)); - return true; - } - return false; - } - - @Override - public PDataType getDataType() { - return children.get(0).getDataType(); - } - - protected RoundingMode getRoundingMode() { - return RoundingMode.HALF_UP; - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - scale = WritableUtils.readVInt(input); - } - - @Override - public void write(DataOutput output) throws IOException { - super.write(output); - WritableUtils.writeVInt(output, scale); - } - - @Override - public String getName() { - return RoundFunction.NAME; - } - - @Override - public OrderPreserving preservesOrder() { - return OrderPreserving.YES; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundFunction.java deleted file mode 100644 index c0ce5000..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundFunction.java +++ /dev/null @@ -1,65 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.parse.RoundParseNode; -import com.salesforce.phoenix.schema.PDataType; - - -/** - * Base class for RoundFunction. - * - * @author jtaylor, samarth.jain - * @since 0.1 - */ -@BuiltInFunction(name = RoundFunction.NAME, - nodeClass = RoundParseNode.class, - args = { - @Argument(allowedTypes={PDataType.TIMESTAMP, PDataType.DECIMAL}), - @Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValue = "null", isConstant=true), - @Argument(allowedTypes={PDataType.INTEGER}, defaultValue="1", isConstant=true) - } - ) -public abstract class RoundFunction extends ScalarFunction { - - public static final String NAME = "ROUND"; - - public RoundFunction(List children) { - super(children); - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundTimestampExpression.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundTimestampExpression.java deleted file mode 100644 index 6cf31b1d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundTimestampExpression.java +++ /dev/null @@ -1,114 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.sql.Timestamp; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.google.common.collect.Lists; -import com.salesforce.phoenix.expression.CoerceExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDataType.PDataCodec; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * - * Class encapsulating the process for rounding off a column/literal of - * type {@link com.salesforce.phoenix.schema.PDataType#TIMESTAMP} - * This class only supports rounding off the milliseconds that is for - * {@link TimeUnit#MILLISECOND}. If you want more options of rounding like - * using {@link TimeUnit#HOUR} use {@link RoundDateExpression} - * - * @author samarth.jain - * @since 3.0.0 - */ - -public class RoundTimestampExpression extends RoundDateExpression { - - private static final long HALF_OF_NANOS_IN_MILLI = java.util.concurrent.TimeUnit.MILLISECONDS.toNanos(1)/2; - - public RoundTimestampExpression() {} - - private RoundTimestampExpression(List children) { - super(children); - } - - public static Expression create (List children) throws SQLException { - Expression firstChild = children.get(0); - PDataType firstChildDataType = firstChild.getDataType(); - String timeUnit = (String)((LiteralExpression)children.get(1)).getValue(); - LiteralExpression multiplierExpr = (LiteralExpression)children.get(2); - - /* - * When rounding off timestamp to milliseconds, nanos play a part only when the multiplier value - * is equal to 1. This is because for cases when multiplier value is greater than 1, number of nanos/multiplier - * will always be less than half the nanos in a millisecond. - */ - if((timeUnit == null || TimeUnit.MILLISECOND.toString().equalsIgnoreCase(timeUnit)) && ((Number)multiplierExpr.getValue()).intValue() == 1) { - return new RoundTimestampExpression(children); - } - // Coerce TIMESTAMP to DATE, as the nanos has no affect - List newChildren = Lists.newArrayListWithExpectedSize(children.size()); - newChildren.add(CoerceExpression.create(firstChild, firstChildDataType == PDataType.TIMESTAMP ? PDataType.DATE : PDataType.UNSIGNED_DATE)); - newChildren.addAll(children.subList(1, children.size())); - return RoundDateExpression.create(newChildren); - } - - @Override - protected PDataCodec getKeyRangeCodec(PDataType columnDataType) { - return columnDataType == PDataType.TIMESTAMP - ? PDataType.DATE.getCodec() - : columnDataType == PDataType.UNSIGNED_TIMESTAMP - ? PDataType.UNSIGNED_DATE.getCodec() - : super.getKeyRangeCodec(columnDataType); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (children.get(0).evaluate(tuple, ptr)) { - ColumnModifier columnModifier = children.get(0).getColumnModifier(); - PDataType dataType = getDataType(); - int nanos = dataType.getNanos(ptr, columnModifier); - if(nanos >= HALF_OF_NANOS_IN_MILLI) { - long timeMillis = dataType.getMillis(ptr, columnModifier); - Timestamp roundedTs = new Timestamp(timeMillis + 1); - byte[] byteValue = dataType.toBytes(roundedTs); - ptr.set(byteValue); - } - return true; // for timestamp we only support rounding up the milliseconds. - } - return false; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SQLTableTypeFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SQLTableTypeFunction.java deleted file mode 100644 index f7405d06..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SQLTableTypeFunction.java +++ /dev/null @@ -1,88 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Function used to get the SQL table type name from the serialized table type. - * Usage: - * SqlTableType('v') will return 'VIEW' based on - * {@link java.sql.DatabaseMetaData#getTableTypes()} - * - * @author jtaylor - * @since 2.2 - */ -@BuiltInFunction(name=SQLTableTypeFunction.NAME, args= { - @Argument(allowedTypes=PDataType.CHAR)} ) -public class SQLTableTypeFunction extends ScalarFunction { - public static final String NAME = "SQLTableType"; - - public SQLTableTypeFunction() { - } - - public SQLTableTypeFunction(List children) throws SQLException { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - Expression child = children.get(0); - if (!child.evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - return true; - } - PTableType tableType = PTableType.fromSerializedValue(ptr.get()[ptr.getOffset()]); - ptr.set(tableType.getValue().getBytes()); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.VARCHAR; - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SQLViewTypeFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SQLViewTypeFunction.java deleted file mode 100644 index 022b533d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SQLViewTypeFunction.java +++ /dev/null @@ -1,88 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PTable.ViewType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Function used to get the SQL view type name from the serialized view type. - * Usage: - * SQLViewType('v') will return 'VIEW' based on - * {@link java.sql.DatabaseMetaData#getTableTypes()} - * - * @author jtaylor - * @since 2.2 - */ -@BuiltInFunction(name=SQLViewTypeFunction.NAME, args= { - @Argument(allowedTypes=PDataType.UNSIGNED_TINYINT)} ) -public class SQLViewTypeFunction extends ScalarFunction { - public static final String NAME = "SQLViewType"; - - public SQLViewTypeFunction() { - } - - public SQLViewTypeFunction(List children) throws SQLException { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - Expression child = children.get(0); - if (!child.evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - return true; - } - ViewType viewType = ViewType.fromSerializedValue(ptr.get()[ptr.getOffset()]); - ptr.set(viewType.getBytes()); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.VARCHAR; - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ScalarFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ScalarFunction.java deleted file mode 100644 index 69ae6e64..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ScalarFunction.java +++ /dev/null @@ -1,80 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import com.salesforce.phoenix.compile.KeyPart; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; - - -public abstract class ScalarFunction extends FunctionExpression { - public static final int NO_TRAVERSAL = -1; - - public ScalarFunction() { - } - - public ScalarFunction(List children) { - super(children); - } - - @Override - public final T accept(ExpressionVisitor visitor) { - List l = acceptChildren(visitor, visitor.visitEnter(this)); - T t = visitor.visitLeave(this, l); - if (t == null) { - t = visitor.defaultReturn(this, l); - } - return t; - } - - /** - * Determines whether or not a function may be used to form - * the start/stop key of a scan - * @return the zero-based position of the argument to traverse - * into to look for a primary key column reference, or - * {@value #NO_TRAVERSAL} if the function cannot be used to - * form the scan key. - */ - public int getKeyFormationTraversalIndex() { - return NO_TRAVERSAL; - } - - /** - * Manufactures a KeyPart used to construct the KeyRange given - * a constant and a comparison operator. - * @param childPart the KeyPart formulated for the child expression - * at the {@link #getKeyFormationTraversalIndex()} position. - * @return the KeyPart for constructing the KeyRange for this - * function. - */ - public KeyPart newKeyPart(KeyPart childPart) { - return null; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SingleAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SingleAggregateFunction.java deleted file mode 100644 index 99455a60..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SingleAggregateFunction.java +++ /dev/null @@ -1,181 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.io.DataInput; -import java.io.IOException; -import java.util.Arrays; -import java.util.Comparator; -import java.util.List; - -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Base class for aggregate functions that calculate an aggregation - * using a single {{@link Aggregator} - * - * @author jtaylor - * @since 0.1 - */ -abstract public class SingleAggregateFunction extends AggregateFunction { - private static final List DEFAULT_EXPRESSION_LIST = Arrays.asList(LiteralExpression.newConstant(1, true)); - protected boolean isConstant; - private Aggregator aggregator; - - /** - * Sort aggregate functions with nullable fields last. This allows us not to have to store trailing null values. - * Within non-nullable/nullable groups, put fixed width values first since we can access those more efficiently - * (i.e. we can skip over groups of them in-mass instead of reading the length of each one to skip over as - * required by a variable length value). - */ - public static final Comparator SCHEMA_COMPARATOR = new Comparator() { - - @Override - public int compare(SingleAggregateFunction o1, SingleAggregateFunction o2) { - boolean isNullable1 = o1.isNullable(); - boolean isNullable2 = o2.isNullable(); - if (isNullable1 != isNullable2) { - return isNullable1 ? 1 : -1; - } - isNullable1 = o1.getAggregatorExpression().isNullable(); - isNullable2 = o2.getAggregatorExpression().isNullable(); - if (isNullable1 != isNullable2) { - return isNullable1 ? 1 : -1; - } - // Ensures COUNT(1) sorts first TODO: unit test for this - boolean isConstant1 = o1.isConstantExpression(); - boolean isConstant2 = o2.isConstantExpression(); - if (isConstant1 != isConstant2) { - return isConstant1 ? 1 : -1; - } - PDataType r1 = o1.getAggregator().getDataType(); - PDataType r2 = o2.getAggregator().getDataType(); - if (r1.isFixedWidth() != r2.isFixedWidth()) { - return r1.isFixedWidth() ? -1 : 1; - } - return r1.compareTo(r2); - } - }; - - protected SingleAggregateFunction() { - this(DEFAULT_EXPRESSION_LIST, true); - } - - public SingleAggregateFunction(List children) { - this(children, children.get(0) instanceof LiteralExpression); - } - - private SingleAggregateFunction(List children, boolean isConstant) { - super(children); - this.isConstant = children.get(0) instanceof LiteralExpression; - this.aggregator = newClientAggregator(); - } - - public boolean isConstantExpression() { - return isConstant; - } - - @Override - public PDataType getDataType() { - return children.get(0).getDataType(); - } - - public Expression getAggregatorExpression() { - return children.get(0); - } - - public Aggregator getAggregator() { - return aggregator; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - return getAggregator().evaluate(tuple, ptr); - } - - /** - * Create the aggregator to do server-side aggregation. - * The data type of the returned Aggregator must match - * the data type returned by {@link #newClientAggregator()} - * @param conf HBase configuration. - * @return the aggregator to use on the server-side - */ - abstract public Aggregator newServerAggregator(Configuration conf); - /** - * Create the aggregator to do client-side aggregation - * based on the results returned from the aggregating - * coprocessor. The data type of the returned Aggregator - * must match the data type returned by {@link #newServerAggregator(Configuration)} - * @return the aggregator to use on the client-side - */ - public Aggregator newClientAggregator() { - return newServerAggregator(null); - } - - public Aggregator newServerAggregator(Configuration config, ImmutableBytesWritable ptr) { - Aggregator agg = newServerAggregator(config); - agg.aggregate(null, ptr); - return agg; - } - - public void readFields(DataInput input, Configuration conf) throws IOException { - super.readFields(input); - aggregator = newServerAggregator(conf); - } - - @Override - public boolean isNullable() { - return true; - } - - protected SingleAggregateFunction getDelegate() { - return this; - } - - @Override - public final T accept(ExpressionVisitor visitor) { - SingleAggregateFunction function = getDelegate(); - List l = acceptChildren(visitor, visitor.visitEnter(function)); - T t = visitor.visitLeave(function, l); - if (t == null) { - t = visitor.defaultReturn(function, l); - } - return t; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SqlTypeNameFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SqlTypeNameFunction.java deleted file mode 100644 index 37b935b5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SqlTypeNameFunction.java +++ /dev/null @@ -1,94 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.IllegalDataException; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; - - -/** - * - * Function used to get the SQL type name from the SQL type integer. - * Usage: - * SqlTypeName(12) - * will return 'VARCHAR' based on {@link java.sql.Types#VARCHAR} being 12 - * - * @author jtaylor - * @since 0.1 - */ -@BuiltInFunction(name=SqlTypeNameFunction.NAME, args= { - @Argument(allowedTypes=PDataType.INTEGER)} ) -public class SqlTypeNameFunction extends ScalarFunction { - public static final String NAME = "SqlTypeName"; - - public SqlTypeNameFunction() { - } - - public SqlTypeNameFunction(List children) throws SQLException { - super(children); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - Expression child = children.get(0); - if (!child.evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - return true; - } - int sqlType = child.getDataType().getCodec().decodeInt(ptr, child.getColumnModifier()); - try { - byte[] sqlTypeNameBytes = PDataType.fromTypeId(sqlType).getSqlTypeNameBytes(); - ptr.set(sqlTypeNameBytes); - } catch (IllegalDataException e) { - ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); - } - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.VARCHAR; - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/StddevPopFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/StddevPopFunction.java deleted file mode 100644 index c0647cca..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/StddevPopFunction.java +++ /dev/null @@ -1,86 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import org.apache.hadoop.conf.Configuration; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.DecimalStddevPopAggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; -import com.salesforce.phoenix.expression.aggregator.StddevPopAggregator; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; - -/** - * - * Built-in function for STDDEV_POP() aggregate function - * - * @author anoopsjohn - * @since 1.2.1 - */ -@BuiltInFunction(name = StddevPopFunction.NAME, args = { @Argument(allowedTypes={PDataType.DECIMAL})}) -public class StddevPopFunction extends DistinctValueWithCountAggregateFunction { - public static final String NAME = "STDDEV_POP"; - - public StddevPopFunction() { - - } - - public StddevPopFunction(List childern) { - super(childern); - } - - @Override - public Aggregator newServerAggregator(Configuration conf) { - return new DistinctValueWithCountServerAggregator(conf); - } - - @Override - public DistinctValueWithCountClientAggregator newClientAggregator() { - if (children.get(0).getDataType() == PDataType.DECIMAL) { - // Special Aggregators for DECIMAL datatype for more precision than double - return new DecimalStddevPopAggregator(children, getAggregatorExpression().getColumnModifier()); - } - return new StddevPopAggregator(children, getAggregatorExpression().getColumnModifier()); - } - - @Override - public String getName() { - return NAME; - } - - @Override - public PDataType getDataType() { - return PDataType.DECIMAL; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/StddevSampFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/StddevSampFunction.java deleted file mode 100644 index 8635915f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/StddevSampFunction.java +++ /dev/null @@ -1,86 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.util.List; - -import org.apache.hadoop.conf.Configuration; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.DecimalStddevSampAggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; -import com.salesforce.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; -import com.salesforce.phoenix.expression.aggregator.StddevSampAggregator; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; - -/** - * - * Built-in function for STDDEV_SAMP() aggregate function - * - * @author anoopsjohn - * @since 1.2.1 - */ -@BuiltInFunction(name = StddevSampFunction.NAME, args = { @Argument(allowedTypes={PDataType.DECIMAL})}) -public class StddevSampFunction extends DistinctValueWithCountAggregateFunction { - public static final String NAME = "STDDEV_SAMP"; - - public StddevSampFunction() { - - } - - public StddevSampFunction(List childern) { - super(childern); - } - - @Override - public Aggregator newServerAggregator(Configuration conf) { - return new DistinctValueWithCountServerAggregator(conf); - } - - @Override - public DistinctValueWithCountClientAggregator newClientAggregator() { - if (children.get(0).getDataType() == PDataType.DECIMAL) { - // Special Aggregators for DECIMAL datatype for more precision than double - return new DecimalStddevSampAggregator(children, getAggregatorExpression().getColumnModifier()); - } - return new StddevSampAggregator(children, getAggregatorExpression().getColumnModifier()); - } - - @Override - public String getName() { - return NAME; - } - - @Override - public PDataType getDataType() { - return PDataType.DECIMAL; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SumAggregateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SumAggregateFunction.java deleted file mode 100644 index 04f8520d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SumAggregateFunction.java +++ /dev/null @@ -1,154 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.math.BigDecimal; -import java.util.List; - -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.DecimalSumAggregator; -import com.salesforce.phoenix.expression.aggregator.DoubleSumAggregator; -import com.salesforce.phoenix.expression.aggregator.NumberSumAggregator; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.parse.SumAggregateParseNode; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Built-in function for SUM aggregation function. - * - * @author jtaylor - * @since 0.1 - */ -@BuiltInFunction(name=SumAggregateFunction.NAME, nodeClass=SumAggregateParseNode.class, args= {@Argument(allowedTypes={PDataType.DECIMAL})} ) -public class SumAggregateFunction extends DelegateConstantToCountAggregateFunction { - public static final String NAME = "SUM"; - - public SumAggregateFunction() { - } - - // TODO: remove when not required at built-in func register time - public SumAggregateFunction(List childExpressions){ - super(childExpressions, null); - } - - public SumAggregateFunction(List childExpressions, CountAggregateFunction delegate){ - super(childExpressions, delegate); - } - - private Aggregator newAggregator(final PDataType type, ColumnModifier columnModifier, ImmutableBytesWritable ptr) { - switch( type ) { - case DECIMAL: - return new DecimalSumAggregator(columnModifier, ptr); - case UNSIGNED_DOUBLE: - case UNSIGNED_FLOAT: - case DOUBLE: - case FLOAT: - return new DoubleSumAggregator(columnModifier, ptr) { - @Override - protected PDataType getInputDataType() { - return type; - } - }; - default: - return new NumberSumAggregator(columnModifier, ptr) { - @Override - protected PDataType getInputDataType() { - return type; - } - }; - } - } - - @Override - public Aggregator newClientAggregator() { - return newAggregator(getDataType(), null, null); - } - - @Override - public Aggregator newServerAggregator(Configuration conf) { - Expression child = getAggregatorExpression(); - return newAggregator(child.getDataType(), child.getColumnModifier(), null); - } - - @Override - public Aggregator newServerAggregator(Configuration conf, ImmutableBytesWritable ptr) { - Expression child = getAggregatorExpression(); - return newAggregator(child.getDataType(), child.getColumnModifier(), ptr); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (!super.evaluate(tuple, ptr)) { - return false; - } - if (isConstantExpression()) { - PDataType type = getDataType(); - Object constantValue = ((LiteralExpression)children.get(0)).getValue(); - if (type == PDataType.DECIMAL) { - BigDecimal value = ((BigDecimal)constantValue).multiply((BigDecimal)PDataType.DECIMAL.toObject(ptr, PDataType.LONG)); - ptr.set(PDataType.DECIMAL.toBytes(value)); - } else { - long constantLongValue = ((Number)constantValue).longValue(); - long value = constantLongValue * type.getCodec().decodeLong(ptr, null); - ptr.set(new byte[type.getByteSize()]); - type.getCodec().encodeLong(value, ptr); - } - } - return true; - } - - @Override - public PDataType getDataType() { - switch(super.getDataType()) { - case DECIMAL: - return PDataType.DECIMAL; - case UNSIGNED_FLOAT: - case UNSIGNED_DOUBLE: - case FLOAT: - case DOUBLE: - return PDataType.DOUBLE; - default: - return PDataType.LONG; - } - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/TimeUnit.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/TimeUnit.java deleted file mode 100644 index 6b782b9b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/TimeUnit.java +++ /dev/null @@ -1,58 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import com.google.common.base.Joiner; - -public enum TimeUnit { - DAY("day"), - HOUR("hour"), - MINUTE("minute"), - SECOND("second"), - MILLISECOND("millisecond"); - - private String value; - - private TimeUnit(String value) { - this.value = value; - } - - public static final String VALID_VALUES = Joiner.on(", ").join(TimeUnit.values()); - - public static TimeUnit getTimeUnit(String timeUnit) { - if(timeUnit == null) { - throw new IllegalArgumentException("No time unit value specified. Only a time unit value that belongs to one of these : " + VALID_VALUES + " is allowed."); - } - for(TimeUnit tu : values()) { - if(timeUnit.equalsIgnoreCase(tu.value)) { - return tu; - } - } - throw new IllegalArgumentException("Invalid value of time unit " + timeUnit + ". Only a time unit value that belongs to one of these : " + VALID_VALUES + " is allowed."); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ToCharFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ToCharFunction.java deleted file mode 100644 index 3dc631ea..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ToCharFunction.java +++ /dev/null @@ -1,143 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.io.*; -import java.sql.SQLException; -import java.text.Format; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.io.WritableUtils; - -import com.google.common.base.Preconditions; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.parse.*; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Implementation of the TO_CHAR(<date>/<number>,[<format-string>] built-in function. - * The first argument must be of type DATE or TIME or TIMESTAMP or DECIMAL or INTEGER, and the second argument must be a constant string. - * - * @author jtaylor - * @since 0.1 - */ -@BuiltInFunction(name=ToCharFunction.NAME, nodeClass=ToCharParseNode.class, args={ - @Argument(allowedTypes={PDataType.TIMESTAMP, PDataType.DECIMAL}), - @Argument(allowedTypes={PDataType.VARCHAR},isConstant=true,defaultValue="null") } ) -public class ToCharFunction extends ScalarFunction { - public static final String NAME = "TO_CHAR"; - private String formatString; - private Format formatter; - private FunctionArgumentType type; - - public ToCharFunction() { - } - - public ToCharFunction(List children, FunctionArgumentType type, String formatString, Format formatter) throws SQLException { - super(children.subList(0, 1)); - Preconditions.checkNotNull(formatString); - Preconditions.checkNotNull(formatter); - Preconditions.checkNotNull(type); - this.type = type; - this.formatString = formatString; - this.formatter = formatter; - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + formatString.hashCode(); - result = prime * result + getExpression().hashCode(); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - ToCharFunction other = (ToCharFunction)obj; - if (!getExpression().equals(other.getExpression())) return false; - if (!formatString.equals(other.formatString)) return false; - return true; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - Expression expression = getExpression(); - if (!expression.evaluate(tuple, ptr)) { - return false; - } - PDataType type = expression.getDataType(); - Object value = formatter.format(type.toObject(ptr, expression.getColumnModifier())); - byte[] b = getDataType().toBytes(value); - ptr.set(b); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.VARCHAR; - } - - @Override - public boolean isNullable() { - return getExpression().isNullable(); - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - formatString = WritableUtils.readString(input); - type = WritableUtils.readEnum(input, FunctionArgumentType.class); - formatter = type.getFormatter(formatString); - } - - @Override - public void write(DataOutput output) throws IOException { - super.write(output); - WritableUtils.writeString(output, formatString); - WritableUtils.writeEnum(output, type); - } - - private Expression getExpression() { - return children.get(0); - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ToDateFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ToDateFunction.java deleted file mode 100644 index 390d509c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ToDateFunction.java +++ /dev/null @@ -1,141 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.io.*; -import java.sql.SQLException; -import java.text.Format; -import java.text.ParseException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.io.WritableUtils; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.*; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.DateUtil; - - -/** - * - * Implementation of the TO_DATE(,[]) built-in function. - * The second argument is optional and defaults to the phoenix.query.dateFormat value - * from the HBase config. If present it must be a constant string. - * - * @author jtaylor - * @since 0.1 - */ -@BuiltInFunction(name=ToDateFunction.NAME, nodeClass=ToDateParseNode.class, args= {@Argument(allowedTypes={PDataType.VARCHAR}),@Argument(allowedTypes={PDataType.VARCHAR},isConstant=true,defaultValue="null")} ) -public class ToDateFunction extends ScalarFunction { - public static final String NAME = "TO_DATE"; - private Format dateParser; - private String dateFormat; - - public ToDateFunction() { - } - - public ToDateFunction(List children, String dateFormat, Format dateParser) throws SQLException { - super(children.subList(0, 1)); - this.dateFormat = dateFormat; - this.dateParser = dateParser; - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + dateFormat.hashCode(); - result = prime * result + getExpression().hashCode(); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - ToDateFunction other = (ToDateFunction)obj; - if (!getExpression().equals(other.getExpression())) return false; - if (!dateFormat.equals(other.dateFormat)) return false; - return true; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - Expression expression = getExpression(); - if (!expression.evaluate(tuple, ptr) || ptr.getLength() == 0) { - return false; - } - PDataType type = expression.getDataType(); - String dateStr = (String)type.toObject(ptr, expression.getColumnModifier()); - try { - Object value = dateParser.parseObject(dateStr); - byte[] byteValue = getDataType().toBytes(value); - ptr.set(byteValue); - return true; - } catch (ParseException e) { - throw new IllegalStateException("to_date('" + dateStr + ")' did not match expected date format of '" + dateFormat + "'."); - } - } - - @Override - public PDataType getDataType() { - return PDataType.DATE; - } - - @Override - public boolean isNullable() { - return getExpression().isNullable(); - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - dateFormat = WritableUtils.readString(input); - dateParser = DateUtil.getDateParser(dateFormat); - } - - @Override - public void write(DataOutput output) throws IOException { - super.write(output); - WritableUtils.writeString(output, dateFormat); - } - - private Expression getExpression() { - return children.get(0); - } - - @Override - public String getName() { - return NAME; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ToNumberFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ToNumberFunction.java deleted file mode 100644 index f7f5b609..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ToNumberFunction.java +++ /dev/null @@ -1,184 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import static com.salesforce.phoenix.util.ByteUtil.EMPTY_BYTE_ARRAY; - -import java.io.*; -import java.math.BigDecimal; -import java.sql.SQLException; -import java.text.*; -import java.util.Date; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.io.WritableUtils; - -import com.google.common.base.Preconditions; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.parse.*; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * - * Implementation of TO_NUMBER(<string>/<date>/<timestamp>, [<pattern-string>]) built-in function. The format for the optional - * pattern_string param is specified in {@link DecimalFormat}. - * - * @author elevine - * @since 0.1 - */ -@BuiltInFunction(name=ToNumberFunction.NAME, nodeClass=ToNumberParseNode.class, args= { - @Argument(allowedTypes={PDataType.VARCHAR, PDataType.TIMESTAMP}), - @Argument(allowedTypes={PDataType.VARCHAR}, isConstant=true, defaultValue="null")} ) -public class ToNumberFunction extends ScalarFunction { - public static final String NAME = "TO_NUMBER"; - - private String formatString = null; - private Format format = null; - private FunctionArgumentType type; - - public ToNumberFunction() {} - - public ToNumberFunction(List children, FunctionArgumentType type, String formatString, Format formatter) throws SQLException { - super(children.subList(0, 1)); - Preconditions.checkNotNull(type); - this.type = type; - this.formatString = formatString; - this.format = formatter; - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - Expression expression = getExpression(); - if (!expression.evaluate(tuple, ptr)) { - return false; - } else if (ptr.getLength() == 0) { - return true; - } - - PDataType type = expression.getDataType(); - if (type.isCoercibleTo(PDataType.TIMESTAMP)) { - Date date = (Date) type.toObject(ptr, expression.getColumnModifier()); - BigDecimal time = new BigDecimal(date.getTime()); - byte[] byteValue = getDataType().toBytes(time); - ptr.set(byteValue); - return true; - } - - String stringValue = (String)type.toObject(ptr, expression.getColumnModifier()); - if (stringValue == null) { - ptr.set(EMPTY_BYTE_ARRAY); - return true; - } - stringValue = stringValue.trim(); - BigDecimal decimalValue; - if (format == null) { - decimalValue = (BigDecimal) getDataType().toObject(stringValue); - } else { - ParsePosition parsePosition = new ParsePosition(0); - Number number = ((DecimalFormat) format).parse(stringValue, parsePosition); - if (parsePosition.getErrorIndex() > -1) { - ptr.set(EMPTY_BYTE_ARRAY); - return true; - } - - if (number instanceof BigDecimal) { - // since we set DecimalFormat.setParseBigDecimal(true) we are guaranteeing result to be - // of type BigDecimal in most cases. see java.text.DecimalFormat.parse() JavaDoc. - decimalValue = (BigDecimal)number; - } else { - ptr.set(EMPTY_BYTE_ARRAY); - return true; - } - } - byte[] byteValue = getDataType().toBytes(decimalValue); - ptr.set(byteValue); - return true; - } - - @Override - public PDataType getDataType() { - return PDataType.DECIMAL; - } - - @Override - public boolean isNullable() { - return getExpression().isNullable(); - } - - private Expression getExpression() { - return children.get(0); - } - - @Override - public String getName() { - return NAME; - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - formatString = WritableUtils.readString(input); - type = WritableUtils.readEnum(input, FunctionArgumentType.class); - if (formatString != null) { - format = type.getFormatter(formatString); - } - } - - @Override - public void write(DataOutput output) throws IOException { - super.write(output); - WritableUtils.writeString(output, formatString); - WritableUtils.writeEnum(output, type); - } - - @Override - public int hashCode() { - final int prime = 31; - int result = super.hashCode(); - result = prime * result + ((formatString == null) ? 0 : formatString.hashCode()); - result = prime * result + getExpression().hashCode(); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (!super.equals(obj)) return false; - if (getClass() != obj.getClass()) return false; - ToNumberFunction other = (ToNumberFunction)obj; - if (formatString == null) { - if (other.formatString != null) return false; - } else if (!formatString.equals(other.formatString)) return false; - if (!getExpression().equals(other.getExpression())) return false; - return true; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/TrimFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/TrimFunction.java deleted file mode 100644 index 6ae134c3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/TrimFunction.java +++ /dev/null @@ -1,116 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.StringUtil; - - -/** - * Implementation of the Trim() build-in function. It removes from both end of - * space character and other function bytes in single byte utf8 characters set. - * - * @author zhuang - * @since 0.1 - */ -@BuiltInFunction(name=TrimFunction.NAME, args={ - @Argument(allowedTypes={PDataType.VARCHAR})} ) -public class TrimFunction extends ScalarFunction { - public static final String NAME = "TRIM"; - - private Integer byteSize; - - public TrimFunction() { } - - public TrimFunction(List children) throws SQLException { - super(children); - if (getStringExpression().getDataType().isFixedWidth()) { - byteSize = getStringExpression().getByteSize(); - } - } - - private Expression getStringExpression() { - return children.get(0); - } - - @Override - public ColumnModifier getColumnModifier() { - return children.get(0).getColumnModifier(); - } - - @Override - public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { - if (!getStringExpression().evaluate(tuple, ptr)) { - return false; - } - if (ptr.getLength() == 0) { - ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); - return true; - } - byte[] string = ptr.get(); - int offset = ptr.getOffset(); - int length = ptr.getLength(); - - ColumnModifier columnModifier = getColumnModifier(); - int end = StringUtil.getFirstNonBlankCharIdxFromEnd(string, offset, length, columnModifier); - if (end == offset - 1) { - ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); - return true; - } - int head = StringUtil.getFirstNonBlankCharIdxFromStart(string, offset, length, columnModifier); - ptr.set(string, head, end - head + 1); - return true; - } - - @Override - public Integer getByteSize() { - return byteSize; - } - - @Override - public PDataType getDataType() { - return PDataType.VARCHAR; - } - - @Override - public String getName() { - return NAME; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/TruncFunction.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/TruncFunction.java deleted file mode 100644 index 9a6ee50b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/TruncFunction.java +++ /dev/null @@ -1,73 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FloorParseNode; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.PDataType; - - -/** - * - * Function used to bucketize date/time values by truncating them to - * an even increment. Usage: - * TRUNC(,<'day'|'hour'|'minute'|'second'|'millisecond'>,[]) - * The integer multiplier is optional and is used to do rollups to a partial time unit (i.e. 10 minute rollup) - * The function returns a {@link com.salesforce.phoenix.schema.PDataType#DATE} - * - * @author jtaylor - * @since 0.1 - */ -@BuiltInFunction(name = TruncFunction.NAME, -nodeClass = FloorParseNode.class, -args = { - @Argument(allowedTypes={PDataType.TIMESTAMP, PDataType.DECIMAL}), - @Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValue = "null", isConstant=true), - @Argument(allowedTypes={PDataType.INTEGER}, defaultValue="1", isConstant=true) - } -) -public abstract class TruncFunction extends ScalarFunction { - - public static final String NAME = "TRUNC"; - - public TruncFunction(List children) throws SQLException { - super(children); - } - - @Override - public String getName() { - return NAME; - } - - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/BaseExpressionVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/BaseExpressionVisitor.java deleted file mode 100644 index 4417954a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/BaseExpressionVisitor.java +++ /dev/null @@ -1,272 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.visitor; - -import java.util.Iterator; -import java.util.List; - -import com.salesforce.phoenix.expression.AddExpression; -import com.salesforce.phoenix.expression.AndExpression; -import com.salesforce.phoenix.expression.ArrayConstructorExpression; -import com.salesforce.phoenix.expression.CaseExpression; -import com.salesforce.phoenix.expression.CoerceExpression; -import com.salesforce.phoenix.expression.ComparisonExpression; -import com.salesforce.phoenix.expression.DivideExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.InListExpression; -import com.salesforce.phoenix.expression.IsNullExpression; -import com.salesforce.phoenix.expression.KeyValueColumnExpression; -import com.salesforce.phoenix.expression.LikeExpression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.MultiplyExpression; -import com.salesforce.phoenix.expression.NotExpression; -import com.salesforce.phoenix.expression.OrExpression; -import com.salesforce.phoenix.expression.ProjectedColumnExpression; -import com.salesforce.phoenix.expression.RowKeyColumnExpression; -import com.salesforce.phoenix.expression.RowValueConstructorExpression; -import com.salesforce.phoenix.expression.StringConcatExpression; -import com.salesforce.phoenix.expression.SubtractExpression; -import com.salesforce.phoenix.expression.function.ScalarFunction; -import com.salesforce.phoenix.expression.function.SingleAggregateFunction; - - -public abstract class BaseExpressionVisitor implements ExpressionVisitor { - @Override - public E visit(Expression node) { - return null; - } - - @Override - public Iterator visitEnter(Expression node) { - return null; - } - - @Override - public E visitLeave(Expression node, List l) { - return null; - } - - @Override - public E defaultReturn(Expression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(AndExpression node) { - return null; - } - - @Override - public E visitLeave(AndExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(OrExpression node) { - return null; - } - - @Override - public E visitLeave(OrExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(ScalarFunction node) { - return null; - } - - @Override - public E visitLeave(ScalarFunction node, List l) { - return null; - } - - @Override - public Iterator visitEnter(ComparisonExpression node) { - return null; - } - - @Override - public E visitLeave(ComparisonExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(LikeExpression node) { - return null; - } - - @Override - public E visitLeave(LikeExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(SingleAggregateFunction node) { - return null; - } - - @Override - public E visitLeave(SingleAggregateFunction node, List l) { - return null; - } - - @Override - public Iterator visitEnter(CaseExpression node) { - return null; - } - - @Override - public E visitLeave(CaseExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(NotExpression node) { - return null; - } - - @Override - public E visitLeave(NotExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(IsNullExpression node) { - return null; - } - - @Override - public E visitLeave(IsNullExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(InListExpression node) { - return null; - } - - @Override - public E visitLeave(InListExpression node, List l) { - return null; - } - - @Override - public E visit(LiteralExpression node) { - return null; - } - - @Override - public E visit(RowKeyColumnExpression node) { - return null; - } - - @Override - public E visit(KeyValueColumnExpression node) { - return null; - } - - @Override - public E visit(ProjectedColumnExpression node) { - return null; - } - - @Override - public Iterator visitEnter(SubtractExpression node) { - return null; - } - - @Override - public E visitLeave(SubtractExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(AddExpression node) { - return null; - } - @Override - public E visitLeave(AddExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(MultiplyExpression node) { - return null; - } - @Override - public E visitLeave(MultiplyExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(DivideExpression node) { - return null; - } - @Override - public E visitLeave(DivideExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(StringConcatExpression node) { - return null; - } - @Override - public E visitLeave(StringConcatExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(RowValueConstructorExpression node) { - return null; - } - @Override - public E visitLeave(RowValueConstructorExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(CoerceExpression node) { - return null; - } - - @Override - public E visitLeave(CoerceExpression node, List l) { - return null; - } - - @Override - public Iterator visitEnter(ArrayConstructorExpression node) { - return null; - } - @Override - public E visitLeave(ArrayConstructorExpression node, List l) { - return null; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/ExpressionVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/ExpressionVisitor.java deleted file mode 100644 index 92aa7a69..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/ExpressionVisitor.java +++ /dev/null @@ -1,150 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.visitor; - -import java.util.Iterator; -import java.util.List; - -import com.salesforce.phoenix.expression.AddExpression; -import com.salesforce.phoenix.expression.AndExpression; -import com.salesforce.phoenix.expression.ArrayConstructorExpression; -import com.salesforce.phoenix.expression.CaseExpression; -import com.salesforce.phoenix.expression.CoerceExpression; -import com.salesforce.phoenix.expression.ComparisonExpression; -import com.salesforce.phoenix.expression.DivideExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.InListExpression; -import com.salesforce.phoenix.expression.IsNullExpression; -import com.salesforce.phoenix.expression.KeyValueColumnExpression; -import com.salesforce.phoenix.expression.LikeExpression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.MultiplyExpression; -import com.salesforce.phoenix.expression.NotExpression; -import com.salesforce.phoenix.expression.OrExpression; -import com.salesforce.phoenix.expression.ProjectedColumnExpression; -import com.salesforce.phoenix.expression.RowKeyColumnExpression; -import com.salesforce.phoenix.expression.RowValueConstructorExpression; -import com.salesforce.phoenix.expression.StringConcatExpression; -import com.salesforce.phoenix.expression.SubtractExpression; -import com.salesforce.phoenix.expression.function.ScalarFunction; -import com.salesforce.phoenix.expression.function.SingleAggregateFunction; - - -/** - * - * Visitor for an expression (which may contain other nested expressions) - * - * @author jtaylor - * @since 0.1 - */ -public interface ExpressionVisitor { - /** - * Default visit method when an expression subclass doesn't - * define an accept method of its own. This will end up calling - * the {@link #defaultIterator(Expression)} to iterate over the - * children calling accept on them - */ - public E visit(Expression node); - /** - * Default visitEnter method when an expression subclass doesn't - * define an accept method of its own. This will end up calling - * the {@link #defaultIterator(Expression)} to iterate over the - * children calling accept on them - */ - public Iterator visitEnter(Expression node); - /** - * Default visitLeave method when an expression subclass doesn't - * define an accept method of its own. This will end up calling - * the {@link #defaultReturn(Expression, List)} with the list from - * the iteration over the children. - */ - public E visitLeave(Expression node, List l); - - public E defaultReturn(Expression node, List l); - public Iterator defaultIterator(Expression node); - - public Iterator visitEnter(AndExpression node); - public E visitLeave(AndExpression node, List l); - - public Iterator visitEnter(OrExpression node); - public E visitLeave(OrExpression node, List l); - - public Iterator visitEnter(ScalarFunction node); - public E visitLeave(ScalarFunction node, List l); - - public Iterator visitEnter(ComparisonExpression node); - public E visitLeave(ComparisonExpression node, List l); - - public Iterator visitEnter(LikeExpression node); - public E visitLeave(LikeExpression node, List l); - - public Iterator visitEnter(SingleAggregateFunction node); - public E visitLeave(SingleAggregateFunction node, List l); - - public Iterator visitEnter(CaseExpression node); - public E visitLeave(CaseExpression node, List l); - - public Iterator visitEnter(NotExpression node); - public E visitLeave(NotExpression node, List l); - - public Iterator visitEnter(InListExpression node); - public E visitLeave(InListExpression node, List l); - - public Iterator visitEnter(IsNullExpression node); - public E visitLeave(IsNullExpression node, List l); - - public Iterator visitEnter(SubtractExpression node); - public E visitLeave(SubtractExpression node, List l); - - public Iterator visitEnter(MultiplyExpression node); - public E visitLeave(MultiplyExpression node, List l); - - public Iterator visitEnter(AddExpression node); - public E visitLeave(AddExpression node, List l); - - public Iterator visitEnter(DivideExpression node); - public E visitLeave(DivideExpression node, List l); - - public Iterator visitEnter(CoerceExpression node); - public E visitLeave(CoerceExpression node, List l); - - public Iterator visitEnter(ArrayConstructorExpression node); - public E visitLeave(ArrayConstructorExpression node, List l); - - public E visit(LiteralExpression node); - public E visit(RowKeyColumnExpression node); - public E visit(KeyValueColumnExpression node); - public E visit(ProjectedColumnExpression node); - - public Iterator visitEnter(StringConcatExpression node); - public E visitLeave(StringConcatExpression node, List l); - - public Iterator visitEnter(RowValueConstructorExpression node); - public E visitLeave(RowValueConstructorExpression node, List l); - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/KeyValueExpressionVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/KeyValueExpressionVisitor.java deleted file mode 100644 index 925f55ff..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/KeyValueExpressionVisitor.java +++ /dev/null @@ -1,46 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.visitor; - -import com.salesforce.phoenix.expression.KeyValueColumnExpression; - - - - -/** - * - * Implementation of ExpressionVisitor where only KeyValueDataAccessor - * is being visited - * - * @author jtaylor - * @since 0.1 - */ -public abstract class KeyValueExpressionVisitor extends TraverseAllExpressionVisitor { - @Override - abstract public Void visit(KeyValueColumnExpression node); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/SingleAggregateFunctionVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/SingleAggregateFunctionVisitor.java deleted file mode 100644 index cc58cf91..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/SingleAggregateFunctionVisitor.java +++ /dev/null @@ -1,48 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.visitor; - -import java.util.Iterator; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.SingleAggregateFunction; - - - -/** - * - * Implementation of ExpressionVisitor where only SingleAggregateFunction - * instances are visited - * - * @author jtaylor - * @since 0.1 - */ -public abstract class SingleAggregateFunctionVisitor extends TraverseAllExpressionVisitor { - @Override - abstract public Iterator visitEnter(SingleAggregateFunction node); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/TraverseAllExpressionVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/TraverseAllExpressionVisitor.java deleted file mode 100644 index 0077e691..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/TraverseAllExpressionVisitor.java +++ /dev/null @@ -1,65 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.visitor; - -import java.util.*; - -import com.salesforce.phoenix.expression.Expression; - - - - -public class TraverseAllExpressionVisitor extends BaseExpressionVisitor { - - @Override - public Iterator defaultIterator(Expression node) { - final List children = node.getChildren(); - return new Iterator() { - private int position; - - @Override - public final boolean hasNext() { - return position < children.size(); - } - - @Override - public final Expression next() { - if (!hasNext()) { - throw new NoSuchElementException(); - } - return children.get(position++); - } - - @Override - public final void remove() { - throw new UnsupportedOperationException(); - } - }; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/TraverseNoExpressionVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/TraverseNoExpressionVisitor.java deleted file mode 100644 index afc08308..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/visitor/TraverseNoExpressionVisitor.java +++ /dev/null @@ -1,43 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.visitor; - -import java.util.Iterator; - - -import com.google.common.collect.Iterators; -import com.salesforce.phoenix.expression.Expression; - -public class TraverseNoExpressionVisitor extends BaseExpressionVisitor { - - @Override - public Iterator defaultIterator(Expression node) { - return Iterators.emptyIterator(); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/BooleanExpressionFilter.java b/phoenix-core/src/main/java/com/salesforce/phoenix/filter/BooleanExpressionFilter.java deleted file mode 100644 index 75b72245..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/BooleanExpressionFilter.java +++ /dev/null @@ -1,125 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.filter; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; - -import org.apache.hadoop.hbase.filter.FilterBase; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.io.WritableUtils; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.ExpressionType; -import com.salesforce.phoenix.schema.IllegalDataException; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Base class for filter that evaluates a WHERE clause expression. - * - * @author jtaylor - * @since 0.1 - */ -abstract public class BooleanExpressionFilter extends FilterBase { - - protected Expression expression; - protected boolean evaluateOnCompletion; - private ImmutableBytesWritable tempPtr = new ImmutableBytesWritable(); - - public BooleanExpressionFilter() { - } - - public BooleanExpressionFilter(Expression expression) { - this.expression = expression; - } - - protected void setEvaluateOnCompletion(boolean evaluateOnCompletion) { - this.evaluateOnCompletion = evaluateOnCompletion; - } - - protected boolean evaluateOnCompletion() { - return evaluateOnCompletion; - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + expression.hashCode(); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - BooleanExpressionFilter other = (BooleanExpressionFilter)obj; - if (!expression.equals(other.expression)) return false; - return true; - } - - @Override - public String toString() { - return expression.toString(); - } - - @edu.umd.cs.findbugs.annotations.SuppressWarnings( - value="NP_BOOLEAN_RETURN_NULL", - justification="Returns null by design.") - protected Boolean evaluate(Tuple input) { - try { - if (!expression.evaluate(input, tempPtr)) { - return null; - } - } catch (IllegalDataException e) { - return Boolean.FALSE; - } - return (Boolean)expression.getDataType().toObject(tempPtr); - } - - @Override - public void readFields(DataInput input) throws IOException { - expression = ExpressionType.values()[WritableUtils.readVInt(input)].newInstance(); - expression.readFields(input); - } - - @Override - public void write(DataOutput output) throws IOException { - WritableUtils.writeVInt(output, ExpressionType.valueOf(expression).ordinal()); - expression.write(output); - } - - @Override - public void reset() { - expression.reset(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/EvaluateOnCompletionVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/filter/EvaluateOnCompletionVisitor.java deleted file mode 100644 index e93b8cfd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/EvaluateOnCompletionVisitor.java +++ /dev/null @@ -1,91 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.filter; - -import java.util.Iterator; - -import com.salesforce.phoenix.expression.ArrayConstructorExpression; -import com.salesforce.phoenix.expression.CaseExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.IsNullExpression; -import com.salesforce.phoenix.expression.RowKeyColumnExpression; -import com.salesforce.phoenix.expression.RowValueConstructorExpression; -import com.salesforce.phoenix.expression.visitor.TraverseAllExpressionVisitor; - - -/** - * - * Implementation of ExpressionVisitor for the expression used by the - * BooleanExpressionFilter that looks for expressions that need to be - * evaluated upon completion. Examples include: - * - CaseExpression with an else clause, since upon completion, the - * else clause would apply if the when clauses could not be evaluated - * due to the absense of a value. - * - IsNullExpression that's not negated, since upon completion, we - * know definitively that a column value was not found. - * - row key columns are used, since we may never have encountered a - * key value column of interest, but the expression may evaluate to true - * just based on the row key columns. - * @author jtaylor - * @since 0.1 - */ -public class EvaluateOnCompletionVisitor extends TraverseAllExpressionVisitor { - private boolean evaluateOnCompletion = false; - - public boolean evaluateOnCompletion() { - return evaluateOnCompletion; - } - - @Override - public Iterator visitEnter(IsNullExpression node) { - evaluateOnCompletion |= !node.isNegate(); - return null; - } - @Override - public Iterator visitEnter(CaseExpression node) { - evaluateOnCompletion |= node.hasElse(); - return null; - } - @Override - public Void visit(RowKeyColumnExpression node) { - evaluateOnCompletion = true; - return null; - } - @Override - public Iterator visitEnter(RowValueConstructorExpression node) { - evaluateOnCompletion = true; - return null; - } - - @Override - public Iterator visitEnter(ArrayConstructorExpression node) { - evaluateOnCompletion = true; - return null; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/MultiCFCQKeyValueComparisonFilter.java b/phoenix-core/src/main/java/com/salesforce/phoenix/filter/MultiCFCQKeyValueComparisonFilter.java deleted file mode 100644 index 4fcabbfc..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/MultiCFCQKeyValueComparisonFilter.java +++ /dev/null @@ -1,142 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.filter; - -import java.util.TreeSet; - -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.phoenix.expression.Expression; - - -/** - * - * Filter that evaluates WHERE clause expression, used in the case where there - * are references to multiple column qualifiers over multiple column families. - * - * @author jtaylor - * @since 0.1 - */ -public class MultiCFCQKeyValueComparisonFilter extends MultiKeyValueComparisonFilter { - private final ImmutablePairBytesPtr ptr = new ImmutablePairBytesPtr(); - private TreeSet cfSet; - - public MultiCFCQKeyValueComparisonFilter() { - } - - public MultiCFCQKeyValueComparisonFilter(Expression expression) { - super(expression); - } - - @Override - protected void init() { - cfSet = new TreeSet(Bytes.BYTES_COMPARATOR); - super.init(); - } - - @Override - protected Object setColumnKey(byte[] cf, int cfOffset, int cfLength, - byte[] cq, int cqOffset, int cqLength) { - ptr.set(cf, cfOffset, cfLength, cq, cqOffset, cqLength); - return ptr; - } - - @Override - protected Object newColumnKey(byte[] cf, int cfOffset, int cfLength, - byte[] cq, int cqOffset, int cqLength) { - - byte[] cfKey; - if (cfOffset == 0 && cf.length == cfLength) { - cfKey = cf; - } else { - // Copy bytes here, but figure cf names are typically a few bytes at most, - // so this will be better than creating an ImmutableBytesPtr - cfKey = new byte[cfLength]; - System.arraycopy(cf, cfOffset, cfKey, 0, cfLength); - } - cfSet.add(cfKey); - return new ImmutablePairBytesPtr(cf, cfOffset, cfLength, cq, cqOffset, cqLength); - } - - private static class ImmutablePairBytesPtr { - private byte[] bytes1; - private int offset1; - private int length1; - private byte[] bytes2; - private int offset2; - private int length2; - private int hashCode; - - private ImmutablePairBytesPtr() { - } - - private ImmutablePairBytesPtr(byte[] bytes1, int offset1, int length1, byte[] bytes2, int offset2, int length2) { - set(bytes1, offset1, length1, bytes2, offset2, length2); - } - - @Override - public int hashCode() { - return hashCode; - } - - public void set(byte[] bytes1, int offset1, int length1, byte[] bytes2, int offset2, int length2) { - this.bytes1 = bytes1; - this.offset1 = offset1; - this.length1 = length1; - this.bytes2 = bytes2; - this.offset2 = offset2; - this.length2 = length2; - int hash = 1; - for (int i = offset1; i < offset1 + length1; i++) - hash = (31 * hash) + bytes1[i]; - for (int i = offset2; i < offset2 + length2; i++) - hash = (31 * hash) + bytes2[i]; - hashCode = hash; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - ImmutablePairBytesPtr that = (ImmutablePairBytesPtr)obj; - if (this.hashCode != that.hashCode) return false; - if (Bytes.compareTo(this.bytes2, this.offset2, this.length2, that.bytes2, that.offset2, that.length2) != 0) return false; - if (Bytes.compareTo(this.bytes1, this.offset1, this.length1, that.bytes1, that.offset1, that.length1) != 0) return false; - return true; - } - } - - - @SuppressWarnings("all") // suppressing missing @Override since this doesn't exist for HBase 0.94.4 - public boolean isFamilyEssential(byte[] name) { - // Only the column families involved in the expression are essential. - // The others are for columns projected in the select expression. - return cfSet.contains(name); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/MultiCQKeyValueComparisonFilter.java b/phoenix-core/src/main/java/com/salesforce/phoenix/filter/MultiCQKeyValueComparisonFilter.java deleted file mode 100644 index 215456cc..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/MultiCQKeyValueComparisonFilter.java +++ /dev/null @@ -1,78 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.filter; - -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.expression.Expression; - -/** - * - * Filter that evaluates WHERE clause expression, used in the case where there - * are references to multiple column qualifiers over a single column family. - * - * @author jtaylor - * @since 0.1 - */ -public class MultiCQKeyValueComparisonFilter extends MultiKeyValueComparisonFilter { - private ImmutableBytesPtr ptr = new ImmutableBytesPtr(); - private byte[] cf; - - public MultiCQKeyValueComparisonFilter() { - } - - public MultiCQKeyValueComparisonFilter(Expression expression) { - super(expression); - } - - @Override - protected Object setColumnKey(byte[] cf, int cfOffset, int cfLength, byte[] cq, int cqOffset, - int cqLength) { - ptr.set(cq, cqOffset, cqLength); - return ptr; - } - - @Override - protected Object newColumnKey(byte[] cf, int cfOffset, int cfLength, byte[] cq, int cqOffset, - int cqLength) { - if (cfOffset == 0 && cf.length == cfLength) { - this.cf = cf; - } else { - this.cf = new byte[cfLength]; - System.arraycopy(cf, cfOffset, this.cf, 0, cfLength); - } - return new ImmutableBytesPtr(cq, cqOffset, cqLength); - } - - - @SuppressWarnings("all") // suppressing missing @Override since this doesn't exist for HBase 0.94.4 - public boolean isFamilyEssential(byte[] name) { - return Bytes.compareTo(cf, name) == 0; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/RowKeyComparisonFilter.java b/phoenix-core/src/main/java/com/salesforce/phoenix/filter/RowKeyComparisonFilter.java deleted file mode 100644 index 03cfd3cd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/RowKeyComparisonFilter.java +++ /dev/null @@ -1,164 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.filter; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; -import org.apache.hadoop.io.WritableUtils; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Filter for use when expressions only reference row key columns - * - * @author jtaylor - * @since 0.1 - */ -public class RowKeyComparisonFilter extends BooleanExpressionFilter { - private static final Logger logger = LoggerFactory.getLogger(RowKeyComparisonFilter.class); - - private boolean evaluate = true; - private boolean keepRow = false; - private RowKeyTuple inputTuple = new RowKeyTuple(); - private byte[] essentialCF; - - public RowKeyComparisonFilter() { - } - - public RowKeyComparisonFilter(Expression expression, byte[] essentialCF) { - super(expression); - this.essentialCF = essentialCF; - } - - @Override - public void reset() { - this.keepRow = false; - this.evaluate = true; - super.reset(); - } - - /** - * Evaluate in filterKeyValue instead of filterRowKey, because HBASE-6562 causes filterRowKey - * to be called with deleted or partial row keys. - */ - @Override - public ReturnCode filterKeyValue(KeyValue v) { - if (evaluate) { - inputTuple.setKey(v.getBuffer(), v.getRowOffset(), v.getRowLength()); - this.keepRow = Boolean.TRUE.equals(evaluate(inputTuple)); - if (logger.isDebugEnabled()) { - logger.debug("RowKeyComparisonFilter: " + (this.keepRow ? "KEEP" : "FILTER") + " row " + inputTuple); - } - evaluate = false; - } - return keepRow ? ReturnCode.INCLUDE : ReturnCode.NEXT_ROW; - } - - private final class RowKeyTuple implements Tuple { - private byte[] buf; - private int offset; - private int length; - - public void setKey(byte[] buf, int offset, int length) { - this.buf = buf; - this.offset = offset; - this.length = length; - } - - @Override - public void getKey(ImmutableBytesWritable ptr) { - ptr.set(buf, offset, length); - } - - @Override - public KeyValue getValue(byte[] cf, byte[] cq) { - return null; - } - - @Override - public boolean isImmutable() { - return true; - } - - @Override - public String toString() { - return Bytes.toStringBinary(buf, offset, length); - } - - @Override - public int size() { - return 0; - } - - @Override - public KeyValue getValue(int index) { - throw new IndexOutOfBoundsException(Integer.toString(index)); - } - - @Override - public boolean getValue(byte[] family, byte[] qualifier, - ImmutableBytesWritable ptr) { - return false; - } - } - - @Override - public boolean filterRow() { - return !this.keepRow; - } - - @SuppressWarnings("all") // suppressing missing @Override since this doesn't exist for HBase 0.94.4 - public boolean isFamilyEssential(byte[] name) { - // We only need our "guaranteed to have a key value" column family, - // which we pass in and serialize through. In the case of a VIEW where - // we don't have this, we have to say that all families are essential. - return this.essentialCF.length == 0 ? true : Bytes.compareTo(this.essentialCF, name) == 0; - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - this.essentialCF = WritableUtils.readCompressedByteArray(input); - } - - @Override - public void write(DataOutput output) throws IOException { - super.write(output); - WritableUtils.writeCompressedByteArray(output, this.essentialCF); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/SingleCFCQKeyValueComparisonFilter.java b/phoenix-core/src/main/java/com/salesforce/phoenix/filter/SingleCFCQKeyValueComparisonFilter.java deleted file mode 100644 index 51f69f25..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/SingleCFCQKeyValueComparisonFilter.java +++ /dev/null @@ -1,59 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.filter; - -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.phoenix.expression.Expression; - - -/** - * - * SingleKeyValueComparisonFilter that needs to compare both the column family and - * column qualifier parts of the key value to disambiguate with another similarly - * named column qualifier in a different column family. - * - * @author jtaylor - * @since 0.1 - */ -public class SingleCFCQKeyValueComparisonFilter extends SingleKeyValueComparisonFilter { - public SingleCFCQKeyValueComparisonFilter() { - } - - public SingleCFCQKeyValueComparisonFilter(Expression expression) { - super(expression); - } - - @Override - protected final int compare(byte[] cfBuf, int cfOffset, int cfLength, byte[] cqBuf, int cqOffset, int cqLength) { - int c = Bytes.compareTo(cf, 0, cf.length, cfBuf, cfOffset, cfLength); - if (c != 0) return c; - return Bytes.compareTo(cq, 0, cq.length, cqBuf, cqOffset, cqLength); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/SingleCQKeyValueComparisonFilter.java b/phoenix-core/src/main/java/com/salesforce/phoenix/filter/SingleCQKeyValueComparisonFilter.java deleted file mode 100644 index 0940ccae..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/SingleCQKeyValueComparisonFilter.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.filter; - -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.phoenix.expression.Expression; - - -/** - * - * SingleKeyValueComparisonFilter that needs to only compare the column qualifier - * part of the key value since the column qualifier is unique across all column - * families. - * - * @author jtaylor - * @since 0.1 - */ -public class SingleCQKeyValueComparisonFilter extends SingleKeyValueComparisonFilter { - public SingleCQKeyValueComparisonFilter() { - } - - public SingleCQKeyValueComparisonFilter(Expression expression) { - super(expression); - } - - @Override - protected final int compare(byte[] cfBuf, int cfOffset, int cfLength, byte[] cqBuf, int cqOffset, int cqLength) { - return Bytes.compareTo(cq, 0, cq.length, cqBuf, cqOffset, cqLength); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/SingleKeyValueComparisonFilter.java b/phoenix-core/src/main/java/com/salesforce/phoenix/filter/SingleKeyValueComparisonFilter.java deleted file mode 100644 index 3c5613ee..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/SingleKeyValueComparisonFilter.java +++ /dev/null @@ -1,155 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.filter; - -import java.io.DataInput; -import java.io.IOException; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.KeyValueColumnExpression; -import com.salesforce.phoenix.schema.tuple.SingleKeyValueTuple; - - - -/** - * - * Modeled after {@link org.apache.hadoop.hbase.filter.SingleColumnValueFilter}, - * but for general expression evaluation in the case where only a single KeyValue - * column is referenced in the expression. - * - * @author jtaylor - * @since 0.1 - */ -public abstract class SingleKeyValueComparisonFilter extends BooleanExpressionFilter { - private final SingleKeyValueTuple inputTuple = new SingleKeyValueTuple(); - private boolean matchedColumn; - protected byte[] cf; - protected byte[] cq; - - public SingleKeyValueComparisonFilter() { - } - - public SingleKeyValueComparisonFilter(Expression expression) { - super(expression); - init(); - } - - protected abstract int compare(byte[] cfBuf, int cfOffset, int cfLength, byte[] cqBuf, int cqOffset, int cqLength); - - private void init() { - EvaluateOnCompletionVisitor visitor = new EvaluateOnCompletionVisitor() { - @Override - public Void visit(KeyValueColumnExpression expression) { - cf = expression.getColumnFamily(); - cq = expression.getColumnName(); - return null; - } - }; - expression.accept(visitor); - this.evaluateOnCompletion = visitor.evaluateOnCompletion(); - } - - private boolean foundColumn() { - return inputTuple.size() > 0; - } - - @Override - public ReturnCode filterKeyValue(KeyValue keyValue) { - if (this.matchedColumn) { - // We already found and matched the single column, all keys now pass - // TODO: why won't this cause earlier versions of a kv to be included? - return ReturnCode.INCLUDE; - } - if (this.foundColumn()) { - // We found all the columns, but did not match the expression, so skip to next row - return ReturnCode.NEXT_ROW; - } - byte[] buf = keyValue.getBuffer(); - if (compare(buf, keyValue.getFamilyOffset(), keyValue.getFamilyLength(), buf, keyValue.getQualifierOffset(), keyValue.getQualifierLength()) != 0) { - // Remember the key in case this is the only key value we see. - // We'll need it if we have row key columns too. - inputTuple.setKey(keyValue); - // This is a key value we're not interested in - // TODO: use NEXT_COL when bug fix comes through that includes the row still - return ReturnCode.INCLUDE; - } - inputTuple.setKeyValue(keyValue); - - // We have the columns, so evaluate here - if (!Boolean.TRUE.equals(evaluate(inputTuple))) { - return ReturnCode.NEXT_ROW; - } - this.matchedColumn = true; - return ReturnCode.INCLUDE; - } - - @Override - public boolean filterRow() { - // If column was found, return false if it was matched, true if it was not. - if (foundColumn()) { - return !this.matchedColumn; - } - // If column was not found, evaluate the expression here upon completion. - // This is required with certain expressions, for example, with IS NULL - // expressions where they'll evaluate to TRUE when the column being - // tested wasn't found. - // Since the filter is called also to position the scan initially, we have - // to guard against this by checking whether or not we've filtered in - // the key value (i.e. filterKeyValue was called and we found the keyValue - // for which we're looking). - if (inputTuple.hasKey() && evaluateOnCompletion()) { - return !Boolean.TRUE.equals(evaluate(inputTuple)); - } - // Finally, if we have no values, and we're not required to re-evaluate it - // just filter the row - return true; - } - - @Override - public void reset() { - inputTuple.reset(); - matchedColumn = false; - super.reset(); - } - - @Override - public void readFields(DataInput input) throws IOException { - super.readFields(input); - init(); - } - - @SuppressWarnings("all") // suppressing missing @Override since this doesn't exist for HBase 0.94.4 - public boolean isFamilyEssential(byte[] name) { - // Only the column families involved in the expression are essential. - // The others are for columns projected in the select expression - return Bytes.compareTo(cf, name) == 0; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/index/BaseIndexCodec.java b/phoenix-core/src/main/java/com/salesforce/phoenix/index/BaseIndexCodec.java deleted file mode 100644 index 923f3c47..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/index/BaseIndexCodec.java +++ /dev/null @@ -1,69 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.index; - -import java.io.IOException; - -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; - -import com.salesforce.hbase.index.covered.IndexCodec; - -/** - * - */ -public abstract class BaseIndexCodec implements IndexCodec { - - @Override - public void initialize(RegionCoprocessorEnvironment env) throws IOException { - // noop - } - - /** - * {@inheritDoc} - *

- * By default, the codec is always enabled. Subclasses should override this method if they want do - * decide to index on a per-mutation basis. - * @throws IOException - */ - @Override - public boolean isEnabled(Mutation m) throws IOException { - return true; - } - - /** - * {@inheritDoc} - *

- * Assumes each mutation is not in a batch. Subclasses that have different batching behavior - * should override this. - */ - @Override - public byte[] getBatchId(Mutation m) { - return null; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/index/IndexMetaDataCacheClient.java b/phoenix-core/src/main/java/com/salesforce/phoenix/index/IndexMetaDataCacheClient.java deleted file mode 100644 index e5853077..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/index/IndexMetaDataCacheClient.java +++ /dev/null @@ -1,94 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.index; - -import static com.salesforce.phoenix.query.QueryServices.INDEX_MUTATE_BATCH_SIZE_THRESHOLD_ATTRIB; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.cache.ServerCacheClient; -import com.salesforce.phoenix.cache.ServerCacheClient.ServerCache; -import com.salesforce.phoenix.compile.ScanRanges; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.join.MaxServerCacheSizeExceededException; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.ReadOnlyProps; -import com.salesforce.phoenix.util.ScanUtil; - -public class IndexMetaDataCacheClient { - - private final ServerCacheClient serverCache; - private TableRef cacheUsingTableRef; - - /** - * Construct client used to send index metadata to each region server - * for caching during batched put for secondary index maintenance. - * @param connection the client connection - * @param cacheUsingTableRef table ref to table that will use the cache during its scan - */ - public IndexMetaDataCacheClient(PhoenixConnection connection, TableRef cacheUsingTableRef) { - serverCache = new ServerCacheClient(connection); - this.cacheUsingTableRef = cacheUsingTableRef; - } - - /** - * Determines whether or not to use the IndexMetaDataCache to send the index metadata - * to the region servers. The alternative is to just set the index metadata as an attribute on - * the mutations. - * @param connection - * @param mutations the list of mutations that will be sent in a batch to server - * @param indexMetaDataByteLength length in bytes of the index metadata cache - */ - public static boolean useIndexMetadataCache(PhoenixConnection connection, List mutations, int indexMetaDataByteLength) { - ReadOnlyProps props = connection.getQueryServices().getProps(); - int threshold = props.getInt(INDEX_MUTATE_BATCH_SIZE_THRESHOLD_ATTRIB, QueryServicesOptions.DEFAULT_INDEX_MUTATE_BATCH_SIZE_THRESHOLD); - return (indexMetaDataByteLength > ServerCacheClient.UUID_LENGTH && mutations.size() > threshold); - } - - /** - * Send the index metadata cahce to all region servers for regions that will handle the mutations. - * @return client-side {@link ServerCache} representing the added index metadata cache - * @throws SQLException - * @throws MaxServerCacheSizeExceededException if size of hash cache exceeds max allowed - * size - */ - public ServerCache addIndexMetadataCache(List mutations, ImmutableBytesWritable ptr) throws SQLException { - /** - * Serialize and compress hashCacheTable - */ - return serverCache.addServerCache(ScanUtil.newScanRanges(mutations), ptr, new IndexMetaDataCacheFactory(), cacheUsingTableRef); - } - - - /** - * Send the index metadata cahce to all region servers for regions that will handle the mutations. - * @return client-side {@link ServerCache} representing the added index metadata cache - * @throws SQLException - * @throws MaxServerCacheSizeExceededException if size of hash cache exceeds max allowed - * size - */ - public ServerCache addIndexMetadataCache(ScanRanges ranges, ImmutableBytesWritable ptr) throws SQLException { - /** - * Serialize and compress hashCacheTable - */ - return serverCache.addServerCache(ranges, ptr, new IndexMetaDataCacheFactory(), cacheUsingTableRef); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/index/IndexMetaDataCacheFactory.java b/phoenix-core/src/main/java/com/salesforce/phoenix/index/IndexMetaDataCacheFactory.java deleted file mode 100644 index b53eacbe..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/index/IndexMetaDataCacheFactory.java +++ /dev/null @@ -1,62 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.index; - -import java.io.Closeable; -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.cache.IndexMetaDataCache; -import com.salesforce.phoenix.client.GenericKeyValueBuilder; -import com.salesforce.phoenix.coprocessor.ServerCachingProtocol.ServerCacheFactory; -import com.salesforce.phoenix.memory.MemoryManager.MemoryChunk; - -public class IndexMetaDataCacheFactory implements ServerCacheFactory { - public IndexMetaDataCacheFactory() { - } - - @Override - public void readFields(DataInput arg0) throws IOException { - } - - @Override - public void write(DataOutput arg0) throws IOException { - } - - @Override - public Closeable newCache (ImmutableBytesWritable cachePtr, final MemoryChunk chunk) throws SQLException { - // just use the standard keyvalue builder - this doesn't really need to be fast - final List maintainers = - IndexMaintainer.deserialize(cachePtr, GenericKeyValueBuilder.INSTANCE); - return new IndexMetaDataCache() { - - @Override - public void close() throws IOException { - chunk.close(); - } - - @Override - public List getIndexMaintainers() { - return maintainers; - } - }; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/index/PhoenixIndexBuilder.java b/phoenix-core/src/main/java/com/salesforce/phoenix/index/PhoenixIndexBuilder.java deleted file mode 100644 index be3267f7..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/index/PhoenixIndexBuilder.java +++ /dev/null @@ -1,92 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.index; - -import java.io.IOException; -import java.util.ArrayList; -import java.util.Collections; -import java.util.List; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.regionserver.HRegion; -import org.apache.hadoop.hbase.regionserver.MiniBatchOperationInProgress; -import org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl; -import org.apache.hadoop.hbase.regionserver.RegionScanner; -import org.apache.hadoop.hbase.util.Pair; - -import com.google.common.collect.Lists; -import com.salesforce.hbase.index.covered.CoveredColumnsIndexBuilder; -import com.salesforce.hbase.index.util.IndexManagementUtil; -import com.salesforce.phoenix.compile.ScanRanges; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.util.SchemaUtil; - -/** - * Index builder for covered-columns index that ties into phoenix for faster use. - */ -public class PhoenixIndexBuilder extends CoveredColumnsIndexBuilder { - - @Override - public void batchStarted(MiniBatchOperationInProgress> miniBatchOp) throws IOException { - // The entire purpose of this method impl is to get the existing rows for the - // table rows being indexed into the block cache, as the index maintenance code - // does a point scan per row - List keys = Lists.newArrayListWithExpectedSize(miniBatchOp.size()); - List maintainers = new ArrayList(); - for (int i = 0; i < miniBatchOp.size(); i++) { - Mutation m = miniBatchOp.getOperation(i).getFirst(); - keys.add(PDataType.VARBINARY.getKeyRange(m.getRow())); - maintainers.addAll(getCodec().getIndexMaintainers(m.getAttributesMap())); - } - Scan scan = IndexManagementUtil.newLocalStateScan(maintainers); - ScanRanges scanRanges = ScanRanges.create(Collections.singletonList(keys), SchemaUtil.VAR_BINARY_SCHEMA); - scanRanges.setScanStartStopRow(scan); - scan.setFilter(scanRanges.getSkipScanFilter()); - HRegion region = this.env.getRegion(); - RegionScanner scanner = region.getScanner(scan); - // Run through the scanner using internal nextRaw method - MultiVersionConsistencyControl.setThreadReadPoint(scanner.getMvccReadPoint()); - region.startRegionOperation(); - try { - boolean hasMore; - do { - List results = Lists.newArrayList(); - // Results are potentially returned even when the return value of s.next is false - // since this is an indication of whether or not there are more values after the - // ones returned - hasMore = scanner.nextRaw(results, null); - } while (hasMore); - } finally { - try { - scanner.close(); - } finally { - region.closeRegionOperation(); - } - } - } - - private PhoenixIndexCodec getCodec() { - return (PhoenixIndexCodec)this.codec; - } - - @Override - public byte[] getBatchId(Mutation m){ - return this.codec.getBatchId(m); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/index/PhoenixIndexCodec.java b/phoenix-core/src/main/java/com/salesforce/phoenix/index/PhoenixIndexCodec.java deleted file mode 100644 index f59310bb..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/index/PhoenixIndexCodec.java +++ /dev/null @@ -1,178 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.index; - -import java.io.IOException; -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; -import java.util.Map; - -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.client.Delete; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.client.Put; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Pair; - -import com.google.common.collect.Lists; -import com.salesforce.hbase.index.ValueGetter; -import com.salesforce.hbase.index.covered.IndexCodec; -import com.salesforce.hbase.index.covered.IndexUpdate; -import com.salesforce.hbase.index.covered.TableState; -import com.salesforce.hbase.index.scanner.Scanner; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.hbase.index.util.IndexManagementUtil; -import com.salesforce.hbase.index.write.IndexWriter; -import com.salesforce.phoenix.cache.GlobalCache; -import com.salesforce.phoenix.cache.IndexMetaDataCache; -import com.salesforce.phoenix.cache.ServerCacheClient; -import com.salesforce.phoenix.cache.TenantCache; -import com.salesforce.phoenix.client.KeyValueBuilder; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.util.PhoenixRuntime; -import com.salesforce.phoenix.util.ServerUtil; - -/** - * Phoenix-based {@link IndexCodec}. Manages all the logic of how to cleanup an index ( - * {@link #getIndexDeletes(TableState)}) as well as what the new index state should be ( - * {@link #getIndexUpserts(TableState)}). - */ -public class PhoenixIndexCodec extends BaseIndexCodec { - public static final String INDEX_MD = "IdxMD"; - public static final String INDEX_UUID = "IdxUUID"; - - private RegionCoprocessorEnvironment env; - private KeyValueBuilder builder; - - @Override - public void initialize(RegionCoprocessorEnvironment env) { - this.env = env; - Configuration conf = env.getConfiguration(); - // Install handler that will attempt to disable the index first before killing the region - // server - conf.setIfUnset(IndexWriter.INDEX_FAILURE_POLICY_CONF_KEY, - PhoenixIndexFailurePolicy.class.getName()); - this.builder = KeyValueBuilder.get(env.getHBaseVersion()); - } - - List getIndexMaintainers(Map attributes) throws IOException{ - if (attributes == null) { - return Collections.emptyList(); - } - byte[] uuid = attributes.get(INDEX_UUID); - if (uuid == null) { - return Collections.emptyList(); - } - byte[] md = attributes.get(INDEX_MD); - List indexMaintainers; - if (md != null) { - indexMaintainers = IndexMaintainer.deserialize(md, builder); - } else { - byte[] tenantIdBytes = attributes.get(PhoenixRuntime.TENANT_ID_ATTRIB); - ImmutableBytesWritable tenantId = - tenantIdBytes == null ? null : new ImmutableBytesWritable(tenantIdBytes); - TenantCache cache = GlobalCache.getTenantCache(env, tenantId); - IndexMetaDataCache indexCache = - (IndexMetaDataCache) cache.getServerCache(new ImmutableBytesPtr(uuid)); - if (indexCache == null) { - String msg = "key="+ServerCacheClient.idToString(uuid) + " region=" + env.getRegion(); - SQLException e = new SQLExceptionInfo.Builder(SQLExceptionCode.INDEX_METADATA_NOT_FOUND) - .setMessage(msg).build().buildException(); - ServerUtil.throwIOException("Index update failed", e); // will not return - } - indexMaintainers = indexCache.getIndexMaintainers(); - } - - return indexMaintainers; - } - - @Override - public Iterable getIndexUpserts(TableState state) throws IOException { - List indexMaintainers = getIndexMaintainers(state.getUpdateAttributes()); - if (indexMaintainers.isEmpty()) { - return Collections.emptyList(); - } - ImmutableBytesWritable ptr = new ImmutableBytesWritable(); - List indexUpdates = Lists.newArrayList(); - // TODO: state.getCurrentRowKey() should take an ImmutableBytesWritable arg to prevent byte copy - byte[] dataRowKey = state.getCurrentRowKey(); - for (IndexMaintainer maintainer : indexMaintainers) { - // Short-circuit building state when we know it's a row deletion - if (maintainer.isRowDeleted(state.getPendingUpdate())) { - continue; - } - - // Get a scanner over the columns this maintainer would like to look at - // Any updates that we would make for those columns are then added to the index update - Pair statePair = state.getIndexedColumnsTableState(maintainer.getAllColumns()); - IndexUpdate indexUpdate = statePair.getSecond(); - Scanner scanner = statePair.getFirst(); - - // get the values from the scanner so we can actually use them - ValueGetter valueGetter = IndexManagementUtil.createGetterFromScanner(scanner, dataRowKey); - ptr.set(dataRowKey); - Put put = maintainer.buildUpdateMutation(valueGetter, ptr, state.getCurrentTimestamp()); - indexUpdate.setTable(maintainer.getIndexTableName()); - indexUpdate.setUpdate(put); - //make sure we close the scanner when we are done - scanner.close(); - indexUpdates.add(indexUpdate); - } - return indexUpdates; - } - - @Override - public Iterable getIndexDeletes(TableState state) throws IOException { - List indexMaintainers = getIndexMaintainers(state.getUpdateAttributes()); - if (indexMaintainers.isEmpty()) { - return Collections.emptyList(); - } - List indexUpdates = Lists.newArrayList(); - ImmutableBytesWritable ptr = new ImmutableBytesWritable(); - // TODO: state.getCurrentRowKey() should take an ImmutableBytesWritable arg to prevent byte copy - byte[] dataRowKey = state.getCurrentRowKey(); - for (IndexMaintainer maintainer : indexMaintainers) { - // TODO: if more efficient, I could do this just once with all columns in all indexes - Pair statePair = state.getIndexedColumnsTableState(maintainer.getAllColumns()); - Scanner scanner = statePair.getFirst(); - IndexUpdate indexUpdate = statePair.getSecond(); - indexUpdate.setTable(maintainer.getIndexTableName()); - ValueGetter valueGetter = IndexManagementUtil.createGetterFromScanner(scanner, dataRowKey); - ptr.set(dataRowKey); - Delete delete = - maintainer.buildDeleteMutation(valueGetter, ptr, state.getPendingUpdate(), - state.getCurrentTimestamp()); - scanner.close(); - indexUpdate.setUpdate(delete); - indexUpdates.add(indexUpdate); - } - return indexUpdates; - } - - @Override - public boolean isEnabled(Mutation m) throws IOException { - return !getIndexMaintainers(m.getAttributesMap()).isEmpty(); - } - - @Override - public byte[] getBatchId(Mutation m) { - Map attributes = m.getAttributesMap(); - return attributes.get(INDEX_UUID); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/index/PhoenixIndexFailurePolicy.java b/phoenix-core/src/main/java/com/salesforce/phoenix/index/PhoenixIndexFailurePolicy.java deleted file mode 100644 index e7cac372..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/index/PhoenixIndexFailurePolicy.java +++ /dev/null @@ -1,98 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.index; - -import java.io.IOException; -import java.util.Collections; -import java.util.List; -import java.util.Set; - -import org.apache.commons.logging.Log; -import org.apache.commons.logging.LogFactory; -import org.apache.hadoop.hbase.DoNotRetryIOException; -import org.apache.hadoop.hbase.Stoppable; -import org.apache.hadoop.hbase.client.HTableInterface; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.client.Put; -import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; - -import com.google.common.collect.Multimap; -import com.salesforce.hbase.index.table.HTableInterfaceReference; -import com.salesforce.hbase.index.write.KillServerOnFailurePolicy; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MutationCode; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.schema.PIndexState; -import com.salesforce.phoenix.util.SchemaUtil; - -/** - * - * Handler called in the event that index updates cannot be written to their - * region server. First attempts to disable the index and failing that falls - * back to the default behavior of killing the region server. - * - * TODO: use delegate pattern instead - * - * @author jtaylor - * @since 2.1 - */ -public class PhoenixIndexFailurePolicy extends KillServerOnFailurePolicy { - private static final Log LOG = LogFactory.getLog(PhoenixIndexFailurePolicy.class); - private RegionCoprocessorEnvironment env; - - public PhoenixIndexFailurePolicy() { - } - - @Override - public void setup(Stoppable parent, RegionCoprocessorEnvironment env) { - super.setup(parent, env); - this.env = env; - } - - @Override - public void handleFailure(Multimap attempted, Exception cause) throws IOException { - Set refs = attempted.asMap().keySet(); - StringBuilder buf = new StringBuilder("Disabled index" + (refs.size() > 1 ? "es " : " ")); - try { - for (HTableInterfaceReference ref : refs) { - // Disable the index by using the updateIndexState method of MetaDataProtocol end point coprocessor. - String indexTableName = ref.getTableName(); - byte[] indexTableKey = SchemaUtil.getTableKeyFromFullName(indexTableName); - HTableInterface systemTable = env.getTable(PhoenixDatabaseMetaData.TYPE_TABLE_NAME_BYTES); - MetaDataProtocol mdProxy = systemTable.coprocessorProxy(MetaDataProtocol.class, indexTableKey); - // Mimic the Put that gets generated by the client on an update of the index state - Put put = new Put(indexTableKey); - put.add(PhoenixDatabaseMetaData.TABLE_FAMILY_BYTES, PhoenixDatabaseMetaData.INDEX_STATE_BYTES, PIndexState.DISABLE.getSerializedBytes()); - List tableMetadata = Collections.singletonList(put); - MetaDataMutationResult result = mdProxy.updateIndexState(tableMetadata); - if (result.getMutationCode() != MutationCode.TABLE_ALREADY_EXISTS) { - LOG.warn("Attempt to disable index " + indexTableName + " failed with code = " + result.getMutationCode() + ". Will use default failure policy instead."); - super.handleFailure(attempted, cause); - } - LOG.info("Successfully disabled index " + indexTableName); - buf.append(indexTableName); - buf.append(','); - } - buf.setLength(buf.length()-1); - buf.append(" due to an exception while writing updates"); - } catch (Throwable t) { - super.handleFailure(attempted, cause); - } - throw new DoNotRetryIOException(buf.toString(), cause); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/AggregatingResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/AggregatingResultIterator.java deleted file mode 100644 index 01649b1c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/AggregatingResultIterator.java +++ /dev/null @@ -1,48 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Interface for scanners that either do aggregation - * or delegate to scanners that do aggregation. - * - * @author jtaylor - * @since 0.1 - */ -public interface AggregatingResultIterator extends ResultIterator { - /** - * Provides a means of re-aggregating a result row. For - * scanners that need to look ahead (i.e. {@link com.salesforce.phoenix.iterate.OrderedAggregatingResultIterator} - * @param result the row to re-aggregate - */ - void aggregate(Tuple result); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/BaseResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/BaseResultIterator.java deleted file mode 100644 index b3e71450..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/BaseResultIterator.java +++ /dev/null @@ -1,51 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -/** - * - * Abstract base class for ResultIterator implementations that - * do nothing on close and have no explain plan steps - * - * @author jtaylor - * @since 1.2 - */ -public abstract class BaseResultIterator implements ResultIterator { - - @Override - public void close() throws SQLException { - } - - @Override - public void explain(List planSteps) { - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ConcatResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ConcatResultIterator.java deleted file mode 100644 index c47f5e71..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ConcatResultIterator.java +++ /dev/null @@ -1,99 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Result iterator that concatenates a list of other iterators. - * - * @author jtaylor - * @since 0.1 - */ -public class ConcatResultIterator implements PeekingResultIterator { - private final ResultIterators resultIterators; - private List iterators; - private int index; - - public ConcatResultIterator(ResultIterators iterators) { - this.resultIterators = iterators; - } - - private List getIterators() throws SQLException { - if (iterators == null) { - iterators = resultIterators.getIterators(); - } - return iterators; - } - - @Override - public void close() throws SQLException { - if (iterators != null) { - for (;index < iterators.size(); index++) { - PeekingResultIterator iterator = iterators.get(index); - iterator.close(); - } - } - } - - - @Override - public void explain(List planSteps) { - resultIterators.explain(planSteps); - } - - private PeekingResultIterator currentIterator() throws SQLException { - List iterators = getIterators(); - while (index < iterators.size()) { - PeekingResultIterator iterator = iterators.get(index); - Tuple r = iterator.peek(); - if (r != null) { - return iterator; - } - iterator.close(); - index++; - } - return EMPTY_ITERATOR; - } - - @Override - public Tuple peek() throws SQLException { - return currentIterator().peek(); - } - - @Override - public Tuple next() throws SQLException { - return currentIterator().next(); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/DelegateResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/DelegateResultIterator.java deleted file mode 100644 index 44ad11db..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/DelegateResultIterator.java +++ /dev/null @@ -1,58 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.schema.tuple.Tuple; - - -public class DelegateResultIterator implements ResultIterator { - private final ResultIterator delegate; - - public DelegateResultIterator(ResultIterator delegate) { - this.delegate = delegate; - } - - @Override - public void close() throws SQLException { - delegate.close(); - } - - @Override - public Tuple next() throws SQLException { - return delegate.next(); - } - - @Override - public void explain(List planSteps) { - delegate.explain(planSteps); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/DistinctAggregatingResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/DistinctAggregatingResultIterator.java deleted file mode 100644 index 01db41ea..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/DistinctAggregatingResultIterator.java +++ /dev/null @@ -1,171 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.*; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.google.common.collect.Iterators; -import com.google.common.collect.Sets; -import com.salesforce.phoenix.compile.ColumnProjector; -import com.salesforce.phoenix.compile.RowProjector; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * Result scanner that dedups the incoming tuples to make them distinct. - *

- * Note that the results are held in memory - * - * @author jtaylor - * @since 1.2 - */ -public class DistinctAggregatingResultIterator implements AggregatingResultIterator { - private final AggregatingResultIterator delegate; - private final RowProjector rowProjector; - private Iterator resultIterator; - private final ImmutableBytesWritable ptr1 = new ImmutableBytesWritable(); - private final ImmutableBytesWritable ptr2 = new ImmutableBytesWritable(); - - private class ResultEntry { - private final int hashCode; - private final Tuple result; - - ResultEntry(Tuple result) { - final int prime = 31; - this.result = result; - int hashCode = 0; - for (ColumnProjector column : rowProjector.getColumnProjectors()) { - Expression e = column.getExpression(); - if (e.evaluate(this.result, ptr1)) { - hashCode = prime * hashCode + ptr1.hashCode(); - } - } - this.hashCode = hashCode; - } - - @Override - public boolean equals(Object o) { - if (o == this) { - return true; - } - if (o == null) { - return false; - } - if (o.getClass() != this.getClass()) { - return false; - } - ResultEntry that = (ResultEntry) o; - for (ColumnProjector column : rowProjector.getColumnProjectors()) { - Expression e = column.getExpression(); - boolean isNull1 = !e.evaluate(this.result, ptr1); - boolean isNull2 = !e.evaluate(that.result, ptr2); - if (isNull1 && isNull2) { - return true; - } - if (isNull1 || isNull2) { - return false; - } - if (ptr1.compareTo(ptr2) != 0) { - return false; - } - } - return true; - } - - @Override - public int hashCode() { - return hashCode; - } - - Tuple getResult() { - return result; - } - } - - protected ResultIterator getDelegate() { - return delegate; - } - - public DistinctAggregatingResultIterator(AggregatingResultIterator delegate, - RowProjector rowProjector) { - this.delegate = delegate; - this.rowProjector = rowProjector; - } - - @Override - public Tuple next() throws SQLException { - Iterator iterator = getResultIterator(); - if (iterator.hasNext()) { - ResultEntry entry = iterator.next(); - Tuple tuple = entry.getResult(); - aggregate(tuple); - return tuple; - } - resultIterator = Iterators.emptyIterator(); - return null; - } - - private Iterator getResultIterator() throws SQLException { - if (resultIterator != null) { - return resultIterator; - } - - Set entries = Sets.newHashSet(); // TODO: size? - try { - for (Tuple result = delegate.next(); result != null; result = delegate.next()) { - ResultEntry entry = new ResultEntry(result); - entries.add(entry); - } - } finally { - delegate.close(); - } - - resultIterator = entries.iterator(); - return resultIterator; - } - - @Override - public void close() { - resultIterator = Iterators.emptyIterator(); - } - - - @Override - public void explain(List planSteps) { - delegate.explain(planSteps); - planSteps.add("CLIENT DISTINCT ON " + rowProjector.toString()); - } - - @Override - public void aggregate(Tuple result) { - delegate.aggregate(result); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/FilterAggregatingResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/FilterAggregatingResultIterator.java deleted file mode 100644 index cced489d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/FilterAggregatingResultIterator.java +++ /dev/null @@ -1,88 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Post aggregation filter for HAVING clause. Due to the way we cache aggregation values - * we cannot have a look ahead for this Iterator, because the expressions in the SELECT - * clause would return values for the peeked row instead of the current row. If we only - * use the Result argument in {@link com.salesforce.phoenix.expression.Expression} - * instead of our cached value in Aggregators, we could have a look ahead. - * - * @author jtaylor - * @since 0.1 - */ -public class FilterAggregatingResultIterator implements AggregatingResultIterator { - private final AggregatingResultIterator delegate; - private final Expression expression; - private final ImmutableBytesWritable ptr = new ImmutableBytesWritable(); - - public FilterAggregatingResultIterator(AggregatingResultIterator delegate, Expression expression) { - this.delegate = delegate; - this.expression = expression; - if (expression.getDataType() != PDataType.BOOLEAN) { - throw new IllegalArgumentException("FilterResultIterator requires a boolean expression, but got " + expression); - } - } - - @Override - public Tuple next() throws SQLException { - Tuple next; - do { - next = delegate.next(); - } while (next != null && expression.evaluate(next, ptr) && Boolean.FALSE.equals(expression.getDataType().toObject(ptr))); - return next; - } - - @Override - public void close() throws SQLException { - delegate.close(); - } - - @Override - public void aggregate(Tuple result) { - delegate.aggregate(result); - } - - @Override - public void explain(List planSteps) { - delegate.explain(planSteps); - planSteps.add("CLIENT FILTER BY " + expression.toString()); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/FilterResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/FilterResultIterator.java deleted file mode 100644 index fcc74319..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/FilterResultIterator.java +++ /dev/null @@ -1,86 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Result scanner that filters out rows based on the results of a boolean - * expression (i.e. filters out if {@link com.salesforce.phoenix.expression.Expression#evaluate(Tuple, ImmutableBytesWritable)} - * returns false or the ptr contains a FALSE value}). May not be used where - * the delegate provided is an {@link com.salesforce.phoenix.iterate.AggregatingResultIterator}. - * For these, the {@link com.salesforce.phoenix.iterate.FilterAggregatingResultIterator} should be used. - * - * @author jtaylor - * @since 0.1 - */ -public class FilterResultIterator extends LookAheadResultIterator { - private final ResultIterator delegate; - private final Expression expression; - private final ImmutableBytesWritable ptr = new ImmutableBytesWritable(); - - public FilterResultIterator(ResultIterator delegate, Expression expression) { - if (delegate instanceof AggregatingResultIterator) { - throw new IllegalArgumentException("FilterResultScanner may not be used with an aggregate delegate. Use phoenix.iterate.FilterAggregateResultScanner instead"); - } - this.delegate = delegate; - this.expression = expression; - if (expression.getDataType() != PDataType.BOOLEAN) { - throw new IllegalArgumentException("FilterResultIterator requires a boolean expression, but got " + expression); - } - } - - @Override - protected Tuple advance() throws SQLException { - Tuple next; - do { - next = delegate.next(); - } while (next != null && expression.evaluate(next, ptr) && Boolean.FALSE.equals(expression.getDataType().toObject(ptr))); - return next; - } - - @Override - public void close() throws SQLException { - delegate.close(); - } - - @Override - public void explain(List planSteps) { - delegate.explain(planSteps); - planSteps.add("CLIENT FILTER BY " + expression.toString()); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/GroupedAggregatingResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/GroupedAggregatingResultIterator.java deleted file mode 100644 index a956cb09..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/GroupedAggregatingResultIterator.java +++ /dev/null @@ -1,114 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import static com.salesforce.phoenix.query.QueryConstants.*; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.Aggregators; -import com.salesforce.phoenix.schema.tuple.SingleKeyValueTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.KeyValueUtil; -import com.salesforce.phoenix.util.TupleUtil; - - - -/** - * - * Result scanner that aggregates the row count value for rows with duplicate keys. - * The rows from the backing result iterator must be in key sorted order. For example, - * given the following input: - * a 1 - * a 2 - * b 1 - * b 3 - * c 1 - * the following will be output: - * a 3 - * b 4 - * c 1 - * - * @author jtaylor - * @since 0.1 - */ -public class GroupedAggregatingResultIterator implements AggregatingResultIterator { - private final ImmutableBytesWritable tempPtr = new ImmutableBytesWritable(); - private final PeekingResultIterator resultIterator; - protected final Aggregators aggregators; - - public GroupedAggregatingResultIterator( PeekingResultIterator resultIterator, Aggregators aggregators) { - if (resultIterator == null) throw new NullPointerException(); - if (aggregators == null) throw new NullPointerException(); - this.resultIterator = resultIterator; - this.aggregators = aggregators; - } - - @Override - public Tuple next() throws SQLException { - Tuple result = resultIterator.next(); - if (result == null) { - return null; - } - Aggregator[] rowAggregators = aggregators.getAggregators(); - aggregators.reset(rowAggregators); - while (true) { - aggregators.aggregate(rowAggregators, result); - Tuple nextResult = resultIterator.peek(); - if (nextResult == null || !TupleUtil.equals(result, nextResult, tempPtr)) { - break; - } - result = resultIterator.next(); - } - - byte[] value = aggregators.toBytes(rowAggregators); - result.getKey(tempPtr); - return new SingleKeyValueTuple(KeyValueUtil.newKeyValue(tempPtr, SINGLE_COLUMN_FAMILY, SINGLE_COLUMN, AGG_TIMESTAMP, value, 0, value.length)); - } - - @Override - public void close() throws SQLException { - resultIterator.close(); - } - - @Override - public void aggregate(Tuple result) { - Aggregator[] rowAggregators = aggregators.getAggregators(); - aggregators.reset(rowAggregators); - aggregators.aggregate(rowAggregators, result); - } - - @Override - public void explain(List planSteps) { - resultIterator.explain(planSteps); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/LimitingResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/LimitingResultIterator.java deleted file mode 100644 index 7a76d427..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/LimitingResultIterator.java +++ /dev/null @@ -1,65 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * - * Iterates through tuples up to a limit - * - * @author jtaylor - * @since 1.2 - */ -public class LimitingResultIterator extends DelegateResultIterator { - private int rowCount; - private final int limit; - - public LimitingResultIterator(ResultIterator delegate, int limit) { - super(delegate); - this.limit = limit; - } - - @Override - public Tuple next() throws SQLException { - if (rowCount++ >= limit) { - return null; - } - return super.next(); - } - - @Override - public void explain(List planSteps) { - super.explain(planSteps); - planSteps.add("CLIENT " + limit + " ROW LIMIT"); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/LookAheadResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/LookAheadResultIterator.java deleted file mode 100644 index 68e3c3f8..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/LookAheadResultIterator.java +++ /dev/null @@ -1,61 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; - -import com.salesforce.phoenix.schema.tuple.ResultTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -abstract public class LookAheadResultIterator implements PeekingResultIterator { - private final static Tuple UNINITIALIZED = new ResultTuple(); - private Tuple next = UNINITIALIZED; - - abstract protected Tuple advance() throws SQLException; - - private void init() throws SQLException { - if (next == UNINITIALIZED) { - next = advance(); - } - } - - @Override - public Tuple next() throws SQLException { - init(); - Tuple next = this.next; - this.next = advance(); - return next; - } - - @Override - public Tuple peek() throws SQLException { - init(); - return next; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MaterializedResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MaterializedResultIterator.java deleted file mode 100644 index 7cd9438e..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MaterializedResultIterator.java +++ /dev/null @@ -1,120 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.*; - -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Fully materialized result iterator backed by the result list provided. - * No copy is made of the backing results collection. - * - * @author jtaylor - * @since 0.1 - */ -public class MaterializedResultIterator implements PeekingResultIterator { - private final PeekingCollectionIterator iterator; - - public MaterializedResultIterator(Collection results) { - iterator = new PeekingCollectionIterator(results); - } - - @Override - public void close() { - } - - @Override - public Tuple next() throws SQLException { - return iterator.nextOrNull(); - } - - @Override - public Tuple peek() throws SQLException { - return iterator.peek(); - } - - private static class PeekingCollectionIterator implements Iterator { - private final Iterator iterator; - private Tuple current; - - private PeekingCollectionIterator(Collection results) { - iterator = results.iterator(); - advance(); - } - - private Tuple advance() { - if (iterator.hasNext()) { - current = iterator.next(); - } else { - current = null; - } - return current; - } - - @Override - public boolean hasNext() { - return current != null; - } - - @Override - public Tuple next() { - Tuple next = nextOrNull(); - if (next == null) { - throw new NoSuchElementException(); - } - return next; - } - - public Tuple nextOrNull() { - if (current == null) { - return null; - } - Tuple next = current; - advance(); - return next; - } - - @Override - public void remove() { - throw new UnsupportedOperationException(); - } - - public Tuple peek() { - return current; - } - - } - - @Override - public void explain(List planSteps) { - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MergeSortResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MergeSortResultIterator.java deleted file mode 100644 index 94014643..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MergeSortResultIterator.java +++ /dev/null @@ -1,103 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.SQLCloseables; - - -/** - * - * Base class for a ResultIterator that does a merge sort on the list of iterators - * provided. - * - * @author jtaylor - * @since 1.2 - */ -public abstract class MergeSortResultIterator implements PeekingResultIterator { - protected final ResultIterators resultIterators; - protected final ImmutableBytesWritable tempPtr = new ImmutableBytesWritable(); - private List iterators; - - public MergeSortResultIterator(ResultIterators iterators) { - this.resultIterators = iterators; - } - - private List getIterators() throws SQLException { - if (iterators == null) { - iterators = resultIterators.getIterators(); - } - return iterators; - } - - @Override - public void close() throws SQLException { - if (iterators != null) { - SQLCloseables.closeAll(iterators); - } - } - - abstract protected int compare(Tuple t1, Tuple t2); - - private PeekingResultIterator minIterator() throws SQLException { - List iterators = getIterators(); - Tuple minResult = null; - PeekingResultIterator minIterator = EMPTY_ITERATOR; - for (int i = iterators.size()-1; i >= 0; i--) { - PeekingResultIterator iterator = iterators.get(i); - Tuple r = iterator.peek(); - if (r != null) { - if (minResult == null || compare(r, minResult) < 0) { - minResult = r; - minIterator = iterator; - } - continue; - } - iterator.close(); - iterators.remove(i); - } - return minIterator; - } - - @Override - public Tuple peek() throws SQLException { - PeekingResultIterator iterator = minIterator(); - return iterator.peek(); - } - - @Override - public Tuple next() throws SQLException { - PeekingResultIterator iterator = minIterator(); - return iterator.next(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MergeSortRowKeyResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MergeSortRowKeyResultIterator.java deleted file mode 100644 index 93c9759f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MergeSortRowKeyResultIterator.java +++ /dev/null @@ -1,67 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.util.List; - -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.TupleUtil; - - -/** - * - * ResultIterator that does a merge sort on the list of iterators provided, - * returning the rows in row key ascending order. The iterators provided - * must be in row key ascending order. - * - * @author jtaylor - * @since 0.1 - */ -public class MergeSortRowKeyResultIterator extends MergeSortResultIterator { - private final int keyOffset; - - public MergeSortRowKeyResultIterator(ResultIterators iterators) { - this(iterators, 0); - } - - public MergeSortRowKeyResultIterator(ResultIterators iterators, int keyOffset) { - super(iterators); - this.keyOffset = keyOffset; - } - - @Override - protected int compare(Tuple t1, Tuple t2) { - return TupleUtil.compare(t1, t2, tempPtr, keyOffset); - } - - @Override - public void explain(List planSteps) { - resultIterators.explain(planSteps); - planSteps.add("CLIENT MERGE SORT"); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MergeSortTopNResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MergeSortTopNResultIterator.java deleted file mode 100644 index ffbe982b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MergeSortTopNResultIterator.java +++ /dev/null @@ -1,108 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.OrderByExpression; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * - * ResultIterator that does a merge sort on the list of iterators provided, - * returning the rows ordered by the OrderByExpression. The input - * iterators must be ordered by the OrderByExpression. - * - * @author jtaylor - * @since 0.1 - */ -public class MergeSortTopNResultIterator extends MergeSortResultIterator { - - private final int limit; - private int count = 0; - private final List orderByColumns; - private final ImmutableBytesWritable ptr1 = new ImmutableBytesWritable(); - private final ImmutableBytesWritable ptr2 = new ImmutableBytesWritable(); - - public MergeSortTopNResultIterator(ResultIterators iterators, Integer limit, List orderByColumns) { - super(iterators); - this.limit = limit == null ? -1 : limit; - this.orderByColumns = orderByColumns; - } - - @Override - protected int compare(Tuple t1, Tuple t2) { - for (int i = 0; i < orderByColumns.size(); i++) { - OrderByExpression order = orderByColumns.get(i); - Expression orderExpr = order.getExpression(); - boolean isNull1 = !orderExpr.evaluate(t1, ptr1) || ptr1.getLength() == 0; - boolean isNull2 = !orderExpr.evaluate(t2, ptr2) || ptr2.getLength() == 0; - if (isNull1 && isNull2) { - continue; - } else if (isNull1) { - return order.isNullsLast() ? 1 : -1; - } else if (isNull2) { - return order.isNullsLast() ? -1 : 1; - } - int cmp = ptr1.compareTo(ptr2); - if (cmp == 0) { - continue; - } - return order.isAscending() ? cmp : -cmp; - } - return 0; - } - - @Override - public Tuple peek() throws SQLException { - if (limit >= 0 && count >= limit) { - return null; - } - return super.peek(); - } - - @Override - public Tuple next() throws SQLException { - if (limit >= 0 && count++ >= limit) { - return null; - } - return super.next(); - } - - - @Override - public void explain(List planSteps) { - resultIterators.explain(planSteps); - planSteps.add(" SERVER TOP " + limit + " ROW" + (limit == 1 ? "" : "S") + " SORTED BY " + orderByColumns.toString()); - planSteps.add("CLIENT MERGE SORT"); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/OrderedAggregatingResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/OrderedAggregatingResultIterator.java deleted file mode 100644 index 3c2c57d7..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/OrderedAggregatingResultIterator.java +++ /dev/null @@ -1,71 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.expression.OrderByExpression; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * Result scanner that sorts aggregated rows by columns specified in the ORDER BY clause. - *

- * Note that currently the sort is entirely done in memory. - * - * @author syyang - * @since 0.1 - */ -public class OrderedAggregatingResultIterator extends OrderedResultIterator implements AggregatingResultIterator { - - public OrderedAggregatingResultIterator(AggregatingResultIterator delegate, - List orderByExpressions, - int thresholdBytes, Integer limit) throws SQLException { - super (delegate, orderByExpressions, thresholdBytes, limit); - } - - @Override - protected AggregatingResultIterator getDelegate() { - return (AggregatingResultIterator)super.getDelegate(); - } - - @Override - public Tuple next() throws SQLException { - Tuple tuple = super.next(); - if (tuple != null) { - aggregate(tuple); - } - return tuple; - } - - @Override - public void aggregate(Tuple result) { - getDelegate().aggregate(result); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ParallelIteratorRegionSplitter.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ParallelIteratorRegionSplitter.java deleted file mode 100644 index 8b2f0b73..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ParallelIteratorRegionSplitter.java +++ /dev/null @@ -1,44 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.query.KeyRange; - - -/** - * Interface for strategies determining how to split regions in ParallelIterators. - * - * @author zhuang - */ -public interface ParallelIteratorRegionSplitter { - - public List getSplits() throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ParallelIteratorRegionSplitterFactory.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ParallelIteratorRegionSplitterFactory.java deleted file mode 100644 index 9992bfc6..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ParallelIteratorRegionSplitterFactory.java +++ /dev/null @@ -1,48 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.parse.HintNode; -import com.salesforce.phoenix.schema.TableRef; - - -/** - * Factory class for the Region Splitter used by the project. - */ -public class ParallelIteratorRegionSplitterFactory { - - public static ParallelIteratorRegionSplitter getSplitter(StatementContext context, TableRef table, HintNode hintNode) throws SQLException { - if (context.getScanRanges().useSkipScanFilter()) { - return SkipRangeParallelIteratorRegionSplitter.getInstance(context, table, hintNode); - } - return DefaultParallelIteratorRegionSplitter.getInstance(context, table, hintNode); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/PeekingResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/PeekingResultIterator.java deleted file mode 100644 index 663bb52e..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/PeekingResultIterator.java +++ /dev/null @@ -1,71 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Interface for iterating through results returned from a scan, adding the - * ability to peek at the next result. - * - * @author jtaylor - * @since 0.1 - */ -public interface PeekingResultIterator extends ResultIterator { - public static final PeekingResultIterator EMPTY_ITERATOR = new PeekingResultIterator() { - - @Override - public Tuple next() throws SQLException { - return null; - } - - @Override - public Tuple peek() { - return null; - } - - @Override - public void close() throws SQLException { - } - - @Override - public void explain(List planSteps) { - } - }; - - /** - * Returns the next result without advancing the iterator - * @throws SQLException - */ - public Tuple peek() throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/RegionScannerResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/RegionScannerResultIterator.java deleted file mode 100644 index 0d0e3f09..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/RegionScannerResultIterator.java +++ /dev/null @@ -1,73 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.io.IOException; -import java.sql.SQLException; -import java.util.ArrayList; -import java.util.List; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl; -import org.apache.hadoop.hbase.regionserver.RegionScanner; - -import com.salesforce.phoenix.schema.tuple.MultiKeyValueTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ServerUtil; - - -public class RegionScannerResultIterator extends BaseResultIterator { - private final RegionScanner scanner; - - public RegionScannerResultIterator(RegionScanner scanner) { - this.scanner = scanner; - MultiVersionConsistencyControl.setThreadReadPoint(scanner.getMvccReadPoint()); - } - - @Override - public Tuple next() throws SQLException { - try { - // TODO: size - List results = new ArrayList(); - // Results are potentially returned even when the return value of s.next is false - // since this is an indication of whether or not there are more values after the - // ones returned - boolean hasMore = scanner.nextRaw(results, null); - if (!hasMore && results.isEmpty()) { - return null; - } - // We instantiate a new tuple because in all cases currently we hang on to it (i.e. - // to compute and hold onto the TopN). - MultiKeyValueTuple tuple = new MultiKeyValueTuple(); - tuple.setKeyValues(results); - return tuple; - } catch (IOException e) { - throw ServerUtil.parseServerException(e); - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ResultIterator.java deleted file mode 100644 index edf7a05b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ResultIterator.java +++ /dev/null @@ -1,62 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.SQLCloseable; - - -public interface ResultIterator extends SQLCloseable { - public static final ResultIterator EMPTY_ITERATOR = new ResultIterator() { - @Override - public void close() throws SQLException { - } - - @Override - public Tuple next() throws SQLException { - return null; - } - - @Override - public void explain(List planSteps) { - } - }; - - /** - * Grab the next row's worth of values. The iterator will return a Tuple. - * @return Tuple object if there is another row, null if the scanner is - * exhausted. - * @throws SQLException e - */ - public Tuple next() throws SQLException; - - public void explain(List planSteps); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ResultIterators.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ResultIterators.java deleted file mode 100644 index 72f39b84..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ResultIterators.java +++ /dev/null @@ -1,37 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -public interface ResultIterators { - public List getIterators() throws SQLException; - public int size(); - public void explain(List planSteps); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ScanningResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ScanningResultIterator.java deleted file mode 100644 index 318bfeab..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ScanningResultIterator.java +++ /dev/null @@ -1,69 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.io.IOException; -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.client.Result; -import org.apache.hadoop.hbase.client.ResultScanner; - -import com.salesforce.phoenix.schema.tuple.ResultTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ServerUtil; - - -public class ScanningResultIterator implements ResultIterator { - private final ResultScanner scanner; - - public ScanningResultIterator(ResultScanner scanner) { - this.scanner = scanner; - } - - @Override - public void close() throws SQLException { - scanner.close(); - } - - @Override - public Tuple next() throws SQLException { - try { - Result result = scanner.next(); - // TODO: use ResultTuple.setResult(result) - // Need to create a new one if holding on to it (i.e. OrderedResultIterator) - return result == null ? null : new ResultTuple(result); - } catch (IOException e) { - throw ServerUtil.parseServerException(e); - } - } - - @Override - public void explain(List planSteps) { - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SequenceResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SequenceResultIterator.java deleted file mode 100644 index 122428c8..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SequenceResultIterator.java +++ /dev/null @@ -1,67 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.SequenceManager; -import com.salesforce.phoenix.schema.tuple.Tuple; - -/** - * - * Iterates through tuples retrieving sequences from the server as needed - * - * @author jtaylor - * @since 3.0 - */ -public class SequenceResultIterator extends DelegateResultIterator { - private final SequenceManager sequenceManager; - - public SequenceResultIterator(ResultIterator delegate, SequenceManager sequenceManager) throws SQLException { - super(delegate); - sequenceManager.initSequences(); - this.sequenceManager = sequenceManager; - } - - @Override - public Tuple next() throws SQLException { - Tuple next = super.next(); - if (next == null) { - return null; - } - sequenceManager.incrementSequenceValues(); - return next; - } - - @Override - public void explain(List planSteps) { - super.explain(planSteps); - planSteps.add("CLIENT RESERVE " + sequenceManager.getSequenceCount() + " SEQUENCES"); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SkipRangeParallelIteratorRegionSplitter.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SkipRangeParallelIteratorRegionSplitter.java deleted file mode 100644 index 439a49fe..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SkipRangeParallelIteratorRegionSplitter.java +++ /dev/null @@ -1,93 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.HRegionLocation; - -import com.google.common.base.Predicate; -import com.google.common.collect.Iterables; -import com.google.common.collect.Lists; -import com.salesforce.phoenix.compile.ScanRanges; -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.parse.HintNode; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.schema.SaltingUtil; -import com.salesforce.phoenix.schema.TableRef; - - -/** - * Split the region according to the information contained in the scan's SkipScanFilter. - */ -public class SkipRangeParallelIteratorRegionSplitter extends DefaultParallelIteratorRegionSplitter { - - public static SkipRangeParallelIteratorRegionSplitter getInstance(StatementContext context, TableRef table, HintNode hintNode) { - return new SkipRangeParallelIteratorRegionSplitter(context, table, hintNode); - } - - protected SkipRangeParallelIteratorRegionSplitter(StatementContext context, TableRef table, HintNode hintNode) { - super(context, table, hintNode); - } - - @Override - protected List getAllRegions() throws SQLException { - List allTableRegions = context.getConnection().getQueryServices().getAllTableRegions(tableRef.getTable().getPhysicalName().getBytes()); - return filterRegions(allTableRegions, context.getScanRanges()); - } - - public List filterRegions(List allTableRegions, final ScanRanges ranges) { - Iterable regions; - if (ranges == ScanRanges.EVERYTHING) { - return allTableRegions; - } else if (ranges == ScanRanges.NOTHING) { // TODO: why not emptyList? - return Lists.newArrayList(); - } else { - regions = Iterables.filter(allTableRegions, - new Predicate() { - @Override - public boolean apply(HRegionLocation region) { - KeyRange minMaxRange = context.getMinMaxRange(); - if (minMaxRange != null) { - KeyRange range = KeyRange.getKeyRange(region.getRegionInfo().getStartKey(), region.getRegionInfo().getEndKey()); - if (tableRef.getTable().getBucketNum() != null) { - // Add salt byte, as minMaxRange won't have it - minMaxRange = SaltingUtil.addSaltByte(region.getRegionInfo().getStartKey(), minMaxRange); - } - range = range.intersect(minMaxRange); - return ranges.intersect(range.getLowerRange(), range.getUpperRange()); - } - return ranges.intersect(region.getRegionInfo().getStartKey(), region.getRegionInfo().getEndKey()); - } - }); - } - return Lists.newArrayList(regions); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SpoolTooBigToDiskException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SpoolTooBigToDiskException.java deleted file mode 100644 index 74e35e6f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SpoolTooBigToDiskException.java +++ /dev/null @@ -1,17 +0,0 @@ -package com.salesforce.phoenix.iterate; - -/** - * Thrown by {@link com.salesforce.phoenix.iterate.SpoolingResultIterator } when - * result is too big to fit into memory and too big to spool to disk. - * - * @author haitaoyao - * - */ -public class SpoolTooBigToDiskException extends RuntimeException { - - private static final long serialVersionUID = 1L; - - public SpoolTooBigToDiskException(String msg) { - super(msg); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/TableResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/TableResultIterator.java deleted file mode 100644 index 1e2f1cf0..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/TableResultIterator.java +++ /dev/null @@ -1,94 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import java.io.IOException; -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.client.HTableInterface; -import org.apache.hadoop.hbase.client.Scan; - -import com.google.common.io.Closeables; -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ServerUtil; - - -/** - * - * Wrapper for ResultScanner creation that closes HTableInterface - * when ResultScanner is closed. - * - * @author jtaylor - * @since 0.1 - */ -public class TableResultIterator extends ExplainTable implements ResultIterator { - private final HTableInterface htable; - private final ResultIterator delegate; - - public TableResultIterator(StatementContext context, TableRef tableRef) throws SQLException { - this(context, tableRef, context.getScan()); - } - - public TableResultIterator(StatementContext context, TableRef tableRef, Scan scan) throws SQLException { - super(context, tableRef); - htable = context.getConnection().getQueryServices().getTable(tableRef.getTable().getPhysicalName().getBytes()); - try { - delegate = new ScanningResultIterator(htable.getScanner(scan)); - } catch (IOException e) { - Closeables.closeQuietly(htable); - throw ServerUtil.parseServerException(e); - } - } - - @Override - public void close() throws SQLException { - try { - delegate.close(); - } finally { - try { - htable.close(); - } catch (IOException e) { - throw ServerUtil.parseServerException(e); - } - } - } - - @Override - public Tuple next() throws SQLException { - return delegate.next(); - } - - @Override - public void explain(List planSteps) { - StringBuilder buf = new StringBuilder(); - explain(buf.toString(),planSteps); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/UngroupedAggregatingResultIterator.java b/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/UngroupedAggregatingResultIterator.java deleted file mode 100644 index e6472c2d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/UngroupedAggregatingResultIterator.java +++ /dev/null @@ -1,64 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; - -import static com.salesforce.phoenix.query.QueryConstants.*; - -import java.sql.SQLException; - -import com.salesforce.phoenix.expression.aggregator.Aggregators; -import com.salesforce.phoenix.schema.tuple.SingleKeyValueTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.KeyValueUtil; - - -public class UngroupedAggregatingResultIterator extends GroupedAggregatingResultIterator { - private boolean hasRows = false; - - public UngroupedAggregatingResultIterator( PeekingResultIterator resultIterator, Aggregators aggregators) { - super(resultIterator, aggregators); - } - - @Override - public Tuple next() throws SQLException { - Tuple result = super.next(); - // Ensure ungrouped aggregregation always returns a row, even if the underlying iterator doesn't. - if (result == null && !hasRows) { - // Generate value using unused ClientAggregators - byte[] value = aggregators.toBytes(aggregators.getAggregators()); - result = new SingleKeyValueTuple( - KeyValueUtil.newKeyValue(UNGROUPED_AGG_ROW_KEY, - SINGLE_COLUMN_FAMILY, - SINGLE_COLUMN, - AGG_TIMESTAMP, - value)); - } - hasRows = true; - return result; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/Jdbc7Shim.java b/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/Jdbc7Shim.java deleted file mode 100644 index 231e4671..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/Jdbc7Shim.java +++ /dev/null @@ -1,74 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.jdbc; - -import java.sql.SQLException; -import java.sql.SQLFeatureNotSupportedException; -import java.util.concurrent.Executor; -import java.util.logging.Logger; - -/** - * Interfaces to be implemented by classes that need to be "JDK7" compliant, - * but also run in JDK6 - */ -public final class Jdbc7Shim { - - public interface Statement { // Note: do not extend "regular" statement or else eclipse 3.7 complains - void closeOnCompletion() throws SQLException; - boolean isCloseOnCompletion() throws SQLException; - } - - public interface CallableStatement extends Statement { - public T getObject(int columnIndex, Class type) throws SQLException; - public T getObject(String columnLabel, Class type) throws SQLException; - } - - public interface Connection { - void setSchema(String schema) throws SQLException; - String getSchema() throws SQLException; - void abort(Executor executor) throws SQLException; - void setNetworkTimeout(Executor executor, int milliseconds) throws SQLException; - int getNetworkTimeout() throws SQLException; - } - - public interface ResultSet { - public T getObject(int columnIndex, Class type) throws SQLException; - public T getObject(String columnLabel, Class type) throws SQLException; - } - - public interface DatabaseMetaData { - java.sql.ResultSet getPseudoColumns(String catalog, String schemaPattern, - String tableNamePattern, String columnNamePattern) - throws SQLException; - boolean generatedKeyAlwaysReturned() throws SQLException; - } - - public interface Driver { - public Logger getParentLogger() throws SQLFeatureNotSupportedException; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixDriver.java b/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixDriver.java deleted file mode 100644 index 934d8820..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixDriver.java +++ /dev/null @@ -1,104 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.jdbc; - -import java.sql.DriverManager; -import java.sql.SQLException; -import java.util.Properties; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ConcurrentMap; - -import com.salesforce.phoenix.query.*; -import com.salesforce.phoenix.util.SQLCloseables; - - -/** - * - * JDBC Driver implementation of Phoenix for production. - * To use this driver, specify the following URL: - * jdbc:phoenix:; - * Only an embedded driver is currently supported (Phoenix client - * runs in the same JVM as the driver). Connections are lightweight - * and are not pooled. The last part of the URL, the hbase zookeeper - * quorum server name, determines the hbase cluster to which queries - * will be routed. - * - * @author jtaylor - * @since 0.1 - */ -public final class PhoenixDriver extends PhoenixEmbeddedDriver { - public static final PhoenixDriver INSTANCE; - static { - try { - DriverManager.registerDriver( INSTANCE = new PhoenixDriver() ); - } catch (SQLException e) { - throw new IllegalStateException("Untable to register " + PhoenixDriver.class.getName() + ": "+ e.getMessage()); - } - } - private final ConcurrentMap connectionQueryServicesMap = new ConcurrentHashMap(3); - - public PhoenixDriver() { // for Squirrel - // Use production services implementation - super(new QueryServicesImpl()); - } - - @Override - public boolean acceptsURL(String url) throws SQLException { - // Accept the url only if test=true attribute not set - return super.acceptsURL(url) && !isTestUrl(url); - } - - @Override - protected ConnectionQueryServices getConnectionQueryServices(String url, Properties info) throws SQLException { - ConnectionInfo connInfo = ConnectionInfo.create(url); - ConnectionInfo normalizedConnInfo = connInfo.normalize(getQueryServices().getProps()); - ConnectionQueryServices connectionQueryServices = connectionQueryServicesMap.get(normalizedConnInfo); - if (connectionQueryServices == null) { - if (normalizedConnInfo.isConnectionless()) { - connectionQueryServices = new ConnectionlessQueryServicesImpl(getQueryServices()); - } else { - connectionQueryServices = new ConnectionQueryServicesImpl(getQueryServices(), normalizedConnInfo); - } - connectionQueryServices.init(url, info); - ConnectionQueryServices prevValue = connectionQueryServicesMap.putIfAbsent(normalizedConnInfo, connectionQueryServices); - if (prevValue != null) { - connectionQueryServices = prevValue; - } - } - return connectionQueryServices; - } - - @Override - public void close() throws SQLException { - try { - SQLCloseables.closeAll(connectionQueryServicesMap.values()); - } finally { - connectionQueryServicesMap.clear(); - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixParameterMetaData.java b/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixParameterMetaData.java deleted file mode 100644 index 57a00cff..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixParameterMetaData.java +++ /dev/null @@ -1,180 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.jdbc; - -import java.sql.ParameterMetaData; -import java.sql.ResultSetMetaData; -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.parse.BindParseNode; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDatum; -import com.salesforce.phoenix.schema.TypeMismatchException; - - - -/** - * - * Implementation of ParameterMetaData for Phoenix - * - * @author jtaylor - * @since 0.1 - */ -public class PhoenixParameterMetaData implements ParameterMetaData { - private final PDatum[] params; - private static final PDatum EMPTY_DATUM = new PDatum() { - @Override - public boolean isNullable() { - return false; - } - - @Override - public Integer getScale() { - return null; - } - - @Override - public Integer getMaxLength() { - return null; - } - - @Override - public PDataType getDataType() { - return null; - } - - @Override - public ColumnModifier getColumnModifier() { - return null; - } - - @Override - public Integer getByteSize() { - return null; - } - }; - public static final PhoenixParameterMetaData EMPTY_PARAMETER_META_DATA = new PhoenixParameterMetaData(0); - public PhoenixParameterMetaData(int paramCount) { - params = new PDatum[paramCount]; - //initialize the params array with the empty_datum marker value. - for(int i = 0; i < paramCount; i++) { - params[i] = EMPTY_DATUM; - } - } - - private PDatum getParam(int index) throws SQLException { - if (index <= 0 || index > params.length) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.PARAM_INDEX_OUT_OF_BOUND) - .setMessage("The index is " + index + ". Must be between 1 and " + params.length) - .build().buildException(); - } - PDatum param = params[index-1]; - - if (param == EMPTY_DATUM) { - //value at params[index-1] was never set. - throw new SQLExceptionInfo.Builder(SQLExceptionCode.PARAM_VALUE_UNBOUND) - .setMessage("Parameter at index " + index + " is unbound").build().buildException(); - } - return param; - } - @Override - public String getParameterClassName(int index) throws SQLException { - PDatum datum = getParam(index); - PDataType type = datum == null ? null : datum.getDataType(); - return type == null ? null : type.getJavaClassName(); - } - - @Override - public int getParameterCount() throws SQLException { - return params.length; - } - - @Override - public int getParameterMode(int index) throws SQLException { - return ParameterMetaData.parameterModeIn; - } - - @Override - public int getParameterType(int index) throws SQLException { - return getParam(index).getDataType().getSqlType(); - } - - @Override - public String getParameterTypeName(int index) throws SQLException { - return getParam(index).getDataType().getSqlTypeName(); - } - - @Override - public int getPrecision(int index) throws SQLException { - return 0; - } - - @Override - public int getScale(int index) throws SQLException { - return 0; - } - - @Override - public int isNullable(int index) throws SQLException { - return getParam(index).isNullable() ? ResultSetMetaData.columnNullable : ResultSetMetaData.columnNoNulls; - } - - @Override - public boolean isSigned(int index) throws SQLException { - @SuppressWarnings("rawtypes") - Class clazz = getParam(index).getDataType().getJavaClass(); - return Number.class.isInstance(clazz); - } - - @Override - public boolean isWrapperFor(Class iface) throws SQLException { - return iface.isInstance(this); - } - - @SuppressWarnings("unchecked") - @Override - public T unwrap(Class iface) throws SQLException { - if (!iface.isInstance(this)) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.CLASS_NOT_UNWRAPPABLE) - .setMessage(this.getClass().getName() + " not unwrappable from " + iface.getName()) - .build().buildException(); - } - return (T)this; - } - - public void addParam(BindParseNode bind, PDatum datum) throws SQLException { - PDatum bindDatum = params[bind.getIndex()]; - if (bindDatum != null && bindDatum.getDataType() != null && !datum.getDataType().isCoercibleTo(bindDatum.getDataType())) { - throw TypeMismatchException.newException(datum.getDataType(), bindDatum.getDataType()); - } - params[bind.getIndex()] = datum; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixStatementFactory.java b/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixStatementFactory.java deleted file mode 100644 index 3a0166a5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixStatementFactory.java +++ /dev/null @@ -1,32 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.jdbc; - -public interface PhoenixStatementFactory { - public PhoenixStatement newStatement(PhoenixConnection connection); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/job/JobManager.java b/phoenix-core/src/main/java/com/salesforce/phoenix/job/JobManager.java deleted file mode 100644 index 8c1fc3eb..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/job/JobManager.java +++ /dev/null @@ -1,123 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.job; - -import java.util.concurrent.*; -import java.util.concurrent.atomic.AtomicLong; - -import com.google.common.util.concurrent.ThreadFactoryBuilder; - -/** - * - * Thread pool executor that executes scans in parallel - * - * @author jtaylor - * @since 0.1 - */ -@SuppressWarnings("rawtypes") -public class JobManager extends AbstractRoundRobinQueue { - - private static final AtomicLong PHOENIX_POOL_INDEX = new AtomicLong(1); - - public JobManager(int maxSize) { - super(maxSize, true); // true -> new producers move to front of queue; this reduces latency. - } - - @Override - protected Object extractProducer(T o) { - return ((JobFutureTask)o).getJobId(); - } - - public static interface JobRunnable extends Runnable { - public Object getJobId(); - } - - public static ThreadPoolExecutor createThreadPoolExec(int keepAliveMs, int size, int queueSize) { - BlockingQueue queue; - if (queueSize == 0) { - queue = new SynchronousQueue(); // Specialized for 0 length. - } else { - queue = new JobManager(queueSize); - } - ThreadFactory threadFactory = new ThreadFactoryBuilder().setNameFormat( - "phoenix-" + PHOENIX_POOL_INDEX.getAndIncrement() - + "-thread-%s").build(); - // For thread pool, set core threads = max threads -- we don't ever want to exceed core threads, but want to go up to core threads *before* using the queue. - ThreadPoolExecutor exec = new ThreadPoolExecutor(size, size, keepAliveMs, TimeUnit.MILLISECONDS, queue, threadFactory) { - @Override - protected RunnableFuture newTaskFor(Callable call) { - // Override this so we can create a JobFutureTask so we can extract out the parentJobId (otherwise, in the default FutureTask, it is private). - return new JobFutureTask(call); - } - - @Override - protected RunnableFuture newTaskFor(Runnable runnable, T value) { - return new JobFutureTask((JobRunnable)runnable, value); - } - - }; - - exec.allowCoreThreadTimeOut(true); // ... and allow core threads to time out. This just keeps things clean when idle, and is nice for ftests modes, etc., where we'd especially like these not to linger. - return exec; - } - - /** - * Subclasses FutureTask for the sole purpose of providing {@link #getCallable()}, which is used to extract the producer in the {@link JobBasedRoundRobinQueue} - */ - static class JobFutureTask extends FutureTask { - private final Object jobId; - - public JobFutureTask(JobRunnable r, T t) { - super(r, t); - this.jobId = r.getJobId(); - } - - public JobFutureTask(Callable c) { - super(c); - // FIXME: this fails when executor used by hbase - if (c instanceof JobCallable) { - this.jobId = ((JobCallable) c).getJobId(); - } else { - this.jobId = this; - } - } - - public Object getJobId() { - return jobId; - } - } - - - /** - * Delegating callable implementation that preserves the parentJobId and sets up thread tracker stuff before delegating to the actual command. - */ - public static interface JobCallable extends Callable { - public Object getJobId(); - } -} - diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/join/HashCacheClient.java b/phoenix-core/src/main/java/com/salesforce/phoenix/join/HashCacheClient.java deleted file mode 100644 index c26c3879..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/join/HashCacheClient.java +++ /dev/null @@ -1,141 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.join; - -import java.io.DataOutputStream; -import java.io.IOException; -import java.sql.SQLException; -import java.util.List; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; -import org.apache.hadoop.io.WritableUtils; -import org.xerial.snappy.Snappy; - -import com.salesforce.phoenix.cache.ServerCacheClient; -import com.salesforce.phoenix.cache.ServerCacheClient.ServerCache; -import com.salesforce.phoenix.compile.ScanRanges; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.ExpressionType; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ServerUtil; -import com.salesforce.phoenix.util.TrustedByteArrayOutputStream; -import com.salesforce.phoenix.util.TupleUtil; - -/** - * - * Client for adding cache of one side of a join to region servers - * - * @author jtaylor - * @since 0.1 - */ -public class HashCacheClient { - private final ServerCacheClient serverCache; - /** - * Construct client used to create a serialized cached snapshot of a table and send it to each region server - * for caching during hash join processing. - * @param connection the client connection - */ - public HashCacheClient(PhoenixConnection connection) { - serverCache = new ServerCacheClient(connection); - } - - /** - * Send the results of scanning through the scanner to all - * region servers for regions of the table that will use the cache - * that intersect with the minMaxKeyRange. - * @param scanner scanner for the table or intermediate results being cached - * @return client-side {@link ServerCache} representing the added hash cache - * @throws SQLException - * @throws MaxServerCacheSizeExceededException if size of hash cache exceeds max allowed - * size - */ - public ServerCache addHashCache(ScanRanges keyRanges, ResultIterator iterator, long estimatedSize, List onExpressions, TableRef cacheUsingTableRef) throws SQLException { - /** - * Serialize and compress hashCacheTable - */ - ImmutableBytesWritable ptr = new ImmutableBytesWritable(); - serialize(ptr, iterator, estimatedSize, onExpressions); - return serverCache.addServerCache(keyRanges, ptr, new HashCacheFactory(), cacheUsingTableRef); - } - - private void serialize(ImmutableBytesWritable ptr, ResultIterator iterator, long estimatedSize, List onExpressions) throws SQLException { - long maxSize = serverCache.getConnection().getQueryServices().getProps().getLong(QueryServices.MAX_SERVER_CACHE_SIZE_ATTRIB, QueryServicesOptions.DEFAULT_MAX_SERVER_CACHE_SIZE); - estimatedSize = Math.min(estimatedSize, maxSize); - if (estimatedSize > Integer.MAX_VALUE) { - throw new IllegalStateException("Estimated size(" + estimatedSize + ") must not be greater than Integer.MAX_VALUE(" + Integer.MAX_VALUE + ")"); - } - try { - TrustedByteArrayOutputStream baOut = new TrustedByteArrayOutputStream((int)estimatedSize); - DataOutputStream out = new DataOutputStream(baOut); - // Write onExpressions first, for hash key evaluation along with deserialization - out.writeInt(onExpressions.size()); - for (Expression expression : onExpressions) { - WritableUtils.writeVInt(out, ExpressionType.valueOf(expression).ordinal()); - expression.write(out); - } - int exprSize = baOut.size() + Bytes.SIZEOF_INT; - out.writeInt(exprSize); - int nRows = 0; - out.writeInt(nRows); // In the end will be replaced with total number of rows - for (Tuple result = iterator.next(); result != null; result = iterator.next()) { - TupleUtil.write(result, out); - if (baOut.size() > maxSize) { - throw new MaxServerCacheSizeExceededException("Size of hash cache (" + baOut.size() + " bytes) exceeds the maximum allowed size (" + maxSize + " bytes)"); - } - nRows++; - } - TrustedByteArrayOutputStream sizeOut = new TrustedByteArrayOutputStream(Bytes.SIZEOF_INT); - DataOutputStream dataOut = new DataOutputStream(sizeOut); - try { - dataOut.writeInt(nRows); - dataOut.flush(); - byte[] cache = baOut.getBuffer(); - // Replace number of rows written above with the correct value. - System.arraycopy(sizeOut.getBuffer(), 0, cache, exprSize, sizeOut.size()); - // Reallocate to actual size plus compressed buffer size (which is allocated below) - int maxCompressedSize = Snappy.maxCompressedLength(baOut.size()); - byte[] compressed = new byte[maxCompressedSize]; // size for worst case - int compressedSize = Snappy.compress(baOut.getBuffer(), 0, baOut.size(), compressed, 0); - // Last realloc to size of compressed buffer. - ptr.set(compressed,0,compressedSize); - } finally { - dataOut.close(); - } - } catch (IOException e) { - throw ServerUtil.parseServerException(e); - } finally { - iterator.close(); - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/join/HashCacheFactory.java b/phoenix-core/src/main/java/com/salesforce/phoenix/join/HashCacheFactory.java deleted file mode 100644 index ed0dab27..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/join/HashCacheFactory.java +++ /dev/null @@ -1,135 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.join; - -import java.io.*; -import java.sql.SQLException; -import java.util.*; - -import net.jcip.annotations.Immutable; - -import org.apache.hadoop.hbase.client.Result; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; -import org.apache.hadoop.io.WritableUtils; -import org.xerial.snappy.Snappy; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.cache.HashCache; -import com.salesforce.phoenix.coprocessor.ServerCachingProtocol.ServerCacheFactory; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.ExpressionType; -import com.salesforce.phoenix.memory.MemoryManager.MemoryChunk; -import com.salesforce.phoenix.schema.tuple.ResultTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.*; - -public class HashCacheFactory implements ServerCacheFactory { - - public HashCacheFactory() { - } - - @Override - public void readFields(DataInput input) throws IOException { - } - - @Override - public void write(DataOutput output) throws IOException { - } - - @Override - public Closeable newCache(ImmutableBytesWritable cachePtr, MemoryChunk chunk) throws SQLException { - try { - int size = Snappy.uncompressedLength(cachePtr.get()); - byte[] uncompressed = new byte[size]; - Snappy.uncompress(cachePtr.get(), 0, cachePtr.getLength(), uncompressed, 0); - return new HashCacheImpl(uncompressed, chunk); - } catch (IOException e) { - throw ServerUtil.parseServerException(e); - } - } - - @Immutable - private class HashCacheImpl implements HashCache { - private final Map> hashCache; - private final MemoryChunk memoryChunk; - - private HashCacheImpl(byte[] hashCacheBytes, MemoryChunk memoryChunk) { - try { - this.memoryChunk = memoryChunk; - byte[] hashCacheByteArray = hashCacheBytes; - int offset = 0; - ByteArrayInputStream input = new ByteArrayInputStream(hashCacheByteArray, offset, hashCacheBytes.length); - DataInputStream dataInput = new DataInputStream(input); - int nExprs = dataInput.readInt(); - List onExpressions = new ArrayList(nExprs); - for (int i = 0; i < nExprs; i++) { - int expressionOrdinal = WritableUtils.readVInt(dataInput); - Expression expression = ExpressionType.values()[expressionOrdinal].newInstance(); - expression.readFields(dataInput); - onExpressions.add(expression); - } - int exprSize = dataInput.readInt(); - offset += exprSize; - int nRows = dataInput.readInt(); - int estimatedSize = SizedUtil.sizeOfMap(nRows, SizedUtil.IMMUTABLE_BYTES_WRITABLE_SIZE, SizedUtil.RESULT_SIZE) + hashCacheBytes.length; - this.memoryChunk.resize(estimatedSize); - HashMap> hashCacheMap = new HashMap>(nRows * 5 / 4); - offset += Bytes.SIZEOF_INT; - // Build Map with evaluated hash key as key and row as value - for (int i = 0; i < nRows; i++) { - int resultSize = (int)Bytes.readVLong(hashCacheByteArray, offset); - offset += WritableUtils.decodeVIntSize(hashCacheByteArray[offset]); - ImmutableBytesWritable value = new ImmutableBytesWritable(hashCacheByteArray,offset,resultSize); - Tuple result = new ResultTuple(new Result(value)); - ImmutableBytesPtr key = TupleUtil.getConcatenatedValue(result, onExpressions); - List tuples = hashCacheMap.get(key); - if (tuples == null) { - tuples = new ArrayList(1); - hashCacheMap.put(key, tuples); - } - tuples.add(result); - offset += resultSize; - } - this.hashCache = Collections.unmodifiableMap(hashCacheMap); - } catch (IOException e) { // Not possible with ByteArrayInputStream - throw new RuntimeException(e); - } - } - - @Override - public void close() { - memoryChunk.close(); - } - - @Override - public List get(ImmutableBytesPtr hashKey) { - return hashCache.get(hashKey); - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/join/MaxServerCacheSizeExceededException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/join/MaxServerCacheSizeExceededException.java deleted file mode 100644 index b6da591c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/join/MaxServerCacheSizeExceededException.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.join; - -/** - * - * Exception thrown when the size of the hash cache exceeds the - * maximum size as specified by the phoenix.query.maxHashCacheBytes - * parameter in the {@link org.apache.hadoop.conf.Configuration} - * - * @author jtaylor - * @since 0.1 - */ -public class MaxServerCacheSizeExceededException extends RuntimeException { - private static final long serialVersionUID = 1L; - - public MaxServerCacheSizeExceededException() { - } - - public MaxServerCacheSizeExceededException(String message) { - super(message); - } - - public MaxServerCacheSizeExceededException(Throwable cause) { - super(cause); - } - - public MaxServerCacheSizeExceededException(String message, Throwable cause) { - super(message, cause); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/join/ScanProjector.java b/phoenix-core/src/main/java/com/salesforce/phoenix/join/ScanProjector.java deleted file mode 100644 index 0da487df..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/join/ScanProjector.java +++ /dev/null @@ -1,250 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.join; - -import java.io.ByteArrayInputStream; -import java.io.ByteArrayOutputStream; -import java.io.DataInputStream; -import java.io.DataOutputStream; -import java.io.IOException; -import java.util.List; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; -import org.apache.hadoop.io.WritableUtils; - -import com.salesforce.phoenix.compile.JoinCompiler.ProjectedPTableWrapper; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.ExpressionType; -import com.salesforce.phoenix.schema.KeyValueSchema; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.ValueBitSet; -import com.salesforce.phoenix.schema.KeyValueSchema.KeyValueSchemaBuilder; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.KeyValueUtil; -import com.salesforce.phoenix.util.SchemaUtil; - -public class ScanProjector { - public static final byte[] VALUE_COLUMN_FAMILY = Bytes.toBytes("_v"); - public static final byte[] VALUE_COLUMN_QUALIFIER = new byte[0]; - - private static final String SCAN_PROJECTOR = "scanProjector"; - - private final KeyValueSchema schema; - private final Expression[] expressions; - private ValueBitSet valueSet; - private final ImmutableBytesWritable ptr = new ImmutableBytesWritable(); - - public ScanProjector(ProjectedPTableWrapper projected) { - List columns = projected.getTable().getColumns(); - expressions = new Expression[columns.size() - projected.getTable().getPKColumns().size()]; - // we do not count minNullableIndex for we might do later merge. - KeyValueSchemaBuilder builder = new KeyValueSchemaBuilder(0); - int i = 0; - for (PColumn column : projected.getTable().getColumns()) { - if (!SchemaUtil.isPKColumn(column)) { - builder.addField(column); - expressions[i++] = projected.getSourceExpression(column); - } - } - schema = builder.build(); - valueSet = ValueBitSet.newInstance(schema); - } - - private ScanProjector(KeyValueSchema schema, Expression[] expressions) { - this.schema = schema; - this.expressions = expressions; - this.valueSet = ValueBitSet.newInstance(schema); - } - - public void setValueBitSet(ValueBitSet bitSet) { - this.valueSet = bitSet; - } - - public static void serializeProjectorIntoScan(Scan scan, ScanProjector projector) { - ByteArrayOutputStream stream = new ByteArrayOutputStream(); - try { - DataOutputStream output = new DataOutputStream(stream); - projector.schema.write(output); - int count = projector.expressions.length; - WritableUtils.writeVInt(output, count); - for (int i = 0; i < count; i++) { - WritableUtils.writeVInt(output, ExpressionType.valueOf(projector.expressions[i]).ordinal()); - projector.expressions[i].write(output); - } - scan.setAttribute(SCAN_PROJECTOR, stream.toByteArray()); - } catch (IOException e) { - throw new RuntimeException(e); - } finally { - try { - stream.close(); - } catch (IOException e) { - throw new RuntimeException(e); - } - } - - } - - public static ScanProjector deserializeProjectorFromScan(Scan scan) { - byte[] proj = scan.getAttribute(SCAN_PROJECTOR); - if (proj == null) { - return null; - } - ByteArrayInputStream stream = new ByteArrayInputStream(proj); - try { - DataInputStream input = new DataInputStream(stream); - KeyValueSchema schema = new KeyValueSchema(); - schema.readFields(input); - int count = WritableUtils.readVInt(input); - Expression[] expressions = new Expression[count]; - for (int i = 0; i < count; i++) { - int ordinal = WritableUtils.readVInt(input); - expressions[i] = ExpressionType.values()[ordinal].newInstance(); - expressions[i].readFields(input); - } - return new ScanProjector(schema, expressions); - } catch (IOException e) { - throw new RuntimeException(e); - } finally { - try { - stream.close(); - } catch (IOException e) { - throw new RuntimeException(e); - } - } - } - - public static class ProjectedValueTuple implements Tuple { - private ImmutableBytesWritable keyPtr = new ImmutableBytesWritable(); - private long timestamp; - private byte[] projectedValue; - private int bitSetLen; - private KeyValue keyValue; - - private ProjectedValueTuple(byte[] keyBuffer, int keyOffset, int keyLength, long timestamp, byte[] projectedValue, int bitSetLen) { - this.keyPtr.set(keyBuffer, keyOffset, keyLength); - this.timestamp = timestamp; - this.projectedValue = projectedValue; - this.bitSetLen = bitSetLen; - } - - public ImmutableBytesWritable getKeyPtr() { - return keyPtr; - } - - public long getTimestamp() { - return timestamp; - } - - public byte[] getProjectedValue() { - return projectedValue; - } - - public int getBitSetLength() { - return bitSetLen; - } - - @Override - public void getKey(ImmutableBytesWritable ptr) { - ptr.set(keyPtr.get(), keyPtr.getOffset(), keyPtr.getLength()); - } - - @Override - public KeyValue getValue(int index) { - if (index != 0) { - throw new IndexOutOfBoundsException(Integer.toString(index)); - } - return getValue(VALUE_COLUMN_FAMILY, VALUE_COLUMN_QUALIFIER); - } - - @Override - public KeyValue getValue(byte[] family, byte[] qualifier) { - if (keyValue == null) { - keyValue = KeyValueUtil.newKeyValue(keyPtr.get(), keyPtr.getOffset(), keyPtr.getLength(), - VALUE_COLUMN_FAMILY, VALUE_COLUMN_QUALIFIER, timestamp, projectedValue, 0, projectedValue.length); - } - return keyValue; - } - - @Override - public boolean getValue(byte[] family, byte[] qualifier, - ImmutableBytesWritable ptr) { - ptr.set(projectedValue); - return true; - } - - @Override - public boolean isImmutable() { - return true; - } - - @Override - public int size() { - return 1; - } - } - - public ProjectedValueTuple projectResults(Tuple tuple) { - byte[] bytesValue = schema.toBytes(tuple, expressions, valueSet, ptr); - KeyValue base = tuple.getValue(0); - return new ProjectedValueTuple(base.getBuffer(), base.getRowOffset(), base.getRowLength(), base.getTimestamp(), bytesValue, valueSet.getEstimatedLength()); - } - - public static void decodeProjectedValue(Tuple tuple, ImmutableBytesWritable ptr) throws IOException { - boolean b = tuple.getValue(VALUE_COLUMN_FAMILY, VALUE_COLUMN_QUALIFIER, ptr); - if (!b) - throw new IOException("Trying to decode a non-projected value."); - } - - public static ProjectedValueTuple mergeProjectedValue(ProjectedValueTuple dest, KeyValueSchema destSchema, ValueBitSet destBitSet, - Tuple src, KeyValueSchema srcSchema, ValueBitSet srcBitSet, int offset) throws IOException { - ImmutableBytesWritable destValue = new ImmutableBytesWritable(dest.getProjectedValue()); - destBitSet.clear(); - destBitSet.or(destValue); - int origDestBitSetLen = dest.getBitSetLength(); - ImmutableBytesWritable srcValue = new ImmutableBytesWritable(); - decodeProjectedValue(src, srcValue); - srcBitSet.clear(); - srcBitSet.or(srcValue); - int origSrcBitSetLen = srcBitSet.getEstimatedLength(); - for (int i = 0; i < srcBitSet.getMaxSetBit(); i++) { - if (srcBitSet.get(i)) { - destBitSet.set(offset + i); - } - } - int destBitSetLen = destBitSet.getEstimatedLength(); - byte[] merged = new byte[destValue.getLength() - origDestBitSetLen + srcValue.getLength() - origSrcBitSetLen + destBitSetLen]; - int o = Bytes.putBytes(merged, 0, destValue.get(), destValue.getOffset(), destValue.getLength() - origDestBitSetLen); - o = Bytes.putBytes(merged, o, srcValue.get(), srcValue.getOffset(), srcValue.getLength() - origSrcBitSetLen); - destBitSet.toBytes(merged, o); - ImmutableBytesWritable keyPtr = dest.getKeyPtr(); - return new ProjectedValueTuple(keyPtr.get(), keyPtr.getOffset(), keyPtr.getLength(), dest.getTimestamp(), merged, destBitSetLen); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/map/reduce/util/ConfigReader.java b/phoenix-core/src/main/java/com/salesforce/phoenix/map/reduce/util/ConfigReader.java deleted file mode 100644 index 46664eb5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/map/reduce/util/ConfigReader.java +++ /dev/null @@ -1,122 +0,0 @@ -/******************************************************************************* -* Copyright (c) 2013, Salesforce.com, Inc. -* All rights reserved. -* -* Redistribution and use in source and binary forms, with or without -* modification, are permitted provided that the following conditions are met: -* -* Redistributions of source code must retain the above copyright notice, -* this list of conditions and the following disclaimer. -* Redistributions in binary form must reproduce the above copyright notice, -* this list of conditions and the following disclaimer in the documentation -* and/or other materials provided with the distribution. -* Neither the name of Salesforce.com nor the names of its contributors may -* be used to endorse or promote products derived from this software without -* specific prior written permission. -* -* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -* DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -* OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -******************************************************************************/ -package com.salesforce.phoenix.map.reduce.util; - -import java.util.Enumeration; -import java.util.HashMap; -import java.util.Map; -import java.util.ResourceBundle; - -/** - * Class to read configs. - * - */ - -public class ConfigReader - { - - private String propertyFile = null; - private boolean loaded = false; - private static final Object _synObj = new Object(); - private Map properties = new HashMap(); - private Exception loadException = null; - - /** - * Retrieves singleton config objects from a hashmap of stored objects, - * creates these objects if they aren't in the hashmap. - */ - - public ConfigReader(String propertyFile) { - this.propertyFile = propertyFile; - } - - public void load() throws Exception { - if (loaded) { - if (loadException != null) { - throw new Exception(loadException); - } - return; - } - synchronized (_synObj) { - if (!loaded) { - try { - String tmpFile = propertyFile.trim(); - if (tmpFile.endsWith(".properties")) { - tmpFile = tmpFile - .substring(0, tmpFile.lastIndexOf(".")); - } - ResourceBundle resource = ResourceBundle.getBundle(tmpFile); - Enumeration enm = resource.getKeys(); - - while (enm.hasMoreElements()) { - String key = enm.nextElement(); - String value = resource.getString(key); - properties.put(key, value); - } - } catch (Exception e) { - System.err - .println("Exception while loading the config.properties file :: " - + e.getMessage()); - loadException = e; - loaded = true; - throw e; - } - loaded = true; - } - } - } - - public void addConfig(String key, String value) { - try { - load(); - } catch (Exception e) { - System.err.println("ERROR :: " + e.getMessage()); - } - properties.put(key, value); - } - - public boolean hasConfig(String key) { - try { - load(); - } catch (Exception e) { - System.err.println("ERROR :: " + e.getMessage()); - } - return properties.containsKey(key); - } - - public String getConfig(String key) throws Exception { - load(); - return properties.get(key); - } - - public Map getAllConfigMap() throws Exception { - load(); - return properties; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/memory/ChildMemoryManager.java b/phoenix-core/src/main/java/com/salesforce/phoenix/memory/ChildMemoryManager.java deleted file mode 100644 index f4912fa5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/memory/ChildMemoryManager.java +++ /dev/null @@ -1,126 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.memory; - -import org.apache.http.annotation.GuardedBy; -import org.apache.http.annotation.ThreadSafe; - -/** - * - * Child memory manager that delegates through to global memory manager, - * but enforces that at most a threshold percentage is used by this - * memory manager. No blocking is done if the threshold is exceeded, - * but the standard blocking will be done by the global memory manager. - * - * @author jtaylor - * @since 0.1 - */ -@ThreadSafe -public class ChildMemoryManager extends DelegatingMemoryManager { - private final Object sync = new Object(); - private final int maxPercOfTotal; - @GuardedBy("sync") - private long allocatedBytes; - - public ChildMemoryManager(MemoryManager mm, int maxPercOfTotal) { - super(mm); - if (mm instanceof ChildMemoryManager) { - throw new IllegalStateException("ChildMemoryManager cannot delegate to another ChildMemoryManager"); - } - this.maxPercOfTotal = maxPercOfTotal; - if (maxPercOfTotal <= 0 || maxPercOfTotal > 100) { - throw new IllegalArgumentException("Max percentage of total memory (" + maxPercOfTotal + "%) must be greater than zero and less than or equal to 100"); - } - } - - - private long adjustAllocation(long minBytes, long reqBytes) { - assert(reqBytes >= minBytes); - long availBytes = getAvailableMemory(); - // Check if this memory managers percentage of allocated bytes exceeds its allowed maximum - if (minBytes > availBytes) { - throw new InsufficientMemoryException("Attempt to allocate more memory than the max allowed of " + maxPercOfTotal + "%"); - } - // Revise reqBytes down to available memory if necessary - return Math.min(reqBytes,availBytes); - } - - @Override - public MemoryChunk allocate(long minBytes, long nBytes) { - synchronized (sync) { - nBytes = adjustAllocation(minBytes, nBytes); - final MemoryChunk chunk = super.allocate(minBytes, nBytes); - allocatedBytes += chunk.getSize(); - // Instantiate delegate chunk to track allocatedBytes correctly - return new MemoryChunk() { - @Override - public void close() { - synchronized (sync) { - allocatedBytes -= chunk.getSize(); - chunk.close(); - } - } - - @Override - public long getSize() { - return chunk.getSize(); - } - - @Override - public void resize(long nBytes) { - synchronized (sync) { - long size = getSize(); - long deltaBytes = nBytes - size; - if (deltaBytes > 0) { - adjustAllocation(deltaBytes,deltaBytes); // Throw if too much memory - } - chunk.resize(nBytes); - allocatedBytes += deltaBytes; - } - } - }; - } - } - - @Override - public long getAvailableMemory() { - synchronized (sync) { - long availBytes = getMaxMemory() - allocatedBytes; - // Sanity check (should never happen) - if (availBytes < 0) { - throw new IllegalStateException("Available memory has become negative: " + availBytes + " bytes. Allocated memory: " + allocatedBytes + " bytes."); - } - return availBytes; - } - } - - @Override - public long getMaxMemory() { - return maxPercOfTotal * super.getMaxMemory() / 100; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/memory/DelegatingMemoryManager.java b/phoenix-core/src/main/java/com/salesforce/phoenix/memory/DelegatingMemoryManager.java deleted file mode 100644 index 3bcccb60..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/memory/DelegatingMemoryManager.java +++ /dev/null @@ -1,69 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.memory; - -/** - * - * Memory manager that delegates through to another memory manager. - * - * @author jtaylor - * @since 0.1 - */ -public class DelegatingMemoryManager implements MemoryManager { - private final MemoryManager parent; - - public DelegatingMemoryManager(MemoryManager globalMemoryManager){ - this.parent = globalMemoryManager; - } - - @Override - public long getAvailableMemory() { - return parent.getAvailableMemory(); - } - - @Override - public long getMaxMemory() { - return parent.getMaxMemory(); - } - - @Override - public MemoryChunk allocate(long minBytes, long reqBytes) { - return parent.allocate(minBytes, reqBytes); - } - - - @Override - public MemoryChunk allocate(long nBytes) { - return allocate(nBytes, nBytes); - } - - public MemoryManager getParent() { - return parent; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/memory/InsufficientMemoryException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/memory/InsufficientMemoryException.java deleted file mode 100644 index 2157f4a4..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/memory/InsufficientMemoryException.java +++ /dev/null @@ -1,55 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.memory; - -/** - * - * Exception thrown by MemoryManager when insufficient memory is available - * - * @author jtaylor - * @since 0.1 - */ -public class InsufficientMemoryException extends RuntimeException { - private static final long serialVersionUID = 1L; - - public InsufficientMemoryException() { - } - - public InsufficientMemoryException(String message) { - super(message); - } - - public InsufficientMemoryException(Throwable cause) { - super(cause); - } - - public InsufficientMemoryException(String message, Throwable cause) { - super(message, cause); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/memory/MemoryManager.java b/phoenix-core/src/main/java/com/salesforce/phoenix/memory/MemoryManager.java deleted file mode 100644 index 5fdd043c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/memory/MemoryManager.java +++ /dev/null @@ -1,105 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.memory; - -import java.io.Closeable; - -/** - * - * Memory manager used to track memory usage. Either throttles - * memory usage by blocking when the max memory is reached or - * allocates up to a maximum without blocking. - * - * @author jtaylor - * @since 0.1 - */ -public interface MemoryManager { - /** - * Get the total amount of memory (in bytes) that may be allocated. - */ - long getMaxMemory(); - - /** - * Get the amount of available memory (in bytes) not yet allocated. - */ - long getAvailableMemory(); - - /** - * Allocate up to reqBytes of memory, dialing the amount down to - * minBytes if full amount is not available. If minBytes is not - * available, then this call will block for a configurable amount - * of time and throw if minBytes does not become available. - * @param minBytes minimum number of bytes required - * @param reqBytes requested number of bytes. Must be greater - * than or equal to minBytes - * @return MemoryChunk that was allocated - * @throws InsufficientMemoryException if unable to allocate minBytes - * during configured amount of time - */ - MemoryChunk allocate(long minBytes, long reqBytes); - - /** - * Equivalent to calling {@link #allocate(long, long)} where - * minBytes and reqBytes being the same. - */ - MemoryChunk allocate(long nBytes); - - /** - * - * Chunk of allocated memory. To reclaim the memory, call {@link #close()} - * - * @author jtaylor - * @since 0.1 - */ - public static interface MemoryChunk extends Closeable { - /** - * Get the size in bytes of the allocated chunk. - */ - long getSize(); - - /** - * Free up the memory associated with this chunk - */ - @Override - void close(); - - /** - * Resize an already allocated memory chunk up or down to a - * new amount. If decreasing allocation, this call will not block. - * If increasing allocation, and nBytes is not available, then - * this call will block for a configurable amount of time and - * throw if nBytes does not become available. Most commonly - * used to adjust the allocation of a memory buffer that was - * originally sized for the worst case scenario. - * @param nBytes new number of bytes required for this chunk - * @throws InsufficientMemoryException if unable to allocate minBytes - * during configured amount of time - */ - void resize(long nBytes); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AddColumnStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AddColumnStatement.java deleted file mode 100644 index 6007b0be..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AddColumnStatement.java +++ /dev/null @@ -1,59 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.Collections; -import java.util.List; -import java.util.Map; - -import com.salesforce.phoenix.schema.PTableType; - -public class AddColumnStatement extends AlterTableStatement { - private final List columnDefs; - private final boolean ifNotExists; - private final Map props; - - protected AddColumnStatement(NamedTableNode table, PTableType tableType, List columnDefs, boolean ifNotExists, Map props) { - super(table, tableType); - this.columnDefs = columnDefs; - this.props = props == null ? Collections.emptyMap() : props; - this.ifNotExists = ifNotExists; - } - - public List getColumnDefs() { - return columnDefs; - } - - public boolean ifNotExists() { - return ifNotExists; - } - - public Map getProps() { - return props; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AddParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AddParseNode.java deleted file mode 100644 index 69f7ee86..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AddParseNode.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing addition in a SQL expression - * - * @author jtaylor - * @since 0.1 - */ -public class AddParseNode extends ArithmeticParseNode { - - AddParseNode(List children) { - super(children); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AggregateFunctionParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AggregateFunctionParseNode.java deleted file mode 100644 index e0a6d388..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AggregateFunctionParseNode.java +++ /dev/null @@ -1,47 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.List; - - -public class AggregateFunctionParseNode extends FunctionParseNode { - - public AggregateFunctionParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - /** - * Aggregate function are not stateless, even though all the args may be stateless, - * for example, COUNT(1) - */ - @Override - public boolean isStateless() { - return false; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AliasedNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AliasedNode.java deleted file mode 100644 index 667be6b1..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AliasedNode.java +++ /dev/null @@ -1,61 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import com.salesforce.phoenix.util.SchemaUtil; - -/** - * - * Node representing an aliased parse node in a SQL select clause - * - * @author jtaylor - * @since 0.1 - */ -public class AliasedNode { - private final String alias; - private final ParseNode node; - private final boolean isCaseSensitve; - - public AliasedNode(String alias, ParseNode node) { - this.isCaseSensitve = alias != null && SchemaUtil.isCaseSensitive(alias); - this.alias = alias == null ? null : SchemaUtil.normalizeIdentifier(alias); - this.node = node; - } - - public String getAlias() { - return alias; - } - - public ParseNode getNode() { - return node; - } - - public boolean isCaseSensitve() { - return isCaseSensitve; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AlterIndexStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AlterIndexStatement.java deleted file mode 100644 index 20765bfe..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AlterIndexStatement.java +++ /dev/null @@ -1,61 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import com.salesforce.phoenix.schema.PIndexState; - -public class AlterIndexStatement extends SingleTableSQLStatement { - private final String dataTableName; - private final boolean ifExists; - private final PIndexState indexState; - - public AlterIndexStatement(NamedTableNode indexTableNode, String dataTableName, boolean ifExists, PIndexState indexState) { - super(indexTableNode,0); - this.dataTableName = dataTableName; - this.ifExists = ifExists; - this.indexState = indexState; - } - - public String getTableName() { - return dataTableName; - } - - @Override - public int getBindCount() { - return 0; - } - - public boolean ifExists() { - return ifExists; - } - - public PIndexState getIndexState() { - return indexState; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AlterTableStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AlterTableStatement.java deleted file mode 100644 index 2590bbf0..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AlterTableStatement.java +++ /dev/null @@ -1,43 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import com.salesforce.phoenix.schema.PTableType; - -public abstract class AlterTableStatement extends SingleTableSQLStatement { - private final PTableType tableType; - - AlterTableStatement(NamedTableNode table, PTableType tableType) { - super(table, 0); - this.tableType = tableType; - } - - public PTableType getTableType() { - return tableType; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AndParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AndParseNode.java deleted file mode 100644 index ca2182f3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AndParseNode.java +++ /dev/null @@ -1,59 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - - -/** - * - * Node representing AND in a SQL expression - * - * @author jtaylor - * @since 0.1 - */ -public class AndParseNode extends CompoundParseNode { - - AndParseNode(List children) { - super(children); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ArithmeticParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ArithmeticParseNode.java deleted file mode 100644 index 5609ab01..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ArithmeticParseNode.java +++ /dev/null @@ -1,38 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.List; - -public abstract class ArithmeticParseNode extends CompoundParseNode { - - public ArithmeticParseNode(List children) { - super(children); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ArrayConstructorNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ArrayConstructorNode.java deleted file mode 100644 index f9ba8c2b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ArrayConstructorNode.java +++ /dev/null @@ -1,52 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2014, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -/** - * Holds the list of array elements that will be used by the upsert stmt with ARRAY column - * - */ -public class ArrayConstructorNode extends CompoundParseNode { - - public ArrayConstructorNode(List children) { - super(children); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AvgAggregateParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AvgAggregateParseNode.java deleted file mode 100644 index 19b35847..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/AvgAggregateParseNode.java +++ /dev/null @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.*; - - -public class AvgAggregateParseNode extends AggregateFunctionParseNode { - - public AvgAggregateParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public FunctionExpression create(List children, StatementContext context) throws SQLException { - SumAggregateFunction sumFunc; - CountAggregateFunction countFunc = (CountAggregateFunction)context.getExpressionManager().addIfAbsent(new CountAggregateFunction(children)); - if (!countFunc.isConstantExpression()) { - sumFunc = (SumAggregateFunction)context.getExpressionManager().addIfAbsent(new SumAggregateFunction(countFunc.getChildren(),null)); - } else { - sumFunc = null; - } - - return new AvgAggregateFunction(children, countFunc, sumFunc); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BaseParseNodeVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BaseParseNodeVisitor.java deleted file mode 100644 index 31d51bb3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BaseParseNodeVisitor.java +++ /dev/null @@ -1,78 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.sql.SQLFeatureNotSupportedException; -import java.util.List; - - - -/** - * - * Base class for parse node visitors. - * - * @author jtaylor - * @since 0.1 - */ -public abstract class BaseParseNodeVisitor implements ParseNodeVisitor { - - /** - * Fall through visitEnter method. Anything coming through - * here means that a more specific method wasn't found - * and thus this CompoundNode is not yet supported. - */ - @Override - public boolean visitEnter(CompoundParseNode expressionNode) throws SQLException { - throw new SQLFeatureNotSupportedException(expressionNode.toString()); - } - - @Override - public E visitLeave(CompoundParseNode expressionNode, List l) throws SQLException { - throw new SQLFeatureNotSupportedException(expressionNode.toString()); - } - - /** - * Fall through visit method. Anything coming through - * here means that a more specific method wasn't found - * and thus this Node is not yet supported. - */ - @Override - public E visit(ParseNode expressionNode) throws SQLException { - throw new SQLFeatureNotSupportedException(expressionNode.toString()); - } - - @Override - public List newElementList(int size) { - return null; - } - - @Override - public void addElement(List l, E element) { - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BetweenParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BetweenParseNode.java deleted file mode 100644 index cd7d5ded..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BetweenParseNode.java +++ /dev/null @@ -1,62 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.*; - - - -/** - * - * Node representing BETWEEN in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class BetweenParseNode extends CompoundParseNode { - private final boolean negate; - - BetweenParseNode(ParseNode l, ParseNode r1, ParseNode r2, boolean negate) { - super(Arrays.asList(l, r1, r2)); - this.negate = negate; - } - - public boolean isNegate() { - return negate; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BinaryParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BinaryParseNode.java deleted file mode 100644 index 37bdf55a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BinaryParseNode.java +++ /dev/null @@ -1,52 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.Arrays; - -/** - * - * Abstract class for operators that operate on exactly two nodes - * - * @author jtaylor - * @since 0.1 - */ -public abstract class BinaryParseNode extends CompoundParseNode { - - BinaryParseNode(ParseNode lhs, ParseNode rhs) { - super(Arrays.asList(lhs, rhs)); - } - - public ParseNode getLHS() { - return getChildren().get(0); - } - - public ParseNode getRHS() { - return getChildren().get(1); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BindParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BindParseNode.java deleted file mode 100644 index 383bcde9..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BindParseNode.java +++ /dev/null @@ -1,69 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; - - - -/** - * - * Node representing a bind variable in a SQL expression - * - * @author jtaylor - * @since 0.1 - */ -public class BindParseNode extends NamedParseNode { - private final int index; - - BindParseNode(String name) { - super(name); - index = Integer.parseInt(name); - } - - public int getIndex() { - return index-1; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - return visitor.visit(this); - } - - - @Override - public boolean isStateless() { - return true; - } - - @Override - public String toString() { - return ":" + index; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BindTableNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BindTableNode.java deleted file mode 100644 index 3c7c8828..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BindTableNode.java +++ /dev/null @@ -1,52 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; - - - -/** - * - * Node representing a TABLE bound using an ARRAY variable - * TODO: modify grammar to support this - * @author jtaylor - * @since 0.1 - */ -public class BindTableNode extends ConcreteTableNode { - - BindTableNode(String alias, TableName name) { - super(alias, name); - } - - @Override - public void accept(TableNodeVisitor visitor) throws SQLException { - visitor.visit(this); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BindableStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BindableStatement.java deleted file mode 100644 index 2ac82809..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/BindableStatement.java +++ /dev/null @@ -1,33 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - - -public interface BindableStatement { - public int getBindCount(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CaseParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CaseParseNode.java deleted file mode 100644 index 96870b6f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CaseParseNode.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing a CASE in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class CaseParseNode extends CompoundParseNode { - - CaseParseNode(List children) { - super(children); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CastParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CastParseNode.java deleted file mode 100644 index 50ab939b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CastParseNode.java +++ /dev/null @@ -1,88 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.RoundDecimalExpression; -import com.salesforce.phoenix.expression.function.RoundTimestampExpression; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.TypeMismatchException; - -/** - * - * Node representing the CAST operator in SQL. - * - * @author samarth.jain - * @since 0.1 - * - */ -public class CastParseNode extends UnaryParseNode { - - private final PDataType dt; - - CastParseNode(ParseNode expr, String dataType) { - super(expr); - dt = PDataType.fromSqlTypeName(dataType); - } - - CastParseNode(ParseNode expr, PDataType dataType) { - super(expr); - dt = dataType; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } - - public PDataType getDataType() { - return dt; - } - - public static Expression convertToRoundExpressionIfNeeded(PDataType fromDataType, PDataType targetDataType, List expressions) throws SQLException { - Expression firstChildExpr = expressions.get(0); - if(fromDataType == targetDataType) { - return firstChildExpr; - } else if(fromDataType == PDataType.DECIMAL && targetDataType.isCoercibleTo(PDataType.LONG)) { - return new RoundDecimalExpression(expressions); - } else if((fromDataType == PDataType.TIMESTAMP || fromDataType == PDataType.UNSIGNED_TIMESTAMP) && targetDataType.isCoercibleTo(PDataType.DATE)) { - return RoundTimestampExpression.create(expressions); - } else if(!fromDataType.isCoercibleTo(targetDataType)) { - throw TypeMismatchException.newException(fromDataType, targetDataType, firstChildExpr.toString()); - } - return firstChildExpr; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CeilParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CeilParseNode.java deleted file mode 100644 index e7f78b81..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CeilParseNode.java +++ /dev/null @@ -1,90 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.CeilDateExpression; -import com.salesforce.phoenix.expression.function.CeilDecimalExpression; -import com.salesforce.phoenix.expression.function.CeilFunction; -import com.salesforce.phoenix.expression.function.CeilTimestampExpression; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.TypeMismatchException; - -/** - * Parse node corresponding to {@link CeilFunction}. - * It also acts as a factory for creating the right kind of - * ceil expression according to the data type of the - * first child. - * - * @author samarth.jain - * @since 3.0.0 - */ -public class CeilParseNode extends FunctionParseNode { - - CeilParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public Expression create(List children, StatementContext context) throws SQLException { - return getCeilExpression(children); - } - - public static Expression getCeilExpression(List children) throws SQLException { - final Expression firstChild = children.get(0); - final PDataType firstChildDataType = firstChild.getDataType(); - if(firstChildDataType.isCoercibleTo(PDataType.DATE)) { - return CeilDateExpression.create(children); - } else if (firstChildDataType == PDataType.TIMESTAMP || firstChildDataType == PDataType.UNSIGNED_TIMESTAMP) { - return CeilTimestampExpression.create(children); - } else if(firstChildDataType.isCoercibleTo(PDataType.DECIMAL)) { - return new CeilDecimalExpression(children); - } else { - throw TypeMismatchException.newException(firstChildDataType, "1"); - } - } - - /** - * When ceiling off decimals, user need not specify the scale. In such cases, - * we need to prevent the function from getting evaluated as null. This is really - * a hack. A better way would have been if {@link com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo} provided a - * way of associating default values for each permissible data type. - * Something like: @ Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValues = {"null", "1"} isConstant=true) - * Till then, this will have to do. - */ - @Override - public boolean evalToNullIfParamIsNull(StatementContext context, int index) throws SQLException { - return index == 0; - } - - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnDef.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnDef.java deleted file mode 100644 index 1bdffe91..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnDef.java +++ /dev/null @@ -1,189 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.util.SchemaUtil; - - -/** - * - * Represents a column definition during DDL - * - * @author jtaylor - * @since 0.1 - */ -public class ColumnDef { - private final ColumnName columnDefName; - private PDataType dataType; - private final boolean isNull; - private final Integer maxLength; - private final Integer scale; - private final boolean isPK; - private final ColumnModifier columnModifier; - private final boolean isArray; - private final Integer arrSize; - - ColumnDef(ColumnName columnDefName, String sqlTypeName, boolean isArray, Integer arrSize, boolean isNull, Integer maxLength, - Integer scale, boolean isPK, ColumnModifier columnModifier) { - try { - PDataType localType = null; - this.columnDefName = columnDefName; - this.isArray = isArray; - if(this.isArray) { - localType = sqlTypeName == null ? null : PDataType.fromTypeId(PDataType.sqlArrayType(SchemaUtil.normalizeIdentifier(sqlTypeName))); - this.dataType = sqlTypeName == null ? null : PDataType.fromSqlTypeName(SchemaUtil.normalizeIdentifier(sqlTypeName)); - } else { - this.dataType = sqlTypeName == null ? null : PDataType.fromSqlTypeName(SchemaUtil.normalizeIdentifier(sqlTypeName)); - } - - // TODO : Add correctness check for arrSize. Should this be ignored as in postgresql - // Also add what is the limit that we would support. Are we going to support a - // fixed size or like postgre allow infinite. May be the datatypes max limit can - // be used for the array size (May be too big) - if(this.isArray) { - this.arrSize = arrSize; - } else { - this.arrSize = 0; - } - this.isNull = isNull; - if (this.dataType == PDataType.CHAR) { - if (maxLength == null) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.MISSING_CHAR_LENGTH) - .setColumnName(columnDefName.getColumnName()).build().buildException(); - } - if (maxLength < 1) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.NONPOSITIVE_CHAR_LENGTH) - .setColumnName(columnDefName.getColumnName()).build().buildException(); - } - scale = null; - } else if (this.dataType == PDataType.VARCHAR) { - if (maxLength != null && maxLength < 1) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.NONPOSITIVE_CHAR_LENGTH) - .setColumnName(columnDefName.getColumnName()).build().buildException(); - } - scale = null; - } else if (this.dataType == PDataType.DECIMAL) { - Integer origMaxLength = maxLength; - maxLength = maxLength == null ? PDataType.MAX_PRECISION : maxLength; - // for deciaml, 1 <= maxLength <= PDataType.MAX_PRECISION; - if (maxLength < 1 || maxLength > PDataType.MAX_PRECISION) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.DECIMAL_PRECISION_OUT_OF_RANGE) - .setColumnName(columnDefName.getColumnName()).build().buildException(); - } - // When a precision is specified and a scale is not specified, it is set to 0. - // - // This is the standard as specified in - // http://docs.oracle.com/cd/B28359_01/server.111/b28318/datatype.htm#CNCPT1832 - // and - // http://docs.oracle.com/javadb/10.6.2.1/ref/rrefsqlj15260.html. - // Otherwise, if scale is bigger than maxLength, just set it to the maxLength; - // - // When neither a precision nor a scale is specified, the precision and scale is - // ignored. All decimal are stored with as much decimal points as possible. - scale = scale == null ? - origMaxLength == null ? null : PDataType.DEFAULT_SCALE : - scale > maxLength ? maxLength : scale; - } else if (this.dataType == PDataType.BINARY) { - if (maxLength == null) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.MISSING_BINARY_LENGTH) - .setColumnName(columnDefName.getColumnName()).build().buildException(); - } - if (maxLength < 1) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.NONPOSITIVE_BINARY_LENGTH) - .setColumnName(columnDefName.getColumnName()).build().buildException(); - } - scale = null; - } else if (this.dataType == PDataType.INTEGER) { - maxLength = PDataType.INT_PRECISION; - scale = PDataType.ZERO; - } else if (this.dataType == PDataType.LONG) { - maxLength = PDataType.LONG_PRECISION; - scale = PDataType.ZERO; - } else { - // ignore maxLength and scale for other types. - maxLength = null; - scale = null; - } - this.maxLength = maxLength; - this.scale = scale; - this.isPK = isPK; - this.columnModifier = columnModifier; - if(this.isArray) { - this.dataType = localType; - } - } catch (SQLException e) { - throw new ParseException(e); - } - } - ColumnDef(ColumnName columnDefName, String sqlTypeName, boolean isNull, Integer maxLength, - Integer scale, boolean isPK, ColumnModifier columnModifier) { - this(columnDefName, sqlTypeName, false, 0, isNull, maxLength, scale, isPK, columnModifier); - } - - public ColumnName getColumnDefName() { - return columnDefName; - } - - public PDataType getDataType() { - return dataType; - } - - public boolean isNull() { - return isNull; - } - - public Integer getMaxLength() { - return maxLength; - } - - public Integer getScale() { - return scale; - } - - public boolean isPK() { - return isPK; - } - - public ColumnModifier getColumnModifier() { - return columnModifier; - } - - public boolean isArray() { - return isArray; - } - - public int getArraySize() { - return arrSize; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnFamilyDef.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnFamilyDef.java deleted file mode 100644 index 17ff7f86..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnFamilyDef.java +++ /dev/null @@ -1,65 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.*; - - -import com.google.common.collect.ImmutableList; -import com.salesforce.phoenix.util.SchemaUtil; - -/** - * - * Definition of a Column Family at DDL time - * - * @author jtaylor - * @since 0.1 - */ -public class ColumnFamilyDef { - private final String name; - private final List columnDefs; - private final Map props; - - ColumnFamilyDef(String name, List columnDefs, Map props) { - this.name = SchemaUtil.normalizeIdentifier(name); - this.columnDefs = ImmutableList.copyOf(columnDefs); - this.props = props == null ? Collections.emptyMap() : props; - } - - public String getName() { - return name; - } - - public List getColumnDefs() { - return columnDefs; - } - - public Map getProps() { - return props; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnName.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnName.java deleted file mode 100644 index ae7804dd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnName.java +++ /dev/null @@ -1,110 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import com.salesforce.phoenix.util.SchemaUtil; - - -public class ColumnName { - private final NamedNode familyNode; - private final NamedNode columnNode; - - public static ColumnName caseSensitiveColumnName(String familyName, String columnName) { - return new ColumnName(NamedNode.caseSensitiveNamedNode(familyName), NamedNode.caseSensitiveNamedNode(columnName)); - } - - public static ColumnName caseSensitiveColumnName(String columnName) { - return new ColumnName(null, NamedNode.caseSensitiveNamedNode(columnName)); - } - - public static ColumnName newColumnName(NamedNode columnName) { - return new ColumnName(null, columnName); - } - - public static ColumnName newColumnName(NamedNode familyName, NamedNode columnName) { - return new ColumnName(familyName, columnName); - } - - private ColumnName(NamedNode familyNode, NamedNode columnNode) { - this.familyNode = familyNode; - this.columnNode = columnNode; - } - - - ColumnName(String familyName, String columnName) { - this.familyNode = familyName == null ? null : new NamedNode(familyName); - this.columnNode = new NamedNode(columnName); - } - - ColumnName(String columnName) { - this(null, columnName); - } - - public String getFamilyName() { - return familyNode == null ? null : familyNode.getName(); - } - - public String getColumnName() { - return columnNode.getName(); - } - - public NamedNode getFamilyNode() { - return familyNode; - } - - public NamedNode getColumnNode() { - return columnNode; - } - - @Override - public String toString() { - return SchemaUtil.getColumnName(getFamilyName(),getColumnName()); - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + columnNode.hashCode(); - result = prime * result + ((familyNode == null) ? 0 : familyNode.hashCode()); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - ColumnName other = (ColumnName)obj; - if (!columnNode.equals(other.columnNode)) return false; - if (familyNode == null) { - if (other.familyNode != null) return false; - } else if (!familyNode.equals(other.familyNode)) return false; - return true; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnParseNode.java deleted file mode 100644 index 88229cbb..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ColumnParseNode.java +++ /dev/null @@ -1,86 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; - -import com.salesforce.phoenix.query.QueryConstants; - -/** - * Node representing a reference to a column in a SQL expression - * - * @author jtaylor - * @since 0.1 - */ -public class ColumnParseNode extends NamedParseNode { - private final TableName tableName; - private final String fullName; - private final String alias; - - public ColumnParseNode(TableName tableName, String name, String alias) { - // Upper case here so our Maps can depend on this (and we don't have to upper case and create a string on every - // lookup - super(name); - this.alias = alias; - this.tableName = tableName; - fullName = tableName == null ? getName() : tableName.toString() + QueryConstants.NAME_SEPARATOR + getName(); - } - - public ColumnParseNode(TableName tableName, String name) { - this(tableName, name, null); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - return visitor.visit(this); - } - - public String getTableName() { - return tableName == null ? null : tableName.getTableName(); - } - - public String getSchemaName() { - return tableName == null ? null : tableName.getSchemaName(); - } - - public String getFullName() { - return fullName; - } - - @Override - public String getAlias() { - return alias; - } - - @Override - public String toString() { - return fullName; - } - - @Override - public int hashCode() { - return fullName.hashCode(); - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - ColumnParseNode other = (ColumnParseNode)obj; - return fullName.equals(other.fullName); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ComparisonParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ComparisonParseNode.java deleted file mode 100644 index c6973f72..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ComparisonParseNode.java +++ /dev/null @@ -1,67 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -import org.apache.hadoop.hbase.filter.CompareFilter; - -/** - * - * Common base class for =, >, >=, <, <=, != - * - * @author jtaylor - * @since 0.1 - */ -public abstract class ComparisonParseNode extends BinaryParseNode { - - ComparisonParseNode(ParseNode lhs, ParseNode rhs) { - super(lhs, rhs); - } - - @Override - public final T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } - - /** - * Return the comparison operator associated with the given comparison expression node - */ - public abstract CompareFilter.CompareOp getFilterOp(); - - /** - * Return the inverted operator for the CompareOp - */ - public abstract CompareFilter.CompareOp getInvertFilterOp(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CompoundParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CompoundParseNode.java deleted file mode 100644 index d82fa936..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CompoundParseNode.java +++ /dev/null @@ -1,83 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Abstract node representing an expression node that has children - * - * @author jtaylor - * @since 0.1 - */ -public abstract class CompoundParseNode extends ParseNode { - private final List children; - private final boolean isStateless; - - CompoundParseNode(List children) { - this.children = Collections.unmodifiableList(children); - boolean isStateless = true; - for (ParseNode child : children) { - isStateless &= child.isStateless(); - if (!isStateless) { - break; - } - } - this.isStateless = isStateless; - } - - @Override - public boolean isStateless() { - return isStateless; - } - - @Override - public final List getChildren() { - return children; - } - - - final List acceptChildren(ParseNodeVisitor visitor) throws SQLException { - List l = visitor.newElementList(children.size()); - for (int i = 0; i < children.size(); i++) { - T e = children.get(i).accept(visitor); - visitor.addElement(l, e); - } - return l; - } - - @Override - public String toString() { - return this.getClass().getName() + children.toString(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ConcreteTableNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ConcreteTableNode.java deleted file mode 100644 index a741745b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ConcreteTableNode.java +++ /dev/null @@ -1,51 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import com.salesforce.phoenix.util.SchemaUtil; - -/** - * - * Abstract node representing a table reference in the FROM clause in SQL - * - * @author jtaylor - * @since 0.1 - */ -public abstract class ConcreteTableNode extends TableNode { - private final TableName name; - - ConcreteTableNode(String alias, TableName name) { - super(SchemaUtil.normalizeIdentifier(alias)); - this.name = name; - } - - public TableName getName() { - return name; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CreateIndexStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CreateIndexStatement.java deleted file mode 100644 index 0e66534f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CreateIndexStatement.java +++ /dev/null @@ -1,82 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.Collections; -import java.util.List; - -import org.apache.hadoop.hbase.util.Pair; - -import com.google.common.collect.ListMultimap; - - -public class CreateIndexStatement extends SingleTableSQLStatement { - private final TableName indexTableName; - private final PrimaryKeyConstraint indexConstraint; - private final List includeColumns; - private final List splitNodes; - private final ListMultimap> props; - private final boolean ifNotExists; - - public CreateIndexStatement(NamedNode indexTableName, NamedTableNode dataTable, - PrimaryKeyConstraint indexConstraint, List includeColumns, List splits, - ListMultimap> props, boolean ifNotExists, int bindCount) { - super(dataTable, bindCount); - this.indexTableName =TableName.createNormalized(dataTable.getName().getSchemaName(),indexTableName.getName()); - this.indexConstraint = indexConstraint == null ? PrimaryKeyConstraint.EMPTY : indexConstraint; - this.includeColumns = includeColumns == null ? Collections.emptyList() : includeColumns; - this.splitNodes = splits == null ? Collections.emptyList() : splits; - this.props = props; - this.ifNotExists = ifNotExists; - } - - public PrimaryKeyConstraint getIndexConstraint() { - return indexConstraint; - } - - public List getIncludeColumns() { - return includeColumns; - } - - public TableName getIndexTableName() { - return indexTableName; - } - - public List getSplitNodes() { - return splitNodes; - } - - public ListMultimap> getProps() { - return props; - } - - public boolean ifNotExists() { - return ifNotExists; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CreateSequenceStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CreateSequenceStatement.java deleted file mode 100644 index e26604e2..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CreateSequenceStatement.java +++ /dev/null @@ -1,72 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -public class CreateSequenceStatement implements BindableStatement { - - private final TableName sequenceName; - private final ParseNode startWith; - private final ParseNode incrementBy; - private final ParseNode cacheSize; - private final boolean ifNotExists; - private final int bindCount; - - protected CreateSequenceStatement(TableName sequenceName, ParseNode startsWith, ParseNode incrementBy, ParseNode cacheSize, boolean ifNotExists, int bindCount) { - this.sequenceName = sequenceName; - this.startWith = startsWith == null ? LiteralParseNode.ONE : startsWith; - this.incrementBy = incrementBy == null ? LiteralParseNode.ONE : incrementBy; - this.cacheSize = cacheSize == null ? null : cacheSize; - this.ifNotExists = ifNotExists; - this.bindCount = bindCount; - } - - @Override - public int getBindCount() { - return this.bindCount; - } - - public ParseNode getIncrementBy() { - return incrementBy; - } - - public TableName getSequenceName() { - return sequenceName; - } - - public ParseNode getCacheSize() { - return cacheSize; - } - - public ParseNode getStartWith() { - return startWith; - } - - public boolean ifNotExists() { - return ifNotExists; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CreateTableStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CreateTableStatement.java deleted file mode 100644 index 155f6132..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CreateTableStatement.java +++ /dev/null @@ -1,108 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.Collections; -import java.util.List; - -import org.apache.hadoop.hbase.util.Pair; - -import com.google.common.collect.ImmutableList; -import com.google.common.collect.ImmutableListMultimap; -import com.google.common.collect.ListMultimap; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.schema.PTableType; - -public class CreateTableStatement implements BindableStatement { - private final TableName tableName; - private final PTableType tableType; - private final List columns; - private final PrimaryKeyConstraint pkConstraint; - private final List splitNodes; - private final int bindCount; - private final ListMultimap> props; - private final boolean ifNotExists; - private final TableName baseTableName; - private final ParseNode whereClause; - - protected CreateTableStatement(TableName tableName, ListMultimap> props, List columns, PrimaryKeyConstraint pkConstraint, - List splitNodes, PTableType tableType, boolean ifNotExists, - TableName baseTableName, ParseNode whereClause, int bindCount) { - this.tableName = tableName; - this.props = props == null ? ImmutableListMultimap.>of() : props; - this.tableType = PhoenixDatabaseMetaData.TYPE_SCHEMA.equals(tableName.getSchemaName()) ? PTableType.SYSTEM : tableType; - this.columns = columns == null ? ImmutableList.of() : ImmutableList.copyOf(columns); - this.pkConstraint = pkConstraint == null ? PrimaryKeyConstraint.EMPTY : pkConstraint; - this.splitNodes = splitNodes == null ? Collections.emptyList() : ImmutableList.copyOf(splitNodes); - this.bindCount = bindCount; - this.ifNotExists = ifNotExists; - this.baseTableName = baseTableName; - this.whereClause = whereClause; - } - - public ParseNode getWhereClause() { - return whereClause; - } - - @Override - public int getBindCount() { - return bindCount; - } - - public TableName getTableName() { - return tableName; - } - - public TableName getBaseTableName() { - return baseTableName; - } - - public List getColumnDefs() { - return columns; - } - - public List getSplitNodes() { - return splitNodes; - } - - public PTableType getTableType() { - return tableType; - } - - public ListMultimap> getProps() { - return props; - } - - public boolean ifNotExists() { - return ifNotExists; - } - - public PrimaryKeyConstraint getPrimaryKeyConstraint() { - return pkConstraint; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CurrentDateParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CurrentDateParseNode.java deleted file mode 100644 index 54e9a386..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CurrentDateParseNode.java +++ /dev/null @@ -1,49 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.CurrentDateFunction; -import com.salesforce.phoenix.expression.function.FunctionExpression; - - -public class CurrentDateParseNode extends FunctionParseNode { - - public CurrentDateParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public FunctionExpression create(List children, StatementContext context) throws SQLException { - return new CurrentDateFunction(context.getCurrentTime()); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CurrentTimeParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CurrentTimeParseNode.java deleted file mode 100644 index 3d169f76..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/CurrentTimeParseNode.java +++ /dev/null @@ -1,49 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.CurrentTimeFunction; -import com.salesforce.phoenix.expression.function.FunctionExpression; - - -public class CurrentTimeParseNode extends FunctionParseNode { - - public CurrentTimeParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public FunctionExpression create(List children, StatementContext context) throws SQLException { - return new CurrentTimeFunction(context.getCurrentTime()); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DelegateConstantToCountParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DelegateConstantToCountParseNode.java deleted file mode 100644 index 265d1215..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DelegateConstantToCountParseNode.java +++ /dev/null @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.CountAggregateFunction; -import com.salesforce.phoenix.expression.function.FunctionExpression; - - -public abstract class DelegateConstantToCountParseNode extends AggregateFunctionParseNode { - - public DelegateConstantToCountParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - protected CountAggregateFunction getDelegateFunction(List children, StatementContext context) { - CountAggregateFunction countFunc = null; - if (getChildren().get(0).isStateless()) { - countFunc = (CountAggregateFunction)context.getExpressionManager().addIfAbsent(new CountAggregateFunction(children)); - } - return countFunc; - } - - @Override - public abstract FunctionExpression create(List children, StatementContext context) throws SQLException; - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DeleteStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DeleteStatement.java deleted file mode 100644 index 317a5cfe..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DeleteStatement.java +++ /dev/null @@ -1,77 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.Collections; -import java.util.List; - -public class DeleteStatement extends SingleTableSQLStatement implements FilterableStatement { - private final ParseNode whereNode; - private final List orderBy; - private final LimitNode limit; - private final HintNode hint; - - public DeleteStatement(NamedTableNode table, HintNode hint, ParseNode whereNode, List orderBy, LimitNode limit, int bindCount) { - super(table, bindCount); - this.whereNode = whereNode; - this.orderBy = orderBy == null ? Collections.emptyList() : orderBy; - this.limit = limit; - this.hint = hint == null ? HintNode.EMPTY_HINT_NODE : hint; - } - - @Override - public ParseNode getWhere() { - return whereNode; - } - - @Override - public List getOrderBy() { - return orderBy; - } - - @Override - public LimitNode getLimit() { - return limit; - } - - @Override - public HintNode getHint() { - return hint; - } - - @Override - public boolean isDistinct() { - return false; - } - - @Override - public boolean isAggregate() { - return false; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DerivedTableNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DerivedTableNode.java deleted file mode 100644 index 5faba22a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DerivedTableNode.java +++ /dev/null @@ -1,59 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; - - - -/** - * - * Node representing a subselect in the FROM clause of SQL - * - * @author jtaylor - * @since 0.1 - */ -public class DerivedTableNode extends TableNode { - - private final SelectStatement select; - - DerivedTableNode(String alias, SelectStatement select) { - super(alias); - this.select = select; - } - - public SelectStatement getSelect() { - return select; - } - - @Override - public void accept(TableNodeVisitor visitor) throws SQLException { - visitor.visit(this); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DistinctCountParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DistinctCountParseNode.java deleted file mode 100644 index e7815e64..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DistinctCountParseNode.java +++ /dev/null @@ -1,54 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.DistinctCountAggregateFunction; -import com.salesforce.phoenix.expression.function.FunctionExpression; - -/** - * - * @author anoopsjohn - * @since 1.2.1 - */ -public class DistinctCountParseNode extends DelegateConstantToCountParseNode { - - public DistinctCountParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public FunctionExpression create(List children, StatementContext context) - throws SQLException { - return new DistinctCountAggregateFunction(children, getDelegateFunction(children, context)); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DivideParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DivideParseNode.java deleted file mode 100644 index 9270a3f8..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DivideParseNode.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing division in a SQL expression - * - * @author jtaylor - * @since 0.1 - */ -public class DivideParseNode extends ArithmeticParseNode { - - DivideParseNode(List children) { - super(children); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropColumnStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropColumnStatement.java deleted file mode 100644 index 18db5b50..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropColumnStatement.java +++ /dev/null @@ -1,51 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.List; - -import com.salesforce.phoenix.schema.PTableType; - -public class DropColumnStatement extends AlterTableStatement { - private final List columnRefs; - private final boolean ifExists; - - protected DropColumnStatement(NamedTableNode table, PTableType tableType, List columnRefs, boolean ifExists) { - super(table, tableType); - this.columnRefs = columnRefs; - this.ifExists = ifExists; - } - - public List getColumnRefs() { - return columnRefs; - } - - public boolean ifExists() { - return ifExists; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropIndexStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropIndexStatement.java deleted file mode 100644 index bae21897..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropIndexStatement.java +++ /dev/null @@ -1,58 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -public class DropIndexStatement implements BindableStatement { - private final TableName tableName; - private final NamedNode indexName; - private final boolean ifExists; - - public DropIndexStatement(NamedNode indexName, TableName tableName, boolean ifExists) { - this.indexName = indexName; - this.tableName = tableName; - this.ifExists = ifExists; - } - - public TableName getTableName() { - return tableName; - } - - public NamedNode getIndexName() { - return indexName; - } - - @Override - public int getBindCount() { - return 0; - } - - public boolean ifExists() { - return ifExists; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropSequenceStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropSequenceStatement.java deleted file mode 100644 index c1182b05..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropSequenceStatement.java +++ /dev/null @@ -1,54 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -public class DropSequenceStatement implements BindableStatement { - - private final TableName sequenceName; - private final boolean ifExists; - private final int bindCount; - - protected DropSequenceStatement(TableName sequenceName, boolean ifExists, int bindCount) { - this.sequenceName = sequenceName; - this.ifExists = ifExists; - this.bindCount = bindCount; - } - - @Override - public int getBindCount() { - return bindCount; - } - - public TableName getSequenceName() { - return sequenceName; - } - - public boolean ifExists() { - return ifExists; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropTableStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropTableStatement.java deleted file mode 100644 index 4cac741b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/DropTableStatement.java +++ /dev/null @@ -1,59 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import com.salesforce.phoenix.schema.PTableType; - -public class DropTableStatement implements BindableStatement { - private final TableName tableName; - private final boolean ifExists; - private final PTableType tableType; - - protected DropTableStatement(TableName tableName, PTableType tableType, boolean ifExists) { - this.tableName = tableName; - this.tableType = tableType; - this.ifExists = ifExists; - } - - @Override - public int getBindCount() { - return 0; // No binds for DROP - } - - public TableName getTableName() { - return tableName; - } - - public PTableType getTableType() { - return tableType; - } - - public boolean ifExists() { - return ifExists; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/EqualParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/EqualParseNode.java deleted file mode 100644 index 62a76581..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/EqualParseNode.java +++ /dev/null @@ -1,55 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; - - -/** - * - * Node representing the equal operator in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class EqualParseNode extends ComparisonParseNode { - - EqualParseNode(ParseNode lhs, ParseNode rhs) { - super(lhs, rhs); - } - - @Override - public CompareOp getFilterOp() { - return CompareOp.EQUAL; - } - - @Override - public CompareOp getInvertFilterOp() { - return CompareOp.EQUAL; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ExistsParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ExistsParseNode.java deleted file mode 100644 index 208f58f1..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ExistsParseNode.java +++ /dev/null @@ -1,63 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing EXISTS and NOT EXISTS expressions in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class ExistsParseNode extends BinaryParseNode { - private final boolean negate; - - ExistsParseNode(ParseNode l, ParseNode r, boolean negate) { - super(l, r); - this.negate = negate; - } - - public boolean isNegate() { - return negate; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ExplainStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ExplainStatement.java deleted file mode 100644 index e3dcbff5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ExplainStatement.java +++ /dev/null @@ -1,45 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -public class ExplainStatement implements BindableStatement { - private final BindableStatement statement; - - public ExplainStatement(BindableStatement statement) { - this.statement = statement; - } - - public BindableStatement getStatement() { - return statement; - } - - @Override - public int getBindCount() { - return statement.getBindCount(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/FamilyWildcardParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/FamilyWildcardParseNode.java deleted file mode 100644 index 36094efb..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/FamilyWildcardParseNode.java +++ /dev/null @@ -1,63 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ - -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; - -/** - * - * Node representing the selection of all columns of a family (cf.*) in the SELECT clause of SQL - * - * @author nmaillard - * @since 1.2 - */ - -public class FamilyWildcardParseNode extends NamedParseNode { - private final boolean isRewrite; - - public FamilyWildcardParseNode(String familyName, boolean isRewrite){ - super(familyName); - this.isRewrite = isRewrite; - } - - public FamilyWildcardParseNode(FamilyWildcardParseNode familyName, boolean isRewrite){ - super(familyName); - this.isRewrite = isRewrite; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - return visitor.visit(this); - } - - public boolean isRewrite() { - return isRewrite; - } -} - diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/FilterableStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/FilterableStatement.java deleted file mode 100644 index ff486239..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/FilterableStatement.java +++ /dev/null @@ -1,39 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.List; - -public interface FilterableStatement extends BindableStatement { - public HintNode getHint(); - public ParseNode getWhere(); - public boolean isDistinct(); - public boolean isAggregate(); - public List getOrderBy(); - public LimitNode getLimit(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/FloorParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/FloorParseNode.java deleted file mode 100644 index fe43d9a6..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/FloorParseNode.java +++ /dev/null @@ -1,89 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.FloorDateExpression; -import com.salesforce.phoenix.expression.function.FloorDecimalExpression; -import com.salesforce.phoenix.expression.function.FloorFunction; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.TypeMismatchException; - -/** - * Parse node corresponding to {@link FloorFunction}. - * It also acts as a factory for creating the right kind of - * floor expression according to the data type of the - * first child. - * - * @author samarth.jain - * @since 3.0.0 - */ -public class FloorParseNode extends FunctionParseNode { - - FloorParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public Expression create(List children, StatementContext context) throws SQLException { - return getFloorExpression(children); - } - - public static Expression getFloorExpression(List children) throws SQLException { - final Expression firstChild = children.get(0); - final PDataType firstChildDataType = firstChild.getDataType(); - - //FLOOR on timestamp doesn't really care about the nanos part i.e. it just sets it to zero. - //Which is exactly what FloorDateExpression does too. - if(firstChildDataType.isCoercibleTo(PDataType.TIMESTAMP)) { - return FloorDateExpression.create(children); - } else if(firstChildDataType.isCoercibleTo(PDataType.DECIMAL)) { - return new FloorDecimalExpression(children); - } else { - throw TypeMismatchException.newException(firstChildDataType, "1"); - } - } - - /** - * When rounding off decimals, user need not specify the scale. In such cases, - * we need to prevent the function from getting evaluated as null. This is really - * a hack. A better way would have been if {@link com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo} provided a - * way of associating default values for each permissible data type. - * Something like: @ Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValues = {"null", "1"} isConstant=true) - * Till then, this will have to do. - */ - @Override - public boolean evalToNullIfParamIsNull(StatementContext context, int index) throws SQLException { - return index == 0; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/GreaterThanOrEqualParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/GreaterThanOrEqualParseNode.java deleted file mode 100644 index 5fcb4b31..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/GreaterThanOrEqualParseNode.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import org.apache.hadoop.hbase.filter.CompareFilter; -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; - - -/** - * - * Node representing the greater than or equal to operator (>=) in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class GreaterThanOrEqualParseNode extends ComparisonParseNode { - - GreaterThanOrEqualParseNode(ParseNode lhs, ParseNode rhs) { - super(lhs, rhs); - } - - @Override - public CompareOp getFilterOp() { - return CompareFilter.CompareOp.GREATER_OR_EQUAL; - } - - - @Override - public CompareOp getInvertFilterOp() { - return CompareOp.LESS_OR_EQUAL; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/GreaterThanParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/GreaterThanParseNode.java deleted file mode 100644 index 2ec6848d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/GreaterThanParseNode.java +++ /dev/null @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import org.apache.hadoop.hbase.filter.CompareFilter; -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; - - -/** - * - * Node representing the greater than operator (>) in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class GreaterThanParseNode extends ComparisonParseNode { - - GreaterThanParseNode(ParseNode lhs, ParseNode rhs) { - super(lhs, rhs); - } - - @Override - public CompareOp getFilterOp() { - return CompareFilter.CompareOp.GREATER; - } - - @Override - public CompareOp getInvertFilterOp() { - return CompareOp.LESS; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/HintNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/HintNode.java deleted file mode 100644 index 809a1f73..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/HintNode.java +++ /dev/null @@ -1,168 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.HashMap; -import java.util.Map; - -import com.google.common.collect.ImmutableMap; -import com.salesforce.phoenix.util.SchemaUtil; - - -/** - * Node representing optimizer hints in SQL - */ -public class HintNode { - public static final HintNode EMPTY_HINT_NODE = new HintNode(); - - public static final char SEPARATOR = ' '; - public static final String PREFIX = "("; - public static final String SUFFIX = ")"; - // Split on whitespace and parenthesis, keeping the parenthesis in the token array - private static final String SPLIT_REGEXP = "\\s+|((?<=\\" + PREFIX + ")|(?=\\" + PREFIX + "))|((?<=\\" + SUFFIX + ")|(?=\\" + SUFFIX + "))"; - - public enum Hint { - /** - * Forces a range scan to be used to process the query. - */ - RANGE_SCAN, - /** - * Forces a skip scan to be used to process the query. - */ - SKIP_SCAN, - /** - * Prevents the spawning of multiple threads during - * query processing. - */ - NO_INTRA_REGION_PARALLELIZATION, - /** - * Prevents the usage of indexes, forcing usage - * of the data table for a query. - */ - NO_INDEX, - /** - * Hint of the form INDEX( ...) - * to suggest usage of the index if possible. The first - * usable index in the list of indexes will be choosen. - * Table and index names may be surrounded by double quotes - * if they are case sensitive. - */ - INDEX, - /** - * All things being equal, use the data table instead of - * the index table when optimizing. - */ - USE_DATA_OVER_INDEX_TABLE, - /** - * All things being equal, use the index table instead of - * the data table when optimizing. - */ - USE_INDEX_OVER_DATA_TABLE, - }; - - private final Map hints; - - public static HintNode create(HintNode hintNode, Hint hint) { - return create(hintNode, hint, ""); - } - - public static HintNode create(HintNode hintNode, Hint hint, String value) { - Map hints = new HashMap(hintNode.hints); - hints.put(hint, value); - return new HintNode(hints); - } - - private HintNode() { - hints = new HashMap(); - } - - private HintNode(Map hints) { - this.hints = ImmutableMap.copyOf(hints); - } - - public HintNode(String hint) { - Map hints = new HashMap(); - // Split on whitespace or parenthesis. We do not need to handle escaped or - // embedded whitespace/parenthesis, since we are parsing what will be HBase - // table names which are not allowed to contain whitespace or parenthesis. - String[] hintWords = hint.split(SPLIT_REGEXP); - for (int i = 0; i < hintWords.length; i++) { - String hintWord = hintWords[i]; - if (hintWord.isEmpty()) { - continue; - } - try { - Hint key = Hint.valueOf(hintWord.toUpperCase()); - String hintValue = ""; - if (i+1 < hintWords.length && PREFIX.equals(hintWords[i+1])) { - StringBuffer hintValueBuf = new StringBuffer(hint.length()); - hintValueBuf.append(PREFIX); - i+=2; - while (i < hintWords.length && !SUFFIX.equals(hintWords[i])) { - hintValueBuf.append(SchemaUtil.normalizeIdentifier(hintWords[i++])); - hintValueBuf.append(SEPARATOR); - } - // Replace trailing separator with suffix - hintValueBuf.replace(hintValueBuf.length()-1, hintValueBuf.length(), SUFFIX); - hintValue = hintValueBuf.toString(); - } - String oldValue = hints.put(key, hintValue); - // Concatenate together any old value with the new value - if (oldValue != null) { - hints.put(key, oldValue + hintValue); - } - } catch (IllegalArgumentException e) { // Ignore unknown/invalid hints - } - } - this.hints = ImmutableMap.copyOf(hints); - } - - public boolean isEmpty() { - return hints.isEmpty(); - } - - /** - * Gets the value of the hint or null if the hint is not present. - * @param hint the hint - * @return the value specified in parenthesis following the hint or null - * if the hint is not present. - * - */ - public String getHint(Hint hint) { - return hints.get(hint); - } - - /** - * Tests for the presence of a hint in a query - * @param hint the hint - * @return true if the hint is present and false otherwise - */ - public boolean hasHint(Hint hint) { - return hints.containsKey(hint); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/InListParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/InListParseNode.java deleted file mode 100644 index 1ba1dbc0..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/InListParseNode.java +++ /dev/null @@ -1,74 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - - - -/** - * - * Node representing the IN literal list expression in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class InListParseNode extends CompoundParseNode { - private final boolean negate; - - InListParseNode(List children, boolean negate) { - super(children); - // All values in the IN must be constant. First child is the LHS - for (int i = 1; i < children.size(); i++) { - ParseNode child = children.get(i); - if (!child.isStateless()) { - throw new ParseException(new SQLExceptionInfo.Builder(SQLExceptionCode.VALUE_IN_LIST_NOT_CONSTANT) - .build().buildException()); - } - } - this.negate = negate; - } - - public boolean isNegate() { - return negate; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/InParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/InParseNode.java deleted file mode 100644 index 29b30f5d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/InParseNode.java +++ /dev/null @@ -1,63 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing IN subquery expression in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class InParseNode extends BinaryParseNode { - private final boolean negate; - - InParseNode(ParseNode l, ParseNode r, boolean negate) { - super(l, r); - this.negate = negate; - } - - public boolean isNegate() { - return negate; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/IndexKeyConstraint.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/IndexKeyConstraint.java deleted file mode 100644 index 804a10f3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/IndexKeyConstraint.java +++ /dev/null @@ -1,47 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.List; - -import org.apache.hadoop.hbase.util.Pair; - -import com.google.common.collect.ImmutableList; -import com.salesforce.phoenix.schema.ColumnModifier; - -public class IndexKeyConstraint { - private final List> columnNameToModifier; - - IndexKeyConstraint(List> columnNameAndModifier) { - this.columnNameToModifier = ImmutableList.copyOf(columnNameAndModifier); - } - - public List> getColumns() { - return columnNameToModifier; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/IsNullParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/IsNullParseNode.java deleted file mode 100644 index 84815838..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/IsNullParseNode.java +++ /dev/null @@ -1,63 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing the IS NULL and IS NOT NULL expressions in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class IsNullParseNode extends UnaryParseNode { - private final boolean negate; - - IsNullParseNode(ParseNode expr, boolean negate) { - super(expr); - this.negate = negate; - } - - public boolean isNegate() { - return negate; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/JoinTableNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/JoinTableNode.java deleted file mode 100644 index 32834ca3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/JoinTableNode.java +++ /dev/null @@ -1,72 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; - - - -/** - * - * Node representing the join specified in the FROM clause of SQL - * - * @author jtaylor - * @since 0.1 - */ -public class JoinTableNode extends TableNode { - public enum JoinType {Inner, Left, Right, Full}; - - private final JoinType type; - private final ParseNode on; - private final TableNode table; - - JoinTableNode(JoinType type, ParseNode on, TableNode table) { - super(table.getAlias()); - this.type = type; - this.on = on; - this.table = table; - } - - public JoinType getType() { - return type; - } - - public ParseNode getOnNode() { - return on; - } - - public TableNode getTable() { - return table; - } - - @Override - public void accept(TableNodeVisitor visitor) throws SQLException { - visitor.visit(this); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LessThanOrEqualParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LessThanOrEqualParseNode.java deleted file mode 100644 index 33fa577c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LessThanOrEqualParseNode.java +++ /dev/null @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import org.apache.hadoop.hbase.filter.CompareFilter; -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; - - -/** - * - * Node representing the less than or equal to operator (<=) in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class LessThanOrEqualParseNode extends ComparisonParseNode { - - LessThanOrEqualParseNode(ParseNode lhs, ParseNode rhs) { - super(lhs, rhs); - } - - @Override - public CompareOp getFilterOp() { - return CompareFilter.CompareOp.LESS_OR_EQUAL; - } - - @Override - public CompareOp getInvertFilterOp() { - return CompareOp.GREATER_OR_EQUAL; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LessThanParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LessThanParseNode.java deleted file mode 100644 index 8e112328..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LessThanParseNode.java +++ /dev/null @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import org.apache.hadoop.hbase.filter.CompareFilter; -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; - - -/** - * - * Node representing the less than operator (<) in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class LessThanParseNode extends ComparisonParseNode { - - LessThanParseNode(ParseNode lhs, ParseNode rhs) { - super(lhs, rhs); - } - - @Override - public CompareOp getFilterOp() { - return CompareFilter.CompareOp.LESS; - } - - @Override - public CompareOp getInvertFilterOp() { - return CompareOp.GREATER; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LikeParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LikeParseNode.java deleted file mode 100644 index 200ec3ba..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LikeParseNode.java +++ /dev/null @@ -1,63 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing LIKE and NOT LIKE in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class LikeParseNode extends BinaryParseNode { - private final boolean negate; - - LikeParseNode(ParseNode lhs, ParseNode rhs, boolean negate) { - super(lhs, rhs); - this.negate = negate; - } - - public boolean isNegate() { - return negate; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LimitNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LimitNode.java deleted file mode 100644 index 00fa46a7..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LimitNode.java +++ /dev/null @@ -1,48 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - - -public class LimitNode { - private final BindParseNode bindNode; - private final LiteralParseNode limitNode; - - LimitNode(BindParseNode bindNode) { - this.bindNode = bindNode; - limitNode = null; - } - - LimitNode(LiteralParseNode limitNode) { - this.limitNode = limitNode; - this.bindNode = null; - } - - public ParseNode getLimitParseNode() { - return bindNode == null ? limitNode : bindNode; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LiteralParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LiteralParseNode.java deleted file mode 100644 index aa2648ea..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/LiteralParseNode.java +++ /dev/null @@ -1,85 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -import com.salesforce.phoenix.schema.PDataType; - - - -/** - * - * Node representing literal expressions such as 1,2.5,'foo', and NULL in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class LiteralParseNode extends TerminalParseNode { - public static final List STAR = Collections.singletonList(new LiteralParseNode(1)); - public static final ParseNode NULL = new LiteralParseNode(null); - public static final ParseNode ZERO = new LiteralParseNode(0); - public static final ParseNode ONE = new LiteralParseNode(1); - - private final Object value; - private final PDataType type; - - public LiteralParseNode(Object value) { - this.value = value; - this.type = PDataType.fromLiteral(value); - } - - public PDataType getType() { - return type; - } - - public Object getValue() { - return value; - } - - @Override - public boolean isStateless() { - return true; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - return visitor.visit(this); - } - - public byte[] getBytes() { - return type == null ? null : type.toBytes(value); - } - - @Override - public String toString() { - return type == PDataType.VARCHAR ? ("'" + value.toString() + "'") : value == null ? "null" : value.toString(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/MaxAggregateParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/MaxAggregateParseNode.java deleted file mode 100644 index 2859f121..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/MaxAggregateParseNode.java +++ /dev/null @@ -1,49 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.FunctionExpression; -import com.salesforce.phoenix.expression.function.MaxAggregateFunction; - - -public class MaxAggregateParseNode extends DelegateConstantToCountParseNode { - - public MaxAggregateParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public FunctionExpression create(List children, StatementContext context) throws SQLException { - return new MaxAggregateFunction(children, getDelegateFunction(children,context)); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/MinAggregateParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/MinAggregateParseNode.java deleted file mode 100644 index 5fc13ac1..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/MinAggregateParseNode.java +++ /dev/null @@ -1,49 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.FunctionExpression; -import com.salesforce.phoenix.expression.function.MinAggregateFunction; - - -public class MinAggregateParseNode extends DelegateConstantToCountParseNode { - - public MinAggregateParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public FunctionExpression create(List children, StatementContext context) throws SQLException { - return new MinAggregateFunction(children, getDelegateFunction(children,context)); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/MultiplyParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/MultiplyParseNode.java deleted file mode 100644 index 4d17a748..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/MultiplyParseNode.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing multiplication in a SQL expression - * - * @author jtaylor - * @since 0.1 - */ -public class MultiplyParseNode extends ArithmeticParseNode { - - MultiplyParseNode(List children) { - super(children); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NamedParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NamedParseNode.java deleted file mode 100644 index 66b5f02a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NamedParseNode.java +++ /dev/null @@ -1,61 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - - -/** - * - * Abstract node representing named nodes such as binds and column expressions in SQL - * - * @author jtaylor - * @since 0.1 - */ -public abstract class NamedParseNode extends TerminalParseNode{ - private final NamedNode namedNode; - - NamedParseNode(NamedParseNode node) { - this.namedNode = node.namedNode; - } - - NamedParseNode(String name) { - this.namedNode = new NamedNode(name); - } - - public String getName() { - return namedNode.getName(); - } - - public boolean isCaseSensitive() { - return namedNode.isCaseSensitive(); - } - - @Override - public String toString() { - return getName(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NamedTableNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NamedTableNode.java deleted file mode 100644 index b53ed1c0..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NamedTableNode.java +++ /dev/null @@ -1,61 +0,0 @@ -/* - * copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - * **************************************************************************** - */ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -import com.google.common.collect.ImmutableList; - -/** - * Node representing an explicit table reference in the FROM clause of SQL - * - * @author jtaylor - * @since 0.1 - */ -public class NamedTableNode extends ConcreteTableNode { - - private final List dynColumns; - - public static NamedTableNode create (String alias, TableName name, List dynColumns) { - return new NamedTableNode(alias, name, dynColumns); - } - - NamedTableNode(String alias, TableName name) { - super(alias, name); - dynColumns = Collections. emptyList(); - } - - NamedTableNode(String alias, TableName name, List dynColumns) { - super(alias, name); - if (dynColumns != null) { - this.dynColumns = ImmutableList.copyOf(dynColumns); - } else { - this.dynColumns = Collections. emptyList(); - } - } - - @Override - public void accept(TableNodeVisitor visitor) throws SQLException { - visitor.visit(this); - } - - public List getDynamicColumns() { - return dynColumns; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NotEqualParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NotEqualParseNode.java deleted file mode 100644 index bb71cf65..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NotEqualParseNode.java +++ /dev/null @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; - - - -/** - * - * Node representing a not equal expression (!=,<>) in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class NotEqualParseNode extends ComparisonParseNode { - - NotEqualParseNode(ParseNode lhs, ParseNode rhs) { - super(lhs, rhs); - } - - @Override - public CompareOp getFilterOp() { - return CompareOp.NOT_EQUAL; - } - - @Override - public CompareOp getInvertFilterOp() { - return CompareOp.NOT_EQUAL; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NotParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NotParseNode.java deleted file mode 100644 index f14171c3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NotParseNode.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing the NOT in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class NotParseNode extends UnaryParseNode { - - NotParseNode(ParseNode expr) { - super(expr); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/OrParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/OrParseNode.java deleted file mode 100644 index 9eab2285..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/OrParseNode.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing an OR in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class OrParseNode extends CompoundParseNode { - - OrParseNode(List children) { - super(children); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/OrderByNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/OrderByNode.java deleted file mode 100644 index 5fea466e..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/OrderByNode.java +++ /dev/null @@ -1,65 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - - -/** - * - * Node representing an ORDER BY clause (including asc/desc and nulls first/last) in SQL - * - * @author jtaylor - * @since 0.1 - */ -public final class OrderByNode { - private final ParseNode child; - private final boolean nullsLast; - private final boolean orderAscending; - - OrderByNode(ParseNode child, boolean nullsLast, boolean orderAscending) { - this.child = child; - this.nullsLast = nullsLast; - this.orderAscending = orderAscending; - } - - public boolean isNullsLast() { - return nullsLast; - } - - public boolean isAscending() { - return orderAscending; - } - - public ParseNode getNode() { - return child; - } - - @Override - public String toString() { - return child.toString() + (orderAscending ? " asc" : " desc") + " nulls " + (nullsLast ? "last" : "first"); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/OuterJoinParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/OuterJoinParseNode.java deleted file mode 100644 index 7d98d135..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/OuterJoinParseNode.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing an outer join qualifier (+) in SQL - * TODO: remove Oracle specific syntax - * - * @author jtaylor - * @since 0.1 - */ -public class OuterJoinParseNode extends UnaryParseNode{ - OuterJoinParseNode(ParseNode node) { - super(node); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseContext.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseContext.java deleted file mode 100644 index 0c43174d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseContext.java +++ /dev/null @@ -1,68 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.List; - -import com.google.common.collect.Lists; - -public class ParseContext { - private boolean isAggregate; - - public ParseContext() { - } - - public boolean isAggregate() { - return isAggregate; - } - - public void setAggregate(boolean isAggregate) { - this.isAggregate |= isAggregate; - } - - public static class Stack { - private final List stack = Lists.newArrayListWithExpectedSize(5); - - public void push(ParseContext context) { - stack.add(context); - } - - public ParseContext pop() { - return stack.remove(stack.size()-1); - } - - public ParseContext peek() { - return stack.get(stack.size()-1); - } - - public boolean isEmpty() { - return stack.isEmpty(); - } - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseException.java deleted file mode 100644 index 1d0d64d8..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseException.java +++ /dev/null @@ -1,55 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -/** - * - * RuntimeException for exceptions occurring during parsing, - * since ANTLR doesn't handle typed exceptions well. - * - * @author jtaylor - * @since 2.0 - */ -public class ParseException extends RuntimeException { - - public ParseException() { - } - - public ParseException(String msg) { - super(msg); - } - - public ParseException(Throwable t) { - super(t); - } - - public ParseException(String msg, Throwable t) { - super(msg, t); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNode.java deleted file mode 100644 index e9dc2bf9..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNode.java +++ /dev/null @@ -1,60 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - - - - -/** - * - * Abstract base class for a parse node in SQL - * - * @author jtaylor - * @since 0.1 - */ -public abstract class ParseNode { - public abstract List getChildren(); - public abstract T accept(ParseNodeVisitor visitor) throws SQLException; - - public boolean isStateless() { - return false; - } - - /** - * Allows node to override what the alias is for a given node. - * Useful for a column reference, as JDBC says that the alias - * name for "a.b" should be "b" - * @return the alias to use for this node or null for no alias - */ - public String getAlias() { - return null; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNodeVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNodeVisitor.java deleted file mode 100644 index 27a58b94..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNodeVisitor.java +++ /dev/null @@ -1,116 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - - - -/** - * - * Visitor for ParseNode in the node tree. Uses composite - * visitor pattern with enter/leave calls for any - * compound expression node. Only supported SQL constructs - * have visit methods. Unsupported constructs fall through - * to {@link #visitEnter(CompoundParseNode)} for - * compound parse nodes and {@link #visit(ParseNode)} - * for terminal parse nodes. - * - * @author jtaylor - * @since 0.1 - */ -public interface ParseNodeVisitor { - public List newElementList(int size); - public void addElement(List a, E element); - - public boolean visitEnter(LikeParseNode node) throws SQLException; - public E visitLeave(LikeParseNode node, List l) throws SQLException; - - public boolean visitEnter(AndParseNode node) throws SQLException; - public E visitLeave(AndParseNode node, List l) throws SQLException; - - public boolean visitEnter(OrParseNode node) throws SQLException; - public E visitLeave(OrParseNode node, List l) throws SQLException; - - public boolean visitEnter(FunctionParseNode node) throws SQLException; - public E visitLeave(FunctionParseNode node, List l) throws SQLException; - - public boolean visitEnter(ComparisonParseNode node) throws SQLException; - public E visitLeave(ComparisonParseNode node, List l) throws SQLException; - - public boolean visitEnter(CaseParseNode node) throws SQLException; - public E visitLeave(CaseParseNode node, List l) throws SQLException; - - public boolean visitEnter(CompoundParseNode node) throws SQLException; - public E visitLeave(CompoundParseNode node, List l) throws SQLException; - - public boolean visitEnter(AddParseNode node) throws SQLException; - public E visitLeave(AddParseNode node, List l) throws SQLException; - - public boolean visitEnter(MultiplyParseNode node) throws SQLException; - public E visitLeave(MultiplyParseNode node, List l) throws SQLException; - - public boolean visitEnter(DivideParseNode node) throws SQLException; - public E visitLeave(DivideParseNode node, List l) throws SQLException; - - public boolean visitEnter(SubtractParseNode node) throws SQLException; - public E visitLeave(SubtractParseNode node, List l) throws SQLException; - - public boolean visitEnter(NotParseNode node) throws SQLException; - public E visitLeave(NotParseNode node, List l) throws SQLException; - - public boolean visitEnter(InListParseNode node) throws SQLException; - public E visitLeave(InListParseNode node, List l) throws SQLException; - - public boolean visitEnter(IsNullParseNode node) throws SQLException; - public E visitLeave(IsNullParseNode node, List l) throws SQLException; - - public E visit(ColumnParseNode node) throws SQLException; - public E visit(LiteralParseNode node) throws SQLException; - public E visit(BindParseNode node) throws SQLException; - public E visit(WildcardParseNode node) throws SQLException; - public E visit(FamilyWildcardParseNode node) throws SQLException; - public E visit(ParseNode node) throws SQLException; - - public boolean visitEnter(StringConcatParseNode node) throws SQLException; - public E visitLeave(StringConcatParseNode node, List l) throws SQLException; - - public boolean visitEnter(BetweenParseNode node) throws SQLException; - public E visitLeave(BetweenParseNode node, List l) throws SQLException; - - public boolean visitEnter(CastParseNode node) throws SQLException; - public E visitLeave(CastParseNode node, List l) throws SQLException; - - public boolean visitEnter(RowValueConstructorParseNode node) throws SQLException; - public E visitLeave(RowValueConstructorParseNode node, List l) throws SQLException; - - public boolean visitEnter(ArrayConstructorNode node) throws SQLException; - public E visitLeave(ArrayConstructorNode node, List l) throws SQLException; - public E visit(SequenceValueParseNode node) throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/PrimaryKeyConstraint.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/PrimaryKeyConstraint.java deleted file mode 100644 index bd1a8063..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/PrimaryKeyConstraint.java +++ /dev/null @@ -1,77 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.Collections; -import java.util.HashMap; -import java.util.List; - -import org.apache.hadoop.hbase.util.Pair; - -import com.google.common.collect.ImmutableList; -import com.google.common.collect.Maps; -import com.salesforce.phoenix.schema.ColumnModifier; - -public class PrimaryKeyConstraint extends NamedNode { - public static final PrimaryKeyConstraint EMPTY = new PrimaryKeyConstraint(null, Collections.>emptyList()); - - private final List> columns; - private final HashMap> columnNameToModifier; - - PrimaryKeyConstraint(String name, List> columns) { - super(name); - this.columns = columns == null ? Collections.>emptyList() : ImmutableList.copyOf(columns); - this.columnNameToModifier = Maps.newHashMapWithExpectedSize(this.columns.size()); - for (Pair p : this.columns) { - this.columnNameToModifier.put(p.getFirst(), p); - } - } - - public List> getColumnNames() { - return columns; - } - - public Pair getColumn(ColumnName columnName) { - return columnNameToModifier.get(columnName); - } - - public boolean contains(ColumnName columnName) { - return columnNameToModifier.containsKey(columnName); - } - - @Override - public int hashCode() { - return super.hashCode(); - } - - @Override - public boolean equals(Object obj) { - return super.equals(obj); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/RoundParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/RoundParseNode.java deleted file mode 100644 index df692e2d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/RoundParseNode.java +++ /dev/null @@ -1,91 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.RoundDateExpression; -import com.salesforce.phoenix.expression.function.RoundDecimalExpression; -import com.salesforce.phoenix.expression.function.RoundFunction; -import com.salesforce.phoenix.expression.function.RoundTimestampExpression; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.TypeMismatchException; - -/** - * - * Parse node corresponding to {@link RoundFunction}. - * It also acts as a factory for creating the right kind of - * round expression according to the data type of the - * first child. - * - * @author samarth.jain - * @since 3.0.0 - */ -public class RoundParseNode extends FunctionParseNode { - - RoundParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public Expression create(List children, StatementContext context) throws SQLException { - return getRoundExpression(children); - } - - public static Expression getRoundExpression(List children) throws SQLException { - final Expression firstChild = children.get(0); - final PDataType firstChildDataType = firstChild.getDataType(); - - if(firstChildDataType.isCoercibleTo(PDataType.DATE)) { - return RoundDateExpression.create(children); // FIXME: remove cast - } else if (firstChildDataType.isCoercibleTo(PDataType.TIMESTAMP)) { - return RoundTimestampExpression.create(children); // FIXME: remove cast - } else if(firstChildDataType.isCoercibleTo(PDataType.DECIMAL)) { - return new RoundDecimalExpression(children); - } else { - throw TypeMismatchException.newException(firstChildDataType, "1"); - } - } - - /** - * When rounding off decimals, user need not specify the scale. In such cases, - * we need to prevent the function from getting evaluated as null. This is really - * a hack. A better way would have been if {@link com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo} provided a - * way of associating default values for each permissible data type. - * Something like: @ Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValues = {"null", "1"} isConstant=true) - * Till then, this will have to do. - */ - @Override - public boolean evalToNullIfParamIsNull(StatementContext context, int index) throws SQLException { - return index == 0; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/RowValueConstructorParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/RowValueConstructorParseNode.java deleted file mode 100644 index f616466a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/RowValueConstructorParseNode.java +++ /dev/null @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - -/** - * - * Node representing a row value constructor in SQL. - * - * @author samarth.jain - * @since 0.1 - */ -public class RowValueConstructorParseNode extends CompoundParseNode { - - public RowValueConstructorParseNode(List l) { - super(l); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SelectStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SelectStatement.java deleted file mode 100644 index 3725014a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SelectStatement.java +++ /dev/null @@ -1,185 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.Arrays; -import java.util.Collections; -import java.util.List; - -import com.salesforce.phoenix.expression.function.CountAggregateFunction; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo; - -/** - * - * Top level node representing a SQL statement - * - * @author jtaylor - * @since 0.1 - */ -public class SelectStatement implements FilterableStatement { - public static final SelectStatement SELECT_ONE = - new SelectStatement( - Collections.emptyList(), null, false, - Collections.singletonList(new AliasedNode(null,new LiteralParseNode(1))), - null, Collections.emptyList(), - null, Collections.emptyList(), - null, 0, false); - public static final SelectStatement COUNT_ONE = - new SelectStatement( - Collections.emptyList(), null, false, - Collections.singletonList( - new AliasedNode(null, - new AggregateFunctionParseNode( - CountAggregateFunction.NORMALIZED_NAME, - LiteralParseNode.STAR, - new BuiltInFunctionInfo(CountAggregateFunction.class, CountAggregateFunction.class.getAnnotation(BuiltInFunction.class))))), - null, Collections.emptyList(), - null, Collections.emptyList(), - null, 0, true); - public static SelectStatement create(SelectStatement select, HintNode hint) { - if (select.getHint() == hint || hint.isEmpty()) { - return select; - } - return new SelectStatement(select.getFrom(), hint, select.isDistinct(), - select.getSelect(), select.getWhere(), select.getGroupBy(), select.getHaving(), - select.getOrderBy(), select.getLimit(), select.getBindCount(), select.isAggregate()); - } - public static SelectStatement create(SelectStatement select, ParseNode where) { - if (where == null) { - return select; - } - if (select.getWhere() != null) { - where = new AndParseNode(Arrays.asList(select.getWhere(), where)); - } - return new SelectStatement(select.getFrom(), select.getHint(), select.isDistinct(), - select.getSelect(), where, select.getGroupBy(), select.getHaving(), - select.getOrderBy(), select.getLimit(), select.getBindCount(), select.isAggregate()); - } - public static SelectStatement create(SelectStatement select, List selects) { - return new SelectStatement(select.getFrom(), select.getHint(), select.isDistinct(), - selects, select.getWhere(), select.getGroupBy(), select.getHaving(), - select.getOrderBy(), select.getLimit(), select.getBindCount(), select.isAggregate()); - } - - private final List fromTable; - private final HintNode hint; - private final boolean isDistinct; - private final List select; - private final ParseNode where; - private final List groupBy; - private final ParseNode having; - private final List orderBy; - private final LimitNode limit; - private final int bindCount; - private final boolean isAggregate; - - // Count constant expressions - private static int countConstants(List nodes) { - int count = 0; - for (int i = 0; i < nodes.size(); i++) { - if (nodes.get(i).isStateless()) { - count++; - } - } - return count; - } - - protected SelectStatement(List from, HintNode hint, boolean isDistinct, List select, ParseNode where, List groupBy, ParseNode having, List orderBy, LimitNode limit, int bindCount, boolean isAggregate) { - this.fromTable = Collections.unmodifiableList(from); - this.hint = hint == null ? HintNode.EMPTY_HINT_NODE : hint; - this.isDistinct = isDistinct; - this.select = Collections.unmodifiableList(select); - this.where = where; - this.groupBy = Collections.unmodifiableList(groupBy); - this.having = having; - this.orderBy = Collections.unmodifiableList(orderBy); - this.limit = limit; - this.bindCount = bindCount; - this.isAggregate = isAggregate || groupBy.size() != countConstants(groupBy) || this.having != null; - } - - @Override - public boolean isDistinct() { - return isDistinct; - } - - @Override - public LimitNode getLimit() { - return limit; - } - - @Override - public int getBindCount() { - return bindCount; - } - - public List getFrom() { - return fromTable; - } - - @Override - public HintNode getHint() { - return hint; - } - - public List getSelect() { - return select; - } - /** - * Gets the where condition, or null if none. - */ - @Override - public ParseNode getWhere() { - return where; - } - - /** - * Gets the group-by, containing at least 1 element, or null, if none. - */ - public List getGroupBy() { - return groupBy; - } - - public ParseNode getHaving() { - return having; - } - - /** - * Gets the order-by, containing at least 1 element, or null, if none. - */ - @Override - public List getOrderBy() { - return orderBy; - } - - @Override - public boolean isAggregate() { - return isAggregate; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SingleTableSQLStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SingleTableSQLStatement.java deleted file mode 100644 index 381b16ba..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SingleTableSQLStatement.java +++ /dev/null @@ -1,47 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -public abstract class SingleTableSQLStatement implements BindableStatement { - private final NamedTableNode table; - private final int bindCount; - - public SingleTableSQLStatement(NamedTableNode table, int bindCount) { - this.table = table; - this.bindCount = bindCount; - } - - public NamedTableNode getTable() { - return table; - } - - @Override - public int getBindCount() { - return bindCount; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/StringConcatParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/StringConcatParseNode.java deleted file mode 100644 index 5308da10..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/StringConcatParseNode.java +++ /dev/null @@ -1,59 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - - -/** - * - * Node representing || String concatenation in a SQL expression - * - * @author kmahadik - * @since 0.1 - */ -public class StringConcatParseNode extends CompoundParseNode { - - StringConcatParseNode(List children) { - super(children); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SubqueryParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SubqueryParseNode.java deleted file mode 100644 index 12c14ad0..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SubqueryParseNode.java +++ /dev/null @@ -1,57 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; - - - -/** - * - * Node representing a subquery in SQL - * - * @author jtaylor - * @since 0.1 - */ -public class SubqueryParseNode extends TerminalParseNode { - private final SelectStatement select; - - SubqueryParseNode(SelectStatement select) { - this.select = select; - } - - public SelectStatement getSelectNode() { - return select; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - return visitor.visit(this); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SubtractParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SubtractParseNode.java deleted file mode 100644 index 8a66dec9..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SubtractParseNode.java +++ /dev/null @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.Collections; -import java.util.List; - - - -/** - * - * Node representing subtraction in a SQL expression - * - * @author jtaylor - * @since 0.1 - */ -public class SubtractParseNode extends ArithmeticParseNode { - SubtractParseNode(List children) { - super(children); - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - List l = Collections.emptyList(); - if (visitor.visitEnter(this)) { - l = acceptChildren(visitor); - } - return visitor.visitLeave(this, l); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SumAggregateParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SumAggregateParseNode.java deleted file mode 100644 index d653d22d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SumAggregateParseNode.java +++ /dev/null @@ -1,49 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.function.FunctionExpression; -import com.salesforce.phoenix.expression.function.SumAggregateFunction; - - -public class SumAggregateParseNode extends DelegateConstantToCountParseNode { - - public SumAggregateParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public FunctionExpression create(List children, StatementContext context) throws SQLException { - return new SumAggregateFunction(children, getDelegateFunction(children,context)); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TableName.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TableName.java deleted file mode 100644 index 67c05a95..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TableName.java +++ /dev/null @@ -1,97 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.util.SchemaUtil; - -public class TableName { - private final String tableName; - private final String schemaName; - - public static TableName createNormalized(String schemaName, String tableName) { - schemaName = schemaName == null ? null : SchemaUtil.normalizeIdentifier(schemaName); - tableName = SchemaUtil.normalizeIdentifier(tableName); - return new TableName(schemaName, tableName); - } - - public static TableName create(String schemaName, String tableName) { - return new TableName(schemaName,tableName); - } - - private TableName(String schemaName, String tableName) { - this.schemaName = schemaName; - this.tableName = tableName; - } - - public String getTableName() { - return tableName; - } - - public String getSchemaName() { - return schemaName; - } - - @Override - public String toString() { - return (schemaName == null ? "" : schemaName + QueryConstants.NAME_SEPARATOR) + tableName; - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result - + ((schemaName == null) ? 0 : schemaName.hashCode()); - result = prime * result - + ((tableName == null) ? 0 : tableName.hashCode()); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) - return true; - if (obj == null) - return false; - if (getClass() != obj.getClass()) - return false; - TableName other = (TableName) obj; - if (schemaName == null) { - if (other.schemaName != null) - return false; - } else if (!schemaName.equals(other.schemaName)) - return false; - if (tableName == null) { - if (other.tableName != null) - return false; - } else if (!tableName.equals(other.tableName)) - return false; - return true; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TableNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TableNode.java deleted file mode 100644 index f38a7e29..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TableNode.java +++ /dev/null @@ -1,53 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; - - - -/** - * - * Abstract base class for FROM clause data sources - * - * @author jtaylor - * @since 0.1 - */ -public abstract class TableNode { - private final String alias; - - TableNode(String alias) { - this.alias = alias; - } - - public String getAlias() { - return alias; - } - - public abstract void accept(TableNodeVisitor visitor) throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TableNodeVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TableNodeVisitor.java deleted file mode 100644 index 5fb616a0..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TableNodeVisitor.java +++ /dev/null @@ -1,45 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; - - -/** - * - * Visitor for nodes in the FROM clause - * - * @author jtaylor - * @since 0.1 - */ -public interface TableNodeVisitor { - void visit(BindTableNode boundTableNode) throws SQLException; - void visit(JoinTableNode joinNode) throws SQLException; - void visit(NamedTableNode namedTableNode) throws SQLException; - void visit(DerivedTableNode subselectNode) throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TerminalParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TerminalParseNode.java deleted file mode 100644 index 84faae19..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TerminalParseNode.java +++ /dev/null @@ -1,45 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.Collections; -import java.util.List; - -/** - * - * Abstract node for expressions that have no children - * - * @author jtaylor - * @since 0.1 - */ -public abstract class TerminalParseNode extends ParseNode { - @Override - public final List getChildren() { - return Collections.emptyList(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ToCharParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ToCharParseNode.java deleted file mode 100644 index bffb96bc..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ToCharParseNode.java +++ /dev/null @@ -1,75 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.text.Format; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.function.FunctionArgumentType; -import com.salesforce.phoenix.expression.function.FunctionExpression; -import com.salesforce.phoenix.expression.function.ToCharFunction; -import com.salesforce.phoenix.schema.PDataType; - - -public class ToCharParseNode extends FunctionParseNode { - - public ToCharParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public FunctionExpression create(List children, StatementContext context) throws SQLException { - PDataType dataType = children.get(0).getDataType(); - String formatString = (String)((LiteralExpression)children.get(1)).getValue(); // either date or number format string - Format formatter; - FunctionArgumentType type; - if (dataType.isCoercibleTo(PDataType.TIMESTAMP)) { - if (formatString == null) { - formatString = context.getDateFormat(); - formatter = context.getDateFormatter(); - } else { - formatter = FunctionArgumentType.TEMPORAL.getFormatter(formatString); - } - type = FunctionArgumentType.TEMPORAL; - } - else if (dataType.isCoercibleTo(PDataType.DECIMAL)) { - if (formatString == null) - formatString = context.getNumberFormat(); - formatter = FunctionArgumentType.NUMERIC.getFormatter(formatString); - type = FunctionArgumentType.NUMERIC; - } - else { - throw new SQLException(dataType + " type is unsupported for TO_CHAR(). Numeric and temporal types are supported."); - } - return new ToCharFunction(children, type, formatString, formatter); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ToDateParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ToDateParseNode.java deleted file mode 100644 index e5040dc6..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ToDateParseNode.java +++ /dev/null @@ -1,59 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.text.Format; -import java.util.List; - -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.function.FunctionExpression; -import com.salesforce.phoenix.expression.function.ToDateFunction; -import com.salesforce.phoenix.util.DateUtil; - - -public class ToDateParseNode extends FunctionParseNode { - public ToDateParseNode(String name, List children, BuiltInFunctionInfo info) { - super(name, children, info); - } - - @Override - public FunctionExpression create(List children, StatementContext context) throws SQLException { - Format dateParser; - String dateFormat = (String)((LiteralExpression)children.get(1)).getValue(); - if (dateFormat == null) { - dateFormat = context.getDateFormat(); - dateParser = context.getDateParser(); - } else { - dateParser = DateUtil.getDateParser(dateFormat); - } - return new ToDateFunction(children, dateFormat, dateParser); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TraverseAllParseNodeVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TraverseAllParseNodeVisitor.java deleted file mode 100644 index 2615324a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TraverseAllParseNodeVisitor.java +++ /dev/null @@ -1,166 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - - -/** - * - * Visitor that traverses into all parse nodes - * - * @author jtaylor - * @since 0.1 - */ -public abstract class TraverseAllParseNodeVisitor extends BaseParseNodeVisitor { - @Override - public boolean visitEnter(AndParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(OrParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(FunctionParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(CaseParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(ComparisonParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(LikeParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(NotParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(CastParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(InListParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(IsNullParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(MultiplyParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(SubtractParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(AddParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(DivideParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(BetweenParseNode node) throws SQLException { - return true; - } - - @Override - public T visit(ColumnParseNode node) throws SQLException { - return null; - } - - @Override - public T visitLeave(IsNullParseNode node, List l) throws SQLException { - return null; - } - - @Override - public T visit(LiteralParseNode node) throws SQLException { - return null; - } - - @Override - public T visit(BindParseNode node) throws SQLException { - return null; - } - - @Override - public T visit(WildcardParseNode node) throws SQLException { - return null; - } - - @Override - public T visit(FamilyWildcardParseNode node) throws SQLException { - return null; - } - - @Override - public boolean visitEnter(StringConcatParseNode node) throws SQLException { - return true; - } - - @Override - public boolean visitEnter(RowValueConstructorParseNode node) throws SQLException { - return true; - } - - @Override - public T visit(SequenceValueParseNode node) throws SQLException { - return null; - } - - @Override - public boolean visitEnter(ArrayConstructorNode node) throws SQLException { - return true; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TraverseNoParseNodeVisitor.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TraverseNoParseNodeVisitor.java deleted file mode 100644 index 34983c68..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/TraverseNoParseNodeVisitor.java +++ /dev/null @@ -1,250 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; -import java.util.List; - - -/** - * - * Visitor that traverses into no parse nodes - * - * @author jtaylor - * @since 0.1 - */ -public abstract class TraverseNoParseNodeVisitor extends BaseParseNodeVisitor { - @Override - public boolean visitEnter(AndParseNode node) throws SQLException { - return false; - } - - @Override - public boolean visitEnter(OrParseNode node) throws SQLException { - return false; - } - - @Override - public boolean visitEnter(FunctionParseNode node) throws SQLException { - return false; - } - - @Override - public boolean visitEnter(ComparisonParseNode node) throws SQLException { - return false; - } - - @Override - public boolean visitEnter(CaseParseNode node) throws SQLException { - return false; - } - - @Override - public boolean visitEnter(LikeParseNode node) throws SQLException { - return false; - } - - @Override - public boolean visitEnter(BetweenParseNode node) throws SQLException { - return false; - } - - @Override - public T visitLeave(LikeParseNode node, List l) throws SQLException { - return null; - } - - @Override - public boolean visitEnter(NotParseNode node) throws SQLException { - return false; - } - - @Override - public boolean visitEnter(CastParseNode node) throws SQLException { - return false; - } - - @Override - public T visitLeave(NotParseNode node, List l) throws SQLException { - return null; - } - - @Override - public T visitLeave(CastParseNode node, List l) throws SQLException { - return null; - } - - @Override - public boolean visitEnter(InListParseNode node) throws SQLException { - return false; - } - - @Override - public T visitLeave(InListParseNode node, List l) throws SQLException { - return null; - } - - @Override - public boolean visitEnter(IsNullParseNode node) throws SQLException { - return false; - } - - @Override - public T visitLeave(IsNullParseNode node, List l) throws SQLException { - return null; - } - - @Override - public T visit(ColumnParseNode node) throws SQLException { - return null; - } - - @Override - public T visit(LiteralParseNode node) throws SQLException { - return null; - } - - @Override - public T visit(BindParseNode node) throws SQLException { - return null; - } - - @Override - public T visit(WildcardParseNode node) throws SQLException { - return null; - } - - @Override - public T visit(FamilyWildcardParseNode node) throws SQLException { - return null; - } - - @Override - public T visitLeave(AndParseNode node, List l) throws SQLException { - return null; - } - - @Override - public T visitLeave(OrParseNode node, List l) throws SQLException { - return null; - } - - @Override - public T visitLeave(FunctionParseNode node, List l) throws SQLException { - return null; - } - - @Override - public T visitLeave(ComparisonParseNode node, List l) throws SQLException { - return null; - } - - @Override - public T visitLeave(CaseParseNode node, List l) throws SQLException { - return null; - } - - @Override - public boolean visitEnter(MultiplyParseNode node) throws SQLException { - return false; - } - - @Override - public T visitLeave(MultiplyParseNode node, List l) throws SQLException { - return null; - } - - @Override - public boolean visitEnter(SubtractParseNode node) throws SQLException { - return false; - } - - @Override - public T visitLeave(SubtractParseNode node, List l) throws SQLException { - return null; - } - - @Override - public boolean visitEnter(AddParseNode node) throws SQLException { - return false; - } - - @Override - public T visitLeave(AddParseNode node, List l) throws SQLException { - return null; - } - - @Override - public boolean visitEnter(DivideParseNode node) throws SQLException { - return false; - } - - @Override - public T visitLeave(DivideParseNode node, List l) throws SQLException { - return null; - } - @Override - public boolean visitEnter(StringConcatParseNode node) throws SQLException { - return false; - } - - @Override - public T visitLeave(StringConcatParseNode node, List l) throws SQLException { - return null; - } - - @Override - public T visitLeave(BetweenParseNode node, List l) throws SQLException { - return null; - } - - @Override - public boolean visitEnter(RowValueConstructorParseNode node) throws SQLException { - return false; - } - - @Override - public T visitLeave(RowValueConstructorParseNode node, List l) throws SQLException { - return null; - } - - @Override - public T visit(SequenceValueParseNode node) throws SQLException { - return null; - } - - @Override - public boolean visitEnter(ArrayConstructorNode node) throws SQLException { - return false; - } - - @Override - public T visitLeave(ArrayConstructorNode node, List l) throws SQLException { - return null; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/UnaryParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/UnaryParseNode.java deleted file mode 100644 index 124ae4f3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/UnaryParseNode.java +++ /dev/null @@ -1,43 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.Collections; - -/** - * - * Abstract node representing an expression that has a single child in SQL - * - * @author jtaylor - * @since 0.1 - */ -public abstract class UnaryParseNode extends CompoundParseNode { - UnaryParseNode(ParseNode expr) { - super(Collections.singletonList(expr)); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/UpsertStatement.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/UpsertStatement.java deleted file mode 100644 index a7479d3d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/UpsertStatement.java +++ /dev/null @@ -1,62 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.util.Collections; -import java.util.List; - -public class UpsertStatement extends SingleTableSQLStatement { - private final List columns; - private final List values; - private final SelectStatement select; - private final HintNode hint; - - public UpsertStatement(NamedTableNode table, HintNode hint, List columns, List values, SelectStatement select, int bindCount) { - super(table, bindCount); - this.columns = columns == null ? Collections.emptyList() : columns; - this.values = values; - this.select = select; - this.hint = hint == null ? HintNode.EMPTY_HINT_NODE : hint; - } - - public List getColumns() { - return columns; - } - - public List getValues() { - return values; - } - - public SelectStatement getSelect() { - return select; - } - - public HintNode getHint() { - return hint; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/WildcardParseNode.java b/phoenix-core/src/main/java/com/salesforce/phoenix/parse/WildcardParseNode.java deleted file mode 100644 index dc165079..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/WildcardParseNode.java +++ /dev/null @@ -1,64 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; - -import java.sql.SQLException; - - - -/** - * - * Node representing the selection of all columns (*) in the SELECT clause of SQL - * - * @author jtaylor - * @since 0.1 - */ -public class WildcardParseNode extends TerminalParseNode { - private final boolean isRewrite; - public static final WildcardParseNode INSTANCE = new WildcardParseNode(false); - public static final WildcardParseNode REWRITE_INSTANCE = new WildcardParseNode(true); - - private WildcardParseNode(boolean isRewrite) { - this.isRewrite = isRewrite; - } - - @Override - public T accept(ParseNodeVisitor visitor) throws SQLException { - return visitor.visit(this); - } - - @Override - public String toString() { - return "*"; - } - - public boolean isRewrite() { - return isRewrite; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/BaseQueryServicesImpl.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/BaseQueryServicesImpl.java deleted file mode 100644 index 2d715af5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/BaseQueryServicesImpl.java +++ /dev/null @@ -1,88 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import java.util.concurrent.ExecutorService; - -import com.salesforce.phoenix.job.JobManager; -import com.salesforce.phoenix.memory.GlobalMemoryManager; -import com.salesforce.phoenix.memory.MemoryManager; -import com.salesforce.phoenix.optimize.QueryOptimizer; -import com.salesforce.phoenix.util.ReadOnlyProps; - - - -/** - * - * Base class for QueryService implementors. - * - * @author jtaylor - * @since 0.1 - */ -public abstract class BaseQueryServicesImpl implements QueryServices { - private final ExecutorService executor; - private final MemoryManager memoryManager; - private final ReadOnlyProps props; - private final QueryOptimizer queryOptimizer; - - public BaseQueryServicesImpl(QueryServicesOptions options) { - this.executor = JobManager.createThreadPoolExec( - options.getKeepAliveMs(), - options.getThreadPoolSize(), - options.getQueueSize()); - this.memoryManager = new GlobalMemoryManager( - Runtime.getRuntime().totalMemory() * options.getMaxMemoryPerc() / 100, - options.getMaxMemoryWaitMs()); - this.props = options.getProps(); - this.queryOptimizer = new QueryOptimizer(this); - } - - @Override - public ExecutorService getExecutor() { - return executor; - } - - @Override - public MemoryManager getMemoryManager() { - return memoryManager; - } - - @Override - public final ReadOnlyProps getProps() { - return props; - } - - @Override - public void close() { - } - - @Override - public QueryOptimizer getOptimizer() { - return queryOptimizer; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/ChildQueryServices.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/ChildQueryServices.java deleted file mode 100644 index 67950bb3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/ChildQueryServices.java +++ /dev/null @@ -1,55 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import com.salesforce.phoenix.memory.ChildMemoryManager; -import com.salesforce.phoenix.memory.MemoryManager; - -/** - * - * Child QueryServices that delegates through to global QueryService. - * Used to track memory used by each org to allow a max percentage threshold. - * - * @author jtaylor - * @since 0.1 - */ -public class ChildQueryServices extends DelegateConnectionQueryServices { - private final MemoryManager memoryManager; - private static final int DEFAULT_MAX_ORG_MEMORY_PERC = 30; - - public ChildQueryServices(ConnectionQueryServices services) { - super(services); - int maxOrgMemPerc = getProps().getInt(MAX_TENANT_MEMORY_PERC_ATTRIB, DEFAULT_MAX_ORG_MEMORY_PERC); - this.memoryManager = new ChildMemoryManager(services.getMemoryManager(), maxOrgMemPerc); - } - - @Override - public MemoryManager getMemoryManager() { - return memoryManager; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/ConfigurationFactory.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/ConfigurationFactory.java deleted file mode 100644 index bd95fadd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/ConfigurationFactory.java +++ /dev/null @@ -1,54 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.HBaseConfiguration; - -/** - * Creates {@link Configuration} instances that contain HBase/Hadoop settings. - * - * @author aaraujo - * @since 2.0 - */ -public interface ConfigurationFactory { - /** - * @return Configuration containing HBase/Hadoop settings - */ - Configuration getConfiguration(); - - /** - * Default implementation uses {@link org.apache.hadoop.hbase.HBaseConfiguration#create()}. - */ - static class ConfigurationFactoryImpl implements ConfigurationFactory { - @Override - public Configuration getConfiguration() { - return HBaseConfiguration.create(); - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/ConnectionQueryServices.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/ConnectionQueryServices.java deleted file mode 100644 index b992e946..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/ConnectionQueryServices.java +++ /dev/null @@ -1,108 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import java.sql.SQLException; -import java.util.List; -import java.util.Map; -import java.util.Properties; - -import org.apache.hadoop.hbase.HRegionLocation; -import org.apache.hadoop.hbase.HTableDescriptor; -import org.apache.hadoop.hbase.client.HBaseAdmin; -import org.apache.hadoop.hbase.client.HTableInterface; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Pair; - -import com.salesforce.phoenix.client.KeyValueBuilder; -import com.salesforce.phoenix.compile.MutationPlan; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.schema.SequenceKey; -import com.salesforce.phoenix.schema.PTableType; - - -public interface ConnectionQueryServices extends QueryServices, MetaDataMutated { - /** - * Get (and create if necessary) a child QueryService for a given tenantId. - * The QueryService will be cached for the lifetime of the parent QueryService - * @param tenantId the organization ID - * @return the child QueryService - */ - public ConnectionQueryServices getChildQueryServices(ImmutableBytesWritable tenantId); - - /** - * Get an HTableInterface by the given name. It is the callers - * responsibility to close the returned HTableInterface. - * @param tableName the name of the HTable - * @return the HTableInterface - * @throws SQLException - */ - public HTableInterface getTable(byte[] tableName) throws SQLException; - - public HTableDescriptor getTableDescriptor(byte[] tableName) throws SQLException; - - public StatsManager getStatsManager(); - - public List getAllTableRegions(byte[] tableName) throws SQLException; - - public PhoenixConnection connect(String url, Properties info) throws SQLException; - - public MetaDataMutationResult getTable(byte[] tenantId, byte[] schemaName, byte[] tableName, long tableTimestamp, long clientTimetamp) throws SQLException; - public MetaDataMutationResult createTable(List tableMetaData, byte[] tableName, PTableType tableType, Map tableProps, List>> families, byte[][] splits) throws SQLException; - public MetaDataMutationResult dropTable(List tableMetadata, PTableType tableType) throws SQLException; - public MetaDataMutationResult addColumn(List tableMetaData, PTableType tableType, List>> families) throws SQLException; - public MetaDataMutationResult dropColumn(List tableMetadata, PTableType tableType) throws SQLException; - public MetaDataMutationResult updateIndexState(List tableMetadata, String parentTableName) throws SQLException; - public MutationState updateData(MutationPlan plan) throws SQLException; - - public void init(String url, Properties props) throws SQLException; - - public int getLowestClusterHBaseVersion(); - public HBaseAdmin getAdmin() throws SQLException; - - void clearTableRegionCache(byte[] tableName) throws SQLException; - - boolean hasInvalidIndexConfiguration(); - - long createSequence(String tenantId, String schemaName, String sequenceName, long startWith, long incrementBy, int batchSize, long timestamp) throws SQLException; - long dropSequence(String tenantId, String schemaName, String sequenceName, long timestamp) throws SQLException; - void reserveSequenceValues(List sequenceKeys, long timestamp, long[] values, SQLException[] exceptions) throws SQLException; - void incrementSequenceValues(List sequenceKeys, long timestamp, long[] values, SQLException[] exceptions) throws SQLException; - long getSequenceValue(SequenceKey sequenceKey, long timestamp) throws SQLException; - void returnSequenceValues(List sequenceKeys, long timestamp, SQLException[] exceptions) throws SQLException; - void addConnection(PhoenixConnection connection) throws SQLException; - void removeConnection(PhoenixConnection connection) throws SQLException; - - /** - * @return the {@link KeyValueBuilder} that is valid for the locally installed version of HBase. - */ - public KeyValueBuilder getKeyValueBuilder(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/DelegateConnectionQueryServices.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/DelegateConnectionQueryServices.java deleted file mode 100644 index 52b6e596..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/DelegateConnectionQueryServices.java +++ /dev/null @@ -1,231 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import java.sql.SQLException; -import java.util.List; -import java.util.Map; -import java.util.Properties; - -import org.apache.hadoop.hbase.HRegionLocation; -import org.apache.hadoop.hbase.HTableDescriptor; -import org.apache.hadoop.hbase.client.HBaseAdmin; -import org.apache.hadoop.hbase.client.HTableInterface; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Pair; - -import com.salesforce.phoenix.client.KeyValueBuilder; -import com.salesforce.phoenix.compile.MutationPlan; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PMetaData; -import com.salesforce.phoenix.schema.SequenceKey; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableType; - - -public class DelegateConnectionQueryServices extends DelegateQueryServices implements ConnectionQueryServices { - - public DelegateConnectionQueryServices(ConnectionQueryServices delegate) { - super(delegate); - } - - @Override - protected ConnectionQueryServices getDelegate() { - return (ConnectionQueryServices)super.getDelegate(); - } - - @Override - public ConnectionQueryServices getChildQueryServices(ImmutableBytesWritable tenantId) { - return getDelegate().getChildQueryServices(tenantId); - } - - @Override - public HTableInterface getTable(byte[] tableName) throws SQLException { - return getDelegate().getTable(tableName); - } - - @Override - public StatsManager getStatsManager() { - return getDelegate().getStatsManager(); - } - - @Override - public List getAllTableRegions(byte[] tableName) throws SQLException { - return getDelegate().getAllTableRegions(tableName); - } - - @Override - public PMetaData addTable(PTable table) throws SQLException { - return getDelegate().addTable(table); - } - - @Override - public PMetaData addColumn(String tableName, List columns, long tableTimeStamp, long tableSeqNum, - boolean isImmutableRows) throws SQLException { - return getDelegate().addColumn(tableName, columns, tableTimeStamp, tableSeqNum, isImmutableRows); - } - - @Override - public PMetaData removeTable(String tableName) - throws SQLException { - return getDelegate().removeTable(tableName); - } - - @Override - public PMetaData removeColumn(String tableName, String familyName, String columnName, long tableTimeStamp, - long tableSeqNum) throws SQLException { - return getDelegate().removeColumn(tableName, familyName, columnName, tableTimeStamp, tableSeqNum); - } - - @Override - public PhoenixConnection connect(String url, Properties info) throws SQLException { - return getDelegate().connect(url, info); - } - - @Override - public MetaDataMutationResult getTable(byte[] tenantId, byte[] schemaBytes, byte[] tableBytes, long tableTimestamp, long clientTimestamp) throws SQLException { - return getDelegate().getTable(tenantId, schemaBytes, tableBytes, tableTimestamp, clientTimestamp); - } - - @Override - public MetaDataMutationResult createTable(List tableMetaData, byte[] tableName, - PTableType tableType, Map tableProps, List>> families, byte[][] splits) - throws SQLException { - return getDelegate().createTable(tableMetaData, tableName, tableType, tableProps, families, splits); - } - - @Override - public MetaDataMutationResult dropTable(List tabeMetaData, PTableType tableType) throws SQLException { - return getDelegate().dropTable(tabeMetaData, tableType); - } - - @Override - public MetaDataMutationResult addColumn(List tabeMetaData, PTableType tableType, List>> families ) throws SQLException { - return getDelegate().addColumn(tabeMetaData, tableType, families); - } - - - @Override - public MetaDataMutationResult dropColumn(List tabeMetaData, PTableType tableType) throws SQLException { - return getDelegate().dropColumn(tabeMetaData, tableType); - } - - @Override - public MetaDataMutationResult updateIndexState(List tableMetadata, String parentTableName) throws SQLException { - return getDelegate().updateIndexState(tableMetadata, parentTableName); - } - - @Override - public void init(String url, Properties props) throws SQLException { - getDelegate().init(url, props); - } - - @Override - public MutationState updateData(MutationPlan plan) throws SQLException { - return getDelegate().updateData(plan); - } - - @Override - public int getLowestClusterHBaseVersion() { - return getDelegate().getLowestClusterHBaseVersion(); - } - - @Override - public HBaseAdmin getAdmin() throws SQLException { - return getDelegate().getAdmin(); - } - - @Override - public HTableDescriptor getTableDescriptor(byte[] tableName) throws SQLException { - return getDelegate().getTableDescriptor(tableName); - } - - @Override - public void clearTableRegionCache(byte[] tableName) throws SQLException { - getDelegate().clearTableRegionCache(tableName); - } - - @Override - public boolean hasInvalidIndexConfiguration() { - return getDelegate().hasInvalidIndexConfiguration(); - } - - @Override - public long createSequence(String tenantId, String schemaName, String sequenceName, long startWith, - long incrementBy, int batchSize, long timestamp) throws SQLException { - return getDelegate().createSequence(tenantId, schemaName, sequenceName, startWith, incrementBy, batchSize, timestamp); - } - - @Override - public long dropSequence(String tenantId, String schemaName, String sequenceName, long timestamp) - throws SQLException { - return getDelegate().dropSequence(tenantId, schemaName, sequenceName, timestamp); - } - - @Override - public void reserveSequenceValues(List sequenceKeys, long timestamp, long[] values, - SQLException[] exceptions) throws SQLException { - getDelegate().reserveSequenceValues(sequenceKeys, timestamp, values, exceptions); - } - - @Override - public void incrementSequenceValues(List sequenceKeys, long timestamp, long[] values, - SQLException[] exceptions) throws SQLException { - getDelegate().incrementSequenceValues(sequenceKeys, timestamp, values, exceptions); - } - - @Override - public long getSequenceValue(SequenceKey sequenceKey, long timestamp) throws SQLException { - return getDelegate().getSequenceValue(sequenceKey, timestamp); - } - - @Override - public void returnSequenceValues(List sequenceKeys, long timestamp, SQLException[] exceptions) - throws SQLException { - getDelegate().returnSequenceValues(sequenceKeys, timestamp, exceptions); - } - - @Override - public void addConnection(PhoenixConnection connection) throws SQLException { - getDelegate().addConnection(connection); - } - - @Override - public void removeConnection(PhoenixConnection connection) throws SQLException { - getDelegate().removeConnection(connection); - } - - @Override - public KeyValueBuilder getKeyValueBuilder() { - return getDelegate().getKeyValueBuilder(); - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/DelegateQueryServices.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/DelegateQueryServices.java deleted file mode 100644 index 706e90bf..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/DelegateQueryServices.java +++ /dev/null @@ -1,82 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import java.sql.SQLException; -import java.util.concurrent.ExecutorService; - -import com.salesforce.phoenix.memory.MemoryManager; -import com.salesforce.phoenix.optimize.QueryOptimizer; -import com.salesforce.phoenix.util.ReadOnlyProps; - - - -/** - * - * Class that delegates QueryService calls through to - * a parent QueryService. - * - * @author jtaylor - * @since 0.1 - */ -public class DelegateQueryServices implements QueryServices { - private final QueryServices parent; - - public DelegateQueryServices(QueryServices queryServices) { - parent = queryServices; - } - - protected QueryServices getDelegate() { - return parent; - } - - @Override - public ExecutorService getExecutor() { - return parent.getExecutor(); - } - - @Override - public MemoryManager getMemoryManager() { - return parent.getMemoryManager(); - } - - @Override - public void close() throws SQLException { - parent.close(); - } - - @Override - public ReadOnlyProps getProps() { - return parent.getProps(); - } - - @Override - public QueryOptimizer getOptimizer() { - return parent.getOptimizer(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/HBaseFactoryProvider.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/HBaseFactoryProvider.java deleted file mode 100644 index 60e0b052..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/HBaseFactoryProvider.java +++ /dev/null @@ -1,65 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import com.salesforce.phoenix.util.InstanceResolver; - -/** - * Manages factories that provide extension points for HBase. - *

- * Dependent modules may register their own implementations of the following using {@link java.util.ServiceLoader}: - *

    - *
  • {@link ConfigurationFactory}
    • - *
  • {@link HTableFactory}
    • - *
  • {@link HConnectionFactory}
    • - *
- * - * If a custom implementation is not registered, the default implementations will be used. - * - * @author aaraujo - * @since 0.2 - */ -public class HBaseFactoryProvider { - - private static final HTableFactory DEFAULT_HTABLE_FACTORY = new HTableFactory.HTableFactoryImpl(); - private static final HConnectionFactory DEFAULT_HCONNECTION_FACTORY = - new HConnectionFactory.HConnectionFactoryImpl(); - private static final ConfigurationFactory DEFAULT_CONFIGURATION_FACTORY = new ConfigurationFactory.ConfigurationFactoryImpl(); - - public static HTableFactory getHTableFactory() { - return InstanceResolver.getSingleton(HTableFactory.class, DEFAULT_HTABLE_FACTORY); - } - - public static HConnectionFactory getHConnectionFactory() { - return InstanceResolver.getSingleton(HConnectionFactory.class, DEFAULT_HCONNECTION_FACTORY); - } - - public static ConfigurationFactory getConfigurationFactory() { - return InstanceResolver.getSingleton(ConfigurationFactory.class, DEFAULT_CONFIGURATION_FACTORY); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/HConnectionFactory.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/HConnectionFactory.java deleted file mode 100644 index 9541a011..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/HConnectionFactory.java +++ /dev/null @@ -1,59 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hbase.ZooKeeperConnectionException; -import org.apache.hadoop.hbase.client.HConnection; -import org.apache.hadoop.hbase.client.HConnectionManager; - -/** - * Factory for creating {@link HConnection} - * - * @author ukuchibhotla - */ -public interface HConnectionFactory { - - /** - * Creates HConnection to access HBase clusters. - * - * @param configuration object - * @return A HConnection instance - */ - HConnection createConnection(Configuration conf) throws ZooKeeperConnectionException; - - /** - * Default implementation. Uses standard HBase HConnections. - */ - static class HConnectionFactoryImpl implements HConnectionFactory { - @Override - public HConnection createConnection(Configuration conf) throws ZooKeeperConnectionException { - return HConnectionManager.createConnection(conf); - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/HTableFactory.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/HTableFactory.java deleted file mode 100644 index f6089c3d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/HTableFactory.java +++ /dev/null @@ -1,64 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import java.io.IOException; -import java.util.concurrent.ExecutorService; - -import org.apache.hadoop.hbase.client.HConnection; -import org.apache.hadoop.hbase.client.HTable; -import org.apache.hadoop.hbase.client.HTableInterface; - -/** - * Creates clients to access HBase tables. - * - * @author aaraujo - * @since 0.2 - */ -public interface HTableFactory { - /** - * Creates an HBase client using an externally managed HConnection and Thread pool. - * - * @param tableName Name of the table. - * @param connection HConnection to use. - * @param pool ExecutorService to use. - * @return An client to access an HBase table. - * @throws IOException if a server or network exception occurs - */ - HTableInterface getTable(byte[] tableName, HConnection connection, ExecutorService pool) throws IOException; - - /** - * Default implementation. Uses standard HBase HTables. - */ - static class HTableFactoryImpl implements HTableFactory { - @Override - public HTableInterface getTable(byte[] tableName, HConnection connection, ExecutorService pool) throws IOException { - return new HTable(tableName, connection, pool); - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/MetaDataMutated.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/MetaDataMutated.java deleted file mode 100644 index e5b47581..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/MetaDataMutated.java +++ /dev/null @@ -1,50 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import java.sql.SQLException; -import java.util.List; - -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PMetaData; -import com.salesforce.phoenix.schema.PTable; - - -/** - * - * Interface for applying schema mutations to our client-side schema cache - * - * @author jtaylor - * @since 0.1 - */ -public interface MetaDataMutated { - PMetaData addTable(PTable table) throws SQLException; - PMetaData removeTable(String tableName) throws SQLException; - PMetaData addColumn(String tableName, List columns, long tableTimeStamp, long tableSeqNum, boolean isImmutableRows) throws SQLException; - PMetaData removeColumn(String tableName, String familyName, String columnName, long tableTimeStamp, long tableSeqNum) throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryConstants.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryConstants.java deleted file mode 100644 index ffacef00..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryConstants.java +++ /dev/null @@ -1,224 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - - -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.BASE_SCHEMA_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.BASE_TABLE_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.BUFFER_LENGTH; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.CACHE_SIZE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.CHAR_OCTET_LENGTH; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_COUNT; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_DEF; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_MODIFIER; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_SIZE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.CURRENT_VALUE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TABLE_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TYPE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DECIMAL_DIGITS; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DEFAULT_COLUMN_FAMILY_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DISABLE_WAL; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.IMMUTABLE_ROWS; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.INCREMENT_BY; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.INDEX_STATE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.IS_AUTOINCREMENT; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.IS_NULLABLE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.MULTI_TENANT; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.NULLABLE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.NUM_PREC_RADIX; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.ORDINAL_POSITION; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.PK_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.REF_GENERATION_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.REMARKS_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SALT_BUCKETS; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SCOPE_CATALOG; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SCOPE_SCHEMA; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SCOPE_TABLE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SELF_REFERENCING_COL_NAME_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SEQUENCE_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SEQUENCE_SCHEMA; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SOURCE_DATA_TYPE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SQL_DATA_TYPE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SQL_DATETIME_SUB; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.START_WITH; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_CAT_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_NAME_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_SCHEM_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_SEQ_NUM; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_TYPE_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TENANT_ID; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_SCHEMA; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_SEQUENCE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_TABLE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_EXPRESSION; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_TYPE; - -import java.math.BigDecimal; - -import org.apache.hadoop.hbase.HConstants; -import org.apache.hadoop.hbase.HTableDescriptor; -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.schema.MetaDataSplitPolicy; -import com.salesforce.phoenix.schema.PName; -import com.salesforce.phoenix.schema.PNormalizedName; - - -/** - * - * Constants used during querying - * - * @author jtaylor - * @since 0.1 - */ -public interface QueryConstants { - public static final String NAME_SEPARATOR = "."; - public final static byte[] NAME_SEPARATOR_BYTES = Bytes.toBytes(NAME_SEPARATOR); - public static final byte NAME_SEPARATOR_BYTE = NAME_SEPARATOR_BYTES[0]; - public static final String NULL_SCHEMA_NAME = ""; - public static final String NULL_DISPLAY_TEXT = ""; - public static final long UNSET_TIMESTAMP = -1; - - public enum JoinType {INNER, LEFT_OUTER} - public final static String PHOENIX_SCHEMA = "system"; - public final static String PHOENIX_METADATA = "table"; - - public final static PName SINGLE_COLUMN_NAME = new PNormalizedName("s"); - public final static PName SINGLE_COLUMN_FAMILY_NAME = new PNormalizedName("s"); - public final static byte[] SINGLE_COLUMN = SINGLE_COLUMN_NAME.getBytes(); - public final static byte[] SINGLE_COLUMN_FAMILY = SINGLE_COLUMN_FAMILY_NAME.getBytes(); - - public static final long AGG_TIMESTAMP = HConstants.LATEST_TIMESTAMP; - /** - * Key used for a single row aggregation where there is no group by - */ - public final static byte[] UNGROUPED_AGG_ROW_KEY = Bytes.toBytes("a"); - public final static PName AGG_COLUMN_NAME = SINGLE_COLUMN_NAME; - public final static PName AGG_COLUMN_FAMILY_NAME = SINGLE_COLUMN_FAMILY_NAME; - - public static final byte[] TRUE = new byte[] {1}; - - /** - * Separator used between variable length keys for a composite key. - * Variable length data types may not use this byte value. - */ - public static final byte SEPARATOR_BYTE = (byte) 0; - public static final byte[] SEPARATOR_BYTE_ARRAY = new byte[] {SEPARATOR_BYTE}; - - public static final String DEFAULT_COPROCESS_PATH = "phoenix.jar"; - public final static int MILLIS_IN_DAY = 1000 * 60 * 60 * 24; - - public static final String EMPTY_COLUMN_NAME = "_0"; - public static final byte[] EMPTY_COLUMN_BYTES = Bytes.toBytes(EMPTY_COLUMN_NAME); - public static final ImmutableBytesPtr EMPTY_COLUMN_BYTES_PTR = new ImmutableBytesPtr( - EMPTY_COLUMN_BYTES); - public static final String DEFAULT_COLUMN_FAMILY = EMPTY_COLUMN_NAME; - public static final byte[] DEFAULT_COLUMN_FAMILY_BYTES = EMPTY_COLUMN_BYTES; - public static final String ALL_FAMILY_PROPERTIES_KEY = ""; - public static final String SYSTEM_TABLE_PK_NAME = "pk"; - - public static final double MILLIS_TO_NANOS_CONVERTOR = Math.pow(10, 6); - public static final BigDecimal BD_MILLIS_NANOS_CONVERSION = BigDecimal.valueOf(MILLIS_TO_NANOS_CONVERTOR); - public static final BigDecimal BD_MILLIS_IN_DAY = BigDecimal.valueOf(QueryConstants.MILLIS_IN_DAY); - - - public static final String CREATE_TABLE_METADATA = - // Do not use IF NOT EXISTS as we sometimes catch the TableAlreadyExists exception - // and add columns to the SYSTEM.TABLE dynamically. - "CREATE TABLE " + TYPE_SCHEMA + ".\"" + TYPE_TABLE + "\"(\n" + - // PK columns - TENANT_ID + " VARCHAR NULL," + - TABLE_SCHEM_NAME + " VARCHAR NULL," + - TABLE_NAME_NAME + " VARCHAR NOT NULL," + - COLUMN_NAME + " VARCHAR NULL," + // null only for table row - TABLE_CAT_NAME + " VARCHAR NULL," + // using for CF - ensures uniqueness for columns - // Table metadata (will be null for column rows) - TABLE_TYPE_NAME + " CHAR(1)," + - REMARKS_NAME + " VARCHAR," + - DATA_TYPE + " INTEGER," + - PK_NAME + " VARCHAR," + - TYPE_NAME + " VARCHAR," + - SELF_REFERENCING_COL_NAME_NAME + " VARCHAR," + - REF_GENERATION_NAME + " VARCHAR," + - TABLE_SEQ_NUM + " BIGINT," + - COLUMN_COUNT + " INTEGER," + - // Column metadata (will be null for table row) - COLUMN_SIZE + " INTEGER," + - BUFFER_LENGTH + " INTEGER," + - DECIMAL_DIGITS + " INTEGER," + - NUM_PREC_RADIX + " INTEGER," + - NULLABLE + " INTEGER," + - COLUMN_DEF + " VARCHAR," + - SQL_DATA_TYPE + " INTEGER," + - SQL_DATETIME_SUB + " INTEGER," + - CHAR_OCTET_LENGTH + " INTEGER," + - ORDINAL_POSITION + " INTEGER," + - IS_NULLABLE + " VARCHAR," + - SCOPE_CATALOG + " VARCHAR," + - SCOPE_SCHEMA + " VARCHAR," + - SCOPE_TABLE + " VARCHAR," + - SOURCE_DATA_TYPE + " INTEGER," + // supposed to be SHORT - IS_AUTOINCREMENT + " VARCHAR," + - // Columns added in 1.2.1 - COLUMN_MODIFIER + " INTEGER," + - SALT_BUCKETS + " INTEGER," + - // Columns added in 2.0.0 - DATA_TABLE_NAME + " VARCHAR," + - INDEX_STATE + " CHAR(1),\n" + - IMMUTABLE_ROWS + " BOOLEAN,\n" + - // Columns added in 3.0.0 - VIEW_EXPRESSION + " VARBINARY,\n" + - DEFAULT_COLUMN_FAMILY_NAME + " VARCHAR,\n" + - DISABLE_WAL + " BOOLEAN,\n" + - MULTI_TENANT + " BOOLEAN,\n" + - VIEW_TYPE + " UNSIGNED_TINYINT,\n" + - BASE_SCHEMA_NAME + " VARCHAR,\n" + - BASE_TABLE_NAME + " VARCHAR,\n" + - "CONSTRAINT " + SYSTEM_TABLE_PK_NAME + " PRIMARY KEY (" + TENANT_ID + "," - + TABLE_SCHEM_NAME + "," + TABLE_NAME_NAME + "," + COLUMN_NAME + "," + TABLE_CAT_NAME + "))\n" + - HConstants.VERSIONS + "=" + MetaDataProtocol.DEFAULT_MAX_META_DATA_VERSIONS + ",\n" + - DEFAULT_COLUMN_FAMILY_NAME + "=" + "'_0'" + ",\n" + // Use original default for b/w compat - HTableDescriptor.SPLIT_POLICY + "='" + MetaDataSplitPolicy.class.getName() + "'\n"; - - public static final String CREATE_SEQUENCE_METADATA = - "CREATE TABLE IF NOT EXISTS " + TYPE_SCHEMA + ".\"" + TYPE_SEQUENCE + "\"(\n" + - TENANT_ID + " VARCHAR NULL," + - SEQUENCE_SCHEMA + " VARCHAR NULL, \n" + - SEQUENCE_NAME + " VARCHAR NOT NULL, \n" + - START_WITH + " BIGINT NOT NULL, \n" + - CURRENT_VALUE + " BIGINT NOT NULL, \n" + - INCREMENT_BY + " BIGINT NOT NULL, \n" + - CACHE_SIZE + " INTEGER NOT NULL \n" + - " CONSTRAINT " + SYSTEM_TABLE_PK_NAME + " PRIMARY KEY (" + TENANT_ID + "," + SEQUENCE_SCHEMA + "," + SEQUENCE_NAME + "))\n" + - HConstants.VERSIONS + "=" + MetaDataProtocol.DEFAULT_MAX_META_DATA_VERSIONS + "\n"; - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryServices.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryServices.java deleted file mode 100644 index 9691bee9..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryServices.java +++ /dev/null @@ -1,127 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import java.util.concurrent.ExecutorService; - -import org.apache.http.annotation.Immutable; - -import com.salesforce.phoenix.iterate.SpoolTooBigToDiskException; -import com.salesforce.phoenix.memory.MemoryManager; -import com.salesforce.phoenix.optimize.QueryOptimizer; -import com.salesforce.phoenix.util.ReadOnlyProps; -import com.salesforce.phoenix.util.SQLCloseable; - - - -/** - * - * Interface to group together services needed during querying. The - * parameters that may be set in {@link org.apache.hadoop.conf.Configuration} - * are documented here: https://github.com/forcedotcom/phoenix/wiki/Tuning - * - * @author jtaylor - * @since 0.1 - */ -@Immutable -public interface QueryServices extends SQLCloseable { - public static final String KEEP_ALIVE_MS_ATTRIB = "phoenix.query.keepAliveMs"; - public static final String THREAD_POOL_SIZE_ATTRIB = "phoenix.query.threadPoolSize"; - public static final String QUEUE_SIZE_ATTRIB = "phoenix.query.queueSize"; - public static final String THREAD_TIMEOUT_MS_ATTRIB = "phoenix.query.timeoutMs"; - public static final String SPOOL_THRESHOLD_BYTES_ATTRIB = "phoenix.query.spoolThresholdBytes"; - - /** - * max size to spool the the result into - * ${java.io.tmpdir}/ResultSpoolerXXX.bin if - * {@link QueryServices#SPOOL_THRESHOLD_BYTES_ATTRIB } is reached. - *

- * default is unlimited(-1) - *

- * if the threshold is reached, a {@link SpoolTooBigToDiskException } will be thrown - */ - public static final String MAX_SPOOL_TO_DISK_BYTES_ATTRIB = "phoenix.query.maxSpoolToDiskBytes"; - - public static final String MAX_MEMORY_PERC_ATTRIB = "phoenix.query.maxGlobalMemoryPercentage"; - public static final String MAX_MEMORY_WAIT_MS_ATTRIB = "phoenix.query.maxGlobalMemoryWaitMs"; - public static final String MAX_TENANT_MEMORY_PERC_ATTRIB = "phoenix.query.maxTenantMemoryPercentage"; - public static final String MAX_SERVER_CACHE_SIZE_ATTRIB = "phoenix.query.maxServerCacheBytes"; - public static final String TARGET_QUERY_CONCURRENCY_ATTRIB = "phoenix.query.targetConcurrency"; - public static final String MAX_QUERY_CONCURRENCY_ATTRIB = "phoenix.query.maxConcurrency"; - public static final String DATE_FORMAT_ATTRIB = "phoenix.query.dateFormat"; - public static final String NUMBER_FORMAT_ATTRIB = "phoenix.query.numberFormat"; - public static final String STATS_UPDATE_FREQ_MS_ATTRIB = "phoenix.query.statsUpdateFrequency"; - public static final String MAX_STATS_AGE_MS_ATTRIB = "phoenix.query.maxStatsAge"; - public static final String CALL_QUEUE_ROUND_ROBIN_ATTRIB = "ipc.server.callqueue.roundrobin"; - public static final String SCAN_CACHE_SIZE_ATTRIB = "hbase.client.scanner.caching"; - public static final String MAX_MUTATION_SIZE_ATTRIB = "phoenix.mutate.maxSize"; - public static final String MUTATE_BATCH_SIZE_ATTRIB = "phoenix.mutate.batchSize"; - public static final String MAX_SERVER_CACHE_TIME_TO_LIVE_MS = "phoenix.coprocessor.maxServerCacheTimeToLiveMs"; - public static final String MAX_INTRA_REGION_PARALLELIZATION_ATTRIB = "phoenix.query.maxIntraRegionParallelization"; - public static final String ROW_KEY_ORDER_SALTED_TABLE_ATTRIB = "phoenix.query.rowKeyOrderSaltedTable"; - public static final String USE_INDEXES_ATTRIB = "phoenix.query.useIndexes"; - public static final String IMMUTABLE_ROWS_ATTRIB = "phoenix.mutate.immutableRows"; - public static final String INDEX_MUTATE_BATCH_SIZE_THRESHOLD_ATTRIB = "phoenix.index.mutableBatchSizeThreshold"; - public static final String DROP_METADATA_ATTRIB = "phoenix.schema.dropMetaData"; - public static final String GROUPBY_SPILLABLE_ATTRIB = "phoenix.groupby.spillable"; - public static final String GROUPBY_SPILL_FILES_ATTRIB = "phoenix.groupby.spillFiles"; - public static final String GROUPBY_MAX_CACHE_SIZE_ATTRIB = "phoenix.groupby.maxCacheSize"; - - public static final String CALL_QUEUE_PRODUCER_ATTRIB_NAME = "CALL_QUEUE_PRODUCER"; - - public static final String MASTER_INFO_PORT_ATTRIB = "hbase.master.info.port"; - public static final String REGIONSERVER_INFO_PORT_ATTRIB = "hbase.regionserver.info.port"; - public static final String REGIONSERVER_LEASE_PERIOD_ATTRIB = "hbase.regionserver.lease.period"; - public static final String RPC_TIMEOUT_ATTRIB = "hbase.rpc.timeout"; - public static final String ZOOKEEPER_QUARUM_ATTRIB = "hbase.zookeeper.quorum"; - public static final String ZOOKEEPER_PORT_ATTRIB = "hbase.zookeeper.property.clientPort"; - public static final String ZOOKEEPER_ROOT_NODE_ATTRIB = "zookeeper.znode.parent"; - public static final String DISTINCT_VALUE_COMPRESS_THRESHOLD_ATTRIB = "phoenix.distinct.value.compress.threshold"; - public static final String SEQUENCE_CACHE_SIZE_ATTRIB = "phoenix.sequence.cacheSize"; - - - /** - * Get executor service used for parallel scans - */ - public ExecutorService getExecutor(); - /** - * Get the memory manager used to track memory usage - */ - public MemoryManager getMemoryManager(); - - /** - * Get the properties from the HBase configuration in a - * read-only structure that avoids any synchronization - */ - public ReadOnlyProps getProps(); - - /** - * Get query optimizer used to choose the best query plan - */ - public QueryOptimizer getOptimizer(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryServicesImpl.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryServicesImpl.java deleted file mode 100644 index 04f5e562..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryServicesImpl.java +++ /dev/null @@ -1,46 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - - - - - -/** - * - * Real implementation of QueryServices for use in runtime and perf testing - * - * @author jtaylor - * @since 0.1 - */ -public final class QueryServicesImpl extends BaseQueryServicesImpl { - - public QueryServicesImpl() { - super(QueryServicesOptions.withDefaults()); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/StatsManager.java b/phoenix-core/src/main/java/com/salesforce/phoenix/query/StatsManager.java deleted file mode 100644 index 44f10647..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/StatsManager.java +++ /dev/null @@ -1,67 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import java.sql.SQLException; - -import com.salesforce.phoenix.schema.TableRef; - - -/** - * - * Interface for managing and caching table statistics. - * The frequency of updating the table statistics are controlled - * by {@link com.salesforce.phoenix.query.QueryServices#STATS_UPDATE_FREQ_MS_ATTRIB}. - * Table stats may also be manually updated through {@link #updateStats(TableRef)}. - * - * - * @author jtaylor - * @since 0.1 - */ -public interface StatsManager { - /** - * Get the minimum key for the given table - * @param table the table - * @return the minimum key or null if unknown - */ - byte[] getMinKey(TableRef table); - - /** - * Get the maximum key for the given table - * @param table the table - * @return the maximum key or null if unknown - */ - byte[] getMaxKey(TableRef table); - - /** - * Manually update the cached table statistics - * @param table the table - * @throws SQLException - */ - void updateStats(TableRef table) throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/AmbiguousColumnException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/AmbiguousColumnException.java deleted file mode 100644 index 53d08d34..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/AmbiguousColumnException.java +++ /dev/null @@ -1,60 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - -/** - * - * Exception thrown when a column name is used without being qualified with an alias - * and more than one table contains that column. - * - * @author jtaylor - * @since 0.1 - */ -public class AmbiguousColumnException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.AMBIGUOUS_COLUMN; - - public AmbiguousColumnException() { - super(new SQLExceptionInfo.Builder(code).build().toString(), code.getSQLState(), code.getErrorCode()); - } - - public AmbiguousColumnException(String columnName) { - super(new SQLExceptionInfo.Builder(code).setColumnName(columnName).build().toString(), - code.getSQLState(), code.getErrorCode()); - } - - public AmbiguousColumnException(String columnName, Throwable cause) { - super(new SQLExceptionInfo.Builder(code).setColumnName(columnName).build().toString(), - code.getSQLState(), code.getErrorCode(), cause); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/AmbiguousTableException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/AmbiguousTableException.java deleted file mode 100644 index 98f1d8d7..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/AmbiguousTableException.java +++ /dev/null @@ -1,60 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - -/** - * - * Exception thrown when a table name is used without being qualified with an alias - * and more than one schema contains that table. - * - * @author jtaylor - * @since 0.1 - */ -public class AmbiguousTableException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.AMBIGUOUS_TABLE; - - public AmbiguousTableException() { - super(new SQLExceptionInfo.Builder(code).build().toString(), code.getSQLState(), code.getErrorCode(), null); - } - - public AmbiguousTableException(String tableName) { - super(new SQLExceptionInfo.Builder(code).setTableName(tableName).toString(), - code.getSQLState(), code.getErrorCode(), null); - } - - public AmbiguousTableException(String tableName, Throwable cause) { - super(new SQLExceptionInfo.Builder(code).setTableName(tableName).toString(), - code.getSQLState(), code.getErrorCode(), cause); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ArgumentTypeMismatchException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ArgumentTypeMismatchException.java deleted file mode 100644 index a3b5aaeb..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ArgumentTypeMismatchException.java +++ /dev/null @@ -1,52 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - -/** - * Exception thrown when we try to use use an argument that has the wrong type. - * - * @author zhuang - * @since 1.0 - */ -public class ArgumentTypeMismatchException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.TYPE_MISMATCH; - - public ArgumentTypeMismatchException(PDataType expected, PDataType actual, String location) { - super(new SQLExceptionInfo.Builder(code).setMessage("expected: " + expected + " but was: " + actual + " at " + location).build().toString(), code.getSQLState(), code.getErrorCode()); - } - - public ArgumentTypeMismatchException(String expected, String actual, String location) { - super(new SQLExceptionInfo.Builder(code).setMessage("expected: " + expected + " but was: " + actual + " at " + location).build().toString(), code.getSQLState(), code.getErrorCode()); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnAlreadyExistsException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnAlreadyExistsException.java deleted file mode 100644 index 4521f0c7..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnAlreadyExistsException.java +++ /dev/null @@ -1,70 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - - -/** - * - * Exception thrown when a column already exists. - * - * @author jtaylor - * @since 0.1 - */ -public class ColumnAlreadyExistsException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.COLUMN_EXIST_IN_DEF; - private final String schemaName; - private final String tableName; - private final String columnName; - - public ColumnAlreadyExistsException(String schemaName, String tableName, String columnName) { - super(new SQLExceptionInfo.Builder(code).setColumnName(columnName) - .setSchemaName(schemaName).setTableName(tableName).build().toString(), - code.getSQLState(), code.getErrorCode(), null); - this.schemaName = schemaName; - this.tableName = tableName; - this.columnName = columnName; - } - - public String getTableName() { - return tableName; - } - - public String getSchemaName() { - return schemaName; - } - - public String getColumnName() { - return columnName; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnFamilyNotFoundException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnFamilyNotFoundException.java deleted file mode 100644 index 9c049e15..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnFamilyNotFoundException.java +++ /dev/null @@ -1,54 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - -/** - * - * Exception thrown when a family name could not be found in the schema - * - * @author jtaylor - * @since 0.1 - */ -public class ColumnFamilyNotFoundException extends MetaDataEntityNotFoundException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.COLUMN_FAMILY_NOT_FOUND; - private final String familyName; - - public ColumnFamilyNotFoundException(String familyName) { - super(new SQLExceptionInfo.Builder(code).setFamilyName(familyName).build().toString(), - code.getSQLState(), code.getErrorCode(), null); - this.familyName = familyName; - } - - public String getFamilyName() { - return familyName; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnModifier.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnModifier.java deleted file mode 100644 index c48ede85..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnModifier.java +++ /dev/null @@ -1,157 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; - -import com.google.common.base.Preconditions; - -/** - * A ColumnModifier implementation modifies how bytes are stored in a primary key column.

- * The {@link ColumnModifier#apply apply} method is called when the bytes for a specific column are first written to HBase and again - * when they are read back. Phoenix attemps to minimize calls to apply when bytes are read out of HBase. - * - * @author simontoens - * @since 1.2 - */ -public enum ColumnModifier { - /** - * Invert the bytes in the src byte array to support descending ordering of row keys. - */ - SORT_DESC(1) { - @Override - public byte[] apply(byte[] src, int srcOffset, byte[] dest, int dstOffset, int length) { - Preconditions.checkNotNull(src); - Preconditions.checkNotNull(dest); - for (int i = 0; i < length; i++) { - dest[dstOffset+i] = (byte)(src[srcOffset+i] ^ 0xFF); - } - return dest; - } - - @Override - public byte apply(byte b) { - return (byte)(b ^ 0xFF); - } - - @Override - public CompareOp transform(CompareOp op) { - switch (op) { - case EQUAL: - return op; - case GREATER: - return CompareOp.LESS; - case GREATER_OR_EQUAL: - return CompareOp.LESS_OR_EQUAL; - case LESS: - return CompareOp.GREATER; - case LESS_OR_EQUAL: - return CompareOp.GREATER_OR_EQUAL; - default: - throw new IllegalArgumentException("Unknown operator " + op); - } - } - - @Override - public byte[] apply(byte[] src, int srcOffset, int length) { - return apply(src, srcOffset, new byte[length], 0, length); - } - }; - - private final int serializationId; - - ColumnModifier(int serializationId) { - this.serializationId = serializationId; - } - - public int getSerializationId() { - return serializationId; - } - /** - * Returns the ColumnModifier for the specified DDL stmt keyword. - */ - public static ColumnModifier fromDDLValue(String modifier) { - if (modifier == null) { - return null; - } else if (modifier.equalsIgnoreCase("ASC")) { - return null; - } else if (modifier.equalsIgnoreCase("DESC")) { - return SORT_DESC; - } else { - return null; - } - } - - /** - * Returns the ColumnModifier for the specified internal value. - */ - public static ColumnModifier fromSystemValue(int value) { - for (ColumnModifier mod : ColumnModifier.values()) { - if (mod.getSerializationId() == value) { - return mod; - } - } - return null; - } - - /** - * Returns an internal value representing the specified ColumnModifier. - */ - public static int toSystemValue(ColumnModifier columnModifier) { - if (columnModifier == null) { - return 0; - } - return columnModifier.getSerializationId(); - } - - /** - * Copies the bytes from source array to destination array and applies the column modifier operation on the bytes - * starting at the specified offsets. The column modifier is applied to the number of bytes matching the - * specified length. - * @param src the source byte array to copy from, cannot be null - * @param srcOffset the offset into the source byte array at which to begin. - * @param dest the destination byte array into which to transfer the modified bytes. - * @param dstOffset the offset into the destination byte array at which to begin - * @param length the number of bytes for which to apply the modification - * @return the destination byte array - */ - public abstract byte[] apply(byte[] src, int srcOffset, byte[] dest, int dstOffset, int length); - /** - * Copies the bytes from source array to a newly allocated destination array and applies the column - * modifier operation on the bytes starting at the specified offsets. The column modifier is applied - * to the number of bytes matching the specified length. - * @param src the source byte array to copy from, cannot be null - * @param srcOffset the offset into the source byte array at which to begin. - * @param length the number of bytes for which to apply the modification - * @return the newly allocated destination byte array - */ - public abstract byte[] apply(byte[] src, int srcOffset, int length); - public abstract byte apply(byte b); - - public abstract CompareOp transform(CompareOp op); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnNotFoundException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnNotFoundException.java deleted file mode 100644 index 661bdc9d..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnNotFoundException.java +++ /dev/null @@ -1,73 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - - -/** - * - * Exception thrown when a column name referenced in a select - * statement cannot be found in any table. - * - * @author jtaylor - * @since 0.1 - */ -public class ColumnNotFoundException extends MetaDataEntityNotFoundException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.COLUMN_NOT_FOUND; - private final String schemaName; - private final String tableName; - private final String columnName; - - public ColumnNotFoundException(String columnName) { - this(null, null, null, columnName); - } - - public ColumnNotFoundException(String schemaName, String tableName, String familyName, String columnName) { - super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName) - .setFamilyName(familyName).setColumnName(columnName).build().toString(), - code.getSQLState(), code.getErrorCode(), null); - this.schemaName = schemaName; - this.tableName = tableName; - this.columnName = columnName; - } - - public String getTableName() { - return tableName; - } - - public String getSchemaName() { - return schemaName; - } - - public String getColumnName() { - return columnName; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnRef.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnRef.java deleted file mode 100644 index e8b4b03c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ColumnRef.java +++ /dev/null @@ -1,143 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import org.apache.http.annotation.Immutable; - -import com.salesforce.phoenix.expression.ColumnExpression; -import com.salesforce.phoenix.expression.IndexKeyValueColumnExpression; -import com.salesforce.phoenix.expression.KeyValueColumnExpression; -import com.salesforce.phoenix.expression.ProjectedColumnExpression; -import com.salesforce.phoenix.expression.RowKeyColumnExpression; -import com.salesforce.phoenix.util.IndexUtil; -import com.salesforce.phoenix.util.SchemaUtil; - - -/** - * - * Class that represents a reference to a PColumn in a PTable - * - * @author jtaylor - * @since 0.1 - */ -@Immutable -public final class ColumnRef { - private final TableRef tableRef; - private final int columnPosition; - private final int pkSlotPosition; - - public ColumnRef(ColumnRef columnRef, long timeStamp) { - this.tableRef = new TableRef(columnRef.tableRef, timeStamp); - this.columnPosition = columnRef.columnPosition; - this.pkSlotPosition = columnRef.pkSlotPosition; - } - - public ColumnRef(TableRef tableRef, int columnPosition) { - if (tableRef == null) { - throw new NullPointerException(); - } - if (columnPosition < 0 || columnPosition >= tableRef.getTable().getColumns().size()) { - throw new IllegalArgumentException("Column position of " + columnPosition + " must be between 0 and " + tableRef.getTable().getColumns().size() + " for table " + tableRef.getTable().getName().getString()); - } - this.tableRef = tableRef; - this.columnPosition = columnPosition; - PColumn column = getColumn(); - int i = -1; - if (SchemaUtil.isPKColumn(column)) { - for (PColumn pkColumn : tableRef.getTable().getPKColumns()) { - i++; - if (pkColumn == column) { - break; - } - } - } - pkSlotPosition = i; - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + columnPosition; - result = prime * result + tableRef.hashCode(); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - ColumnRef other = (ColumnRef)obj; - if (columnPosition != other.columnPosition) return false; - if (!tableRef.equals(other.tableRef)) return false; - return true; - } - - public ColumnExpression newColumnExpression() { - boolean isIndex = tableRef.getTable().getType() == PTableType.INDEX; - if (SchemaUtil.isPKColumn(this.getColumn())) { - String name = this.getColumn().getName().getString(); - if (isIndex) { - name = IndexUtil.getDataColumnName(name); - } - return new RowKeyColumnExpression( - getColumn(), - new RowKeyValueAccessor(this.getTable().getPKColumns(), pkSlotPosition), - name); - } - - if (isIndex) - return new IndexKeyValueColumnExpression(getColumn()); - - if (tableRef.getTable().getType() == PTableType.JOIN) - return new ProjectedColumnExpression(getColumn(), tableRef.getTable()); - - return new KeyValueColumnExpression(getColumn()); - } - - public int getColumnPosition() { - return columnPosition; - } - - public int getPKSlotPosition() { - return pkSlotPosition; - } - - public PColumn getColumn() { - return tableRef.getTable().getColumns().get(columnPosition); - } - - public PTable getTable() { - return tableRef.getTable(); - } - - public TableRef getTableRef() { - return tableRef; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ConcurrentTableMutationException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ConcurrentTableMutationException.java deleted file mode 100644 index 511aa5d1..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ConcurrentTableMutationException.java +++ /dev/null @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - - -public class ConcurrentTableMutationException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.CONCURRENT_TABLE_MUTATION; - private final String schemaName; - private final String tableName; - - public ConcurrentTableMutationException(String schemaName, String tableName) { - super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).build().toString(), - code.getSQLState(), code.getErrorCode()); - this.schemaName = schemaName; - this.tableName = tableName; - } - - public String getTableName() { - return tableName; - } - - public String getSchemaName() { - return schemaName; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ConstraintViolationException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ConstraintViolationException.java deleted file mode 100644 index 121ecaa2..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ConstraintViolationException.java +++ /dev/null @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -/** - * - * Exception thrown when a schema constraint is violated at the - * time of data insertion. - * - * @author jtaylor - * @since 180 - */ -public class ConstraintViolationException extends RuntimeException { - private static final long serialVersionUID = 1L; - - public ConstraintViolationException() { - } - - public ConstraintViolationException(String message) { - super(message); - } - - public ConstraintViolationException(Throwable cause) { - super(cause); - } - - public ConstraintViolationException(String message, Throwable cause) { - super(message, cause); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/DelegateColumn.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/DelegateColumn.java deleted file mode 100644 index 5bbec9ac..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/DelegateColumn.java +++ /dev/null @@ -1,72 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.io.*; - -public class DelegateColumn extends DelegateDatum implements PColumn { - - public DelegateColumn(PColumn delegate) { - super(delegate); - } - - @Override - protected PColumn getDelegate() { - return (PColumn)super.getDelegate(); - } - - @Override - public PName getName() { - return getDelegate().getName(); - } - - @Override - public ColumnModifier getColumnModifier() { - return getDelegate().getColumnModifier(); - } - - @Override - public PName getFamilyName() { - return getDelegate().getFamilyName(); - } - - @Override - public void readFields(DataInput input) throws IOException { - getDelegate().readFields(input); - } - - @Override - public void write(DataOutput output) throws IOException { - getDelegate().write(output); - } - - @Override - public int getPosition() { - return getDelegate().getPosition(); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/DelegateDatum.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/DelegateDatum.java deleted file mode 100644 index d628efbd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/DelegateDatum.java +++ /dev/null @@ -1,70 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -public class DelegateDatum implements PDatum { - private final PDatum delegate; - - public DelegateDatum(PDatum delegate) { - this.delegate = delegate; - } - - @Override - public boolean isNullable() { - return delegate.isNullable(); - } - - @Override - public PDataType getDataType() { - return delegate.getDataType(); - } - - @Override - public Integer getByteSize() { - return delegate.getByteSize(); - } - - @Override - public Integer getMaxLength() { - return delegate.getByteSize(); - } - - @Override - public Integer getScale() { - return delegate.getScale(); - } - - @Override - public ColumnModifier getColumnModifier() { - return delegate.getColumnModifier(); - } - - protected PDatum getDelegate() { - return delegate; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ExecuteQueryNotApplicableException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ExecuteQueryNotApplicableException.java deleted file mode 100644 index 0096753a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ExecuteQueryNotApplicableException.java +++ /dev/null @@ -1,47 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - -public class ExecuteQueryNotApplicableException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.EXECUTE_QUERY_NOT_APPLICABLE; - - public ExecuteQueryNotApplicableException(String query) { - super(new SQLExceptionInfo.Builder(code).setMessage("Query: " + query).build().toString(), code.getSQLState(), code.getErrorCode()); - } - - public ExecuteQueryNotApplicableException(String command, String statement) { - super(new SQLExceptionInfo.Builder(code).setMessage("Command: " + command + ". Statement: " + statement).build().toString(), code.getSQLState(), code.getErrorCode()); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ExecuteUpdateNotApplicableException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ExecuteUpdateNotApplicableException.java deleted file mode 100644 index dfcb3efa..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ExecuteUpdateNotApplicableException.java +++ /dev/null @@ -1,46 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - -public class ExecuteUpdateNotApplicableException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.EXECUTE_QUERY_NOT_APPLICABLE; - - public ExecuteUpdateNotApplicableException(String query) { - super(new SQLExceptionInfo.Builder(code).setMessage("Query: " + query).build().toString(), code.getSQLState(), code.getErrorCode()); - } - - public ExecuteUpdateNotApplicableException(String command, String statement) { - super(new SQLExceptionInfo.Builder(code).setMessage("Command: " + command + ". Statement: " + statement).build().toString(), code.getSQLState(), code.getErrorCode()); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/IllegalDataException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/IllegalDataException.java deleted file mode 100644 index 8074de72..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/IllegalDataException.java +++ /dev/null @@ -1,55 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -/** - * - * Exception thrown when an invalid or illegal data value is found - * - * @author jtaylor - * @since 0.1 - */ -public class IllegalDataException extends ConstraintViolationException { - private static final long serialVersionUID = 1L; - - public IllegalDataException() { - } - - public IllegalDataException(String message) { - super(message); - } - - public IllegalDataException(Throwable cause) { - super(cause); - } - - public IllegalDataException(String message, Throwable cause) { - super(message, cause); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/MetaDataEntityNotFoundException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/MetaDataEntityNotFoundException.java deleted file mode 100644 index fbffc0dc..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/MetaDataEntityNotFoundException.java +++ /dev/null @@ -1,38 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -public abstract class MetaDataEntityNotFoundException extends SQLException { - private static final long serialVersionUID = 1L; - - public MetaDataEntityNotFoundException(String reason, String sqlState, int code, Throwable cause) { - super(reason, sqlState, code, cause); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/MetaDataSplitPolicy.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/MetaDataSplitPolicy.java deleted file mode 100644 index 0b459f13..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/MetaDataSplitPolicy.java +++ /dev/null @@ -1,59 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy; - -import com.salesforce.phoenix.util.SchemaUtil; - - -public class MetaDataSplitPolicy extends ConstantSizeRegionSplitPolicy { - - @Override - protected byte[] getSplitPoint() { - byte[] splitPoint = super.getSplitPoint(); - int offset = SchemaUtil.getVarCharLength(splitPoint, 0, splitPoint.length); - // Split only on Phoenix schema name, so this is ok b/c we won't be splitting - // in the middle of a Phoenix table. - if (offset == splitPoint.length) { - return splitPoint; - } -// offset = SchemaUtil.getVarCharLength(splitPoint, offset+1, splitPoint.length-offset-1); -// // Split only on Phoenix schema and table name, so this is ok b/c we won't be splitting -// // in the middle of a Phoenix table. -// if (offset == splitPoint.length) { -// return splitPoint; -// } - // Otherwise, an attempt is being made to split in the middle of a table. - // Just return a split point at the schema boundary instead - byte[] newSplitPoint = new byte[offset + 1]; - System.arraycopy(splitPoint, 0, newSplitPoint, 0, offset+1); - return newSplitPoint; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/NewerTableAlreadyExistsException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/NewerTableAlreadyExistsException.java deleted file mode 100644 index 545345d9..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/NewerTableAlreadyExistsException.java +++ /dev/null @@ -1,37 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -public class NewerTableAlreadyExistsException extends TableAlreadyExistsException { - private static final long serialVersionUID = 1L; - - public NewerTableAlreadyExistsException(String schemaName, String tableName) { - super(schemaName, tableName); - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PBaseColumn.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PBaseColumn.java deleted file mode 100644 index 119a5220..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PBaseColumn.java +++ /dev/null @@ -1,60 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - - -/** - * - * Base class for PColumn implementors that provides - * some reasonable default implementations. - * - * @author jtaylor - * @since 0.1 - */ -public abstract class PBaseColumn implements PColumn { - - @Override - public final Integer getByteSize() { - return getDataType().getByteSize(); - } - - @Override - public final Integer getMaxLength() { - return null; - } - - @Override - public final Integer getScale() { - return null; - } - - @Override - public boolean isNullable() { - return false; - } - } diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumn.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumn.java deleted file mode 100644 index ad8bd934..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumn.java +++ /dev/null @@ -1,54 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import org.apache.hadoop.io.Writable; - -/** - * Definition of a Phoenix column - * - * @author wmacklem,jtaylor - * @since 0.1 - */ -public interface PColumn extends PDatum, Writable { - - /** - * @return the name of the column qualifier - */ - PName getName(); - - /** - * @return the name of the column family - */ - PName getFamilyName(); - - /** - * @return the zero-based ordinal position of the column - */ - int getPosition(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumnFamily.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumnFamily.java deleted file mode 100644 index 1ef61174..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumnFamily.java +++ /dev/null @@ -1,62 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.util.Collection; - -/** - * - * Definition of a Phoenix Column Family - * - * @author jtaylor - * @since 0.1 - */ -public interface PColumnFamily { - - /** - * @return The column family name. - */ - PName getName(); - - /** - * @return All the PColumns in this column family. - */ - Collection getColumns(); - - /** - * @return The PColumn for the specified column qualifier. - * @throws ColumnNotFoundException if the column cannot be found - */ - PColumn getColumn(byte[] qualifier) throws ColumnNotFoundException; - - /** - * @return The PColumn for the specified column qualifier. - * @throws ColumnNotFoundException if the column cannot be found - */ - PColumn getColumn(String name) throws ColumnNotFoundException; -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumnFamilyImpl.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumnFamilyImpl.java deleted file mode 100644 index 3c01c5cc..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumnFamilyImpl.java +++ /dev/null @@ -1,83 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.util.List; -import java.util.Map; - -import org.apache.hadoop.hbase.util.Bytes; - -import com.google.common.collect.*; - -public class PColumnFamilyImpl implements PColumnFamily { - private final PName name; - private final List columns; - private final Map columnByString; - private final Map columnByBytes; - - public PColumnFamilyImpl(PName name, List columns) { - this.name = name; - this.columns = ImmutableList.copyOf(columns); - ImmutableMap.Builder columnByStringBuilder = ImmutableMap.builder(); - ImmutableSortedMap.Builder columnByBytesBuilder = ImmutableSortedMap.orderedBy(Bytes.BYTES_COMPARATOR); - for (PColumn column : columns) { - columnByBytesBuilder.put(column.getName().getBytes(), column); - columnByStringBuilder.put(column.getName().getString(), column); - } - this.columnByBytes = columnByBytesBuilder.build(); - this.columnByString = columnByStringBuilder.build(); - } - - @Override - public PName getName() { - return name; - } - - @Override - public List getColumns() { - return columns; - } - - @Override - public PColumn getColumn(byte[] qualifier) throws ColumnNotFoundException { - PColumn column = columnByBytes.get(qualifier); - if (column == null) { - throw new ColumnNotFoundException(Bytes.toString(qualifier)); - } - return column; - } - - @Override - public PColumn getColumn(String name) throws ColumnNotFoundException { - PColumn column = columnByString.get(name); - if (column == null) { - throw new ColumnNotFoundException(name); - } - return column; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumnImpl.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumnImpl.java deleted file mode 100644 index fed452de..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PColumnImpl.java +++ /dev/null @@ -1,205 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.io.DataInput; -import java.io.DataOutput; -import java.io.IOException; - -import org.apache.hadoop.hbase.util.Bytes; -import org.apache.hadoop.io.WritableUtils; - -import com.google.common.base.Preconditions; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.util.ByteUtil; - - -public class PColumnImpl implements PColumn { - private static final Integer NO_MAXLENGTH = Integer.MIN_VALUE; - private static final Integer NO_SCALE = Integer.MIN_VALUE; - - private PName name; - private PName familyName; - private PDataType dataType; - private Integer maxLength; - private Integer scale; - private boolean nullable; - private int position; - private ColumnModifier columnModifier; - - public PColumnImpl() { - } - - public PColumnImpl(PName name, - PName familyName, - PDataType dataType, - Integer maxLength, - Integer scale, - boolean nullable, - int position, - ColumnModifier sortOrder) { - init(name, familyName, dataType, maxLength, scale, nullable, position, sortOrder); - } - - public PColumnImpl(PColumn column, int position) { - this(column.getName(), column.getFamilyName(), column.getDataType(), column.getMaxLength(), - column.getScale(), column.isNullable(), position, column.getColumnModifier()); - } - - private void init(PName name, - PName familyName, - PDataType dataType, - Integer maxLength, - Integer scale, - boolean nullable, - int position, - ColumnModifier columnModifier) { - this.dataType = dataType; - if (familyName == null) { - // Allow nullable columns in PK, but only if they're variable length. - // Variable length types may be null, since we use a null-byte terminator - // (which is a disallowed character in variable length types). However, - // fixed width types do not have a way of representing null. - // TODO: we may be able to allow this for columns at the end of the PK - Preconditions.checkArgument(!nullable || !dataType.isFixedWidth(), - "PK columns may not be both fixed width and nullable: " + name.getString()); - } - this.name = name; - this.familyName = familyName == null ? null : familyName; - this.maxLength = maxLength; - this.scale = scale; - this.nullable = nullable; - this.position = position; - this.columnModifier = columnModifier; - } - - @Override - public PName getName() { - return name; - } - - @Override - public PName getFamilyName() { - return familyName; - } - - @Override - public PDataType getDataType() { - return dataType; - } - - @Override - public Integer getMaxLength() { - return maxLength; - } - - @Override - public Integer getScale() { - return scale; - } - - @Override - public Integer getByteSize() { - Integer dataTypeMaxLength = dataType.getByteSize(); - return dataTypeMaxLength == null ? dataType.estimateByteSizeFromLength(maxLength) - : dataTypeMaxLength; - } - - @Override - public boolean isNullable() { - return nullable; - } - - @Override - public int getPosition() { - return position; - } - - @Override - public ColumnModifier getColumnModifier() { - return columnModifier; - } - - @Override - public String toString() { - return (familyName == null ? "" : familyName.toString() + QueryConstants.NAME_SEPARATOR) + name.toString(); - } - - @Override - public void readFields(DataInput input) throws IOException { - byte[] columnNameBytes = Bytes.readByteArray(input); - PName columnName = PNameFactory.newName(columnNameBytes); - byte[] familyNameBytes = Bytes.readByteArray(input); - PName familyName = familyNameBytes.length == 0 ? null : PNameFactory.newName(familyNameBytes); - // TODO: optimize the reading/writing of this b/c it could likely all fit in a single byte or two - PDataType dataType = PDataType.values()[WritableUtils.readVInt(input)]; - int maxLength = WritableUtils.readVInt(input); - int scale = WritableUtils.readVInt(input); - boolean nullable = input.readBoolean(); - int position = WritableUtils.readVInt(input); - ColumnModifier columnModifier = ColumnModifier.fromSystemValue(WritableUtils.readVInt(input)); - init(columnName, familyName, dataType, maxLength == NO_MAXLENGTH ? null : maxLength, - scale == NO_SCALE ? null : scale, nullable, position, columnModifier); - } - - @Override - public void write(DataOutput output) throws IOException { - Bytes.writeByteArray(output, name.getBytes()); - Bytes.writeByteArray(output, familyName == null ? ByteUtil.EMPTY_BYTE_ARRAY : familyName.getBytes()); - WritableUtils.writeVInt(output, dataType.ordinal()); - WritableUtils.writeVInt(output, maxLength == null ? NO_MAXLENGTH : maxLength); - WritableUtils.writeVInt(output, scale == null ? NO_SCALE : scale); - output.writeBoolean(nullable); - WritableUtils.writeVInt(output, position); - WritableUtils.writeVInt(output, ColumnModifier.toSystemValue(columnModifier)); - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + ((familyName == null) ? 0 : familyName.hashCode()); - result = prime * result + ((name == null) ? 0 : name.hashCode()); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - PColumnImpl other = (PColumnImpl)obj; - if (familyName == null) { - if (other.familyName != null) return false; - } else if (!familyName.equals(other.familyName)) return false; - if (name == null) { - if (other.name != null) return false; - } else if (!name.equals(other.name)) return false; - return true; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PDateColumn.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PDateColumn.java deleted file mode 100644 index ba9c314b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PDateColumn.java +++ /dev/null @@ -1,35 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -abstract public class PDateColumn extends PBaseColumn { - @Override - public PDataType getDataType() { - return PDataType.DATE; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PDatum.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PDatum.java deleted file mode 100644 index bd9131b3..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PDatum.java +++ /dev/null @@ -1,62 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - - -public interface PDatum { - /** - * @return is this column nullable? - */ - boolean isNullable(); - - /** - * @return data type of the column - */ - PDataType getDataType(); - - /** - * @return maximum byte length of the column - */ - Integer getByteSize(); - - /** - * @return the actual length of the column. For decimal, it would be its precision. For char or - * varchar, it would be the maximum length as specified during schema definition. - */ - Integer getMaxLength(); - - /** - * @return scale of a decimal number. - */ - Integer getScale(); - - /** - * @return The modifier for this column or null if it doesn't have a modifier - */ - ColumnModifier getColumnModifier(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PIndexState.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PIndexState.java deleted file mode 100644 index bcea82b5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PIndexState.java +++ /dev/null @@ -1,100 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - - -public enum PIndexState { - BUILDING("b"), - USABLE("e"), - UNUSABLE("d"), - ACTIVE("a"), - INACTIVE("i"), - DISABLE("x"), - REBUILD("r"); - - private final String serializedValue; - private final byte[] serializedBytes; - private final byte[] nameBytesValue; - - private PIndexState(String value) { - this.serializedValue = value; - this.serializedBytes = PDataType.VARCHAR.toBytes(value); - this.nameBytesValue = PDataType.VARCHAR.toBytes(this.toString()); - } - - public String getSerializedValue() { - return serializedValue; - } - - public byte[] getSerializedBytes() { - return serializedBytes; - } - - public byte[] toBytes() { - return nameBytesValue; - } - - private static final PIndexState[] FROM_VALUE; - private static final int FROM_VALUE_OFFSET; - static { - int minChar = Integer.MAX_VALUE; - int maxChar = Integer.MIN_VALUE; - for (PIndexState state: PIndexState.values()) { - char c = state.getSerializedValue().charAt(0); - if (c < minChar) { - minChar = c; - } - if (c > maxChar) { - maxChar = c; - } - } - FROM_VALUE_OFFSET = minChar; - FROM_VALUE = new PIndexState[maxChar - minChar + 1]; - for (PIndexState state: PIndexState.values()) { - FROM_VALUE[state.getSerializedValue().charAt(0) - minChar] = state; - } - } - - public static PIndexState fromSerializedValue(String serializedValue) { - if (serializedValue.length() == 1) { - int i = serializedValue.charAt(0) - FROM_VALUE_OFFSET; - if (i >= 0 && i < FROM_VALUE.length && FROM_VALUE[i] != null) { - return FROM_VALUE[i]; - } - } - throw new IllegalArgumentException("Unable to PIndexState enum for serialized value of '" + serializedValue + "'"); - } - - public static PIndexState fromSerializedValue(byte serializedByte) { - int i = serializedByte - FROM_VALUE_OFFSET; - if (i >= 0 && i < FROM_VALUE.length && FROM_VALUE[i] != null) { - return FROM_VALUE[i]; - } - throw new IllegalArgumentException("Unable to PIndexState enum for serialized value of '" + (char)serializedByte + "'"); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PIntegerColumn.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PIntegerColumn.java deleted file mode 100644 index eb8fdc81..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PIntegerColumn.java +++ /dev/null @@ -1,43 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - - -/** - * - * Base class for PColumn implementors of type Integer. - * - * @author jtaylor - * @since 0.1 - */ -public abstract class PIntegerColumn extends PBaseColumn { - @Override - public final PDataType getDataType() { - return PDataType.INTEGER; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PLongColumn.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PLongColumn.java deleted file mode 100644 index 62a3a7da..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PLongColumn.java +++ /dev/null @@ -1,43 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - - -/** - * - * Base class for PColumn implementors of type Long. - * - * @author jtaylor - * @since 0.1 - */ -public abstract class PLongColumn extends PBaseColumn { - @Override - public final PDataType getDataType() { - return PDataType.LONG; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PMetaData.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PMetaData.java deleted file mode 100644 index f47dccc0..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PMetaData.java +++ /dev/null @@ -1,38 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.util.Map; - -import com.salesforce.phoenix.query.MetaDataMutated; - - -public interface PMetaData extends MetaDataMutated { - public PTable getTable(String name) throws TableNotFoundException; - public Map getTables(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PName.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PName.java deleted file mode 100644 index a0876a18..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PName.java +++ /dev/null @@ -1,104 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.util.ByteUtil; - - -/** - * - * Interface to encapsulate both the client-side name - * together with the server-side name for a named object - * - * @author jtaylor - * @since 0.1 - */ -public interface PName { - public static PName EMPTY_NAME = new PName() { - @Override - public String getString() { - return ""; - } - - @Override - public byte[] getBytes() { - return ByteUtil.EMPTY_BYTE_ARRAY; - } - - @Override - public String toString() { - return getString(); - } - - @Override - public ImmutableBytesPtr getBytesPtr() { - return ByteUtil.EMPTY_BYTE_ARRAY_PTR; - } - }; - public static PName EMPTY_COLUMN_NAME = new PName() { - @Override - public String getString() { - return QueryConstants.EMPTY_COLUMN_NAME; - } - - @Override - public byte[] getBytes() { - return QueryConstants.EMPTY_COLUMN_BYTES; - } - - @Override - public String toString() { - return getString(); - } - - @Override - public ImmutableBytesPtr getBytesPtr() { - return QueryConstants.EMPTY_COLUMN_BYTES_PTR; - } - }; - /** - * Get the client-side, normalized name as referenced - * in a SQL statement. - * @return the normalized string name - */ - String getString(); - - /** - * Get the server-side name as referenced in HBase-related - * APIs such as Scan, Filter, etc. - * @return the name as a byte array - */ - byte[] getBytes(); - - /** - * @return a pointer to the underlying bytes - */ - ImmutableBytesPtr getBytesPtr(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PNormalizedName.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PNormalizedName.java deleted file mode 100644 index 08686b8c..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PNormalizedName.java +++ /dev/null @@ -1,61 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import org.apache.http.annotation.Immutable; - -import com.salesforce.phoenix.util.SchemaUtil; - - -@Immutable -public class PNormalizedName extends PNameImpl { - - public PNormalizedName(String nonNormalizedName) { - super(SchemaUtil.normalizeIdentifier(nonNormalizedName)); - } - - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + getString().hashCode(); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - PNormalizedName other = (PNormalizedName)obj; - // Compare normalized stringName for equality, since bytesName - // may differ since it remains case sensitive. - if (!getString().equals(other.getString())) return false; - return true; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PRow.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PRow.java deleted file mode 100644 index 9e2a05ab..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PRow.java +++ /dev/null @@ -1,83 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.util.List; -import java.util.Map; - -import org.apache.hadoop.hbase.client.Mutation; - -import com.google.common.collect.ImmutableMap; - -/** - * - * Provide a client API for updating rows. The updates are processed in - * the calling order. Calling setValue after calling delete will cause the - * delete to be canceled. Conversely, calling delete after calling - * setValue will cause all prior setValue calls to be canceled. - * - * @author jtaylor - * @since 0.1 - */ -public interface PRow { - Map DELETE_MARKER = ImmutableMap.of(); - - /** - * Get the list of {@link org.apache.hadoop.hbase.client.Mutation} used to - * update an HTable after all mutations through calls to - * {@link #setValue(PColumn, Object)} or {@link #delete()}. - * @return the list of mutations representing all changes made to a row - * @throws ConstraintViolationException if row data violates schema - * constraint - */ - public List toRowMutations(); - - /** - * Set a column value in the row - * @param col the column for which the value is being set - * @param value the value - * @throws ConstraintViolationException if row data violates schema - * constraint - */ - public void setValue(PColumn col, Object value); - - /** - * Set a column value in the row - * @param col the column for which the value is being set - * @param value the value - * @throws ConstraintViolationException if row data violates schema - * constraint - */ - public void setValue(PColumn col, byte[] value); - - /** - * Delete the row. Note that a delete take precedence over any - * values that may have been set before or after the delete call. - */ - public void delete(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PStringColumn.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PStringColumn.java deleted file mode 100644 index c41bc850..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PStringColumn.java +++ /dev/null @@ -1,44 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -/** - * - * Abstract class for columns of type {@link com.salesforce.phoenix.schema.PDataType#VARCHAR} - * - * @author jtaylor - * @since 0.1 - */ -public abstract class PStringColumn extends PBaseColumn { - - @Override - public PDataType getDataType() { - return PDataType.VARCHAR; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PTable.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PTable.java deleted file mode 100644 index ae544c07..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PTable.java +++ /dev/null @@ -1,261 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.util.List; - -import javax.annotation.Nullable; - -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; -import org.apache.hadoop.io.Writable; - -import com.salesforce.phoenix.client.KeyValueBuilder; -import com.salesforce.phoenix.index.IndexMaintainer; -import com.salesforce.phoenix.schema.stat.PTableStats; - - -/** - * Definition of a Phoenix table - * - * @author wmacklem,jtaylor - * @since 0.1 - */ -public interface PTable extends Writable { - public static final long INITIAL_SEQ_NUM = 0; - public static final String IS_IMMUTABLE_ROWS_PROP_NAME = "IMMUTABLE_ROWS"; - public static final boolean DEFAULT_DISABLE_WAL = false; - - public enum ViewType { - MAPPED((byte)1), - READ_ONLY((byte)2), - UPDATABLE((byte)3); - - private final byte[] byteValue; - private final byte serializedValue; - - ViewType(byte serializedValue) { - this.serializedValue = serializedValue; - this.byteValue = Bytes.toBytes(this.name()); - } - - public byte[] getBytes() { - return byteValue; - } - - public boolean isReadOnly() { - return this != UPDATABLE; - } - - public byte getSerializedValue() { - return this.serializedValue; - } - - public static ViewType fromSerializedValue(byte serializedValue) { - if (serializedValue < 1 || serializedValue > ViewType.values().length) { - throw new IllegalArgumentException("Invalid ViewType " + serializedValue); - } - return ViewType.values()[serializedValue-1]; - } - - public static ViewType combine(ViewType type1, ViewType type2) { - if (type1 == null) { - return type2; - } - if (type2 == null) { - return type1; - } - if (type1 == UPDATABLE && type2 == UPDATABLE) { - return UPDATABLE; - } - return READ_ONLY; - } - } - - long getTimeStamp(); - long getSequenceNumber(); - /** - * @return table name - */ - PName getName(); - PName getSchemaName(); - PName getTableName(); - - /** - * @return the table type - */ - PTableType getType(); - - PName getPKName(); - - /** - * Get the PK columns ordered by position. - * @return a list of the PK columns - */ - List getPKColumns(); - - /** - * Get all columns ordered by position. - * @return a list of all columns - */ - List getColumns(); - - /** - * @return A list of the column families of this table - * ordered by position. - */ - List getColumnFamilies(); - - /** - * Get the column family with the given name - * @param family the column family name - * @return the PColumnFamily with the given name - * @throws ColumnFamilyNotFoundException if the column family cannot be found - */ - PColumnFamily getColumnFamily(byte[] family) throws ColumnFamilyNotFoundException; - - PColumnFamily getColumnFamily(String family) throws ColumnFamilyNotFoundException; - - /** - * Get the column with the given string name. - * @param name the column name - * @return the PColumn with the given name - * @throws ColumnNotFoundException if no column with the given name - * can be found - * @throws AmbiguousColumnException if multiple columns are found with the given name - */ - PColumn getColumn(String name) throws ColumnNotFoundException, AmbiguousColumnException; - - /** - * Get the PK column with the given name. - * @param name the column name - * @return the PColumn with the given name - * @throws ColumnNotFoundException if no PK column with the given name - * can be found - * @throws ColumnNotFoundException - */ - PColumn getPKColumn(String name) throws ColumnNotFoundException; - - /** - * Creates a new row at the specified timestamp using the key - * for the PK values (from {@link #newKey(ImmutableBytesWritable, byte[][])} - * and the optional key values specified using values. - * @param ts the timestamp that the key value will have when committed - * @param key the row key of the key value - * @param values the optional key values - * @return the new row. Use {@link com.salesforce.phoenix.schema.PRow#toRowMutations()} to - * generate the Row to send to the HBase server. - * @throws ConstraintViolationException if row data violates schema - * constraint - */ - PRow newRow(KeyValueBuilder builder, long ts, ImmutableBytesWritable key, byte[]... values); - - /** - * Creates a new row for the PK values (from {@link #newKey(ImmutableBytesWritable, byte[][])} - * and the optional key values specified using values. The timestamp of the key value - * will be set by the HBase server. - * @param key the row key of the key value - * @param values the optional key values - * @return the new row. Use {@link com.salesforce.phoenix.schema.PRow#toRowMutations()} to - * generate the row to send to the HBase server. - * @throws ConstraintViolationException if row data violates schema - * constraint - */ - PRow newRow(KeyValueBuilder builder, ImmutableBytesWritable key, byte[]... values); - - /** - * Formulates a row key using the values provided. The values must be in - * the same order as {@link #getPKColumns()}. - * @param key bytes pointer that will be filled in with the row key - * @param values the PK column values - * @return the number of values that were used from values to set - * the row key - */ - int newKey(ImmutableBytesWritable key, byte[][] values); - - /** - * Return the statistics table associated with this PTable. - * @return the statistics table. - */ - PTableStats getTableStats(); - - RowKeySchema getRowKeySchema(); - - /** - * Return the number of buckets used by this table for salting. If the table does - * not use salting, returns null. - * @return number of buckets used by this table for salting, or null if salting is not used. - */ - Integer getBucketNum(); - - /** - * Return the list of indexes defined on this table. - * @return the list of indexes. - */ - List getIndexes(); - - /** - * For a table of index type, return the state of the table. - * @return the state of the index. - */ - PIndexState getIndexState(); - - /** - * Gets the full name of the data table for an index table. - * @return the name of the data table that this index is on - * or null if not an index. - */ - PName getParentName(); - /** - * Gets the table name of the data table for an index table. - * @return the table name of the data table that this index is - * on or null if not an index. - */ - PName getParentTableName(); - - /** - * For a tenant-specific table, return the name of table in Phoenix that physically stores data. - * @return the name of the data table that tenant-specific table points to or null if this table is not tenant-specifidc. - * @see #isDerivedTable() - */ - @Nullable PName getBaseName(); - @Nullable PName getBaseSchemaName(); - @Nullable PName getBaseTableName(); - - PName getPhysicalName(); - boolean isImmutableRows(); - - void getIndexMaintainers(ImmutableBytesWritable ptr); - IndexMaintainer getIndexMaintainer(PTable dataTable); - PName getDefaultFamilyName(); - String getViewExpression(); - - boolean isWALDisabled(); - boolean isMultiTenant(); - ViewType getViewType(); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PTableType.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PTableType.java deleted file mode 100644 index 684165c6..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PTableType.java +++ /dev/null @@ -1,119 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.util.Map; - -import com.google.common.collect.Maps; - - -public enum PTableType { - SYSTEM("s", "SYSTEM TABLE"), - TABLE("u", "TABLE"), - VIEW("v", "VIEW"), - INDEX("i", "INDEX"), - JOIN("j", "JOIN"); - - private final PName value; - private final String serializedValue; - - private PTableType(String serializedValue, String value) { - this.serializedValue = serializedValue; - this.value = PNameFactory.newName(value); - } - - public String getSerializedValue() { - return serializedValue; - } - - public PName getValue() { - return value; - } - - @Override - public String toString() { - return value.getString(); - } - - private static final PTableType[] FROM_SERIALIZED_VALUE; - private static final int FROM_SERIALIZED_VALUE_OFFSET; - private static final Map FROM_VALUE = Maps.newHashMapWithExpectedSize(PTableType.values().length); - - static { - int minChar = Integer.MAX_VALUE; - int maxChar = Integer.MIN_VALUE; - for (PTableType type : PTableType.values()) { - char c = type.getSerializedValue().charAt(0); - if (c < minChar) { - minChar = c; - } - if (c > maxChar) { - maxChar = c; - } - } - FROM_SERIALIZED_VALUE_OFFSET = minChar; - FROM_SERIALIZED_VALUE = new PTableType[maxChar - minChar + 1]; - for (PTableType type : PTableType.values()) { - FROM_SERIALIZED_VALUE[type.getSerializedValue().charAt(0) - minChar] = type; - } - } - - static { - for (PTableType type : PTableType.values()) { - if (FROM_VALUE.put(type.getValue().getString(),type) != null) { - throw new IllegalStateException("Duplicate PTableType value of " + type.getValue().getString() + " is not allowed"); - } - } - } - - public static PTableType fromValue(String value) { - PTableType type = FROM_VALUE.get(value); - if (type == null) { - throw new IllegalArgumentException("Unable to PTableType enum for value of '" + value + "'"); - } - return type; - } - - public static PTableType fromSerializedValue(String serializedValue) { - if (serializedValue.length() == 1) { - int i = serializedValue.charAt(0) - FROM_SERIALIZED_VALUE_OFFSET; - if (i >= 0 && i < FROM_SERIALIZED_VALUE.length && FROM_SERIALIZED_VALUE[i] != null) { - return FROM_SERIALIZED_VALUE[i]; - } - } - throw new IllegalArgumentException("Unable to PTableType enum for serialized value of '" + serializedValue + "'"); - } - - public static PTableType fromSerializedValue(byte serializedByte) { - int i = serializedByte - FROM_SERIALIZED_VALUE_OFFSET; - if (i >= 0 && i < FROM_SERIALIZED_VALUE.length && FROM_SERIALIZED_VALUE[i] != null) { - return FROM_SERIALIZED_VALUE[i]; - } - throw new IllegalArgumentException("Unable to PTableType enum for serialized value of '" + (char)serializedByte + "'"); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ReadOnlyTableException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ReadOnlyTableException.java deleted file mode 100644 index 7ed6e3b7..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ReadOnlyTableException.java +++ /dev/null @@ -1,65 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - -/** - * - * Exception thrown when an attempt is made to modify or write to a read-only table. - * - * @author jtaylor - * @since 0.1 - */ -public class ReadOnlyTableException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.READ_ONLY_TABLE; - - public ReadOnlyTableException(String schemaName, String tableName) { - super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).toString(), - code.getSQLState(), code.getErrorCode()); - } - - public ReadOnlyTableException(String message, String schemaName, String tableName) { - super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).setMessage(message).toString(), - code.getSQLState(), code.getErrorCode()); - } - - public ReadOnlyTableException(String message, String schemaName, String tableName, Throwable cause) { - super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).setRootCause(cause).setMessage(message).toString(), - code.getSQLState(), code.getErrorCode(), cause); - } - - public ReadOnlyTableException(String message, String schemaName, String tableName, String familyName) { - super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).setFamilyName(familyName).setMessage(message).toString(), - code.getSQLState(), code.getErrorCode()); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/SequenceAlreadyExistsException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/SequenceAlreadyExistsException.java deleted file mode 100644 index 4612c4da..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/SequenceAlreadyExistsException.java +++ /dev/null @@ -1,56 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - - -public class SequenceAlreadyExistsException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.SEQUENCE_ALREADY_EXIST; - private final String schemaName; - private final String sequenceName; - - - public SequenceAlreadyExistsException(String schemaName, String sequenceName) { - super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(sequenceName).build().toString(), - code.getSQLState(), code.getErrorCode()); - this.schemaName = schemaName; - this.sequenceName = sequenceName; - } - - public String getSequenceName() { - return sequenceName; - } - - public String getSchemaName() { - return schemaName; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/SequenceKey.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/SequenceKey.java deleted file mode 100644 index bfc33b60..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/SequenceKey.java +++ /dev/null @@ -1,84 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - - -public class SequenceKey implements Comparable { - @Override - public int hashCode() { - final int prime = 31; - int result = 1; - result = prime * result + ((tenantId == null) ? 0 : tenantId.hashCode()); - result = prime * result + ((schemaName == null) ? 0 : schemaName.hashCode()); - result = prime * result + sequenceName.hashCode(); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - SequenceKey other = (SequenceKey)obj; - return this.compareTo(other) == 0; - } - - private final String tenantId; - private final String schemaName; - private final String sequenceName; - - public SequenceKey(String tenantId, String schemaName, String sequenceName) { - this.tenantId = tenantId; - this.schemaName = schemaName; - this.sequenceName = sequenceName; - } - - public String getTenantId() { - return tenantId; - } - - public String getSchemaName() { - return schemaName; - } - - public String getSequenceName() { - return sequenceName; - } - - @Override - public int compareTo(SequenceKey that) { - int c = this.tenantId == that.getTenantId() ? 0 : this.tenantId == null ? -1 : that.getTenantId() == null ? 1 : this.tenantId.compareTo(that.getTenantId()); - if (c == 0) { - c = this.schemaName == that.getSchemaName() ? 0 : this.schemaName == null ? -1 : that.getSchemaName() == null ? 1 : this.schemaName.compareTo(that.getSchemaName()); - if (c == 0) { - return sequenceName.compareTo(that.getSequenceName()); - } - } - return c; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/SequenceNotFoundException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/SequenceNotFoundException.java deleted file mode 100644 index 56f11fc5..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/SequenceNotFoundException.java +++ /dev/null @@ -1,58 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - - -public class SequenceNotFoundException extends MetaDataEntityNotFoundException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.SEQUENCE_UNDEFINED; - private final String schemaName; - private final String tableName; - - public SequenceNotFoundException(String tableName) { - this(null, tableName); - } - - public SequenceNotFoundException(String schemaName, String tableName) { - super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).build().toString(), - code.getSQLState(), code.getErrorCode(), null); - this.tableName = tableName; - this.schemaName = schemaName; - } - - public String getTableName() { - return tableName; - } - - public String getSchemaName() { - return schemaName; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TableAlreadyExistsException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TableAlreadyExistsException.java deleted file mode 100644 index 7bcbe477..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TableAlreadyExistsException.java +++ /dev/null @@ -1,63 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - - -/** - * - * Exception thrown when a table name could not be found in the schema - * - * @author jtaylor - * @since 0.1 - */ -public class TableAlreadyExistsException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.TABLE_ALREADY_EXIST; - private final String schemaName; - private final String tableName; - - public TableAlreadyExistsException(String schemaName, String tableName) { - super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).build().toString(), - code.getSQLState(), code.getErrorCode()); - this.schemaName = schemaName; - this.tableName = tableName; - } - - public String getTableName() { - return tableName; - } - - public String getSchemaName() { - return schemaName; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TableNotFoundException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TableNotFoundException.java deleted file mode 100644 index ae604d7a..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TableNotFoundException.java +++ /dev/null @@ -1,81 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import org.apache.hadoop.hbase.HConstants; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - - -/** - * - * Exception thrown when a table name could not be found in the schema - * - * @author jtaylor - * @since 0.1 - */ -public class TableNotFoundException extends MetaDataEntityNotFoundException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.TABLE_UNDEFINED; - private final String schemaName; - private final String tableName; - private final long timestamp; - - public TableNotFoundException(TableNotFoundException e, long timestamp) { - this(e.schemaName,e.tableName, timestamp); - } - - public TableNotFoundException(String tableName) { - this(null, tableName); - } - - public TableNotFoundException(String schemaName, String tableName) { - this(schemaName, tableName, HConstants.LATEST_TIMESTAMP); - } - - public TableNotFoundException(String schemaName, String tableName, long timestamp) { - super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).build().toString(), - code.getSQLState(), code.getErrorCode(), null); - this.tableName = tableName; - this.schemaName = schemaName; - this.timestamp = timestamp; - } - - public String getTableName() { - return tableName; - } - - public String getSchemaName() { - return schemaName; - } - - public long getTimeStamp() { - return timestamp; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TableRef.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TableRef.java deleted file mode 100644 index b91c4a19..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TableRef.java +++ /dev/null @@ -1,90 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import org.apache.hadoop.hbase.HConstants; - - - -public final class TableRef { - private final PTable table; - private final String alias; - private final long timeStamp; - private final boolean hasDynamicCols; - - public TableRef(TableRef tableRef, long timeStamp) { - this(tableRef.alias, tableRef.table, timeStamp, tableRef.hasDynamicCols); - } - - public TableRef(PTable table) { - this(null, table, HConstants.LATEST_TIMESTAMP, false); - } - - public TableRef(String alias, PTable table, long timeStamp, boolean hasDynamicCols) { - this.alias = alias; - this.table = table; - this.timeStamp = timeStamp; - this.hasDynamicCols = hasDynamicCols; - } - - public PTable getTable() { - return table; - } - - public String getTableAlias() { - return alias; - } - - @Override - public int hashCode() { - final int prime = 31; - int result = alias == null ? 0 : alias.hashCode(); - result = prime * result + this.table.getName().getString().hashCode(); - return result; - } - - @Override - public boolean equals(Object obj) { - if (this == obj) return true; - if (obj == null) return false; - if (getClass() != obj.getClass()) return false; - TableRef other = (TableRef)obj; - if ((alias == null && other.alias != null) || (alias != null && !alias.equals(other.alias))) return false; - if (!table.getName().getString().equals(other.table.getName().getString())) return false; - return true; - } - - public long getTimeStamp() { - return timeStamp; - } - - public boolean hasDynamicCols() { - return hasDynamicCols; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TypeMismatchException.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TypeMismatchException.java deleted file mode 100644 index 019776cd..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/TypeMismatchException.java +++ /dev/null @@ -1,68 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - -/** - * Exception thrown when we try to convert one type into a different incompatible type. - * - * @author zhuang - * @since 1.0 - */ -public class TypeMismatchException extends SQLException { - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.TYPE_MISMATCH; - - public TypeMismatchException(String msg) { - super(new SQLExceptionInfo.Builder(code).setMessage(msg).build().toString(), code.getSQLState(), code.getErrorCode()); - } - - public static TypeMismatchException newException(PDataType lhs) { - return new TypeMismatchException(getMessage(lhs,null,null)); - } - - public static TypeMismatchException newException(PDataType lhs, String location) { - return new TypeMismatchException(getMessage(lhs,null,location)); - } - - public static TypeMismatchException newException(PDataType lhs, PDataType rhs) { - return new TypeMismatchException(getMessage(lhs,rhs,null)); - } - - public static TypeMismatchException newException(PDataType lhs, PDataType rhs, String location) { - return new TypeMismatchException(getMessage(lhs,rhs,location)); - } - - public static String getMessage(PDataType lhs, PDataType rhs, String location) { - return lhs + (rhs == null ? "" : " and " + rhs) + (location == null ? "" : " for " + location); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ValueRangeExcpetion.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ValueRangeExcpetion.java deleted file mode 100644 index 85a3c713..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ValueRangeExcpetion.java +++ /dev/null @@ -1,48 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.sql.SQLException; - -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; - -/** - * Exception thrown when we try to use use an argument that has the wrong type. - * - * @author anoopsjohn - * @since 1.1.2 - */ -public class ValueRangeExcpetion extends SQLException{ - private static final long serialVersionUID = 1L; - private static SQLExceptionCode code = SQLExceptionCode.VALUE_OUTSIDE_RANGE; - - public ValueRangeExcpetion(Object minValue, Object maxValue, Object actualValue, String location){ - super(new SQLExceptionInfo.Builder(code).setMessage("expected: [" + minValue + " , " + maxValue + "] but was: " + actualValue + " at " + location).build().toString(), code.getSQLState(), code.getErrorCode()); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/stat/PTableStats.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/stat/PTableStats.java deleted file mode 100644 index 80519d18..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/stat/PTableStats.java +++ /dev/null @@ -1,55 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema.stat; - -import java.io.DataOutput; -import java.io.IOException; - -import org.apache.hadoop.hbase.HRegionInfo; - - -/** - * Interface for Phoenix table statistics. Statistics is collected on the server - * side and can be used for various purpose like splitting region for scanning, etc. - * - * The table is defined on the client side, but it is populated on the server side. The client - * should not populate any data to the statistics object. - */ -public interface PTableStats { - - /** - * Given the region info, returns an array of bytes that is the current estimate of key - * distribution inside that region. The keys should split that region into equal chunks. - * - * @param region - * @return array of keys - */ - byte[][] getRegionGuidePosts(HRegionInfo region); - - void write(DataOutput output) throws IOException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/stat/PTableStatsImpl.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/stat/PTableStatsImpl.java deleted file mode 100644 index 637c1cb9..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/stat/PTableStatsImpl.java +++ /dev/null @@ -1,78 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema.stat; - -import java.io.DataOutput; -import java.io.IOException; -import java.util.Map; -import java.util.Map.Entry; - -import org.apache.hadoop.hbase.HRegionInfo; -import org.apache.hadoop.hbase.util.Bytes; -import org.apache.hadoop.io.WritableUtils; - -import com.google.common.collect.ImmutableMap; - - -/** - * Implementation for PTableStats. - */ -public class PTableStatsImpl implements PTableStats { - - // The map for guide posts should be immutable. We only take the current snapshot from outside - // method call and store it. - private Map regionGuidePosts; - - public PTableStatsImpl() { } - - public PTableStatsImpl(Map stats) { - regionGuidePosts = ImmutableMap.copyOf(stats); - } - - @Override - public byte[][] getRegionGuidePosts(HRegionInfo region) { - return regionGuidePosts.get(region.getRegionNameAsString()); - } - - @Override - public void write(DataOutput output) throws IOException { - if (regionGuidePosts == null) { - WritableUtils.writeVInt(output, 0); - return; - } - WritableUtils.writeVInt(output, regionGuidePosts.size()); - for (Entry entry : regionGuidePosts.entrySet()) { - WritableUtils.writeString(output, entry.getKey()); - byte[][] value = entry.getValue(); - WritableUtils.writeVInt(output, value.length); - for (int i=0; i values; - - public MultiKeyValueTuple(List values) { - setKeyValues(values); - } - - public MultiKeyValueTuple() { - } - - public void setKeyValues(List values) { - this.values = ImmutableList.copyOf(values); - } - - @Override - public void getKey(ImmutableBytesWritable ptr) { - KeyValue value = values.get(0); - ptr.set(value.getBuffer(), value.getRowOffset(), value.getRowLength()); - } - - @Override - public boolean isImmutable() { - return true; - } - - @Override - public KeyValue getValue(byte[] family, byte[] qualifier) { - return KeyValueUtil.getColumnLatest(values, family, qualifier); - } - - @Override - public String toString() { - return values.toString(); - } - - @Override - public int size() { - return values.size(); - } - - @Override - public KeyValue getValue(int index) { - return values.get(index); - } - - @Override - public boolean getValue(byte[] family, byte[] qualifier, - ImmutableBytesWritable ptr) { - KeyValue kv = getValue(family, qualifier); - if (kv == null) - return false; - ptr.set(kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); - return true; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/tuple/ResultTuple.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/tuple/ResultTuple.java deleted file mode 100644 index 1b28c48b..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/tuple/ResultTuple.java +++ /dev/null @@ -1,112 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema.tuple; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.client.Result; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.phoenix.util.ResultUtil; - - -public class ResultTuple implements Tuple { - private Result result; - - public ResultTuple(Result result) { - this.result = result; - } - - public ResultTuple() { - } - - public Result getResult() { - return this.result; - } - - public void setResult(Result result) { - this.result = result; - } - - @Override - public void getKey(ImmutableBytesWritable ptr) { - ResultUtil.getKey(result, ptr); - } - - @Override - public boolean isImmutable() { - return true; - } - - @Override - public KeyValue getValue(byte[] family, byte[] qualifier) { - return result.getColumnLatest(family, qualifier); - } - - @Override - public String toString() { - StringBuilder sb = new StringBuilder(); - sb.append("keyvalues="); - if(this.result.isEmpty()) { - sb.append("NONE"); - return sb.toString(); - } - sb.append("{"); - boolean moreThanOne = false; - for(KeyValue kv : this.result.list()) { - if(moreThanOne) { - sb.append(", \n"); - } else { - moreThanOne = true; - } - sb.append(kv.toString()+"/value="+Bytes.toString(kv.getValue())); - } - sb.append("}\n"); - return sb.toString(); - } - - @Override - public int size() { - return result.size(); - } - - @Override - public KeyValue getValue(int index) { - return result.raw()[index]; - } - - @Override - public boolean getValue(byte[] family, byte[] qualifier, - ImmutableBytesWritable ptr) { - KeyValue kv = getValue(family, qualifier); - if (kv == null) - return false; - ptr.set(kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); - return true; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/tuple/SingleKeyValueTuple.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/tuple/SingleKeyValueTuple.java deleted file mode 100644 index f08b0786..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/tuple/SingleKeyValueTuple.java +++ /dev/null @@ -1,119 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema.tuple; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; - - -public class SingleKeyValueTuple implements Tuple { - private static final byte[] UNITIALIZED_KEY_BUFFER = new byte[0]; - private KeyValue keyValue; - private ImmutableBytesWritable keyPtr = new ImmutableBytesWritable(UNITIALIZED_KEY_BUFFER); - - public SingleKeyValueTuple() { - } - - public SingleKeyValueTuple(KeyValue keyValue) { - if (keyValue == null) { - throw new NullPointerException(); - } - setKeyValue(keyValue); - } - - public boolean hasKey() { - return keyPtr.get() != UNITIALIZED_KEY_BUFFER; - } - - public void reset() { - this.keyValue = null; - keyPtr.set(UNITIALIZED_KEY_BUFFER); - } - - public void setKeyValue(KeyValue keyValue) { - if (keyValue == null) { - throw new IllegalArgumentException(); - } - this.keyValue = keyValue; - setKey(keyValue); - } - - public void setKey(ImmutableBytesWritable ptr) { - keyPtr.set(ptr.get(), ptr.getOffset(), ptr.getLength()); - } - - public void setKey(KeyValue keyValue) { - if (keyValue == null) { - throw new IllegalArgumentException(); - } - keyPtr.set(keyValue.getBuffer(), keyValue.getRowOffset(), keyValue.getRowLength()); - } - - @Override - public void getKey(ImmutableBytesWritable ptr) { - ptr.set(keyPtr.get(), keyPtr.getOffset(), keyPtr.getLength()); - } - - @Override - public KeyValue getValue(byte[] cf, byte[] cq) { - return keyValue; - } - - @Override - public boolean isImmutable() { - return true; - } - - @Override - public String toString() { - return "SingleKeyValueTuple[" + keyValue == null ? keyPtr.get() == UNITIALIZED_KEY_BUFFER ? "null" : Bytes.toStringBinary(keyPtr.get(),keyPtr.getOffset(),keyPtr.getLength()) : keyValue.toString() + "]"; - } - - @Override - public int size() { - return keyValue == null ? 0 : 1; - } - - @Override - public KeyValue getValue(int index) { - if (index != 0 || keyValue == null) { - throw new IndexOutOfBoundsException(Integer.toString(index)); - } - return keyValue; - } - - @Override - public boolean getValue(byte[] family, byte[] qualifier, - ImmutableBytesWritable ptr) { - if (keyValue == null) - return false; - ptr.set(keyValue.getBuffer(), keyValue.getValueOffset(), keyValue.getValueLength()); - return true; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/tuple/Tuple.java b/phoenix-core/src/main/java/com/salesforce/phoenix/schema/tuple/Tuple.java deleted file mode 100644 index ee3781bb..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/tuple/Tuple.java +++ /dev/null @@ -1,92 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema.tuple; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; - -/** - * - * Interface representing an ordered list of KeyValues returned as the - * result of a query. Each tuple represents a row (i.e. all its KeyValues - * will have the same key), and each KeyValue represents a column value. - * - * @author jtaylor - * @since 0.1 - */ -public interface Tuple { - /** - * @return Number of KeyValues contained by the Tuple. - */ - public int size(); - - /** - * Determines whether or not the Tuple is immutable (the typical case) - * or will potentially have additional KeyValues added to it (the case - * during filter evaluation when we see one KeyValue at a time). - * @return true if Tuple is immutable and false otherwise. - */ - public boolean isImmutable(); - - /** - * Get the row key for the Tuple - * @param ptr the bytes pointer that will be updated to point to - * the key buffer. - */ - public void getKey(ImmutableBytesWritable ptr); - - /** - * Get the KeyValue at the given index. - * @param index the zero-based KeyValue index between 0 and {@link #size()} exclusive - * @return the KeyValue at the given index - * @throws IndexOutOfBoundsException if an invalid index is used - */ - public KeyValue getValue(int index); - - /*** - * Get the KeyValue contained by the Tuple with the given family and - * qualifier name. - * @param family the column family of the KeyValue being retrieved - * @param qualifier the column qualify of the KeyValue being retrieved - * @return the KeyValue with the given family and qualifier name or - * null if not found. - */ - public KeyValue getValue(byte [] family, byte [] qualifier); - - /*** - * Get the value byte array of the KeyValue contained by the Tuple with - * the given family and qualifier name. - * @param family the column family of the KeyValue being retrieved - * @param qualifier the column qualify of the KeyValue being retrieved - * @param ptr the bytes pointer that will be updated to point to the - * value buffer. - * @return true if the KeyValue with the given family and qualifier name - * exists; otherwise false. - */ - public boolean getValue(byte [] family, byte [] qualifier, ImmutableBytesWritable ptr); -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/BigDecimalUtil.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/BigDecimalUtil.java deleted file mode 100644 index 80e09409..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/BigDecimalUtil.java +++ /dev/null @@ -1,78 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import org.apache.hadoop.hbase.util.Pair; - -/** - * - * @author anoopsjohn - * @since 1.2.1 - */ -public class BigDecimalUtil { - - /** - * Calculates the precision and scale for BigDecimal arithmetic operation results. It uses the algorithm mentioned - * here - * @param lp precision of the left operand - * @param ls scale of the left operand - * @param rp precision of the right operand - * @param rs scale of the right operand - * @param op The operation type - * @return {@link Pair} comprising of the precision and scale. - */ - public static Pair getResultPrecisionScale(int lp, int ls, int rp, int rs, Operation op) { - int resultPrec = 0, resultScale = 0; - switch (op) { - case MULTIPLY: - resultPrec = lp + rp; - resultScale = ls + rs; - break; - case DIVIDE: - resultPrec = lp - ls + rp + Math.max(ls + rp - rs + 1, 4); - resultScale = 31 - lp + ls - rs; - break; - case ADD: - resultPrec = 2 * (lp - ls) + ls; // Is this correct? The page says addition -> 2 * (p - s) + s. - resultScale = Math.max(ls, rs); - break; - case AVG: - resultPrec = Math.max(lp - ls, rp - rs) + 1 + Math.max(ls, rs); - resultScale = Math.max(Math.max(ls, rs), 4); - break; - case OTHERS: - resultPrec = Math.max(lp - ls, rp - rs) + 1 + Math.max(ls, rs); - resultScale = Math.max(ls, rs); - } - return new Pair(resultPrec, resultScale); - } - - public static enum Operation { - MULTIPLY, DIVIDE, ADD, AVG, OTHERS; - } -} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/Closeables.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/Closeables.java deleted file mode 100644 index 925d1d2f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/Closeables.java +++ /dev/null @@ -1,132 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import java.io.Closeable; -import java.io.IOException; -import java.util.*; - -import com.google.common.collect.Iterables; - - -/** - * Utilities for operating on {@link Closeable}s. - * - */ -public class Closeables { - /** Not constructed */ - private Closeables() { } - - /** - * Allows you to close as many of the {@link Closeable}s as possible. - * - * If any of the close's fail with an IOException, those exception(s) will - * be thrown after attempting to close all of the inputs. - */ - public static void closeAll(Iterable iterable) throws IOException { - IOException ex = closeAllQuietly(iterable); - if (ex != null) throw ex; - } - - public static IOException closeAllQuietly(Iterable iterable) { - if (iterable == null) return null; - - LinkedList exceptions = null; - for (Closeable closeable : iterable) { - try { - closeable.close(); - } catch (IOException x) { - if (exceptions == null) exceptions = new LinkedList(); - exceptions.add(x); - } - } - - IOException ex = MultipleCausesIOException.fromIOExceptions(exceptions); - return ex; - } - - static private class MultipleCausesIOException extends IOException { - private static final long serialVersionUID = 1L; - - static IOException fromIOExceptions(Collection exceptions) { - if (exceptions == null || exceptions.isEmpty()) return null; - if (exceptions.size() == 1) return Iterables.getOnlyElement(exceptions); - - return new MultipleCausesIOException(exceptions); - } - - private final Collection exceptions; - private boolean hasSetStackTrace; - - /** - * Use the {@link #fromIOExceptions(Collection) factory}. - */ - private MultipleCausesIOException(Collection exceptions) { - this.exceptions = exceptions; - } - - @Override - public String getMessage() { - StringBuilder sb = new StringBuilder(this.exceptions.size() * 50); - int exceptionNum = 0; - for (IOException ex : this.exceptions) { - sb.append("Cause Number " + exceptionNum + ": " + ex.getMessage() + "\n"); - exceptionNum++; - } - return sb.toString(); - } - - @Override - public StackTraceElement[] getStackTrace() { - if (!this.hasSetStackTrace) { - ArrayList frames = new ArrayList(this.exceptions.size() * 20); - - int exceptionNum = 0; - for (IOException exception : this.exceptions) { - StackTraceElement header = new StackTraceElement(MultipleCausesIOException.class.getName(), - "Exception Number " + exceptionNum, - "", - 0); - - frames.add(header); - for (StackTraceElement ste : exception.getStackTrace()) { - frames.add(ste); - } - exceptionNum++; - } - - setStackTrace(frames.toArray(new StackTraceElement[frames.size()])); - this.hasSetStackTrace = true; - } - - return super.getStackTrace(); - } - - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/DateUtil.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/DateUtil.java deleted file mode 100644 index 197e8676..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/DateUtil.java +++ /dev/null @@ -1,162 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import java.math.BigDecimal; -import java.sql.Date; -import java.sql.Time; -import java.sql.Timestamp; -import java.text.Format; -import java.text.ParseException; -import java.text.SimpleDateFormat; -import java.util.TimeZone; - -import org.apache.commons.lang.time.FastDateFormat; - -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.IllegalDataException; - - - -@SuppressWarnings("serial") -public class DateUtil { - public static final TimeZone DATE_TIME_ZONE = TimeZone.getTimeZone("GMT"); - public static final String DEFAULT_DATE_FORMAT = "yyyy-MM-dd HH:mm:ss"; // This is the format the app sets in NLS settings for every connection. - public static final Format DEFAULT_DATE_FORMATTER = FastDateFormat.getInstance(DEFAULT_DATE_FORMAT, DATE_TIME_ZONE); - - public static final String DEFAULT_MS_DATE_FORMAT = "yyyy-MM-dd HH:mm:ss.SSS"; - public static final Format DEFAULT_MS_DATE_FORMATTER = FastDateFormat.getInstance(DEFAULT_MS_DATE_FORMAT, DATE_TIME_ZONE); - - private DateUtil() { - } - - public static Format getDateParser(String pattern) { - SimpleDateFormat format = new SimpleDateFormat(pattern) { - @Override - public java.util.Date parseObject(String source) throws ParseException { - java.util.Date date = super.parse(source); - return new java.sql.Date(date.getTime()); - } - }; - format.setTimeZone(DateUtil.DATE_TIME_ZONE); - return format; - } - - public static Format getTimeParser(String pattern) { - SimpleDateFormat format = new SimpleDateFormat(pattern) { - @Override - public java.util.Date parseObject(String source) throws ParseException { - java.util.Date date = super.parse(source); - return new java.sql.Time(date.getTime()); - } - }; - format.setTimeZone(DateUtil.DATE_TIME_ZONE); - return format; - } - - public static Format getTimestampParser(String pattern) { - SimpleDateFormat format = new SimpleDateFormat(pattern) { - @Override - public java.util.Date parseObject(String source) throws ParseException { - java.util.Date date = super.parse(source); - return new java.sql.Timestamp(date.getTime()); - } - }; - format.setTimeZone(DateUtil.DATE_TIME_ZONE); - return format; - } - - public static Format getDateFormatter(String pattern) { - return DateUtil.DEFAULT_DATE_FORMAT.equals(pattern) ? DateUtil.DEFAULT_DATE_FORMATTER : FastDateFormat.getInstance(pattern, DateUtil.DATE_TIME_ZONE); - } - - private static ThreadLocal dateFormat = - new ThreadLocal < Format > () { - @Override protected Format initialValue() { - return getDateParser(DEFAULT_DATE_FORMAT); - } - }; - - public static Date parseDate(String dateValue) { - try { - return (Date)dateFormat.get().parseObject(dateValue); - } catch (ParseException e) { - throw new IllegalDataException(e); - } - } - - private static ThreadLocal timeFormat = - new ThreadLocal < Format > () { - @Override protected Format initialValue() { - return getTimeParser(DEFAULT_DATE_FORMAT); - } - }; - - public static Time parseTime(String timeValue) { - try { - return (Time)timeFormat.get().parseObject(timeValue); - } catch (ParseException e) { - throw new IllegalDataException(e); - } - } - - private static ThreadLocal timestampFormat = - new ThreadLocal < Format > () { - @Override protected Format initialValue() { - return getTimestampParser(DEFAULT_DATE_FORMAT); - } - }; - - public static Timestamp parseTimestamp(String timeValue) { - try { - return (Timestamp)timestampFormat.get().parseObject(timeValue); - } catch (ParseException e) { - throw new IllegalDataException(e); - } - } - - /** - * Utility function to work around the weirdness of the {@link Timestamp} constructor. - * This method takes the milli-seconds that spills over to the nanos part as part of - * constructing the {@link Timestamp} object. - * If we just set the nanos part of timestamp to the nanos passed in param, we - * end up losing the sub-second part of timestamp. - */ - public static Timestamp getTimestamp(long millis, int nanos) { - Timestamp ts = new Timestamp(millis); - ts.setNanos(ts.getNanos() + nanos); - return ts; - } - - /** - * Utility function to convert a {@link BigDecimal} value to {@link Timestamp}. - */ - public static Timestamp getTimestamp(BigDecimal bd) { - return DateUtil.getTimestamp(bd.longValue(), ((bd.remainder(BigDecimal.ONE).multiply(BigDecimal.valueOf(QueryConstants.MILLIS_TO_NANOS_CONVERTOR))).intValue())); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/InstanceResolver.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/InstanceResolver.java deleted file mode 100644 index 9b06e719..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/InstanceResolver.java +++ /dev/null @@ -1,74 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import java.util.ServiceLoader; -import java.util.concurrent.ConcurrentHashMap; - -/** - * Resolves object instances registered using the JDK 6+ {@link java.util.ServiceLoader}. - * - * @author aaraujo - * @since 2.0 - */ -public class InstanceResolver { - private static final ConcurrentHashMap RESOLVED_SINGLETONS = new ConcurrentHashMap(); - - private InstanceResolver() {/* not allowed */} - - /** - * Resolves an instance of the specified class if it has not already been resolved. - * @param clazz The type of instance to resolve - * @param defaultInstance The instance to use if a custom instance has not been registered - * @return The resolved instance or the default instance provided. - * {@code null} if an instance is not registered and a default is not provided. - */ - @SuppressWarnings("unchecked") - public static T getSingleton(Class clazz, T defaultInstance) { - Object obj = RESOLVED_SINGLETONS.get(clazz); - if(obj != null) { - return (T)obj; - } - if (defaultInstance != null && !clazz.isInstance(defaultInstance)) throw new IllegalArgumentException("defaultInstance is not of type " + clazz.getName()); - final Object o = resolveSingleton(clazz, defaultInstance); - obj = RESOLVED_SINGLETONS.putIfAbsent(clazz, o); - if(obj == null) { - obj = o; - } - return (T)obj; - } - - private synchronized static T resolveSingleton(Class clazz, T defaultInstance) { - ServiceLoader loader = ServiceLoader.load(clazz); - // returns the first registered instance found - for (T singleton : loader) { - return singleton; - } - return defaultInstance; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/JDBCUtil.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/JDBCUtil.java deleted file mode 100644 index 24e2a298..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/JDBCUtil.java +++ /dev/null @@ -1,91 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import java.sql.SQLException; -import java.util.Properties; - -import javax.annotation.Nullable; - -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.PName; -import com.salesforce.phoenix.schema.PNameFactory; - - - -/** - * Utilities for JDBC - * - * @author jtaylor - * @since 178 - */ -public class JDBCUtil { - - private JDBCUtil() { - } - - /** - * Find the propName by first looking in the url string and if not found, - * next in the info properties. If not found, null is returned. - * @param url JDBC connection URL - * @param info JDBC connection properties - * @param propName the name of the property to find - * @return the property value or null if not found - */ - public static String findProperty(String url, Properties info, String propName) { - String urlPropName = ";" + propName + "="; - String propValue = info.getProperty(propName); - if (propValue == null) { - int begIndex = url.indexOf(urlPropName); - if (begIndex >= 0) { - int endIndex = url.indexOf(';',begIndex + urlPropName.length()); - if (endIndex < 0) { - endIndex = url.length(); - } - propValue = url.substring(begIndex + urlPropName.length(), endIndex); - } - } - return propValue; - } - - public static Long getCurrentSCN(String url, Properties info) throws SQLException { - String scnStr = findProperty(url, info, PhoenixRuntime.CURRENT_SCN_ATTRIB); - return (scnStr == null ? null : Long.parseLong(scnStr)); - } - - public static int getMutateBatchSize(String url, Properties info, ReadOnlyProps props) throws SQLException { - String batchSizeStr = findProperty(url, info, PhoenixRuntime.UPSERT_BATCH_SIZE_ATTRIB); - return (batchSizeStr == null ? props.getInt(QueryServices.MUTATE_BATCH_SIZE_ATTRIB, QueryServicesOptions.DEFAULT_MUTATE_BATCH_SIZE) : Integer.parseInt(batchSizeStr)); - } - - public static @Nullable PName getTenantId(String url, Properties info) throws SQLException { - String tenantId = findProperty(url, info, PhoenixRuntime.TENANT_ID_ATTRIB); - return (tenantId == null ? null : PNameFactory.newName(tenantId)); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/KeyValueUtil.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/KeyValueUtil.java deleted file mode 100644 index 7d496229..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/KeyValueUtil.java +++ /dev/null @@ -1,178 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import java.util.Collections; -import java.util.Comparator; -import java.util.List; - -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.KeyValue.Type; -import org.apache.hadoop.hbase.client.Result; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; - -/** - * - * Utilities for KeyValue. Where there's duplication with KeyValue methods, - * these avoid creating new objects when not necessary (primary preventing - * byte array copying). - * - * @author jtaylor - * @since 0.1 - */ -public class KeyValueUtil { - private KeyValueUtil() { - } - - public static KeyValue newKeyValue(Result r, byte[] cf, byte[] cq, long ts, byte[] value, int valueOffset, int valueLength) { - byte[] bytes = ResultUtil.getRawBytes(r); - return new KeyValue(bytes, ResultUtil.getKeyOffset(r), ResultUtil.getKeyLength(r), - cf, 0, cf.length, - cq, 0, cq.length, - ts, Type.Put, - value, valueOffset, valueLength); - } - - public static KeyValue newKeyValue(byte[] key, byte[] cf, byte[] cq, long ts, byte[] value, int valueOffset, int valueLength) { - return new KeyValue(key, 0, key.length, - cf, 0, cf.length, - cq, 0, cq.length, - ts, Type.Put, - value, valueOffset, valueLength); - } - - public static KeyValue newKeyValue(ImmutableBytesWritable key, byte[] cf, byte[] cq, long ts, byte[] value, int valueOffset, int valueLength) { - return new KeyValue(key.get(), key.getOffset(), key.getLength(), - cf, 0, cf.length, - cq, 0, cq.length, - ts, Type.Put, - value, valueOffset, valueLength); - } - - public static KeyValue newKeyValue(byte[] key, int keyOffset, int keyLength, byte[] cf, byte[] cq, long ts, byte[] value, int valueOffset, int valueLength) { - return new KeyValue(key, keyOffset, keyLength, - cf, 0, cf.length, - cq, 0, cq.length, - ts, Type.Put, - value, valueOffset, valueLength); - } - - public static KeyValue newKeyValue(byte[] key, byte[] cf, byte[] cq, long ts, byte[] value) { - return newKeyValue(key,cf,cq,ts,value,0,value.length); - } - - public static KeyValue newKeyValue(Result r, byte[] cf, byte[] cq, long ts, byte[] value) { - return newKeyValue(r,cf,cq,ts,value,0,value.length); - } - - /** - * Binary search for latest column value without allocating memory in the process - * @param kvs - * @param family - * @param qualifier - */ - public static KeyValue getColumnLatest(Listkvs, byte[] family, byte[] qualifier) { - if (kvs.size() == 0) { - return null; - } - KeyValue row = kvs.get(0); - Comparator comp = new SearchComparator(row.getBuffer(), row.getRowOffset(), row.getRowLength(), family, qualifier); - // pos === ( -(insertion point) - 1) - int pos = Collections.binarySearch(kvs, null, comp); - // never will exact match - if (pos < 0) { - pos = (pos+1) * -1; - // pos is now insertion point - } - if (pos == kvs.size()) { - return null; // doesn't exist - } - - KeyValue kv = kvs.get(pos); - if (Bytes.compareTo(kv.getBuffer(), kv.getFamilyOffset(), kv.getFamilyLength(), - family, 0, family.length) != 0) { - return null; - } - if (Bytes.compareTo(kv.getBuffer(), kv.getQualifierOffset(), kv.getQualifierLength(), - qualifier, 0, qualifier.length) != 0) { - return null; - } - return kv; - } - - /* - * Special comparator, *only* works for binary search. - * Current JDKs only uses the search term on the right side, - * Making use of that saves instanceof checks, and allows us - * to inline the search term in the comparator - */ - private static class SearchComparator implements Comparator { - private final byte[] row; - private final byte[] family; - private final byte[] qualifier; - private final int rowOff; - private final int rowLen; - - public SearchComparator(byte[] r, int rOff, int rLen, byte[] f, byte[] q) { - row = r; - family = f; - qualifier = q; - rowOff = rOff; - rowLen = rLen; - } - - @Override - public int compare(final KeyValue l, final KeyValue ignored) { - assert ignored == null; - final byte[] buf = l.getBuffer(); - final int rOff = l.getRowOffset(); - final short rLen = l.getRowLength(); - // row - int val = Bytes.compareTo(buf, rOff, rLen, row, rowOff, rowLen); - if (val != 0) { - return val; - } - // family - final int fOff = l.getFamilyOffset(rLen); - final byte fLen = l.getFamilyLength(fOff); - val = Bytes.compareTo(buf, fOff, fLen, family, 0, family.length); - if (val != 0) { - return val; - } - // qualifier - val = Bytes.compareTo(buf, l.getQualifierOffset(fOff), - l.getQualifierLength(rLen, fLen), qualifier, 0, qualifier.length); - if (val != 0) { - return val; - } - // want latest TS and type, so we get the first - return 1; - } - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/NumberUtil.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/NumberUtil.java deleted file mode 100644 index 25823a33..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/NumberUtil.java +++ /dev/null @@ -1,62 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import java.math.BigDecimal; - -import com.salesforce.phoenix.schema.PDataType; - -/** - * Utility methods for numbers like decimal, long, etc. - * - * @author elevine - * @since 0.1 - */ -public class NumberUtil { - - public static final String DEFAULT_NUMBER_FORMAT = "#,##0.###"; - - /** - * Strip all trailing zeros to ensure that no digit will be zero and - * round using our default context to ensure precision doesn't exceed max allowed. - * @return new {@link BigDecimal} instance - */ - public static BigDecimal normalize(BigDecimal bigDecimal) { - return bigDecimal.stripTrailingZeros().round(PDataType.DEFAULT_MATH_CONTEXT); - } - - public static BigDecimal setDecimalWidthAndScale(BigDecimal decimal, int precision, int scale) { - // If we could not fit all the digits before decimal point into the new desired precision and - // scale, return null and the caller method should handle the error. - if (((precision - scale) < (decimal.precision() - decimal.scale()))){ - return null; - } - decimal = decimal.setScale(scale, BigDecimal.ROUND_DOWN); - return decimal; - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/SQLCloseable.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/SQLCloseable.java deleted file mode 100644 index e2350119..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/SQLCloseable.java +++ /dev/null @@ -1,42 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import java.sql.SQLException; - -/** - * - * Interface for a SQL resource that should be closed - * after it is no longer in use. - * - * @author jtaylor - * @since 0.1 - */ -public interface SQLCloseable { - void close() throws SQLException; -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/SQLCloseables.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/SQLCloseables.java deleted file mode 100644 index 455da245..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/SQLCloseables.java +++ /dev/null @@ -1,141 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import java.sql.SQLException; -import java.util.*; - -import com.google.common.collect.Iterables; - - -/** - * Utilities for operating on {@link SQLCloseable}s. - * - * @author jtaylor - * @since 0.1 - */ -public class SQLCloseables { - /** Not constructed */ - private SQLCloseables() { } - - /** - * Allows you to close as many of the {@link SQLCloseable}s as possible. - * - * If any of the close's fail with an IOException, those exception(s) will - * be thrown after attempting to close all of the inputs. - */ - public static void closeAll(Iterable iterable) throws SQLException { - SQLException ex = closeAllQuietly(iterable); - if (ex != null) throw ex; - } - - public static SQLException closeAllQuietly(Iterable iterable) { - if (iterable == null) return null; - - LinkedList exceptions = null; - for (SQLCloseable closeable : iterable) { - try { - closeable.close(); - } catch (SQLException x) { - if (exceptions == null) exceptions = new LinkedList(); - exceptions.add(x); - } - } - - SQLException ex = MultipleCausesSQLException.fromSQLExceptions(exceptions); - return ex; - } - - /** - * A subclass of {@link SQLException} that allows you to chain multiple - * causes together. - * - * @author jtaylor - * @since 0.1 - * @see SQLCloseables - */ - static private class MultipleCausesSQLException extends SQLException { - private static final long serialVersionUID = 1L; - - static SQLException fromSQLExceptions(Collection exceptions) { - if (exceptions == null || exceptions.isEmpty()) return null; - if (exceptions.size() == 1) return Iterables.getOnlyElement(exceptions); - - return new MultipleCausesSQLException(exceptions); - } - - private final Collection exceptions; - private boolean hasSetStackTrace; - - /** - * Use the {@link #fromIOExceptions(Collection) factory}. - */ - private MultipleCausesSQLException(Collection exceptions) { - this.exceptions = exceptions; - } - - @Override - public String getMessage() { - StringBuilder sb = new StringBuilder(this.exceptions.size() * 50); - int exceptionNum = 0; - for (SQLException ex : this.exceptions) { - sb.append("Cause Number " + exceptionNum + ": " + ex.getMessage() + "\n"); - exceptionNum++; - } - return sb.toString(); - } - - @Override - public StackTraceElement[] getStackTrace() { - if (!this.hasSetStackTrace) { - ArrayList frames = new ArrayList(this.exceptions.size() * 20); - - int exceptionNum = 0; - for (SQLException exception : this.exceptions) { - StackTraceElement header = new StackTraceElement(MultipleCausesSQLException.class.getName(), - "Exception Number " + exceptionNum, - "", - 0); - - frames.add(header); - for (StackTraceElement ste : exception.getStackTrace()) { - frames.add(ste); - } - exceptionNum++; - } - - setStackTrace(frames.toArray(new StackTraceElement[frames.size()])); - this.hasSetStackTrace = true; - } - - return super.getStackTrace(); - } - - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/SchemaUtil.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/SchemaUtil.java deleted file mode 100644 index 7a2a85dc..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/SchemaUtil.java +++ /dev/null @@ -1,650 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_TABLE_NAME_BYTES; - -import java.io.IOException; -import java.sql.SQLException; -import java.sql.Statement; -import java.util.ArrayList; -import java.util.Iterator; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Properties; - -import org.apache.hadoop.hbase.HConstants; -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.KeyValue.Type; -import org.apache.hadoop.hbase.client.Delete; -import org.apache.hadoop.hbase.client.Mutation; -import org.apache.hadoop.hbase.client.Put; -import org.apache.hadoop.hbase.client.Scan; -import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding; -import org.apache.hadoop.hbase.regionserver.HRegion; -import org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl; -import org.apache.hadoop.hbase.regionserver.RegionScanner; -import org.apache.hadoop.hbase.util.Bytes; - -import com.google.common.collect.Lists; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.AmbiguousColumnException; -import com.salesforce.phoenix.schema.ColumnFamilyNotFoundException; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PColumnFamily; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDatum; -import com.salesforce.phoenix.schema.PMetaData; -import com.salesforce.phoenix.schema.PName; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.RowKeySchema; -import com.salesforce.phoenix.schema.RowKeySchema.RowKeySchemaBuilder; -import com.salesforce.phoenix.schema.SaltingUtil; - - - -/** - * - * Static class for various schema-related utilities - * - * @author jtaylor - * @since 0.1 - */ -public class SchemaUtil { - private static final int VAR_LENGTH_ESTIMATE = 10; - - public static final DataBlockEncoding DEFAULT_DATA_BLOCK_ENCODING = DataBlockEncoding.FAST_DIFF; - public static final RowKeySchema VAR_BINARY_SCHEMA = new RowKeySchemaBuilder(1).addField(new PDatum() { - - @Override - public boolean isNullable() { - return false; - } - - @Override - public PDataType getDataType() { - return PDataType.VARBINARY; - } - - @Override - public Integer getByteSize() { - return null; - } - - @Override - public Integer getMaxLength() { - return null; - } - - @Override - public Integer getScale() { - return null; - } - - @Override - public ColumnModifier getColumnModifier() { - return null; - } - - }, false, null).build(); - - /** - * May not be instantiated - */ - private SchemaUtil() { - } - - public static boolean isPKColumn(PColumn column) { - return column.getFamilyName() == null; - } - - /** - * Estimate the max key length in bytes of the PK for a given table - * @param table the table - * @return the max PK length - */ - public static int estimateKeyLength(PTable table) { - int maxKeyLength = 0; - // Calculate the max length of a key (each part must currently be of a fixed width) - int i = 0; - List columns = table.getPKColumns(); - while (i < columns.size()) { - PColumn keyColumn = columns.get(i++); - Integer byteSize = keyColumn.getByteSize(); - maxKeyLength += (byteSize == null) ? VAR_LENGTH_ESTIMATE : byteSize; - } - return maxKeyLength; - } - - /** - * Normalize an identifier. If name is surrounded by double quotes, - * it is used as-is, otherwise the name is upper caased. - * @param name the parsed identifier - * @return the normalized identifier - */ - public static String normalizeIdentifier(String name) { - if (name == null) { - return name; - } - if (isCaseSensitive(name)) { - // Don't upper case if in quotes - return name.substring(1, name.length()-1); - } - return name.toUpperCase(); - } - - public static boolean isCaseSensitive(String name) { - return name.length() > 0 && name.charAt(0)=='"'; - } - - public static List concat(List l1, List l2) { - int size1 = l1.size(); - if (size1 == 0) { - return l2; - } - int size2 = l2.size(); - if (size2 == 0) { - return l1; - } - List l3 = new ArrayList(size1 + size2); - l3.addAll(l1); - l3.addAll(l2); - return l3; - } - - public static byte[] getSequenceKey(byte[] tenantId, byte[] schemaName, byte[] sequenceName) { - return getTableKey(tenantId, schemaName, sequenceName); - } - - public static byte[] getSequenceKey(String tenantId, String schemaName, String sequenceName) { - return getTableKey(tenantId, schemaName, sequenceName); - } - - /** - * Get the key used in the Phoenix metadata row for a table definition - * @param schemaName - * @param tableName - */ - public static byte[] getTableKey(byte[] tenantId, byte[] schemaName, byte[] tableName) { - return ByteUtil.concat(tenantId, QueryConstants.SEPARATOR_BYTE_ARRAY, schemaName, QueryConstants.SEPARATOR_BYTE_ARRAY, tableName); - } - - public static byte[] getTableKey(String tenantId, String schemaName, String tableName) { - return ByteUtil.concat(tenantId == null ? ByteUtil.EMPTY_BYTE_ARRAY : Bytes.toBytes(tenantId), QueryConstants.SEPARATOR_BYTE_ARRAY, schemaName == null ? ByteUtil.EMPTY_BYTE_ARRAY : Bytes.toBytes(schemaName), QueryConstants.SEPARATOR_BYTE_ARRAY, Bytes.toBytes(tableName)); - } - - public static String getTableName(String schemaName, String tableName) { - return getName(schemaName,tableName); - } - - private static String getName(String optionalQualifier, String name) { - if (optionalQualifier == null || optionalQualifier.isEmpty()) { - return name; - } - return optionalQualifier + QueryConstants.NAME_SEPARATOR + name; - } - - public static String getTableName(byte[] schemaName, byte[] tableName) { - return getName(schemaName, tableName); - } - - public static String getColumnDisplayName(byte[] cf, byte[] cq) { - return getName(cf == null || cf.length == 0 || Bytes.compareTo(cf, QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES) == 0 ? ByteUtil.EMPTY_BYTE_ARRAY : cf, cq); - } - - public static String getColumnDisplayName(String cf, String cq) { - return getName(cf == null || cf.isEmpty() || QueryConstants.DEFAULT_COLUMN_FAMILY.equals(cf) ? null : cf, cq); - } - - public static String getMetaDataEntityName(String schemaName, String tableName, String familyName, String columnName) { - if ((schemaName == null || schemaName.isEmpty()) && (tableName == null || tableName.isEmpty())) { - return getName(familyName, columnName); - } - if ((familyName == null || familyName.isEmpty()) && (columnName == null || columnName.isEmpty())) { - return getName(schemaName, tableName); - } - return getName(getName(schemaName, tableName), getName(familyName, columnName)); - } - - public static String getColumnName(String familyName, String columnName) { - return getName(familyName, columnName); - } - - public static byte[] getTableNameAsBytes(String schemaName, String tableName) { - if (schemaName == null || schemaName.length() == 0) { - return StringUtil.toBytes(tableName); - } - return getTableNameAsBytes(StringUtil.toBytes(schemaName),StringUtil.toBytes(tableName)); - } - - public static byte[] getTableNameAsBytes(byte[] schemaName, byte[] tableName) { - return getNameAsBytes(schemaName, tableName); - } - - private static byte[] getNameAsBytes(byte[] nameOne, byte[] nameTwo) { - if (nameOne == null || nameOne.length == 0) { - return nameTwo; - } else if ((nameTwo == null || nameTwo.length == 0)) { - return nameOne; - } else { - return ByteUtil.concat(nameOne, QueryConstants.NAME_SEPARATOR_BYTES, nameTwo); - } - } - - public static String getName(byte[] nameOne, byte[] nameTwo) { - return Bytes.toString(getNameAsBytes(nameOne,nameTwo)); - } - - public static int getVarCharLength(byte[] buf, int keyOffset, int maxLength) { - return getVarCharLength(buf, keyOffset, maxLength, 1); - } - - public static int getVarCharLength(byte[] buf, int keyOffset, int maxLength, int skipCount) { - int length = 0; - for (int i=0; i 0; i++) { - int length = getVarCharLength(rowKey, offset, keyLength); - byte[] b = new byte[length]; - System.arraycopy(rowKey, offset, b, 0, length); - offset += length + 1; - keyLength -= length + 1; - colMetaData[i] = b; - } - return i; - } - - public static String findExistingColumn(PTable table, List columns) { - for (PColumn column : columns) { - PName familyName = column.getFamilyName(); - if (familyName == null) { - try { - return table.getPKColumn(column.getName().getString()).getName().getString(); - } catch (ColumnNotFoundException e) { - continue; - } - } else { - try { - return table.getColumnFamily(familyName.getString()).getColumn(column.getName().getString()).getName().getString(); - } catch (ColumnFamilyNotFoundException e) { - continue; // Shouldn't happen - } catch (ColumnNotFoundException e) { - continue; - } - } - } - return null; - } - - public static String toString(byte[][] values) { - if (values == null) { - return "null"; - } - StringBuilder buf = new StringBuilder("["); - for (byte[] value : values) { - buf.append(Bytes.toStringBinary(value)); - buf.append(','); - } - buf.setCharAt(buf.length()-1, ']'); - return buf.toString(); - } - - public static String toString(PDataType type, byte[] value) { - boolean isString = type.isCoercibleTo(PDataType.VARCHAR); - return isString ? ("'" + type.toObject(value).toString() + "'") : type.toObject(value).toString(); - } - - public static byte[] getEmptyColumnFamily(List families) { - return families.isEmpty() ? QueryConstants.EMPTY_COLUMN_BYTES : families.get(0).getName().getBytes(); - } - - public static ImmutableBytesPtr getEmptyColumnFamilyPtr(List families) { - return families.isEmpty() ? QueryConstants.EMPTY_COLUMN_BYTES_PTR : families.get(0) - .getName().getBytesPtr(); - } - - public static boolean isMetaTable(byte[] tableName) { - return Bytes.compareTo(tableName, TYPE_TABLE_NAME_BYTES) == 0; - } - - public static boolean isSequenceTable(byte[] tableName) { - return Bytes.compareTo(tableName, PhoenixDatabaseMetaData.SEQUENCE_TABLE_NAME_BYTES) == 0; - } - - public static boolean isMetaTable(PTable table) { - return PhoenixDatabaseMetaData.TYPE_SCHEMA.equals(table.getSchemaName().getString()) && PhoenixDatabaseMetaData.TYPE_TABLE.equals(table.getTableName().getString()); - } - - public static boolean isMetaTable(byte[] schemaName, byte[] tableName) { - return Bytes.compareTo(schemaName, PhoenixDatabaseMetaData.TYPE_SCHEMA_BYTES) == 0 && Bytes.compareTo(tableName, PhoenixDatabaseMetaData.TYPE_TABLE_BYTES) == 0; - } - - public static boolean isMetaTable(String schemaName, String tableName) { - return PhoenixDatabaseMetaData.TYPE_SCHEMA.equals(schemaName) && PhoenixDatabaseMetaData.TYPE_TABLE.equals(tableName); - } - - // Given the splits and the rowKeySchema, find out the keys that - public static byte[][] processSplits(byte[][] splits, LinkedHashSet pkColumns, Integer saltBucketNum, boolean defaultRowKeyOrder) throws SQLException { - // FIXME: shouldn't this return if splits.length == 0? - if (splits == null) return null; - // We do not accept user specified splits if the table is salted and we specify defaultRowKeyOrder. In this case, - // throw an exception. - if (splits.length > 0 && saltBucketNum != null && defaultRowKeyOrder) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.NO_SPLITS_ON_SALTED_TABLE).build().buildException(); - } - // If the splits are not specified and table is salted, pre-split the table. - if (splits.length == 0 && saltBucketNum != null) { - splits = SaltingUtil.getSalteByteSplitPoints(saltBucketNum); - } - byte[][] newSplits = new byte[splits.length][]; - for (int i=0; i pkColumns) { - int pos = 0, offset = 0, maxOffset = split.length; - Iterator iterator = pkColumns.iterator(); - while (pos < pkColumns.size()) { - PColumn column = iterator.next(); - if (column.getDataType().isFixedWidth()) { // Fixed width - int length = column.getByteSize(); - if (maxOffset - offset < length) { - // The split truncates the field. Fill in the rest of the part and any fields that - // are missing after this field. - int fillInLength = length - (maxOffset - offset); - fillInLength += estimatePartLength(pos + 1, iterator); - return ByteUtil.fillKey(split, split.length + fillInLength); - } - // Account for this field, move to next position; - offset += length; - pos++; - } else { // Variable length - // If we are the last slot, then we are done. Nothing needs to be filled in. - if (pos == pkColumns.size() - 1) { - break; - } - while (offset < maxOffset && split[offset] != QueryConstants.SEPARATOR_BYTE) { - offset++; - } - if (offset == maxOffset) { - // The var-length field does not end with a separator and it's not the last field. - int fillInLength = 1; // SEPARATOR byte for the current var-length slot. - fillInLength += estimatePartLength(pos + 1, iterator); - return ByteUtil.fillKey(split, split.length + fillInLength); - } - // Move to the next position; - offset += 1; // skip separator; - pos++; - } - } - return split; - } - - // Estimate the key length after pos slot for schema. - private static int estimatePartLength(int pos, Iterator iterator) { - int length = 0; - while (iterator.hasNext()) { - PColumn column = iterator.next(); - if (column.getDataType().isFixedWidth()) { - length += column.getByteSize(); - } else { - length += 1; // SEPARATOR byte. - } - } - return length; - } - - public static final String UPGRADE_TO_2_0 = "UpgradeTo20"; - public static final Integer SYSTEM_TABLE_NULLABLE_VAR_LENGTH_COLUMNS = 3; - public static final String UPGRADE_TO_2_1 = "UpgradeTo21"; - public static final String UPGRADE_TO_3_0 = "UpgradeTo30"; - - private static final byte[] ORIG_DEF_CF_NAME = Bytes.toBytes("_0"); - - public static void upgradeTo3(HRegion region, List mutations) throws IOException { - Scan scan = new Scan(); - scan.setTimeRange(MetaDataProtocol.MIN_TABLE_TIMESTAMP, HConstants.LATEST_TIMESTAMP); - scan.setRaw(true); - scan.setMaxVersions(MetaDataProtocol.DEFAULT_MAX_META_DATA_VERSIONS); - RegionScanner scanner = region.getScanner(scan); - MultiVersionConsistencyControl.setThreadReadPoint(scanner.getMvccReadPoint()); - List result; - region.startRegionOperation(); - try { - byte[] systemTableKeyPrefix = ByteUtil.concat(PhoenixDatabaseMetaData.TYPE_SCHEMA_BYTES, QueryConstants.SEPARATOR_BYTE_ARRAY, PhoenixDatabaseMetaData.TYPE_TABLE_BYTES); - do { - result = Lists.newArrayList(); - scanner.nextRaw(result, null); - for (KeyValue keyValue : result) { - byte[] buf = keyValue.getBuffer(); - int rowOffset = keyValue.getRowOffset(); - int rowLength = keyValue.getRowLength(); - if (Type.codeToType(keyValue.getType()) == Type.Put) { - // Delete old value - Delete delete = new Delete(keyValue.getRow()); - KeyValue deleteKeyValue = new KeyValue(buf, keyValue.getRowOffset(), keyValue.getRowLength(), - buf, keyValue.getFamilyOffset(), keyValue.getFamilyLength(), - buf, keyValue.getQualifierOffset(), keyValue.getQualifierLength(), - keyValue.getTimestamp(), Type.Delete, - ByteUtil.EMPTY_BYTE_ARRAY,0,0); - delete.addDeleteMarker(deleteKeyValue); - mutations.add(delete); - - // If not a row in our system.table, then add a new key value prefixed with a null byte for the empty tenant_id. - // Otherwise, we already have the mutations for the new system.table in the mutation list, so we don't add them here. - if (! ( Bytes.compareTo(buf, rowOffset, rowLength, systemTableKeyPrefix, 0, systemTableKeyPrefix.length) == 0 && - ( rowLength == systemTableKeyPrefix.length || - buf[rowOffset + systemTableKeyPrefix.length] == QueryConstants.SEPARATOR_BYTE)) ) { - KeyValue newKeyValue = SchemaUtil.upgradeTo3(keyValue); - // Put new value - Put put = new Put(newKeyValue.getRow()); - put.add(newKeyValue); - mutations.add(put); - - // Add a new DEFAULT_COLUMN_FAMILY_NAME key value as a table header column. - // We can match here on any table column header we know will occur for any - // table row. - if (Bytes.compareTo( - newKeyValue.getBuffer(), newKeyValue.getQualifierOffset(), newKeyValue.getQualifierLength(), - PhoenixDatabaseMetaData.COLUMN_COUNT_BYTES, 0, PhoenixDatabaseMetaData.COLUMN_COUNT_BYTES.length) == 0) { - KeyValue defCFNameKeyValue = new KeyValue(newKeyValue.getBuffer(), newKeyValue.getRowOffset(), newKeyValue.getRowLength(), - buf, keyValue.getFamilyOffset(), keyValue.getFamilyLength(), - PhoenixDatabaseMetaData.DEFAULT_COLUMN_FAMILY_NAME_BYTES, 0, PhoenixDatabaseMetaData.DEFAULT_COLUMN_FAMILY_NAME_BYTES.length, - newKeyValue.getTimestamp(), Type.Put, - ORIG_DEF_CF_NAME,0,ORIG_DEF_CF_NAME.length); - Put defCFNamePut = new Put(defCFNameKeyValue.getRow()); - defCFNamePut.add(defCFNameKeyValue); - mutations.add(defCFNamePut); - } - } - } else if (Type.codeToType(keyValue.getType()) == Type.Delete){ - KeyValue newKeyValue = SchemaUtil.upgradeTo3(keyValue); - // Copy delete marker using new key so that it continues - // to delete the key value preceding it that will be updated - // as well. - Delete delete = new Delete(newKeyValue.getRow()); - // What timestamp will be used? - // delete.setTimestamp(keyValue.getTimestamp()); - delete.addDeleteMarker(newKeyValue); - mutations.add(delete); - } - } - } while (!result.isEmpty()); - } finally { - region.closeRegionOperation(); - } - } - - private static final byte[][] OLD_TO_NEW_DATA_TYPE_3_0 = new byte[PDataType.TIMESTAMP.getSqlType() + 1][]; - static { - OLD_TO_NEW_DATA_TYPE_3_0[PDataType.TIMESTAMP.getSqlType()] = PDataType.INTEGER.toBytes(PDataType.UNSIGNED_TIMESTAMP.getSqlType()); - OLD_TO_NEW_DATA_TYPE_3_0[PDataType.TIME.getSqlType()] = PDataType.INTEGER.toBytes(PDataType.UNSIGNED_TIME.getSqlType()); - OLD_TO_NEW_DATA_TYPE_3_0[PDataType.DATE.getSqlType()] = PDataType.INTEGER.toBytes(PDataType.UNSIGNED_DATE.getSqlType()); - } - - - private static byte[] updateValueIfNecessary(KeyValue keyValue) { - byte[] buf = keyValue.getBuffer(); - if (Bytes.compareTo( - buf, keyValue.getQualifierOffset(), keyValue.getQualifierLength(), - PhoenixDatabaseMetaData.DATA_TYPE_BYTES, 0, PhoenixDatabaseMetaData.DATA_TYPE_BYTES.length) == 0) { - int sqlType = PDataType.INTEGER.getCodec().decodeInt(buf, keyValue.getValueOffset(), null); - // Switch the DATA_TYPE index for TIME types, as they currently don't support negative values - // In 3.0, we'll switch them to our serialized LONG type instead, as that way we can easily - // support negative types going forward. - if (sqlType >= 0 && sqlType < OLD_TO_NEW_DATA_TYPE_3_0.length && OLD_TO_NEW_DATA_TYPE_3_0[sqlType] != null) { - return OLD_TO_NEW_DATA_TYPE_3_0[sqlType]; - } - } - return buf; - } - private static KeyValue upgradeTo3(KeyValue keyValue) { - byte[] buf = keyValue.getBuffer(); - int newLength = keyValue.getRowLength() + 1; - byte[] newKey = new byte[newLength]; - newKey[0] = QueryConstants.SEPARATOR_BYTE; - System.arraycopy(buf, keyValue.getRowOffset(), newKey, 1, keyValue.getRowLength()); - byte[] valueBuf = updateValueIfNecessary(keyValue); - int valueOffset = keyValue.getValueOffset(); - int valueLength = keyValue.getValueLength(); - if (valueBuf != buf) { - valueOffset = 0; - valueLength = valueBuf.length; - } - return new KeyValue(newKey, 0, newLength, - buf, keyValue.getFamilyOffset(), keyValue.getFamilyLength(), - buf, keyValue.getQualifierOffset(), keyValue.getQualifierLength(), - keyValue.getTimestamp(), Type.codeToType(keyValue.getType()), - valueBuf, valueOffset, valueLength); - } - - public static String getEscapedTableName(String schemaName, String tableName) { - if (schemaName == null || schemaName.length() == 0) { - return "\"" + tableName + "\""; - } - return "\"" + schemaName + "\"." + "\"" + tableName + "\""; - } - - protected static PhoenixConnection addMetaDataColumn(PhoenixConnection conn, long scn, String columnDef) throws SQLException { - String url = conn.getURL(); - Properties props = conn.getClientInfo(); - PMetaData metaData = conn.getPMetaData(); - props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(scn)); - PhoenixConnection metaConnection = null; - - Statement stmt = null; - try { - metaConnection = new PhoenixConnection(conn.getQueryServices(), url, props, metaData); - try { - stmt = metaConnection.createStatement(); - stmt.executeUpdate("ALTER TABLE SYSTEM.\"TABLE\" ADD IF NOT EXISTS " + columnDef); - return metaConnection; - } finally { - if(stmt != null) { - stmt.close(); - } - } - } finally { - if(metaConnection != null) { - metaConnection.close(); - } - } - } - - public static boolean columnExists(PTable table, String columnName) { - try { - table.getColumn(columnName); - return true; - } catch (ColumnNotFoundException e) { - return false; - } catch (AmbiguousColumnException e) { - return true; - } - } - - public static String getSchemaNameFromFullName(String tableName) { - int index = tableName.indexOf(QueryConstants.NAME_SEPARATOR); - if (index < 0) { - return ""; - } - return tableName.substring(0, index); - } - - public static String getTableNameFromFullName(String tableName) { - int index = tableName.indexOf(QueryConstants.NAME_SEPARATOR); - if (index < 0) { - return tableName; - } - return tableName.substring(index+1, tableName.length()); - } - - public static byte[] getTableKeyFromFullName(String fullTableName) { - int index = fullTableName.indexOf(QueryConstants.NAME_SEPARATOR); - if (index < 0) { - return getTableKey(null, null, fullTableName); - } - String schemaName = fullTableName.substring(0, index); - String tableName = fullTableName.substring(index+1); - return getTableKey(null, schemaName, tableName); - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ServerUtil.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/ServerUtil.java deleted file mode 100644 index 1c716fee..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ServerUtil.java +++ /dev/null @@ -1,131 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import java.io.IOException; -import java.sql.SQLException; -import java.util.HashMap; -import java.util.Map; -import java.util.regex.Matcher; -import java.util.regex.Pattern; - -import org.apache.hadoop.hbase.DoNotRetryIOException; - -import com.salesforce.phoenix.exception.PhoenixIOException; -import com.salesforce.phoenix.exception.SQLExceptionCode; - - -public class ServerUtil { - - private static final String FORMAT = "ERROR %d (%s): %s"; - private static final Pattern PATTERN = Pattern.compile("ERROR (\\d+) \\((\\w+)\\): (.*)"); - private static final Map, SQLExceptionCode> errorcodeMap - = new HashMap, SQLExceptionCode>(); - static { - // Map a normal exception into a corresponding SQLException. - errorcodeMap.put(ArithmeticException.class, SQLExceptionCode.SERVER_ARITHMETIC_ERROR); - } - - public static void throwIOException(String msg, Throwable t) throws IOException { - // First unwrap SQLExceptions if it's root cause is an IOException. - if (t instanceof SQLException) { - Throwable cause = t.getCause(); - if (cause instanceof IOException) { - t = cause; - } - } - // Throw immediately if DoNotRetryIOException - if (t instanceof DoNotRetryIOException) { - throw (DoNotRetryIOException)t; - } else if (t instanceof IOException) { - // If the IOException does not wrap any exception, then bubble it up. - Throwable cause = t.getCause(); - if (cause == null || cause instanceof IOException) { - throw (IOException)t; - } - // Else assume it's been wrapped, so throw as DoNotRetryIOException to prevent client hanging while retrying - throw new DoNotRetryIOException(t.getMessage(), cause); - } else if (t instanceof SQLException) { - // If it's already an SQLException, construct an error message so we can parse and reconstruct on the client side. - throw new DoNotRetryIOException(constructSQLErrorMessage((SQLException) t, msg), t); - } else { - // Not a DoNotRetryIOException, IOException or SQLException. Map the exception type to a general SQLException - // and construct the error message so it can be reconstruct on the client side. - // - // If no mapping exists, rethrow it as a generic exception. - SQLExceptionCode code = errorcodeMap.get(t.getClass()); - if (code == null) { - throw new DoNotRetryIOException(msg + ": " + t.getMessage(), t); - } else { - throw new DoNotRetryIOException(constructSQLErrorMessage(code, t, msg), t); - } - } - } - - private static String constructSQLErrorMessage(SQLExceptionCode code, Throwable e, String message) { - return constructSQLErrorMessage(code.getErrorCode(), code.getSQLState(), code.getMessage() + " " + e.getMessage() + " " + message); - } - - private static String constructSQLErrorMessage(SQLException e, String message) { - return constructSQLErrorMessage(e.getErrorCode(), e.getSQLState(), e.getMessage() + " " + message); - } - - private static String constructSQLErrorMessage(int errorCode, String SQLState, String message) { - return String.format(FORMAT, errorCode, SQLState, message); - } - - public static SQLException parseServerException(Throwable t) { - SQLException e = parseServerExceptionOrNull(t); - if (e != null) { - return e; - } - return new PhoenixIOException(t); - } - - public static SQLException parseServerExceptionOrNull(Throwable t) { - while (t.getCause() != null) { - t = t.getCause(); - } - return parseRemoteException(t); - } - - private static SQLException parseRemoteException(Throwable t) { - String message = t.getLocalizedMessage(); - if (message != null) { - // If the message matches the standard pattern, recover the SQLException and throw it. - Matcher matcher = PATTERN.matcher(t.getLocalizedMessage()); - if (matcher.find()) { - int errorCode = Integer.parseInt(matcher.group(1)); - String sqlState = matcher.group(2); - return new SQLException(matcher.group(), sqlState, errorCode, t); - } - } - return null; - } - -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/SizedUtil.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/SizedUtil.java deleted file mode 100644 index f60ad1e6..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/SizedUtil.java +++ /dev/null @@ -1,71 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - - -/** - * Utilities for computing an object's size. All size measurements are in bytes. - * Note, all of the sizes here, but especially OBJECT_SIZE and ARRAY_SIZE are estimates and will - * depend on the JVM itself (and which JVM, 64bit vs. 32bit, etc). - * The current values are based on: - * Java HotSpot(TM) 64-Bit Server VM/14.2-b01 - * - * (Uncomment out and run the main w/ appropriate object to test) - * Also, see this link: - * https://sites.google.com/a/salesforce.com/development/Home/old-wiki-home-page/i-wish-i-knew#TOC-How-to-measure-the-size-of-a-Java-O - * For another way to measure. - */ -public class SizedUtil { - public static final int POINTER_SIZE = 8; // 64 bit jvm. - public static final int OBJECT_SIZE = 16; // measured, see class comment. - public static final int ARRAY_SIZE = 24; // measured, see class comment. - public static final int CHAR_SIZE = 2; - public static final int INT_SIZE = 4; - public static final int LONG_SIZE = 8; - - public static final int MAP_ENTRY_SIZE = OBJECT_SIZE + 3 * POINTER_SIZE + INT_SIZE; - public static final int IMMUTABLE_BYTES_WRITABLE_SIZE = OBJECT_SIZE + INT_SIZE * 2 + ARRAY_SIZE; - public static final int IMMUTABLE_BYTES_PTR_SIZE = IMMUTABLE_BYTES_WRITABLE_SIZE + INT_SIZE;// Extra is an int field which caches hashcode. - public static final int KEY_VALUE_SIZE = 2 * INT_SIZE + LONG_SIZE + 2 * ARRAY_SIZE; - public static final int RESULT_SIZE = OBJECT_SIZE + 3 * POINTER_SIZE + IMMUTABLE_BYTES_WRITABLE_SIZE; - public static final int INT_OBJECT_SIZE = INT_SIZE + OBJECT_SIZE; - public static final int LONG_OBJECT_SIZE = LONG_SIZE + OBJECT_SIZE; - public static final int BIG_DECIMAL_SIZE = - OBJECT_SIZE + 2 * INT_SIZE + LONG_SIZE + 2 * POINTER_SIZE + - OBJECT_SIZE /* BigInteger */ + 5 * INT_SIZE + ARRAY_SIZE /*mag[]*/ + 2 * INT_SIZE /* est mag[2] */; - - private SizedUtil() { - } - - public static int sizeOfMap(int nRows, int keySize, int valueSize) { - return nRows * ( - SizedUtil.MAP_ENTRY_SIZE + // entry - keySize + // key size - valueSize); // value size - } -} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/TrustedByteArrayOutputStream.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/TrustedByteArrayOutputStream.java deleted file mode 100644 index 94ff0228..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/TrustedByteArrayOutputStream.java +++ /dev/null @@ -1,64 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import java.io.ByteArrayOutputStream; -import java.io.IOException; - -/** - * - * Derived version of {@link java.io.ByteArrayOutputStream} that provides access - * to underlying byte array buffer so that it doesn't have to be copied - * - * @author jtaylor - * @since 0.1 - */ -public class TrustedByteArrayOutputStream extends ByteArrayOutputStream { - public TrustedByteArrayOutputStream(int initialSize) { - super(initialSize); - } - public byte[] getBuffer() { - return buf; - } - @Override - public byte[] toByteArray() { - if (buf.length == size()) { - return buf; - } - return super.toByteArray(); - } - @Override - public void write(byte[] b) { - try { - super.write(b); - } catch (IOException e) { - throw new RuntimeException(e); // Impossible - } - } -} - diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/TupleUtil.java b/phoenix-core/src/main/java/com/salesforce/phoenix/util/TupleUtil.java deleted file mode 100644 index ac65e44f..00000000 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/TupleUtil.java +++ /dev/null @@ -1,163 +0,0 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import static com.salesforce.phoenix.query.QueryConstants.SINGLE_COLUMN; -import static com.salesforce.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY; - -import java.io.DataOutput; -import java.io.IOException; -import java.util.List; - -import org.apache.hadoop.hbase.DoNotRetryIOException; -import org.apache.hadoop.hbase.KeyValue; -import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import org.apache.hadoop.hbase.util.Bytes; -import org.apache.hadoop.io.WritableUtils; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.tuple.Tuple; - - -/** - * - * Utilities for Tuple - * - * @author jtaylor - * @since 0.1 - */ -public class TupleUtil { - private TupleUtil() { - } - - public static boolean equals(Tuple t1, Tuple t2, ImmutableBytesWritable ptr) { - t1.getKey(ptr); - byte[] buf = ptr.get(); - int offset = ptr.getOffset(); - int length = ptr.getLength(); - t2.getKey(ptr); - return Bytes.compareTo(buf, offset, length, ptr.get(), ptr.getOffset(), ptr.getLength()) == 0; - } - - public static int compare(Tuple t1, Tuple t2, ImmutableBytesWritable ptr) { - return compare(t1, t2, ptr, 0); - } - - public static int compare(Tuple t1, Tuple t2, ImmutableBytesWritable ptr, int keyOffset) { - t1.getKey(ptr); - byte[] buf = ptr.get(); - int offset = ptr.getOffset() + keyOffset; - int length = ptr.getLength() - keyOffset; - t2.getKey(ptr); - return Bytes.compareTo(buf, offset, length, ptr.get(), ptr.getOffset() + keyOffset, ptr.getLength() - keyOffset); - } - - /** - * Set ptr to point to the value contained in the first KeyValue without - * exploding Result into KeyValue array. - * @param r - * @param ptr - */ - public static void getAggregateValue(Tuple r, ImmutableBytesWritable ptr) { - if (r.size() == 1) { - KeyValue kv = r.getValue(0); // Just one KV for aggregation - if (Bytes.compareTo(SINGLE_COLUMN_FAMILY, 0, SINGLE_COLUMN_FAMILY.length, kv.getBuffer(), kv.getFamilyOffset(), kv.getFamilyLength()) == 0) { - if (Bytes.compareTo(SINGLE_COLUMN, 0, SINGLE_COLUMN.length, kv.getBuffer(), kv.getQualifierOffset(), kv.getQualifierLength()) == 0) { - ptr.set(kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); - return; - } - } - } - throw new IllegalStateException("Expected single, aggregated KeyValue from coprocessor, but instead received " + r + ". Ensure aggregating coprocessors are loaded correctly on server"); - } - - /** Concatenate results evaluated against a list of expressions - * - * @param result the tuple for expression evaluation - * @param expressions - * @return the concatenated byte array as ImmutableBytesWritable - * @throws IOException - */ - public static ImmutableBytesPtr getConcatenatedValue(Tuple result, List expressions) throws IOException { - ImmutableBytesPtr value = new ImmutableBytesPtr(ByteUtil.EMPTY_BYTE_ARRAY); - Expression expression = expressions.get(0); - boolean evaluated = expression.evaluate(result, value); - - if (expressions.size() == 1) { - if (!evaluated) { - value.set(ByteUtil.EMPTY_BYTE_ARRAY); - } - return value; - } else { - TrustedByteArrayOutputStream output = new TrustedByteArrayOutputStream(value.getLength() * expressions.size()); - try { - if (evaluated) { - output.write(value.get(), value.getOffset(), value.getLength()); - } - for (int i = 1; i < expressions.size(); i++) { - if (!expression.getDataType().isFixedWidth()) { - output.write(QueryConstants.SEPARATOR_BYTE); - } - expression = expressions.get(i); - // TODO: should we track trailing null values and omit the separator bytes? - if (expression.evaluate(result, value)) { - output.write(value.get(), value.getOffset(), value.getLength()); - } else if (i < expressions.size()-1 && expression.getDataType().isFixedWidth()) { - // This should never happen, because any non terminating nullable fixed width type (i.e. INT or LONG) is - // converted to a variable length type (i.e. DECIMAL) to allow an empty byte array to represent null. - throw new DoNotRetryIOException("Non terminating null value found for fixed width expression (" + expression + ") in row: " + result); - } - } - byte[] outputBytes = output.getBuffer(); - value.set(outputBytes, 0, output.size()); - return value; - } finally { - output.close(); - } - } - } - - public static int write(Tuple result, DataOutput out) throws IOException { - int size = 0; - for(int i = 0; i < result.size(); i++) { - KeyValue kv = result.getValue(i); - size += kv.getLength(); - size += Bytes.SIZEOF_INT; // kv.getLength - } - - WritableUtils.writeVInt(out, size); - for(int i = 0; i < result.size(); i++) { - KeyValue kv = result.getValue(i); - out.writeInt(kv.getLength()); - out.write(kv.getBuffer(), kv.getOffset(), kv.getLength()); - } - return size; - } -} diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/CapturingAbortable.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/CapturingAbortable.java new file mode 100644 index 00000000..5de5428d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/CapturingAbortable.java @@ -0,0 +1,68 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index; + +import org.apache.hadoop.hbase.Abortable; + +/** + * {@link Abortable} that can rethrow the cause of the abort. + */ +public class CapturingAbortable implements Abortable { + + private Abortable delegate; + private Throwable cause; + private String why; + + public CapturingAbortable(Abortable delegate) { + this.delegate = delegate; + } + + @Override + public void abort(String why, Throwable e) { + if (delegate.isAborted()) { + return; + } + this.why = why; + this.cause = e; + delegate.abort(why, e); + + } + + @Override + public boolean isAborted() { + return delegate.isAborted(); + } + + /** + * Throw the cause of the abort, if this was aborted. If there was an exception causing + * the abort, re-throws that. Otherwise, just throws a generic {@link Exception} with the reason + * why the abort was caused. + * @throws Throwable the cause of the abort. + */ + public void throwCauseIfAborted() throws Throwable { + if (!this.isAborted()) { + return; + } + if (cause == null) { + throw new Exception(why); + } + throw cause; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/IndexLogRollSynchronizer.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/IndexLogRollSynchronizer.java new file mode 100644 index 00000000..96e94cae --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/IndexLogRollSynchronizer.java @@ -0,0 +1,125 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index; + +import java.io.IOException; +import java.util.concurrent.locks.ReentrantReadWriteLock.WriteLock; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.fs.Path; +import org.apache.hadoop.hbase.HRegionInfo; +import org.apache.hadoop.hbase.HTableDescriptor; +import org.apache.hadoop.hbase.regionserver.wal.HLogKey; +import org.apache.hadoop.hbase.regionserver.wal.WALActionsListener; +import org.apache.hadoop.hbase.regionserver.wal.WALEdit; + +/** + * Ensure that the log isn't rolled while we are the in middle of doing a pending index write. + *

+ * The problem we are trying to solve is the following sequence: + *

    + *
  1. Write to the indexed table
    2. + *
  2. Write the index-containing WALEdit
    3. + *
  3. Start writing to the index tables in the postXXX hook
    4. + *
  4. WAL gets rolled and archived
    5. + *
  5. An index update fails, in which case we should kill ourselves to get WAL replay
    6. + *
  6. Since the WAL got archived, we won't get the replay of the index writes
    7. + *
+ *

+ * The usual course of events should be: + *

    + *
  1. In a preXXX hook, + *
      + *
    1. Build the {@link WALEdit} + index information
      2. + *
    2. Lock the {@link IndexLogRollSynchronizer#logArchiveLock}
      3. + *
        + *
      • This is a reentrant readlock on the WAL archiving, so we can make multiple WAL/index updates + * concurrently
        • + *
      + * + *
    + *
    2. + *
  2. Pass that {@link WALEdit} to the WAL, ensuring its durable and replayable
    3. + *
  3. In the corresponding postXXX, + *
      + *
    1. make the updates to the index tables
      2. + *
    2. Unlock {@link IndexLogRollSynchronizer#logArchiveLock}
      3. + *
    + *
+ *

+ * this should be added as a {@link WALActionsListener} by updating + */ +public class IndexLogRollSynchronizer implements WALActionsListener { + + private static final Log LOG = LogFactory.getLog(IndexLogRollSynchronizer.class); + private WriteLock logArchiveLock; + + public IndexLogRollSynchronizer(WriteLock logWriteLock){ + this.logArchiveLock = logWriteLock; + } + + + @Override + public void preLogArchive(Path oldPath, Path newPath) throws IOException { + //take a write lock on the index - any pending index updates will complete before we finish + LOG.debug("Taking INDEX_UPDATE writelock"); + logArchiveLock.lock(); + LOG.debug("Got the INDEX_UPDATE writelock"); + } + + @Override + public void postLogArchive(Path oldPath, Path newPath) throws IOException { + // done archiving the logs, any WAL updates will be replayed on failure + LOG.debug("Releasing INDEX_UPDATE writelock"); + logArchiveLock.unlock(); + } + + @Override + public void logCloseRequested() { + // don't care- before this is called, all the HRegions are closed, so we can't get any new + // requests and all pending request can finish before the WAL closes. + } + + @Override + public void preLogRoll(Path oldPath, Path newPath) throws IOException { + // noop + } + + @Override + public void postLogRoll(Path oldPath, Path newPath) throws IOException { + // noop + } + + @Override + public void logRollRequested() { + // noop + } + + @Override + public void visitLogEntryBeforeWrite(HRegionInfo info, HLogKey logKey, WALEdit logEdit) { + // noop + } + + @Override + public void visitLogEntryBeforeWrite(HTableDescriptor htd, HLogKey logKey, WALEdit logEdit) { + // noop + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/Indexer.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/Indexer.java similarity index 90% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/Indexer.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/Indexer.java index f68d9ed9..ce7ef87e 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/Indexer.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/Indexer.java @@ -1,33 +1,25 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index; - -import static com.salesforce.hbase.index.util.IndexManagementUtil.rethrowIndexingException; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index; + +import static org.apache.hadoop.hbase.index.util.IndexManagementUtil.rethrowIndexingException; import java.io.DataInput; import java.io.DataOutput; @@ -68,19 +60,19 @@ import org.apache.hadoop.hbase.util.Pair; import com.google.common.collect.Multimap; -import com.salesforce.hbase.index.builder.IndexBuildManager; -import com.salesforce.hbase.index.builder.IndexBuilder; -import com.salesforce.hbase.index.builder.IndexBuildingFailureException; -import com.salesforce.hbase.index.table.HTableInterfaceReference; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.hbase.index.util.IndexManagementUtil; -import com.salesforce.hbase.index.wal.IndexedKeyValue; -import com.salesforce.hbase.index.write.IndexFailurePolicy; -import com.salesforce.hbase.index.write.IndexWriter; -import com.salesforce.hbase.index.write.recovery.PerRegionIndexWriteCache; -import com.salesforce.hbase.index.write.recovery.StoreFailuresInCachePolicy; -import com.salesforce.hbase.index.write.recovery.TrackingParallelWriterIndexCommitter; -import com.salesforce.phoenix.util.MetaDataUtil; +import org.apache.hadoop.hbase.index.builder.IndexBuildManager; +import org.apache.hadoop.hbase.index.builder.IndexBuilder; +import org.apache.hadoop.hbase.index.builder.IndexBuildingFailureException; +import org.apache.hadoop.hbase.index.table.HTableInterfaceReference; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.util.IndexManagementUtil; +import org.apache.hadoop.hbase.index.wal.IndexedKeyValue; +import org.apache.hadoop.hbase.index.write.IndexFailurePolicy; +import org.apache.hadoop.hbase.index.write.IndexWriter; +import org.apache.hadoop.hbase.index.write.recovery.PerRegionIndexWriteCache; +import org.apache.hadoop.hbase.index.write.recovery.StoreFailuresInCachePolicy; +import org.apache.hadoop.hbase.index.write.recovery.TrackingParallelWriterIndexCommitter; +import org.apache.phoenix.util.MetaDataUtil; /** * Do all the work of managing index updates from a single coprocessor. All Puts/Delets are passed @@ -122,7 +114,7 @@ public class Indexer extends BaseRegionObserver { */ public static final String CHECK_VERSION_CONF_KEY = "com.saleforce.hbase.index.checkversion"; - private static final String INDEX_RECOVERY_FAILURE_POLICY_KEY = "com.salesforce.hbase.index.recovery.failurepolicy"; + private static final String INDEX_RECOVERY_FAILURE_POLICY_KEY = "org.apache.hadoop.hbase.index.recovery.failurepolicy"; /** * Marker {@link KeyValue} to indicate that we are doing a batch operation. Needed because the diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/ValueGetter.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/ValueGetter.java new file mode 100644 index 00000000..54c4fbcd --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/ValueGetter.java @@ -0,0 +1,38 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index; + +import java.io.IOException; + +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; + +public interface ValueGetter { + + /** + * Get the most recent (largest timestamp) for the given column reference + * @param ref to match against an underlying key value. Uses the passed object to match the + * keyValue via {@link ColumnReference#matches} + * @return the stored value for the given {@link ColumnReference}, or null if no value is + * present. + * @throws IOException if there is an error accessing the underlying data storage + */ + public ImmutableBytesPtr getLatestValue(ColumnReference ref) throws IOException; +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/BaseIndexBuilder.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/BaseIndexBuilder.java new file mode 100644 index 00000000..73b74216 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/BaseIndexBuilder.java @@ -0,0 +1,99 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.builder; + +import java.io.IOException; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; +import org.apache.hadoop.hbase.regionserver.MiniBatchOperationInProgress; +import org.apache.hadoop.hbase.util.Pair; + +import org.apache.hadoop.hbase.index.covered.CoveredColumnsIndexBuilder; + +/** + * Basic implementation of the {@link IndexBuilder} that doesn't do any actual work of indexing. + *

+ * You should extend this class, rather than implementing IndexBuilder directly to maintain + * compatability going forward. + *

+ * Generally, you should consider using one of the implemented IndexBuilders (e.g + * {@link CoveredColumnsIndexBuilder}) as there is a lot of work required to keep an index table + * up-to-date. + */ +public abstract class BaseIndexBuilder implements IndexBuilder { + + private static final Log LOG = LogFactory.getLog(BaseIndexBuilder.class); + protected boolean stopped; + + @Override + public void extendBaseIndexBuilderInstead() { } + + @Override + public void setup(RegionCoprocessorEnvironment conf) throws IOException { + // noop + } + + @Override + public void batchStarted(MiniBatchOperationInProgress> miniBatchOp) throws IOException { + // noop + } + + @Override + public void batchCompleted(MiniBatchOperationInProgress> miniBatchOp) { + // noop + } + + /** + * By default, we always attempt to index the mutation. Commonly this can be slow (because the + * framework spends the time to do the indexing, only to realize that you don't need it) or not + * ideal (if you want to turn on/off indexing on a table without completely reloading it). + * @throws IOException + */ + @Override + public boolean isEnabled(Mutation m) throws IOException { + return true; + } + + /** + * {@inheritDoc} + *

+ * By default, assumes that all mutations should not be batched. That is to say, each + * mutation always applies to different rows, even if they are in the same batch, or are + * independent updates. + */ + @Override + public byte[] getBatchId(Mutation m) { + return null; + } + + @Override + public void stop(String why) { + LOG.debug("Stopping because: " + why); + this.stopped = true; + } + + @Override + public boolean isStopped() { + return this.stopped; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/builder/IndexBuildManager.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/IndexBuildManager.java similarity index 77% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/builder/IndexBuildManager.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/IndexBuildManager.java index 62f12e45..0752446d 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/builder/IndexBuildManager.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/IndexBuildManager.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.builder; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.builder; import java.io.IOException; import java.util.ArrayList; @@ -45,12 +37,12 @@ import org.apache.hadoop.hbase.regionserver.MiniBatchOperationInProgress; import org.apache.hadoop.hbase.util.Pair; -import com.salesforce.hbase.index.Indexer; -import com.salesforce.hbase.index.parallel.QuickFailingTaskRunner; -import com.salesforce.hbase.index.parallel.Task; -import com.salesforce.hbase.index.parallel.TaskBatch; -import com.salesforce.hbase.index.parallel.ThreadPoolBuilder; -import com.salesforce.hbase.index.parallel.ThreadPoolManager; +import org.apache.hadoop.hbase.index.Indexer; +import org.apache.hadoop.hbase.index.parallel.QuickFailingTaskRunner; +import org.apache.hadoop.hbase.index.parallel.Task; +import org.apache.hadoop.hbase.index.parallel.TaskBatch; +import org.apache.hadoop.hbase.index.parallel.ThreadPoolBuilder; +import org.apache.hadoop.hbase.index.parallel.ThreadPoolManager; /** * Manage the building of index updates from primary table updates. diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/builder/IndexBuilder.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/IndexBuilder.java similarity index 76% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/builder/IndexBuilder.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/IndexBuilder.java index d19d0158..8e49d6d8 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/builder/IndexBuilder.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/IndexBuilder.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.builder; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.builder; import java.io.IOException; import java.util.Collection; @@ -41,7 +33,7 @@ import org.apache.hadoop.hbase.regionserver.MiniBatchOperationInProgress; import org.apache.hadoop.hbase.util.Pair; -import com.salesforce.hbase.index.Indexer; +import org.apache.hadoop.hbase.index.Indexer; /** * Interface to build updates ({@link Mutation}s) to the index tables, based on the primary table diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/IndexBuildingFailureException.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/IndexBuildingFailureException.java new file mode 100644 index 00000000..1c89b7f9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/builder/IndexBuildingFailureException.java @@ -0,0 +1,50 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.builder; + +import java.io.IOException; + +import org.apache.hadoop.hbase.DoNotRetryIOException; + +/** + * Unexpected failure while building index updates that wasn't caused by an {@link IOException}. + * This should be used if there is some basic issue with indexing - and no matter of retries will + * fix it. + */ +@SuppressWarnings("serial") +public class IndexBuildingFailureException extends DoNotRetryIOException { + + /** + * Constructor for over the wire propagation. Generally, shouldn't be used since index failure + * should have an underlying cause to propagate. + * @param msg reason for the failure + */ + public IndexBuildingFailureException(String msg) { + super(msg); + } + + /** + * @param msg reason + * @param cause underlying cause for the failure + */ + public IndexBuildingFailureException(String msg, Throwable cause) { + super(msg, cause); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/Batch.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/Batch.java new file mode 100644 index 00000000..8b7815c8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/Batch.java @@ -0,0 +1,62 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered; + +import java.util.ArrayList; +import java.util.List; + +import org.apache.hadoop.hbase.KeyValue; + +/** + * A collection of {@link KeyValue KeyValues} to the primary table + */ +public class Batch { + + private static final long pointDeleteCode = KeyValue.Type.Delete.getCode(); + private final long timestamp; + private List batch = new ArrayList(); + private boolean allPointDeletes = true; + + /** + * @param ts + */ + public Batch(long ts) { + this.timestamp = ts; + } + + public void add(KeyValue kv){ + if (pointDeleteCode != kv.getType()) { + allPointDeletes = false; + } + batch.add(kv); + } + + public boolean isAllPointDeletes() { + return allPointDeletes; + } + + public long getTimestamp() { + return this.timestamp; + } + + public List getKvs() { + return this.batch; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/CoveredColumns.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/CoveredColumns.java new file mode 100644 index 00000000..4846dc86 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/CoveredColumns.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashSet; +import java.util.List; +import java.util.Set; + +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; + +/** + * Manage a set of {@link ColumnReference}s for the {@link LocalTableState}. + */ +public class CoveredColumns { + + Set columns = new HashSet(); + + public Collection findNonCoveredColumns( + Collection columns2) { + List uncovered = new ArrayList(); + for (ColumnReference column : columns2) { + if (!columns.contains(column)) { + uncovered.add(column); + } + } + return uncovered; + } + + public void addColumn(ColumnReference column) { + this.columns.add(column); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/CoveredColumnsIndexBuilder.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/CoveredColumnsIndexBuilder.java similarity index 90% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/CoveredColumnsIndexBuilder.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/CoveredColumnsIndexBuilder.java index 6a0dbe02..1e48d5c4 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/CoveredColumnsIndexBuilder.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/CoveredColumnsIndexBuilder.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered; import java.io.IOException; import java.lang.reflect.Constructor; @@ -52,12 +44,12 @@ import com.google.common.collect.Lists; import com.google.common.primitives.Longs; -import com.salesforce.hbase.index.builder.BaseIndexBuilder; -import com.salesforce.hbase.index.covered.data.LocalHBaseState; -import com.salesforce.hbase.index.covered.data.LocalTable; -import com.salesforce.hbase.index.covered.update.ColumnTracker; -import com.salesforce.hbase.index.covered.update.IndexUpdateManager; -import com.salesforce.hbase.index.covered.update.IndexedColumnGroup; +import org.apache.hadoop.hbase.index.builder.BaseIndexBuilder; +import org.apache.hadoop.hbase.index.covered.data.LocalHBaseState; +import org.apache.hadoop.hbase.index.covered.data.LocalTable; +import org.apache.hadoop.hbase.index.covered.update.ColumnTracker; +import org.apache.hadoop.hbase.index.covered.update.IndexUpdateManager; +import org.apache.hadoop.hbase.index.covered.update.IndexedColumnGroup; /** * Build covered indexes for phoenix updates. @@ -71,7 +63,7 @@ public class CoveredColumnsIndexBuilder extends BaseIndexBuilder { private static final Log LOG = LogFactory.getLog(CoveredColumnsIndexBuilder.class); - public static final String CODEC_CLASS_NAME_KEY = "com.salesforce.hbase.index.codec.class"; + public static final String CODEC_CLASS_NAME_KEY = "org.apache.hadoop.hbase.index.codec.class"; protected RegionCoprocessorEnvironment env; protected IndexCodec codec; diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/IndexCodec.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/IndexCodec.java new file mode 100644 index 00000000..ba053074 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/IndexCodec.java @@ -0,0 +1,112 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered; + +import java.io.IOException; + +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.client.Put; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; + +import org.apache.phoenix.index.BaseIndexCodec; + + +/** + * Codec for creating index updates from the current state of a table. + *

+ * Generally, you should extend {@link BaseIndexCodec} instead, so help maintain compatibility as + * features need to be added to the codec, as well as potentially not haivng to implement some + * methods. + */ +public interface IndexCodec { + + /** + * Do any code initialization necessary + * @param env environment in which the codec is operating + * @throws IOException if the codec cannot be initalized correctly + */ + public void initialize(RegionCoprocessorEnvironment env) throws IOException; + + /** + * Get the index cleanup entries. Currently, this must return just single row deletes (where just + * the row-key is specified and no columns are returned) mapped to the table name. For instance, + * to you have an index 'myIndex' with row : + * + * 

+   * v1,v2,v3 | CF:CQ0  | rowkey
+   *          | CF:CQ1  | rowkey
+   *
+ * + * To then cleanup this entry, you would just return 'v1,v2,v3', 'myIndex'. + * @param state the current state of the table that needs to be cleaned up. Generally, you only + * care about the latest column values, for each column you are indexing for each index + * table. + * @return the pairs of (deletes, index table name) that should be applied. + * @throws IOException + */ + public Iterable getIndexDeletes(TableState state) throws IOException; + + // table state has the pending update already applied, before calling + // get the new index entries + /** + * Get the index updates for the primary table state, for each index table. The returned + * {@link Put}s need to be fully specified (including timestamp) to minimize passes over the same + * key-values multiple times. + *

+ * You must specify the same timestamps on the Put as {@link TableState#getCurrentTimestamp()} so + * the index entries match the primary table row. This could be managed at a higher level, but + * would require iterating all the kvs in the Put again - very inefficient when compared to the + * current interface where you must provide a timestamp anyways (so you might as well provide the + * right one). + * @param state the current state of the table that needs to an index update Generally, you only + * care about the latest column values, for each column you are indexing for each index + * table. + * @return the pairs of (updates,index table name) that should be applied. + * @throws IOException + */ + public Iterable getIndexUpserts(TableState state) throws IOException; + + /** + * This allows the codec to dynamically change whether or not indexing should take place for a + * table. If it doesn't take place, we can save a lot of time on the regular Put patch. By making + * it dynamic, we can save offlining and then onlining a table just to turn indexing on. + *

+ * We can also be smart about even indexing a given update here too - if the update doesn't + * contain any columns that we care about indexing, we can save the effort of analyzing the put + * and further. + * @param m mutation that should be indexed. + * @return true if indexing is enabled for the given table. This should be on a per-table + * basis, as each codec is instantiated per-region. + * @throws IOException + */ + public boolean isEnabled(Mutation m) throws IOException; + + /** + * Get the batch identifier of the given mutation. Generally, updates to the table will take place + * in a batch of updates; if we know that the mutation is part of a batch, we can build the state + * much more intelligently. + *

+ * If you have batches that have multiple updates to the same row state, you must specify a + * batch id for each batch. Otherwise, we cannot guarantee index correctness + * @param m mutation that may or may not be part of the batch + * @return null if the mutation is not part of a batch or an id for the batch. + */ + public byte[] getBatchId(Mutation m); +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/IndexUpdate.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/IndexUpdate.java new file mode 100644 index 00000000..3a1866ed --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/IndexUpdate.java @@ -0,0 +1,79 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered; + +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.client.Put; +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.hadoop.hbase.index.covered.update.ColumnTracker; + +/** + * Update to make to the index table. + */ +public class IndexUpdate { + Mutation update; + byte[] tableName; + ColumnTracker columns; + + IndexUpdate(ColumnTracker tracker) { + this.columns = tracker; + } + + public void setUpdate(Mutation p) { + this.update = p; + } + + public void setTable(byte[] tableName) { + this.tableName = tableName; + } + + public Mutation getUpdate() { + return update; + } + + public byte[] getTableName() { + return tableName; + } + + public ColumnTracker getIndexedColumns() { + return columns; + } + + @Override + public String toString() { + return "IndexUpdate: \n\ttable - " + Bytes.toString(tableName) + "\n\tupdate: " + update + + "\n\tcolumns: " + columns; + } + + public static IndexUpdate createIndexUpdateForTesting(ColumnTracker tracker, byte[] table, Put p) { + IndexUpdate update = new IndexUpdate(tracker); + update.setTable(table); + update.setUpdate(p); + return update; + } + + /** + * @return true if the necessary state for a valid index update has been set. + */ + public boolean isValid() { + return this.tableName != null && this.update != null; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/KeyValueStore.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/KeyValueStore.java new file mode 100644 index 00000000..1048831b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/KeyValueStore.java @@ -0,0 +1,35 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.regionserver.KeyValueScanner; + +/** + * Store a collection of KeyValues in memory. + */ +public interface KeyValueStore { + + public void add(KeyValue kv, boolean overwrite); + + public KeyValueScanner getScanner(); + + public void rollback(KeyValue kv); +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/LocalTableState.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/LocalTableState.java new file mode 100644 index 00000000..1950864f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/LocalTableState.java @@ -0,0 +1,244 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collection; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.client.Result; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; +import org.apache.hadoop.hbase.regionserver.KeyValueScanner; +import org.apache.hadoop.hbase.util.Pair; + +import org.apache.hadoop.hbase.index.covered.data.IndexMemStore; +import org.apache.hadoop.hbase.index.covered.data.LocalHBaseState; +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; +import org.apache.hadoop.hbase.index.covered.update.ColumnTracker; +import org.apache.hadoop.hbase.index.covered.update.IndexedColumnGroup; +import org.apache.hadoop.hbase.index.scanner.Scanner; +import org.apache.hadoop.hbase.index.scanner.ScannerBuilder; + +/** + * Manage the state of the HRegion's view of the table, for the single row. + *

+ * Currently, this is a single-use object - you need to create a new one for each row that you need + * to manage. In the future, we could make this object reusable, but for the moment its easier to + * manage as a throw-away object. + *

+ * This class is not thread-safe - it requires external synchronization is access + * concurrently. + */ +public class LocalTableState implements TableState { + + private long ts; + private RegionCoprocessorEnvironment env; + private KeyValueStore memstore; + private LocalHBaseState table; + private Mutation update; + private Set trackedColumns = new HashSet(); + private ScannerBuilder scannerBuilder; + private List kvs = new ArrayList(); + private List hints; + private CoveredColumns columnSet; + + public LocalTableState(RegionCoprocessorEnvironment environment, LocalHBaseState table, Mutation update) { + this.env = environment; + this.table = table; + this.update = update; + this.memstore = new IndexMemStore(); + this.scannerBuilder = new ScannerBuilder(memstore, update); + this.columnSet = new CoveredColumns(); + } + + public void addPendingUpdates(KeyValue... kvs) { + if (kvs == null) return; + addPendingUpdates(Arrays.asList(kvs)); + } + + public void addPendingUpdates(List kvs) { + if(kvs == null) return; + setPendingUpdates(kvs); + addUpdate(kvs); + } + + private void addUpdate(List list) { + addUpdate(list, true); + } + + private void addUpdate(List list, boolean overwrite) { + if (list == null) return; + for (KeyValue kv : list) { + this.memstore.add(kv, overwrite); + } + } + + @Override + public RegionCoprocessorEnvironment getEnvironment() { + return this.env; + } + + @Override + public long getCurrentTimestamp() { + return this.ts; + } + + @Override + public void setCurrentTimestamp(long timestamp) { + this.ts = timestamp; + } + + public void resetTrackedColumns() { + this.trackedColumns.clear(); + } + + public Set getTrackedColumns() { + return this.trackedColumns; + } + + @Override + public Pair getIndexedColumnsTableState( + Collection indexedColumns) throws IOException { + ensureLocalStateInitialized(indexedColumns); + // filter out things with a newer timestamp and track the column references to which it applies + ColumnTracker tracker = new ColumnTracker(indexedColumns); + synchronized (this.trackedColumns) { + // we haven't seen this set of columns before, so we need to create a new tracker + if (!this.trackedColumns.contains(tracker)) { + this.trackedColumns.add(tracker); + } + } + + Scanner scanner = + this.scannerBuilder.buildIndexedColumnScanner(indexedColumns, tracker, ts); + + return new Pair(scanner, new IndexUpdate(tracker)); + } + + /** + * Initialize the managed local state. Generally, this will only be called by + * {@link #getNonIndexedColumnsTableState(List)}, which is unlikely to be called concurrently from the outside. + * Even then, there is still fairly low contention as each new Put/Delete will have its own table + * state. + */ + private synchronized void ensureLocalStateInitialized( + Collection columns) throws IOException { + // check to see if we haven't initialized any columns yet + Collection toCover = this.columnSet.findNonCoveredColumns(columns); + // we have all the columns loaded, so we are good to go. + if (toCover.isEmpty()) { + return; + } + + // add the current state of the row + this.addUpdate(this.table.getCurrentRowState(update, toCover).list(), false); + + // add the covered columns to the set + for (ColumnReference ref : toCover) { + this.columnSet.addColumn(ref); + } + } + + @Override + public Map getUpdateAttributes() { + return this.update.getAttributesMap(); + } + + @Override + public byte[] getCurrentRowKey() { + return this.update.getRow(); + } + + public Result getCurrentRowState() { + KeyValueScanner scanner = this.memstore.getScanner(); + List kvs = new ArrayList(); + while (scanner.peek() != null) { + try { + kvs.add(scanner.next()); + } catch (IOException e) { + // this should never happen - something has gone terribly arwy if it has + throw new RuntimeException("Local MemStore threw IOException!"); + } + } + return new Result(kvs); + } + + /** + * Helper to add a {@link Mutation} to the values stored for the current row + * @param pendingUpdate update to apply + */ + public void addUpdateForTesting(Mutation pendingUpdate) { + for (Map.Entry> e : pendingUpdate.getFamilyMap().entrySet()) { + List edits = e.getValue(); + addUpdate(edits); + } + } + + /** + * @param hints + */ + public void setHints(List hints) { + this.hints = hints; + } + + @Override + public List getIndexColumnHints() { + return this.hints; + } + + @Override + public Collection getPendingUpdate() { + return this.kvs; + } + + /** + * Set the {@link KeyValue}s in the update for which we are currently building an index update, + * but don't actually apply them. + * @param update pending {@link KeyValue}s + */ + public void setPendingUpdates(Collection update) { + this.kvs.clear(); + this.kvs.addAll(update); + } + + /** + * Apply the {@link KeyValue}s set in {@link #setPendingUpdates(Collection)}. + */ + public void applyPendingUpdates() { + this.addUpdate(kvs); + } + + /** + * Rollback all the given values from the underlying state. + * @param values + */ + public void rollback(Collection values) { + for (KeyValue kv : values) { + this.memstore.rollback(kv); + } + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/TableState.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/TableState.java similarity index 94% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/TableState.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/TableState.java index a93639c3..52d4e851 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/TableState.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/TableState.java @@ -1,4 +1,4 @@ -package com.salesforce.hbase.index.covered; +package org.apache.hadoop.hbase.index.covered; import java.io.IOException; import java.util.Collection; @@ -10,9 +10,9 @@ import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; import org.apache.hadoop.hbase.util.Pair; -import com.salesforce.hbase.index.covered.update.ColumnReference; -import com.salesforce.hbase.index.covered.update.IndexedColumnGroup; -import com.salesforce.hbase.index.scanner.Scanner; +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; +import org.apache.hadoop.hbase.index.covered.update.IndexedColumnGroup; +import org.apache.hadoop.hbase.index.scanner.Scanner; /** * Interface for the current state of the table. This is generally going to be as of a timestamp - a diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/IndexMemStore.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/IndexMemStore.java similarity index 84% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/IndexMemStore.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/IndexMemStore.java index f6637846..ac29dd1c 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/data/IndexMemStore.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/IndexMemStore.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.data; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.data; import java.util.Comparator; import java.util.Iterator; @@ -42,8 +34,8 @@ import org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.hbase.index.covered.KeyValueStore; -import com.salesforce.hbase.index.covered.LocalTableState; +import org.apache.hadoop.hbase.index.covered.KeyValueStore; +import org.apache.hadoop.hbase.index.covered.LocalTableState; /** * Like the HBase {@link MemStore}, but without all that extra work around maintaining snapshots and @@ -320,5 +312,22 @@ public boolean shouldUseScanner(Scan scan, SortedSet columns, long oldes throw new UnsupportedOperationException(this.getClass().getName() + " doesn't support checking to see if it should use a scanner!"); } + + /* + @Override + public boolean backwardSeek(KeyValue arg0) throws IOException { + throw new UnsupportedOperationException(); + } + + @Override + public boolean seekToLastRow() throws IOException { + throw new UnsupportedOperationException(); + } + + @Override + public boolean seekToPreviousRow(KeyValue arg0) throws IOException { + throw new UnsupportedOperationException(); + } + */ } } \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/LazyValueGetter.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/LazyValueGetter.java new file mode 100644 index 00000000..619a4cfb --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/LazyValueGetter.java @@ -0,0 +1,89 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.data; + +import java.io.IOException; +import java.util.Collections; +import java.util.HashMap; +import java.util.Map; + +import org.apache.hadoop.hbase.KeyValue; + +import org.apache.hadoop.hbase.index.ValueGetter; +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; +import org.apache.hadoop.hbase.index.scanner.Scanner; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; + +/** + * {@link ValueGetter} that uses lazy initialization to get the value for the given + * {@link ColumnReference}. Once stored, the mapping for that reference is retained. + */ +public class LazyValueGetter implements ValueGetter { + + private Scanner scan; + private volatile Map values; + private byte[] row; + + /** + * Back the getter with a {@link Scanner} to actually access the local data. + * @param scan backing scanner + * @param currentRow row key for the row to seek in the scanner + */ + public LazyValueGetter(Scanner scan, byte[] currentRow) { + this.scan = scan; + this.row = currentRow; + } + + @Override + public ImmutableBytesPtr getLatestValue(ColumnReference ref) throws IOException { + // ensure we have a backing map + if (values == null) { + synchronized (this) { + values = Collections.synchronizedMap(new HashMap()); + } + } + + // check the value in the map + ImmutableBytesPtr value = values.get(ref); + if (value == null) { + value = get(ref); + values.put(ref, value); + } + + return value; + } + + /** + * @param ref + * @return the first value on the scanner for the given column + */ + private ImmutableBytesPtr get(ColumnReference ref) throws IOException { + KeyValue first = ref.getFirstKeyValueForRow(row); + if (!scan.seek(first)) { + return null; + } + // there is a next value - we only care about the current value, so we can just snag that + KeyValue next = scan.next(); + if (ref.matches(next)) { + return new ImmutableBytesPtr(next.getBuffer(), next.getValueOffset(), next.getValueLength()); + } + return null; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/LocalHBaseState.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/LocalHBaseState.java new file mode 100644 index 00000000..0058efaf --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/LocalHBaseState.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.data; + +import java.io.IOException; +import java.util.Collection; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.client.Result; + +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; + +/** + * Access the current state of the row in the local HBase table, given a mutation + */ +public interface LocalHBaseState { + + /** + * @param m mutation for which we should get the current table state + * @param toCover all the columns the current row state needs to cover; hint the underlying lookup + * to save getting all the columns for the row + * @return the full state of the given row. Includes all current versions (even if they are not + * usually visible to the client (unless they are also doing a raw scan)). Never returns a + * null {@link Result} - instead, when there is not data for the row, returns a + * {@link Result} with no stored {@link KeyValue}s. + * @throws IOException if there is an issue reading the row + */ + public Result getCurrentRowState(Mutation m, Collection toCover) + throws IOException; + +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/LocalTable.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/LocalTable.java new file mode 100644 index 00000000..27905238 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/data/LocalTable.java @@ -0,0 +1,74 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.data; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.List; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.client.Result; +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; +import org.apache.hadoop.hbase.regionserver.HRegion; +import org.apache.hadoop.hbase.regionserver.RegionScanner; + +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; +import org.apache.hadoop.hbase.index.util.IndexManagementUtil; + +/** + * Wrapper around a lazily instantiated, local HTable. + *

+ * Previously, we had used various row and batch caches. However, this ends up being very + * complicated when attempting manage updating and invalidating the cache with no real gain as any + * row accessed multiple times will likely be in HBase's block cache, invalidating any extra caching + * we are doing here. In the end, its simpler and about as efficient to just get the current state + * of the row from HBase and let HBase manage caching the row from disk on its own. + */ +public class LocalTable implements LocalHBaseState { + + private RegionCoprocessorEnvironment env; + + public LocalTable(RegionCoprocessorEnvironment env) { + this.env = env; + } + + @Override + public Result getCurrentRowState(Mutation m, Collection columns) + throws IOException { + byte[] row = m.getRow(); + // need to use a scan here so we can get raw state, which Get doesn't provide. + Scan s = IndexManagementUtil.newLocalStateScan(Collections.singletonList(columns)); + s.setStartRow(row); + s.setStopRow(row); + HRegion region = this.env.getRegion(); + RegionScanner scanner = region.getScanner(s); + List kvs = new ArrayList(1); + boolean more = scanner.next(kvs); + assert !more : "Got more than one result when scanning" + " a single row in the primary table!"; + + Result r = new Result(kvs); + scanner.close(); + return r; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/ColumnGroup.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/ColumnGroup.java similarity index 97% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/ColumnGroup.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/ColumnGroup.java index 001a809c..9f923b10 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/ColumnGroup.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/ColumnGroup.java @@ -1,5 +1,5 @@ -package com.salesforce.hbase.index.covered.example; +package org.apache.hadoop.hbase.index.covered.example; import java.util.ArrayList; import java.util.Iterator; diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumn.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumn.java similarity index 95% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumn.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumn.java index 8c674dbb..ef4b2bac 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumn.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumn.java @@ -1,8 +1,8 @@ -package com.salesforce.hbase.index.covered.example; +package org.apache.hadoop.hbase.index.covered.example; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.hbase.index.covered.update.ColumnReference; +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; /** * A single Column (either a Column Family or a full Family:Qualifier pair) in a {@link ColumnGroup} diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumnIndexCodec.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumnIndexCodec.java similarity index 98% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumnIndexCodec.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumnIndexCodec.java index 0567b3ab..8f0ee994 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumnIndexCodec.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumnIndexCodec.java @@ -15,7 +15,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package com.salesforce.hbase.index.covered.example; +package org.apache.hadoop.hbase.index.covered.example; import java.io.IOException; import java.util.ArrayList; @@ -33,10 +33,10 @@ import org.apache.hadoop.hbase.util.Pair; import com.google.common.collect.Lists; -import com.salesforce.hbase.index.covered.IndexUpdate; -import com.salesforce.hbase.index.covered.TableState; -import com.salesforce.hbase.index.scanner.Scanner; -import com.salesforce.phoenix.index.BaseIndexCodec; +import org.apache.hadoop.hbase.index.covered.IndexUpdate; +import org.apache.hadoop.hbase.index.covered.TableState; +import org.apache.hadoop.hbase.index.scanner.Scanner; +import org.apache.phoenix.index.BaseIndexCodec; /** * diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumnIndexSpecifierBuilder.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumnIndexSpecifierBuilder.java similarity index 95% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumnIndexSpecifierBuilder.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumnIndexSpecifierBuilder.java index abed62e6..0c9f1559 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumnIndexSpecifierBuilder.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumnIndexSpecifierBuilder.java @@ -1,4 +1,4 @@ -package com.salesforce.hbase.index.covered.example; +package org.apache.hadoop.hbase.index.covered.example; import java.io.IOException; import java.util.ArrayList; @@ -10,9 +10,9 @@ import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.hbase.HTableDescriptor; -import com.salesforce.hbase.index.Indexer; -import com.salesforce.hbase.index.covered.CoveredColumnsIndexBuilder; -import com.salesforce.hbase.index.covered.IndexCodec; +import org.apache.hadoop.hbase.index.Indexer; +import org.apache.hadoop.hbase.index.covered.CoveredColumnsIndexBuilder; +import org.apache.hadoop.hbase.index.covered.IndexCodec; /** * Helper to build the configuration for the {@link CoveredColumnIndexer}. diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumnIndexer.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumnIndexer.java similarity index 95% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumnIndexer.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumnIndexer.java index 107bc9bf..c7019c4e 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/example/CoveredColumnIndexer.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/example/CoveredColumnIndexer.java @@ -1,4 +1,4 @@ -package com.salesforce.hbase.index.covered.example; +package org.apache.hadoop.hbase.index.covered.example; import java.io.IOException; import java.util.Collection; @@ -17,10 +17,10 @@ import org.apache.hadoop.hbase.util.Bytes; import org.apache.hadoop.hbase.util.Pair; -import com.salesforce.hbase.index.covered.Batch; -import com.salesforce.hbase.index.covered.CoveredColumnsIndexBuilder; -import com.salesforce.hbase.index.covered.LocalTableState; -import com.salesforce.hbase.index.covered.update.IndexUpdateManager; +import org.apache.hadoop.hbase.index.covered.Batch; +import org.apache.hadoop.hbase.index.covered.CoveredColumnsIndexBuilder; +import org.apache.hadoop.hbase.index.covered.LocalTableState; +import org.apache.hadoop.hbase.index.covered.update.IndexUpdateManager; /** * Index maintainer that maintains multiple indexes based on '{@link ColumnGroup}s'. Each group is a diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/ApplyAndFilterDeletesFilter.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/ApplyAndFilterDeletesFilter.java similarity index 84% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/ApplyAndFilterDeletesFilter.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/ApplyAndFilterDeletesFilter.java index bb91bf99..ebec75a9 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/ApplyAndFilterDeletesFilter.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/ApplyAndFilterDeletesFilter.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.filter; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.filter; import java.io.DataInput; import java.io.DataOutput; @@ -41,7 +33,7 @@ import org.apache.hadoop.hbase.client.Put; import org.apache.hadoop.hbase.filter.FilterBase; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; /** * Only allow the 'latest' timestamp of each family:qualifier pair, ensuring that they aren't diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/ColumnTrackingNextLargestTimestampFilter.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/ColumnTrackingNextLargestTimestampFilter.java new file mode 100644 index 00000000..b23368d4 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/ColumnTrackingNextLargestTimestampFilter.java @@ -0,0 +1,74 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.filter; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.filter.FilterBase; + +import org.apache.hadoop.hbase.index.covered.update.ColumnTracker; + +/** + * Similar to the {@link MaxTimestampFilter}, but also updates the 'next largest' timestamp seen + * that is not skipped by the below criteria. Note that it isn't as quick as the + * {@link MaxTimestampFilter} as we can't just seek ahead to a key with the matching timestamp, but + * have to iterate each kv until we find the right one with an allowed timestamp. + *

+ * Inclusively filter on the maximum timestamp allowed. Excludes all elements greater than (but not + * equal to) the given timestamp, so given ts = 5, a {@link KeyValue} with ts 6 is excluded, but not + * one with ts = 5. + *

+ * This filter generally doesn't make sense on its own - it should follow a per-column filter and + * possible a per-delete filter to only track the most recent (but not exposed to the user) + * timestamp. + */ +public class ColumnTrackingNextLargestTimestampFilter extends FilterBase { + + private long ts; + private ColumnTracker column; + + public ColumnTrackingNextLargestTimestampFilter(long maxTime, ColumnTracker toTrack) { + this.ts = maxTime; + this.column = toTrack; + } + + @Override + public ReturnCode filterKeyValue(KeyValue v) { + long timestamp = v.getTimestamp(); + if (timestamp > ts) { + this.column.setTs(timestamp); + return ReturnCode.SKIP; + } + return ReturnCode.INCLUDE; + } + + @Override + public void write(DataOutput out) throws IOException { + throw new UnsupportedOperationException("Server-side only filter, cannot be serialized!"); + } + + @Override + public void readFields(DataInput in) throws IOException { + throw new UnsupportedOperationException("Server-side only filter, cannot be deserialized!"); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/FamilyOnlyFilter.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/FamilyOnlyFilter.java similarity index 97% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/FamilyOnlyFilter.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/FamilyOnlyFilter.java index 0f4c5342..8591f88d 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/FamilyOnlyFilter.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/FamilyOnlyFilter.java @@ -15,7 +15,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package com.salesforce.hbase.index.covered.filter; +package org.apache.hadoop.hbase.index.covered.filter; import org.apache.hadoop.hbase.KeyValue; import org.apache.hadoop.hbase.filter.BinaryComparator; diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/MaxTimestampFilter.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/MaxTimestampFilter.java new file mode 100644 index 00000000..5b765791 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/MaxTimestampFilter.java @@ -0,0 +1,76 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.filter; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.filter.FilterBase; +import org.apache.hadoop.hbase.util.Bytes; + +/** + * Inclusive filter on the maximum timestamp allowed. Excludes all elements greater than (but not + * equal to) the given timestamp, so given ts = 5, a {@link KeyValue} with ts 6 is excluded, but not + * one with ts = 5. + */ +public class MaxTimestampFilter extends FilterBase { + + private long ts; + + public MaxTimestampFilter(long maxTime) { + this.ts = maxTime; + } + + @Override + public KeyValue getNextKeyHint(KeyValue currentKV) { + // this might be a little excessive right now - better safe than sorry though, so we don't mess + // with other filters too much. + KeyValue kv = currentKV.deepCopy(); + int offset =kv.getTimestampOffset(); + //set the timestamp in the buffer + byte[] buffer = kv.getBuffer(); + byte[] ts = Bytes.toBytes(this.ts); + System.arraycopy(ts, 0, buffer, offset, ts.length); + + return kv; + } + + @Override + public ReturnCode filterKeyValue(KeyValue v) { + long timestamp = v.getTimestamp(); + if (timestamp > ts) { + return ReturnCode.SEEK_NEXT_USING_HINT; + } + return ReturnCode.INCLUDE; + } + + @Override + public void write(DataOutput out) throws IOException { + throw new UnsupportedOperationException("Server-side only filter, cannot be serialized!"); + + } + + @Override + public void readFields(DataInput in) throws IOException { + throw new UnsupportedOperationException("Server-side only filter, cannot be deserialized!"); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/NewerTimestampFilter.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/NewerTimestampFilter.java similarity index 95% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/NewerTimestampFilter.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/NewerTimestampFilter.java index 5140a9b1..560cdd83 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/filter/NewerTimestampFilter.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/filter/NewerTimestampFilter.java @@ -1,4 +1,4 @@ -package com.salesforce.hbase.index.covered.filter; +package org.apache.hadoop.hbase.index.covered.filter; import java.io.DataInput; import java.io.DataOutput; diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/ColumnReference.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/ColumnReference.java new file mode 100644 index 00000000..0525028a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/ColumnReference.java @@ -0,0 +1,168 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.update; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; + +/** + * + */ +public class ColumnReference implements Comparable { + + public static final byte[] ALL_QUALIFIERS = new byte[0]; + + private static int calcHashCode(byte[] family, byte[] qualifier) { + final int prime = 31; + int result = 1; + result = prime * result + Bytes.hashCode(family); + result = prime * result + Bytes.hashCode(qualifier); + return result; + } + + private final int hashCode; + protected final byte[] family; + protected final byte[] qualifier; + private volatile ImmutableBytesWritable familyPtr; + private volatile ImmutableBytesWritable qualifierPtr; + + public ColumnReference(byte[] family, byte[] qualifier) { + this.family = family; + this.qualifier = qualifier; + this.hashCode = calcHashCode(family, qualifier); + } + + public byte[] getFamily() { + return this.family; + } + + public byte[] getQualifier() { + return this.qualifier; + } + + public ImmutableBytesWritable getFamilyWritable() { + if (this.familyPtr == null) { + synchronized (this.family) { + if (this.familyPtr == null) { + this.familyPtr = new ImmutableBytesPtr(this.family); + } + } + } + return this.familyPtr; + } + + public ImmutableBytesWritable getQualifierWritable() { + if (this.qualifierPtr == null) { + synchronized (this.qualifier) { + if (this.qualifierPtr == null) { + this.qualifierPtr = new ImmutableBytesPtr(this.qualifier); + } + } + } + return this.qualifierPtr; + } + + public boolean matches(KeyValue kv) { + if (matchesFamily(kv.getBuffer(), kv.getFamilyOffset(), kv.getFamilyLength())) { + return matchesQualifier(kv.getBuffer(), kv.getQualifierOffset(), kv.getQualifierLength()); + } + return false; + } + + /** + * @param qual to check against + * @return true if this column covers the given qualifier. + */ + public boolean matchesQualifier(byte[] qual) { + return matchesQualifier(qual, 0, qual.length); + } + + public boolean matchesQualifier(byte[] bytes, int offset, int length) { + return allColumns() ? true : match(bytes, offset, length, qualifier); + } + + /** + * @param family to check against + * @return true if this column covers the given family. + */ + public boolean matchesFamily(byte[] family) { + return matchesFamily(family, 0, family.length); + } + + public boolean matchesFamily(byte[] bytes, int offset, int length) { + return match(bytes, offset, length, family); + } + + /** + * @return true if this should include all column qualifiers, false otherwise + */ + public boolean allColumns() { + return this.qualifier == ALL_QUALIFIERS; + } + + /** + * Check to see if the passed bytes match the stored bytes + * @param first + * @param storedKey the stored byte[], should never be null + * @return true if they are byte-equal + */ + private boolean match(byte[] first, int offset, int length, byte[] storedKey) { + return first == null ? false : Bytes.equals(first, offset, length, storedKey, 0, + storedKey.length); + } + + public KeyValue getFirstKeyValueForRow(byte[] row) { + return KeyValue.createFirstOnRow(row, family, qualifier == ALL_QUALIFIERS ? null : qualifier); + } + + @Override + public int compareTo(ColumnReference o) { + int c = Bytes.compareTo(family, o.family); + if (c == 0) { + // matching families, compare qualifiers + c = Bytes.compareTo(qualifier, o.qualifier); + } + return c; + } + + @Override + public boolean equals(Object o) { + if (o instanceof ColumnReference) { + ColumnReference other = (ColumnReference) o; + if (hashCode == other.hashCode && Bytes.equals(family, other.family)) { + return Bytes.equals(qualifier, other.qualifier); + } + } + return false; + } + + @Override + public int hashCode() { + return hashCode; + } + + @Override + public String toString() { + return "ColumnReference - " + Bytes.toString(family) + ":" + Bytes.toString(qualifier); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/ColumnTracker.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/ColumnTracker.java new file mode 100644 index 00000000..e41c3149 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/ColumnTracker.java @@ -0,0 +1,117 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.update; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.List; + + +/** + * Simple POJO for tracking a bunch of column references and the next-newest timestamp for those + * columns + *

+ * Two {@link ColumnTracker}s are considered equal if they track the same columns, even if their + * timestamps are different. + */ +public class ColumnTracker implements IndexedColumnGroup { + + public static final long NO_NEWER_PRIMARY_TABLE_ENTRY_TIMESTAMP = Long.MAX_VALUE; + public static final long GUARANTEED_NEWER_UPDATES = Long.MIN_VALUE; + private final List columns; + private long ts = NO_NEWER_PRIMARY_TABLE_ENTRY_TIMESTAMP; + private final int hashCode; + + private static int calcHashCode(List columns) { + return columns.hashCode(); + } + + public ColumnTracker(Collection columns) { + this.columns = new ArrayList(columns); + // sort the columns + Collections.sort(this.columns); + this.hashCode = calcHashCode(this.columns); + } + + /** + * Set the current timestamp, only if the passed timestamp is strictly less than the currently + * stored timestamp + * @param ts the timestmap to potentially store. + * @return the currently stored timestamp. + */ + public long setTs(long ts) { + this.ts = this.ts > ts ? ts : this.ts; + return this.ts; + } + + public long getTS() { + return this.ts; + } + + @Override + public int hashCode() { + return hashCode; + } + + @Override + public boolean equals(Object o){ + if(!(o instanceof ColumnTracker)){ + return false; + } + ColumnTracker other = (ColumnTracker)o; + if (hashCode != other.hashCode) { + return false; + } + if (other.columns.size() != columns.size()) { + return false; + } + + // check each column to see if they match + for (int i = 0; i < columns.size(); i++) { + if (!columns.get(i).equals(other.columns.get(i))) { + return false; + } + } + + return true; + } + + @Override + public List getColumns() { + return this.columns; + } + + /** + * @return true if this set of columns has seen a column with a timestamp newer than the + * requested timestamp, false otherwise. + */ + public boolean hasNewerTimestamps() { + return !isNewestTime(this.ts); + } + + /** + * @param ts timestamp to check + * @return true if the timestamp is at the most recent timestamp for a column + */ + public static boolean isNewestTime(long ts) { + return ts == NO_NEWER_PRIMARY_TABLE_ENTRY_TIMESTAMP; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/IndexUpdateManager.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/IndexUpdateManager.java similarity index 79% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/IndexUpdateManager.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/IndexUpdateManager.java index 1a9d8c6f..52eb5dd1 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/covered/update/IndexUpdateManager.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/IndexUpdateManager.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.covered.update; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.update; import java.util.Collection; import java.util.Comparator; @@ -43,7 +35,7 @@ import com.google.common.collect.Lists; import com.google.common.primitives.Longs; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; /** * Keeps track of the index updates diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/IndexedColumnGroup.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/IndexedColumnGroup.java new file mode 100644 index 00000000..535742bb --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/IndexedColumnGroup.java @@ -0,0 +1,30 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.update; + +import java.util.List; + +/** + * Group of columns that were requested to build an index + */ +public interface IndexedColumnGroup { + + public List getColumns(); +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/SortedCollection.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/SortedCollection.java new file mode 100644 index 00000000..b228e63b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/covered/update/SortedCollection.java @@ -0,0 +1,130 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.covered.update; + +import java.util.Arrays; +import java.util.Collection; +import java.util.Comparator; +import java.util.Iterator; +import java.util.PriorityQueue; + +import com.google.common.collect.Iterators; + +/** + * A collection whose elements are stored and returned sorted. + *

+ * We can't just use something like a {@link PriorityQueue} because it doesn't return the + * underlying values in sorted order. + * @param + */ +class SortedCollection implements Collection, Iterable { + + private PriorityQueue queue; + private Comparator comparator; + + /** + * Use the given comparator to compare all keys for sorting + * @param comparator + */ + public SortedCollection(Comparator comparator) { + this.queue = new PriorityQueue(1, comparator); + this.comparator = comparator; + } + + /** + * All passed elements are expected to be {@link Comparable} + */ + public SortedCollection() { + this.queue = new PriorityQueue(); + } + + @Override + public int size() { + return this.queue.size(); + } + + @Override + public boolean isEmpty() { + return this.queue.isEmpty(); + } + + @Override + public boolean contains(Object o) { + return this.queue.contains(o); + } + + @Override + public Iterator iterator() { + @SuppressWarnings("unchecked") + T[] array = (T[]) this.queue.toArray(); + if (this.comparator == null) { + Arrays.sort(array); + } else { + Arrays.sort( + array, this.comparator);} + return Iterators.forArray(array); + } + + @Override + public Object[] toArray() { + return this.queue.toArray(); + } + + @SuppressWarnings("hiding") + @Override + public T[] toArray(T[] a) { + return this.queue.toArray(a); + } + + @Override + public boolean add(T e) { + return this.queue.add(e); + } + + @Override + public boolean remove(Object o) { + return this.queue.remove(o); + } + + @Override + public boolean containsAll(Collection c) { + return this.queue.containsAll(c); + } + + @Override + public boolean addAll(Collection c) { + return this.queue.addAll(c); + } + + @Override + public boolean removeAll(Collection c) { + return queue.removeAll(c); + } + + @Override + public boolean retainAll(Collection c) { + return this.queue.retainAll(c); + } + + @Override + public void clear() { + this.queue.clear(); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/exception/IndexWriteException.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/exception/IndexWriteException.java new file mode 100644 index 00000000..16fa1bca --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/exception/IndexWriteException.java @@ -0,0 +1,45 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.exception; + +import org.apache.hadoop.hbase.HBaseIOException; + +/** + * Generic {@link Exception} that an index write has failed + */ +@SuppressWarnings("serial") +public class IndexWriteException extends HBaseIOException { + + public IndexWriteException() { + super(); + } + + public IndexWriteException(String message, Throwable cause) { + super(message, cause); + } + + public IndexWriteException(String message) { + super(message); + } + + public IndexWriteException(Throwable cause) { + super(cause); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/exception/MultiIndexWriteFailureException.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/exception/MultiIndexWriteFailureException.java new file mode 100644 index 00000000..6af43bc7 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/exception/MultiIndexWriteFailureException.java @@ -0,0 +1,46 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.exception; + +import java.util.List; + +import org.apache.hadoop.hbase.index.table.HTableInterfaceReference; + +/** + * Indicate a failure to write to multiple index tables. + */ +@SuppressWarnings("serial") +public class MultiIndexWriteFailureException extends IndexWriteException { + + private List failures; + + /** + * @param failures the tables to which the index write did not succeed + */ + public MultiIndexWriteFailureException(List failures) { + super("Failed to write to multiple index tables"); + this.failures = failures; + + } + + public List getFailedTables() { + return this.failures; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/exception/SingleIndexWriteFailureException.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/exception/SingleIndexWriteFailureException.java new file mode 100644 index 00000000..f0fcb4ba --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/exception/SingleIndexWriteFailureException.java @@ -0,0 +1,63 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.exception; + +import java.util.List; + +import org.apache.hadoop.hbase.client.Mutation; + +/** + * Exception thrown if we cannot successfully write to an index table. + */ +@SuppressWarnings("serial") +public class SingleIndexWriteFailureException extends IndexWriteException { + + private String table; + + /** + * Cannot reach the index, but not sure of the table or the mutations that caused the failure + * @param msg more description of what happened + * @param cause original cause + */ + public SingleIndexWriteFailureException(String msg, Throwable cause) { + super(msg, cause); + } + + /** + * Failed to write the passed mutations to an index table for some reason. + * @param targetTableName index table to which we attempted to write + * @param mutations mutations that were attempted + * @param cause underlying reason for the failure + */ + public SingleIndexWriteFailureException(String targetTableName, List mutations, + Exception cause) { + super("Failed to make index update:\n\t table: " + targetTableName + "\n\t edits: " + mutations + + "\n\tcause: " + cause == null ? "UNKNOWN" : cause.getMessage(), cause); + this.table = targetTableName; + } + + /** + * @return The table to which we failed to write the index updates. If unknown, returns + * null + */ + public String getTableName() { + return this.table; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/BaseTaskRunner.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/BaseTaskRunner.java new file mode 100644 index 00000000..da6f7950 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/BaseTaskRunner.java @@ -0,0 +1,131 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.parallel; + +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.CancellationException; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.ExecutorService; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.hbase.Abortable; + +import com.google.common.util.concurrent.ListenableFuture; +import com.google.common.util.concurrent.ListeningExecutorService; +import com.google.common.util.concurrent.MoreExecutors; + +/** + * {@link TaskRunner} that just manages the underlying thread pool. On called to + * {@link #stop(String)}, the thread pool is shutdown immediately - all pending tasks are cancelled + * and running tasks receive and interrupt. + *

+ * If we find a failure the failure is propagated to the {@link TaskBatch} so any {@link Task} that + * is interested can kill itself as well. + */ +public abstract class BaseTaskRunner implements TaskRunner { + + private static final Log LOG = LogFactory.getLog(BaseTaskRunner.class); + protected ListeningExecutorService writerPool; + private boolean stopped; + + public BaseTaskRunner(ExecutorService service) { + this.writerPool = MoreExecutors.listeningDecorator(service); + } + + @Override + public List submit(TaskBatch tasks) throws CancellationException, ExecutionException, + InterruptedException { + // submit each task to the pool and queue it up to be watched + List> futures = new ArrayList>(tasks.size()); + for (Task task : tasks.getTasks()) { + futures.add(this.writerPool.submit(task)); + } + try { + // This logic is actually much more synchronized than the previous logic. Now we rely on a + // synchronization around the status to tell us when we are done. While this does have the + // advantage of being (1) less code, and (2) supported as part of a library, it is just that + // little bit slower. If push comes to shove, we can revert back to the previous + // implementation, but for right now, this works just fine. + return submitTasks(futures).get(); + } catch (CancellationException e) { + // propagate the failure back out + logAndNotifyAbort(e, tasks); + throw e; + } catch (ExecutionException e) { + // propagate the failure back out + logAndNotifyAbort(e, tasks); + throw e; + } + } + + private void logAndNotifyAbort(Exception e, Abortable abort) { + String msg = "Found a failed task because: " + e.getMessage(); + LOG.error(msg, e); + abort.abort(msg, e.getCause()); + } + + /** + * Build a ListenableFuture for the tasks. Implementing classes can determine return behaviors on + * the given tasks + * @param futures to wait on + * @return a single {@link ListenableFuture} that completes based on the passes tasks. + */ + protected abstract ListenableFuture> submitTasks(List> futures); + + @Override + public List submitUninterruptible(TaskBatch tasks) throws EarlyExitFailure, + ExecutionException { + boolean interrupted = false; + try { + while (!this.isStopped()) { + try { + return this.submit(tasks); + } catch (InterruptedException e) { + interrupted = true; + } + } + } finally { + // restore the interrupted status + if (interrupted) { + Thread.currentThread().interrupt(); + } + } + + // should only get here if we are interrupted while waiting for a result and have been told to + // shutdown by an external source + throw new EarlyExitFailure("Interrupted and stopped before computation was complete!"); + } + + @Override + public void stop(String why) { + if (this.stopped) { + return; + } + LOG.info("Shutting down task runner because " + why); + this.writerPool.shutdownNow(); + } + + @Override + public boolean isStopped() { + return this.stopped; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/EarlyExitFailure.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/EarlyExitFailure.java new file mode 100644 index 00000000..ab2ffd5f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/EarlyExitFailure.java @@ -0,0 +1,36 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.parallel; + +import java.io.IOException; + +/** + * Exception denoting a need to early-exit a task (or group of tasks) due to external notification + */ +@SuppressWarnings("serial") +public class EarlyExitFailure extends IOException { + + /** + * @param msg reason for the early exit + */ + public EarlyExitFailure(String msg) { + super(msg); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/QuickFailingTaskRunner.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/QuickFailingTaskRunner.java new file mode 100644 index 00000000..6a194a6e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/QuickFailingTaskRunner.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.parallel; + +import java.util.List; +import java.util.concurrent.ExecutorService; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; + +import com.google.common.util.concurrent.Futures; +import com.google.common.util.concurrent.ListenableFuture; + +/** + * {@link TaskRunner} that attempts to run all tasks passed, but quits early if any {@link Task} + * fails, not waiting for the remaining {@link Task}s to complete. + */ +public class QuickFailingTaskRunner extends BaseTaskRunner { + + static final Log LOG = LogFactory.getLog(QuickFailingTaskRunner.class); + + /** + * @param service thread pool to which {@link Task}s are submitted. This service is then 'owned' + * by this and will be shutdown on calls to {@link #stop(String)}. + */ + public QuickFailingTaskRunner(ExecutorService service) { + super(service); + } + + @Override + protected ListenableFuture> submitTasks(List> futures) { + return Futures.allAsList(futures); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/Task.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/Task.java new file mode 100644 index 00000000..a052759e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/Task.java @@ -0,0 +1,42 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.parallel; + +import java.util.concurrent.Callable; + +import org.apache.hadoop.hbase.Abortable; + +/** + * Like a {@link Callable}, but supports an internal {@link Abortable} that can be checked + * periodically to determine if the batch should abort + * @param expected result of the task + */ +public abstract class Task implements Callable { + + private Abortable batch; + + void setBatchMonitor(Abortable abort) { + this.batch = abort; + } + + protected boolean isBatchFailed() { + return this.batch.isAborted(); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/TaskBatch.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/TaskBatch.java new file mode 100644 index 00000000..49d96bf9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/TaskBatch.java @@ -0,0 +1,76 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.parallel; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; +import java.util.concurrent.atomic.AtomicBoolean; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.hbase.Abortable; + +/** + * A group of {@link Task}s. The tasks are all bound together using the same {@link Abortable} ( + * this) to ensure that all tasks are aware when any of the other tasks fails. + * @param expected result type from all the tasks + */ +public class TaskBatch implements Abortable { + private static final Log LOG = LogFactory.getLog(TaskBatch.class); + private AtomicBoolean aborted = new AtomicBoolean(); + private List> tasks; + + /** + * @param size expected number of tasks + */ + public TaskBatch(int size) { + this.tasks = new ArrayList>(size); + } + + public void add(Task task) { + this.tasks.add(task); + task.setBatchMonitor(this); + } + + public Collection> getTasks() { + return this.tasks; + } + + @Override + public void abort(String why, Throwable e) { + if (this.aborted.getAndSet(true)) { + return; + } + LOG.info("Aborting batch of tasks because " + why); + } + + @Override + public boolean isAborted() { + return this.aborted.get(); + } + + /** + * @return the number of tasks assigned to this batch + */ + public int size() { + return this.tasks.size(); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/TaskRunner.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/TaskRunner.java new file mode 100644 index 00000000..bc8749c5 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/TaskRunner.java @@ -0,0 +1,62 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.parallel; + +import java.util.List; +import java.util.concurrent.ExecutionException; + +import org.apache.hadoop.hbase.Stoppable; + +/** + * + */ +public interface TaskRunner extends Stoppable { + + /** + * Submit the given tasks to the pool and wait for them to complete. fail. + *

+ * Non-interruptible method. To stop any running tasks call {@link #stop(String)} - this will + * shutdown the thread pool, causing any pending tasks to be failed early (whose failure will be + * ignored) and interrupt any running tasks. It is up to the passed tasks to respect the interrupt + * notification + * @param tasks to run + * @return the result from each task + * @throws ExecutionException if any of the tasks fails. Wraps the underyling failure, which can + * be retrieved via {@link ExecutionException#getCause()}. + * @throws InterruptedException if the current thread is interrupted while waiting for the batch + * to complete + */ + public List submit(TaskBatch tasks) throws + ExecutionException, InterruptedException; + + /** + * Similar to {@link #submit(TaskBatch)}, but is not interruptible. If an interrupt is found while + * waiting for results, we ignore it and only stop is {@link #stop(String)} has been called. On + * return from the method, the interrupt status of the thread is restored. + * @param tasks to run + * @return the result from each task + * @throws EarlyExitFailure if there are still tasks to submit to the pool, but there is a stop + * notification + * @throws ExecutionException if any of the tasks fails. Wraps the underyling failure, which can + * be retrieved via {@link ExecutionException#getCause()}. + */ + public List submitUninterruptible(TaskBatch tasks) throws EarlyExitFailure, + ExecutionException; +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/ThreadPoolBuilder.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/ThreadPoolBuilder.java new file mode 100644 index 00000000..869ccb8a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/ThreadPoolBuilder.java @@ -0,0 +1,92 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.parallel; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.util.Pair; + +/** + * Helper utility to make a thread pool from a configuration based on reasonable defaults and passed + * configuration keys. + */ +public class ThreadPoolBuilder { + + private static final Log LOG = LogFactory.getLog(ThreadPoolBuilder.class); + private static final long DEFAULT_TIMEOUT = 60; + private static final int DEFAULT_MAX_THREADS = 1;// is there a better default? + private Pair timeout; + private Pair maxThreads; + private String name; + private Configuration conf; + + public ThreadPoolBuilder(String poolName, Configuration conf) { + this.name = poolName; + this.conf = conf; + } + + public ThreadPoolBuilder setCoreTimeout(String confkey, long defaultTime) { + if (defaultTime <= 0) { + defaultTime = DEFAULT_TIMEOUT; + } + this.timeout = new Pair(confkey, defaultTime); + return this; + } + + public ThreadPoolBuilder setCoreTimeout(String confKey) { + return this.setCoreTimeout(confKey, DEFAULT_TIMEOUT); + } + + public ThreadPoolBuilder setMaxThread(String confkey, int defaultThreads) { + if (defaultThreads <= 0) { + defaultThreads = DEFAULT_MAX_THREADS; + } + this.maxThreads = new Pair(confkey, defaultThreads); + return this; + } + + String getName() { + return this.name; + } + + int getMaxThreads() { + int maxThreads = DEFAULT_MAX_THREADS; + if (this.maxThreads != null) { + String key = this.maxThreads.getFirst(); + maxThreads = + key == null ? this.maxThreads.getSecond() : conf.getInt(key, this.maxThreads.getSecond()); + } + LOG.trace("Creating pool builder with max " + maxThreads + " threads "); + return maxThreads; + } + + long getKeepAliveTime() { + long timeout =DEFAULT_TIMEOUT; + if (this.timeout != null) { + String key = this.timeout.getFirst(); + timeout = + key == null ? this.timeout.getSecond() : conf.getLong(key, this.timeout.getSecond()); + } + + LOG.trace("Creating pool builder with core thread timeout of " + timeout + " seconds "); + return timeout; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/ThreadPoolManager.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/ThreadPoolManager.java new file mode 100644 index 00000000..7a081055 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/ThreadPoolManager.java @@ -0,0 +1,148 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.parallel; + +import java.util.Collections; +import java.util.List; +import java.util.Map; +import java.util.concurrent.BlockingQueue; +import java.util.concurrent.LinkedBlockingQueue; +import java.util.concurrent.ThreadFactory; +import java.util.concurrent.ThreadPoolExecutor; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; +import org.apache.hadoop.hbase.util.Threads; + +/** + * Manage access to thread pools + */ +public class ThreadPoolManager { + + private static final Log LOG = LogFactory.getLog(ThreadPoolManager.class); + + /** + * Get an executor for the given name, based on the passed {@link Configuration}. If a thread pool + * already exists with that name, it will be returned. + * @param builder + * @param env + * @return a {@link ThreadPoolExecutor} for the given name. Thread pool that only shuts down when + * there are no more explicit references to it. You do not need to shutdown the threadpool + * on your own - it is managed for you. When you are done, you merely need to release your + * reference. If you do attempt to shutdown the pool, you should be careful to call + * {@link ThreadPoolExecutor#shutdown()} XOR {@link ThreadPoolExecutor#shutdownNow()} - extra calls to either can lead to + * early shutdown of the pool. + */ + public static synchronized ThreadPoolExecutor getExecutor(ThreadPoolBuilder builder, + RegionCoprocessorEnvironment env) { + return getExecutor(builder, env.getSharedData()); + } + + static synchronized ThreadPoolExecutor getExecutor(ThreadPoolBuilder builder, + Map poolCache) { + ThreadPoolExecutor pool = (ThreadPoolExecutor) poolCache.get(builder.getName()); + if (pool == null || pool.isTerminating() || pool.isShutdown()) { + pool = getDefaultExecutor(builder); + LOG.info("Creating new pool for " + builder.getName()); + poolCache.put(builder.getName(), pool); + } + ((ShutdownOnUnusedThreadPoolExecutor) pool).addReference(); + + return pool; + } + + /** + * @param conf + * @return + */ + private static ShutdownOnUnusedThreadPoolExecutor getDefaultExecutor(ThreadPoolBuilder builder) { + int maxThreads = builder.getMaxThreads(); + long keepAliveTime = builder.getKeepAliveTime(); + + // we prefer starting a new thread to queuing (the opposite of the usual ThreadPoolExecutor) + // since we are probably writing to a bunch of index tables in this case. Any pending requests + // are then queued up in an infinite (Integer.MAX_VALUE) queue. However, we allow core threads + // to timeout, to we tune up/down for bursty situations. We could be a bit smarter and more + // closely manage the core-thread pool size to handle the bursty traffic (so we can always keep + // some core threads on hand, rather than starting from scratch each time), but that would take + // even more time. If we shutdown the pool, but are still putting new tasks, we can just do the + // usual policy and throw a RejectedExecutionException because we are shutting down anyways and + // the worst thing is that this gets unloaded. + ShutdownOnUnusedThreadPoolExecutor pool = + new ShutdownOnUnusedThreadPoolExecutor(maxThreads, maxThreads, keepAliveTime, + TimeUnit.SECONDS, new LinkedBlockingQueue(), + Threads.newDaemonThreadFactory(builder.getName() + "-"), builder.getName()); + pool.allowCoreThreadTimeOut(true); + return pool; + } + + /** + * Thread pool that only shuts down when there are no more explicit references to it. A reference + * is when obtained and released on calls to {@link #shutdown()} or {@link #shutdownNow()}. + * Therefore, users should be careful to call {@link #shutdown()} XOR {@link #shutdownNow()} - + * extra calls to either can lead to early shutdown of the pool. + */ + private static class ShutdownOnUnusedThreadPoolExecutor extends ThreadPoolExecutor { + + private AtomicInteger references; + private String poolName; + + public ShutdownOnUnusedThreadPoolExecutor(int coreThreads, int maxThreads, long keepAliveTime, + TimeUnit timeUnit, BlockingQueue workQueue, ThreadFactory threadFactory, + String poolName) { + super(coreThreads, maxThreads, keepAliveTime, timeUnit, workQueue, threadFactory); + this.references = new AtomicInteger(); + this.poolName = poolName; + } + + public void addReference() { + this.references.incrementAndGet(); + } + + @Override + protected void finalize() { + // override references counter if we go out of scope - ensures the pool gets cleaned up + LOG.info("Shutting down pool '" + poolName + "' because no more references"); + super.finalize(); + } + + @Override + public void shutdown() { + if (references.decrementAndGet() <= 0) { + LOG.debug("Shutting down pool " + this.poolName); + super.shutdown(); + } + } + + @Override + public List shutdownNow() { + if (references.decrementAndGet() <= 0) { + LOG.debug("Shutting down pool " + this.poolName + " NOW!"); + return super.shutdownNow(); + } + return Collections.emptyList(); + } + + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/WaitForCompletionTaskRunner.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/WaitForCompletionTaskRunner.java new file mode 100644 index 00000000..fe9f3ac9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/parallel/WaitForCompletionTaskRunner.java @@ -0,0 +1,53 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.parallel; + +import java.util.List; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.ExecutorService; + +import com.google.common.util.concurrent.Futures; +import com.google.common.util.concurrent.ListenableFuture; + +/** + * A {@link TaskRunner} that ensures that all the tasks have been attempted before we return, even + * if some of the tasks cause failures. + *

+ * Because we wait until the entire batch is complete to see the failure, checking for failure of + * the {@link TaskBatch} on the submitted tasks will not help - they will never see the failure of + * the other tasks. You will need to provide an external mechanism to propagate the error. + *

+ * Does not throw an {@link ExecutionException} if any of the tasks fail. + */ +public class WaitForCompletionTaskRunner extends BaseTaskRunner { + + /** + * @param service thread pool to which {@link Task}s are submitted. This service is then 'owned' + * by this and will be shutdown on calls to {@link #stop(String)}. + */ + public WaitForCompletionTaskRunner(ExecutorService service) { + super(service); + } + + @Override + public ListenableFuture> submitTasks(List> futures) { + return Futures.successfulAsList(futures); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/EmptyScanner.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/EmptyScanner.java similarity index 91% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/EmptyScanner.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/EmptyScanner.java index 51cb8a36..491a8950 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/EmptyScanner.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/EmptyScanner.java @@ -1,4 +1,4 @@ -package com.salesforce.hbase.index.scanner; +package org.apache.hadoop.hbase.index.scanner; import java.io.IOException; diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/FilteredKeyValueScanner.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/FilteredKeyValueScanner.java new file mode 100644 index 00000000..0f7fed3e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/FilteredKeyValueScanner.java @@ -0,0 +1,144 @@ +package org.apache.hadoop.hbase.index.scanner; + +import java.io.IOException; +import java.util.SortedSet; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.filter.Filter; +import org.apache.hadoop.hbase.filter.Filter.ReturnCode; +import org.apache.hadoop.hbase.regionserver.KeyValueScanner; + +import org.apache.hadoop.hbase.index.covered.KeyValueStore; + +/** + * Combine a simplified version of the logic in the ScanQueryMatcher and the KeyValueScanner. We can get away with this + * here because we are only concerned with a single MemStore for the index; we don't need to worry about multiple column + * families or minimizing seeking through file - we just want to iterate the kvs quickly, in-memory. + */ +public class FilteredKeyValueScanner implements KeyValueScanner { + + private KeyValueScanner delegate; + private Filter filter; + + public FilteredKeyValueScanner(Filter filter, KeyValueStore store) { + this(filter, store.getScanner()); + } + + private FilteredKeyValueScanner(Filter filter, KeyValueScanner delegate) { + this.delegate = delegate; + this.filter = filter; + } + + @Override + public KeyValue peek() { + return delegate.peek(); + } + + /** + * Same a {@link KeyValueScanner#next()} except that we filter out the next {@link KeyValue} until we find one that + * passes the filter. + * + * @return the next {@link KeyValue} or null if no next {@link KeyValue} is present and passes all the + * filters. + */ + @Override + public KeyValue next() throws IOException { + seekToNextUnfilteredKeyValue(); + return delegate.next(); + } + + @Override + public boolean seek(KeyValue key) throws IOException { + if (filter.filterAllRemaining()) { return false; } + // see if we can seek to the next key + if (!delegate.seek(key)) { return false; } + + return seekToNextUnfilteredKeyValue(); + } + + private boolean seekToNextUnfilteredKeyValue() throws IOException { + while (true) { + KeyValue peeked = delegate.peek(); + // no more key values, so we are done + if (peeked == null) { return false; } + + // filter the peeked value to see if it should be served + ReturnCode code = filter.filterKeyValue(peeked); + switch (code) { + // included, so we are done + case INCLUDE: + case INCLUDE_AND_NEXT_COL: + return true; + // not included, so we need to go to the next row + case SKIP: + case NEXT_COL: + case NEXT_ROW: + delegate.next(); + break; + // use a seek hint to find out where we should go + case SEEK_NEXT_USING_HINT: + delegate.seek(filter.getNextKeyHint(peeked)); + } + } + } + + @Override + public boolean reseek(KeyValue key) throws IOException { + this.delegate.reseek(key); + return this.seekToNextUnfilteredKeyValue(); + } + + @Override + public boolean requestSeek(KeyValue kv, boolean forward, boolean useBloom) throws IOException { + return this.reseek(kv); + } + + @Override + public boolean isFileScanner() { + return false; + } + + @Override + public long getSequenceID() { + return this.delegate.getSequenceID(); + } + + @Override + public boolean shouldUseScanner(Scan scan, SortedSet columns, long oldestUnexpiredTS) { + throw new UnsupportedOperationException(this.getClass().getName() + + " doesn't support checking to see if it should use a scanner!"); + } + + @Override + public boolean realSeekDone() { + return this.delegate.realSeekDone(); + } + + @Override + public void enforceSeek() throws IOException { + this.delegate.enforceSeek(); + } + + @Override + public void close() { + this.delegate.close(); + } + + /* + @Override + public boolean backwardSeek(KeyValue arg0) throws IOException { + return this.delegate.backwardSeek(arg0); + } + + @Override + public boolean seekToLastRow() throws IOException { + return this.delegate.seekToLastRow(); + } + + @Override + public boolean seekToPreviousRow(KeyValue arg0) throws IOException { + return this.delegate.seekToPreviousRow(arg0); + } + */ +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/Scanner.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/Scanner.java similarity index 96% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/Scanner.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/Scanner.java index b412b995..9df96cd0 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/Scanner.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/Scanner.java @@ -1,4 +1,4 @@ -package com.salesforce.hbase.index.scanner; +package org.apache.hadoop.hbase.index.scanner; import java.io.Closeable; import java.io.IOException; diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/ScannerBuilder.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/ScannerBuilder.java similarity index 91% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/ScannerBuilder.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/ScannerBuilder.java index 8d681dc3..bbecb9a3 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/scanner/ScannerBuilder.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/scanner/ScannerBuilder.java @@ -1,4 +1,4 @@ -package com.salesforce.hbase.index.scanner; +package org.apache.hadoop.hbase.index.scanner; import java.io.IOException; import java.util.Collection; @@ -16,12 +16,12 @@ import org.apache.hadoop.hbase.util.Bytes; import com.google.common.collect.Lists; -import com.salesforce.hbase.index.covered.KeyValueStore; -import com.salesforce.hbase.index.covered.filter.ApplyAndFilterDeletesFilter; -import com.salesforce.hbase.index.covered.filter.ColumnTrackingNextLargestTimestampFilter; -import com.salesforce.hbase.index.covered.update.ColumnReference; -import com.salesforce.hbase.index.covered.update.ColumnTracker; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.covered.KeyValueStore; +import org.apache.hadoop.hbase.index.covered.filter.ApplyAndFilterDeletesFilter; +import org.apache.hadoop.hbase.index.covered.filter.ColumnTrackingNextLargestTimestampFilter; +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; +import org.apache.hadoop.hbase.index.covered.update.ColumnTracker; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; /** * diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/CachingHTableFactory.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/CachingHTableFactory.java new file mode 100644 index 00000000..d95e67a9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/CachingHTableFactory.java @@ -0,0 +1,112 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.table; + +import java.io.IOException; +import java.util.Map; + +import org.apache.commons.collections.map.LRUMap; +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.client.HTableInterface; +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; + +/** + * A simple cache that just uses usual GC mechanisms to cleanup unused {@link HTableInterface}s. + * When requesting an {@link HTableInterface} via {@link #getTable}, you may get the same table as + * last time, or it may be a new table. + *

+ * You should not call {@link HTableInterface#close()} that is handled when the table goes + * out of scope. Along the same lines, you must ensure to not keep a reference to the table for + * longer than necessary - this leak will ensure that the table never gets closed. + */ +public class CachingHTableFactory implements HTableFactory { + + /** + * LRUMap that closes the {@link HTableInterface} when the table is evicted + */ + @SuppressWarnings("serial") + public class HTableInterfaceLRUMap extends LRUMap { + + public HTableInterfaceLRUMap(int cacheSize) { + super(cacheSize); + } + + @Override + protected boolean removeLRU(LinkEntry entry) { + HTableInterface table = (HTableInterface) entry.getValue(); + if (LOG.isDebugEnabled()) { + LOG.debug("Closing connection to table: " + Bytes.toString(table.getTableName()) + + " because it was evicted from the cache."); + } + try { + table.close(); + } catch (IOException e) { + LOG.info("Failed to correctly close HTable: " + Bytes.toString(table.getTableName()) + + " ignoring since being removed from queue."); + } + return true; + } + } + + public static int getCacheSize(Configuration conf) { + return conf.getInt(CACHE_SIZE_KEY, DEFAULT_CACHE_SIZE); + } + + private static final Log LOG = LogFactory.getLog(CachingHTableFactory.class); + private static final String CACHE_SIZE_KEY = "index.tablefactory.cache.size"; + private static final int DEFAULT_CACHE_SIZE = 10; + + private HTableFactory delegate; + + @SuppressWarnings("rawtypes") + Map openTables; + + public CachingHTableFactory(HTableFactory tableFactory, Configuration conf) { + this(tableFactory, getCacheSize(conf)); + } + + public CachingHTableFactory(HTableFactory factory, int cacheSize) { + this.delegate = factory; + openTables = new HTableInterfaceLRUMap(cacheSize); + } + + @Override + @SuppressWarnings("unchecked") + public HTableInterface getTable(ImmutableBytesPtr tablename) throws IOException { + ImmutableBytesPtr tableBytes = new ImmutableBytesPtr(tablename); + synchronized (openTables) { + HTableInterface table = (HTableInterface) openTables.get(tableBytes); + if (table == null) { + table = delegate.getTable(tablename); + openTables.put(tableBytes, table); + } + return table; + } + } + + @Override + public void shutdown() { + this.delegate.shutdown(); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/table/CoprocessorHTableFactory.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/CoprocessorHTableFactory.java similarity index 91% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/table/CoprocessorHTableFactory.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/CoprocessorHTableFactory.java index 757b1335..5ded8793 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/table/CoprocessorHTableFactory.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/CoprocessorHTableFactory.java @@ -1,4 +1,4 @@ -package com.salesforce.hbase.index.table; +package org.apache.hadoop.hbase.index.table; import java.io.IOException; @@ -10,8 +10,8 @@ import org.apache.hadoop.hbase.client.HTableInterface; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.hbase.index.util.IndexManagementUtil; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.util.IndexManagementUtil; public class CoprocessorHTableFactory implements HTableFactory { diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/HTableFactory.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/HTableFactory.java new file mode 100644 index 00000000..a27a2380 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/HTableFactory.java @@ -0,0 +1,14 @@ +package org.apache.hadoop.hbase.index.table; + +import java.io.IOException; + +import org.apache.hadoop.hbase.client.HTableInterface; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; + +public interface HTableFactory { + + public HTableInterface getTable(ImmutableBytesPtr tablename) throws IOException; + + public void shutdown(); +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/table/HTableInterfaceReference.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/HTableInterfaceReference.java similarity index 90% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/table/HTableInterfaceReference.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/HTableInterfaceReference.java index b3a5f7be..134006f0 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/table/HTableInterfaceReference.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/table/HTableInterfaceReference.java @@ -1,8 +1,8 @@ -package com.salesforce.hbase.index.table; +package org.apache.hadoop.hbase.index.table; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; /** * Reference to an HTableInterface. Currently, its pretty simple in that it is just a wrapper around diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/util/ImmutableBytesPtr.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/util/ImmutableBytesPtr.java new file mode 100644 index 00000000..4ae6afbd --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/util/ImmutableBytesPtr.java @@ -0,0 +1,113 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.util; + +import java.io.DataInput; +import java.io.IOException; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; + +public class ImmutableBytesPtr extends ImmutableBytesWritable { + private int hashCode; + + public ImmutableBytesPtr() { + } + + public ImmutableBytesPtr(byte[] bytes) { + super(bytes); + hashCode = super.hashCode(); + } + + public ImmutableBytesPtr(ImmutableBytesWritable ibw) { + super(ibw.get(), ibw.getOffset(), ibw.getLength()); + hashCode = super.hashCode(); + } + + public ImmutableBytesPtr(ImmutableBytesPtr ibp) { + super(ibp.get(), ibp.getOffset(), ibp.getLength()); + hashCode = ibp.hashCode; + } + + public ImmutableBytesPtr(byte[] bytes, int offset, int length) { + super(bytes, offset, length); + hashCode = super.hashCode(); + } + + @Override + public int hashCode() { + return hashCode; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + ImmutableBytesPtr that = (ImmutableBytesPtr)obj; + if (this.hashCode != that.hashCode) return false; + if (Bytes.compareTo(this.get(), this.getOffset(), this.getLength(), that.get(), that.getOffset(), that.getLength()) != 0) return false; + return true; + } + + public void set(ImmutableBytesWritable ptr) { + set(ptr.get(),ptr.getOffset(),ptr.getLength()); + } + + /** + * @param b Use passed bytes as backing array for this instance. + */ + @Override + public void set(final byte [] b) { + super.set(b); + hashCode = super.hashCode(); + } + + /** + * @param b Use passed bytes as backing array for this instance. + * @param offset + * @param length + */ + @Override + public void set(final byte [] b, final int offset, final int length) { + super.set(b,offset,length); + hashCode = super.hashCode(); + } + + @Override + public void readFields(final DataInput in) throws IOException { + super.readFields(in); + hashCode = super.hashCode(); + } + + /** + * @return the backing byte array, copying only if necessary + */ + public byte[] copyBytesIfNecessary() { + return copyBytesIfNecessary(this); + } + + public static byte[] copyBytesIfNecessary(ImmutableBytesWritable ptr) { + if (ptr.getOffset() == 0 && ptr.getLength() == ptr.get().length) { + return ptr.get(); + } + return ptr.copyBytes(); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/util/IndexManagementUtil.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/util/IndexManagementUtil.java similarity index 82% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/util/IndexManagementUtil.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/util/IndexManagementUtil.java index f472741e..59e07cc7 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/util/IndexManagementUtil.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/util/IndexManagementUtil.java @@ -1,19 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.util; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.util; import java.io.IOException; import java.util.Collection; @@ -28,11 +32,11 @@ import org.apache.hadoop.hbase.client.Scan; import com.google.common.collect.Maps; -import com.salesforce.hbase.index.ValueGetter; -import com.salesforce.hbase.index.builder.IndexBuildingFailureException; -import com.salesforce.hbase.index.covered.data.LazyValueGetter; -import com.salesforce.hbase.index.covered.update.ColumnReference; -import com.salesforce.hbase.index.scanner.Scanner; +import org.apache.hadoop.hbase.index.ValueGetter; +import org.apache.hadoop.hbase.index.builder.IndexBuildingFailureException; +import org.apache.hadoop.hbase.index.covered.data.LazyValueGetter; +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; +import org.apache.hadoop.hbase.index.scanner.Scanner; /** * Utility class to help manage indexes diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/wal/IndexedKeyValue.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/wal/IndexedKeyValue.java similarity index 98% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/wal/IndexedKeyValue.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/wal/IndexedKeyValue.java index 09babf67..a7f4e825 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/wal/IndexedKeyValue.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/wal/IndexedKeyValue.java @@ -1,4 +1,4 @@ -package com.salesforce.hbase.index.wal; +package org.apache.hadoop.hbase.index.wal; import java.io.ByteArrayOutputStream; import java.io.DataInput; @@ -12,7 +12,7 @@ import org.apache.hadoop.hbase.regionserver.wal.HLog; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; public class IndexedKeyValue extends KeyValue { private static int calcHashCode(ImmutableBytesPtr indexTableName, Mutation mutation) { diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/wal/KeyValueCodec.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/wal/KeyValueCodec.java similarity index 98% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/wal/KeyValueCodec.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/wal/KeyValueCodec.java index d6d75bf4..0abdf8de 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/wal/KeyValueCodec.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/wal/KeyValueCodec.java @@ -1,4 +1,4 @@ -package com.salesforce.hbase.index.wal; +package org.apache.hadoop.hbase.index.wal; import java.io.DataInput; import java.io.DataOutput; diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexCommitter.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexCommitter.java new file mode 100644 index 00000000..aed4b896 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexCommitter.java @@ -0,0 +1,39 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.write; + +import org.apache.hadoop.hbase.Stoppable; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; + +import com.google.common.collect.Multimap; +import org.apache.hadoop.hbase.index.exception.IndexWriteException; +import org.apache.hadoop.hbase.index.table.HTableInterfaceReference; + +/** + * Write the index updates to the index tables + */ +public interface IndexCommitter extends Stoppable { + + void setup(IndexWriter parent, RegionCoprocessorEnvironment env, String name); + + public void write(Multimap toWrite) + throws IndexWriteException; +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexFailurePolicy.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexFailurePolicy.java new file mode 100644 index 00000000..683efd5f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexFailurePolicy.java @@ -0,0 +1,46 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.write; + +import java.io.IOException; + +import org.apache.hadoop.hbase.Stoppable; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; + +import com.google.common.collect.Multimap; +import org.apache.hadoop.hbase.index.table.HTableInterfaceReference; + +/** + * Handle failures to write to the index tables. + */ +public interface IndexFailurePolicy extends Stoppable { + + public void setup(Stoppable parent, RegionCoprocessorEnvironment env); + + /** + * Handle the failure of the attempted index updates + * @param attempted map of index table -> mutations to apply + * @param cause reason why there was a failure + * @throws IOException + */ + public void + handleFailure(Multimap attempted, Exception cause) throws IOException; +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexWriter.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexWriter.java similarity index 79% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexWriter.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexWriter.java index f3403937..8c30642e 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/IndexWriter.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexWriter.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.write; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.write; import java.io.IOException; import java.util.Collection; @@ -43,9 +35,9 @@ import com.google.common.collect.ArrayListMultimap; import com.google.common.collect.Multimap; -import com.salesforce.hbase.index.exception.IndexWriteException; -import com.salesforce.hbase.index.table.HTableInterfaceReference; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.exception.IndexWriteException; +import org.apache.hadoop.hbase.index.table.HTableInterfaceReference; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; /** * Do the actual work of writing to the index tables. Ensures that if we do fail to write to the diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexWriterUtils.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexWriterUtils.java new file mode 100644 index 00000000..bee89539 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/IndexWriterUtils.java @@ -0,0 +1,74 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.write; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.CoprocessorEnvironment; + +import org.apache.hadoop.hbase.index.table.CoprocessorHTableFactory; +import org.apache.hadoop.hbase.index.table.HTableFactory; +import org.apache.hadoop.hbase.index.util.IndexManagementUtil; + +public class IndexWriterUtils { + + private static final Log LOG = LogFactory.getLog(IndexWriterUtils.class); + + /** + * Maximum number of threads to allow per-table when writing. Each writer thread (from + * {@link IndexWriterUtils#NUM_CONCURRENT_INDEX_WRITER_THREADS_CONF_KEY}) has a single HTable. + * However, each table is backed by a threadpool to manage the updates to that table. this + * specifies the number of threads to allow in each of those tables. Generally, you shouldn't need + * to change this, unless you have a small number of indexes to which most of the writes go. + * Defaults to: {@value #DEFAULT_NUM_PER_TABLE_THREADS}. + *

+ * For tables to which there are not a lot of writes, the thread pool automatically will decrease + * the number of threads to one (though it can burst up to the specified max for any given table), + * so increasing this to meet the max case is reasonable. + *

+ * Setting this value too small can cause catastrophic cluster failure. The way HTable's + * underlying pool works is such that is does direct hand-off of tasks to threads. This works fine + * because HTables are assumed to work in a single-threaded context, so we never get more threads + * than regionservers. In a multi-threaded context, we can easily grow to more than that number of + * threads. Currently, HBase doesn't support a custom thread-pool to back the HTable via the + * coprocesor hooks, so we can't modify this behavior. + */ + private static final String INDEX_WRITER_PER_TABLE_THREADS_CONF_KEY = + "index.writer.threads.pertable.max"; + private static final int DEFAULT_NUM_PER_TABLE_THREADS = Integer.MAX_VALUE; + + /** Configuration key that HBase uses to set the max number of threads for an HTable */ + public static final String HTABLE_THREAD_KEY = "hbase.htable.threads.max"; + private IndexWriterUtils() { + // private ctor for utilites + } + + public static HTableFactory getDefaultDelegateHTableFactory(CoprocessorEnvironment env) { + // create a simple delegate factory, setup the way we need + Configuration conf = env.getConfiguration(); + // set the number of threads allowed per table. + int htableThreads = + conf.getInt(IndexWriterUtils.INDEX_WRITER_PER_TABLE_THREADS_CONF_KEY, IndexWriterUtils.DEFAULT_NUM_PER_TABLE_THREADS); + LOG.trace("Creating HTableFactory with " + htableThreads + " threads for each HTable."); + IndexManagementUtil.setIfNotSet(conf, HTABLE_THREAD_KEY, htableThreads); + return new CoprocessorHTableFactory(env); + } +} diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/KillServerOnFailurePolicy.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/KillServerOnFailurePolicy.java new file mode 100644 index 00000000..76bde1ab --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/KillServerOnFailurePolicy.java @@ -0,0 +1,82 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.write; + +import java.io.IOException; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.hbase.Abortable; +import org.apache.hadoop.hbase.Stoppable; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; + +import com.google.common.collect.Multimap; +import org.apache.hadoop.hbase.index.table.HTableInterfaceReference; + +/** + * Naive failure policy - kills the server on which it resides + */ +public class KillServerOnFailurePolicy implements IndexFailurePolicy { + + private static final Log LOG = LogFactory.getLog(KillServerOnFailurePolicy.class); + private Abortable abortable; + private Stoppable stoppable; + + @Override + public void setup(Stoppable parent, RegionCoprocessorEnvironment env) { + setup(parent, env.getRegionServerServices()); + } + + public void setup(Stoppable parent, Abortable abort) { + this.stoppable = parent; + this.abortable = abort; + } + + @Override + public void stop(String why) { + // noop + } + + @Override + public boolean isStopped() { + return this.stoppable.isStopped(); + } + + @Override + public void + handleFailure(Multimap attempted, Exception cause) throws IOException { + // cleanup resources + this.stop("Killing ourselves because of an error:" + cause); + // notify the regionserver of the failure + String msg = + "Could not update the index table, killing server region because couldn't write to an index table"; + LOG.error(msg, cause); + try { + this.abortable.abort(msg, cause); + } catch (Exception e) { + LOG.fatal("Couldn't abort this server to preserve index writes, " + + "attempting to hard kill the server"); + System.exit(1); + } + + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/ParallelWriterIndexCommitter.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/ParallelWriterIndexCommitter.java similarity index 76% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/write/ParallelWriterIndexCommitter.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/ParallelWriterIndexCommitter.java index 4adbcb12..86abde48 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/ParallelWriterIndexCommitter.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/ParallelWriterIndexCommitter.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.hbase.index.write; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.write; import java.io.IOException; import java.util.Collection; @@ -45,16 +37,16 @@ import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; import com.google.common.collect.Multimap; -import com.salesforce.hbase.index.exception.SingleIndexWriteFailureException; -import com.salesforce.hbase.index.parallel.EarlyExitFailure; -import com.salesforce.hbase.index.parallel.QuickFailingTaskRunner; -import com.salesforce.hbase.index.parallel.Task; -import com.salesforce.hbase.index.parallel.TaskBatch; -import com.salesforce.hbase.index.parallel.ThreadPoolBuilder; -import com.salesforce.hbase.index.parallel.ThreadPoolManager; -import com.salesforce.hbase.index.table.CachingHTableFactory; -import com.salesforce.hbase.index.table.HTableFactory; -import com.salesforce.hbase.index.table.HTableInterfaceReference; +import org.apache.hadoop.hbase.index.exception.SingleIndexWriteFailureException; +import org.apache.hadoop.hbase.index.parallel.EarlyExitFailure; +import org.apache.hadoop.hbase.index.parallel.QuickFailingTaskRunner; +import org.apache.hadoop.hbase.index.parallel.Task; +import org.apache.hadoop.hbase.index.parallel.TaskBatch; +import org.apache.hadoop.hbase.index.parallel.ThreadPoolBuilder; +import org.apache.hadoop.hbase.index.parallel.ThreadPoolManager; +import org.apache.hadoop.hbase.index.table.CachingHTableFactory; +import org.apache.hadoop.hbase.index.table.HTableFactory; +import org.apache.hadoop.hbase.index.table.HTableInterfaceReference; /** * Write index updates to the index tables in parallel. We attempt to early exit from the writes if diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/recovery/PerRegionIndexWriteCache.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/recovery/PerRegionIndexWriteCache.java new file mode 100644 index 00000000..3d77a837 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/recovery/PerRegionIndexWriteCache.java @@ -0,0 +1,65 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.write.recovery; + +import java.util.Collection; +import java.util.HashMap; +import java.util.Map; + +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.regionserver.HRegion; + +import com.google.common.collect.ArrayListMultimap; +import com.google.common.collect.Multimap; +import org.apache.hadoop.hbase.index.table.HTableInterfaceReference; + + +public class PerRegionIndexWriteCache { + + private Map> cache = + new HashMap>(); + + + /** + * Get the edits for the current region. Removes the edits from the cache. To add them back, call + * {@link #addEdits(HRegion, HTableInterfaceReference, Collection)}. + * @param region + * @return Get the edits for the given region. Returns null if there are no pending edits + * for the region + */ + public Multimap getEdits(HRegion region) { + return cache.remove(region); + } + + /** + * @param region + * @param table + * @param collection + */ + public void addEdits(HRegion region, HTableInterfaceReference table, + Collection collection) { + Multimap edits = cache.get(region); + if (edits == null) { + edits = ArrayListMultimap. create(); + cache.put(region, edits); + } + edits.putAll(table, collection); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/recovery/StoreFailuresInCachePolicy.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/recovery/StoreFailuresInCachePolicy.java similarity index 89% rename from phoenix-core/src/main/java/com/salesforce/hbase/index/write/recovery/StoreFailuresInCachePolicy.java rename to phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/recovery/StoreFailuresInCachePolicy.java index a792c10f..a17395e0 100644 --- a/phoenix-core/src/main/java/com/salesforce/hbase/index/write/recovery/StoreFailuresInCachePolicy.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/recovery/StoreFailuresInCachePolicy.java @@ -15,7 +15,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package com.salesforce.hbase.index.write.recovery; +package org.apache.hadoop.hbase.index.write.recovery; import java.io.IOException; import java.util.List; @@ -26,10 +26,10 @@ import org.apache.hadoop.hbase.regionserver.HRegion; import com.google.common.collect.Multimap; -import com.salesforce.hbase.index.exception.MultiIndexWriteFailureException; -import com.salesforce.hbase.index.table.HTableInterfaceReference; -import com.salesforce.hbase.index.write.IndexFailurePolicy; -import com.salesforce.hbase.index.write.KillServerOnFailurePolicy; +import org.apache.hadoop.hbase.index.exception.MultiIndexWriteFailureException; +import org.apache.hadoop.hbase.index.table.HTableInterfaceReference; +import org.apache.hadoop.hbase.index.write.IndexFailurePolicy; +import org.apache.hadoop.hbase.index.write.KillServerOnFailurePolicy; /** * Tracks any failed writes in The {@link PerRegionIndexWriteCache}, given a diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/recovery/TrackingParallelWriterIndexCommitter.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/recovery/TrackingParallelWriterIndexCommitter.java new file mode 100644 index 00000000..46a771b5 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/index/write/recovery/TrackingParallelWriterIndexCommitter.java @@ -0,0 +1,228 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hbase.index.write.recovery; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.List; +import java.util.Map.Entry; +import java.util.Set; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.ExecutorService; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.Abortable; +import org.apache.hadoop.hbase.Stoppable; +import org.apache.hadoop.hbase.client.HTableInterface; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; + +import com.google.common.collect.Multimap; +import org.apache.hadoop.hbase.index.CapturingAbortable; +import org.apache.hadoop.hbase.index.exception.MultiIndexWriteFailureException; +import org.apache.hadoop.hbase.index.exception.SingleIndexWriteFailureException; +import org.apache.hadoop.hbase.index.parallel.EarlyExitFailure; +import org.apache.hadoop.hbase.index.parallel.Task; +import org.apache.hadoop.hbase.index.parallel.TaskBatch; +import org.apache.hadoop.hbase.index.parallel.TaskRunner; +import org.apache.hadoop.hbase.index.parallel.ThreadPoolBuilder; +import org.apache.hadoop.hbase.index.parallel.ThreadPoolManager; +import org.apache.hadoop.hbase.index.parallel.WaitForCompletionTaskRunner; +import org.apache.hadoop.hbase.index.table.CachingHTableFactory; +import org.apache.hadoop.hbase.index.table.HTableFactory; +import org.apache.hadoop.hbase.index.table.HTableInterfaceReference; +import org.apache.hadoop.hbase.index.write.IndexCommitter; +import org.apache.hadoop.hbase.index.write.IndexWriter; +import org.apache.hadoop.hbase.index.write.IndexWriterUtils; +import org.apache.hadoop.hbase.index.write.ParallelWriterIndexCommitter; + +/** + * Like the {@link ParallelWriterIndexCommitter}, but blocks until all writes have attempted to + * allow the caller to retrieve the failed and succeeded index updates. Therefore, this class will + * be a lot slower, in the face of failures, when compared to the + * {@link ParallelWriterIndexCommitter} (though as fast for writes), so it should be used only when + * you need to at least attempt all writes and know their result; for instance, this is fine for + * doing WAL recovery - it's not a performance intensive situation and we want to limit the the + * edits we need to retry. + *

+ * On failure to {@link #write(Multimap)}, we return a {@link MultiIndexWriteFailureException} that + * contains the list of {@link HTableInterfaceReference} that didn't complete successfully. + *

+ * Failures to write to the index can happen several different ways: + *

    + *
  1. this is {@link #stop(String) stopped} or aborted (via the passed {@link Abortable}. + * This causing any pending tasks to fail whatever they are doing as fast as possible. Any writes + * that have not begun are not even attempted and marked as failures.
    2. + *
  2. A batch write fails. This is the generic HBase write failure - it may occur because the index + * table is not available, .META. or -ROOT- is unavailable, or any other (of many) possible HBase + * exceptions.
    3. + *
+ * Regardless of how the write fails, we still wait for all writes to complete before passing the + * failure back to the client. + */ +public class TrackingParallelWriterIndexCommitter implements IndexCommitter { + private static final Log LOG = LogFactory.getLog(TrackingParallelWriterIndexCommitter.class); + + public static final String NUM_CONCURRENT_INDEX_WRITER_THREADS_CONF_KEY = "index.trackingwriter.threads.max"; + private static final int DEFAULT_CONCURRENT_INDEX_WRITER_THREADS = 10; + private static final String INDEX_WRITER_KEEP_ALIVE_TIME_CONF_KEY = + "index.trackingwriter.threads.keepalivetime"; + + private TaskRunner pool; + private HTableFactory factory; + private CapturingAbortable abortable; + private Stoppable stopped; + + @Override + public void setup(IndexWriter parent, RegionCoprocessorEnvironment env, String name) { + Configuration conf = env.getConfiguration(); + setup(IndexWriterUtils.getDefaultDelegateHTableFactory(env), + ThreadPoolManager.getExecutor( + new ThreadPoolBuilder(name, conf). + setMaxThread(NUM_CONCURRENT_INDEX_WRITER_THREADS_CONF_KEY, + DEFAULT_CONCURRENT_INDEX_WRITER_THREADS). + setCoreTimeout(INDEX_WRITER_KEEP_ALIVE_TIME_CONF_KEY), env), + env.getRegionServerServices(), parent, CachingHTableFactory.getCacheSize(conf)); + } + + /** + * Setup this. + *

+ * Exposed for TESTING + */ + void setup(HTableFactory factory, ExecutorService pool, Abortable abortable, Stoppable stop, + int cacheSize) { + this.pool = new WaitForCompletionTaskRunner(pool); + this.factory = new CachingHTableFactory(factory, cacheSize); + this.abortable = new CapturingAbortable(abortable); + this.stopped = stop; + } + + @Override + public void write(Multimap toWrite) + throws MultiIndexWriteFailureException { + Set>> entries = toWrite.asMap().entrySet(); + TaskBatch tasks = new TaskBatch(entries.size()); + List tables = new ArrayList(entries.size()); + for (Entry> entry : entries) { + // get the mutations for each table. We leak the implementation here a little bit to save + // doing a complete copy over of all the index update for each table. + final List mutations = (List) entry.getValue(); + // track each reference so we can get at it easily later, when determing failures + final HTableInterfaceReference tableReference = entry.getKey(); + tables.add(tableReference); + + /* + * Write a batch of index updates to an index table. This operation stops (is cancelable) via + * two mechanisms: (1) setting aborted or stopped on the IndexWriter or, (2) interrupting the + * running thread. The former will only work if we are not in the midst of writing the current + * batch to the table, though we do check these status variables before starting and before + * writing the batch. The latter usage, interrupting the thread, will work in the previous + * situations as was at some points while writing the batch, depending on the underlying + * writer implementation (HTableInterface#batch is blocking, but doesn't elaborate when is + * supports an interrupt). + */ + tasks.add(new Task() { + + /** + * Do the actual write to the primary table. We don't need to worry about closing the table + * because that is handled the {@link CachingHTableFactory}. + */ + @Override + public Boolean call() throws Exception { + try { + // this may have been queued, but there was an abort/stop so we try to early exit + throwFailureIfDone(); + + if (LOG.isDebugEnabled()) { + LOG.debug("Writing index update:" + mutations + " to table: " + tableReference); + } + HTableInterface table = factory.getTable(tableReference.get()); + throwFailureIfDone(); + table.batch(mutations); + } catch (InterruptedException e) { + // reset the interrupt status on the thread + Thread.currentThread().interrupt(); + throw e; + } catch (Exception e) { + throw e; + } + return Boolean.TRUE; + } + + private void throwFailureIfDone() throws SingleIndexWriteFailureException { + if (stopped.isStopped() || abortable.isAborted() + || Thread.currentThread().isInterrupted()) { + throw new SingleIndexWriteFailureException( + "Pool closed, not attempting to write to the index!", null); + } + + } + }); + } + + List results = null; + try { + LOG.debug("Waiting on index update tasks to complete..."); + results = this.pool.submitUninterruptible(tasks); + } catch (ExecutionException e) { + throw new RuntimeException( + "Should not fail on the results while using a WaitForCompletionTaskRunner", e); + } catch (EarlyExitFailure e) { + throw new RuntimeException("Stopped while waiting for batch, quiting!", e); + } + + // track the failures. We only ever access this on return from our calls, so no extra + // synchronization is needed. We could update all the failures as we find them, but that add a + // lot of locking overhead, and just doing the copy later is about as efficient. + List failures = new ArrayList(); + int index = 0; + for (Boolean result : results) { + // there was a failure + if (result == null) { + // we know which table failed by the index of the result + failures.add(tables.get(index)); + } + index++; + } + + // if any of the tasks failed, then we need to propagate the failure + if (failures.size() > 0) { + // make the list unmodifiable to avoid any more synchronization concerns + throw new MultiIndexWriteFailureException(Collections.unmodifiableList(failures)); + } + return; + } + + @Override + public void stop(String why) { + LOG.info("Shutting down " + this.getClass().getSimpleName()); + this.pool.stop(why); + this.factory.shutdown(); + } + + @Override + public boolean isStopped() { + return this.stopped.isStopped(); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/IndexKeyValueSkipListSet.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/IndexKeyValueSkipListSet.java index 6ad31043..8314fef6 100644 --- a/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/IndexKeyValueSkipListSet.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/IndexKeyValueSkipListSet.java @@ -1,30 +1,22 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ package org.apache.hadoop.hbase.regionserver; import java.util.Comparator; diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/wal/IndexedWALEdit.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/wal/IndexedWALEdit.java index e7e2cb53..6749cc93 100644 --- a/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/wal/IndexedWALEdit.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/wal/IndexedWALEdit.java @@ -10,7 +10,7 @@ import org.apache.hadoop.hbase.KeyValue; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.hbase.index.wal.KeyValueCodec; +import org.apache.hadoop.hbase.index.wal.KeyValueCodec; /** * Read in data for a delegate {@link WALEdit}. This should only be used in concert with an IndexedHLogReader diff --git a/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/wal/IndexedWALEditCodec.java b/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/wal/IndexedWALEditCodec.java index cba13724..01d73904 100644 --- a/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/wal/IndexedWALEditCodec.java +++ b/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/wal/IndexedWALEditCodec.java @@ -13,8 +13,8 @@ import org.apache.hadoop.hbase.codec.Decoder; import org.apache.hadoop.hbase.codec.Encoder; -import com.salesforce.hbase.index.wal.IndexedKeyValue; -import com.salesforce.hbase.index.wal.KeyValueCodec; +import org.apache.hadoop.hbase.index.wal.IndexedKeyValue; +import org.apache.hadoop.hbase.index.wal.KeyValueCodec; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/cache/GlobalCache.java b/phoenix-core/src/main/java/org/apache/phoenix/cache/GlobalCache.java new file mode 100644 index 00000000..77684c5c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/cache/GlobalCache.java @@ -0,0 +1,114 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.cache; + +import static org.apache.phoenix.query.QueryServices.MAX_MEMORY_PERC_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MAX_MEMORY_WAIT_MS_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MAX_TENANT_MEMORY_PERC_ATTRIB; + +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.ConcurrentMap; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.memory.ChildMemoryManager; +import org.apache.phoenix.memory.GlobalMemoryManager; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.PTable; + + +/** + * + * Global root cache for the server. Each tenant is managed as a child tenant cache of this one. Queries + * not associated with a particular tenant use this as their tenant cache. + * + * @author jtaylor + * @since 0.1 + */ +public class GlobalCache extends TenantCacheImpl { + private static GlobalCache INSTANCE; + + private final Configuration config; + // TODO: Use Guava cache with auto removal after lack of access + private final ConcurrentMap perTenantCacheMap = new ConcurrentHashMap(); + // Cache for lastest PTable for a given Phoenix table + private final ConcurrentHashMap metaDataCacheMap = new ConcurrentHashMap(); + + public static synchronized GlobalCache getInstance(RegionCoprocessorEnvironment env) { + // See http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html + // for explanation of why double locking doesn't work. + if (INSTANCE == null) { + INSTANCE = new GlobalCache(env.getConfiguration()); + } + return INSTANCE; + } + + public ConcurrentHashMap getMetaDataCache() { + return metaDataCacheMap; + } + + /** + * Get the tenant cache associated with the tenantId. If tenantId is not applicable, null may be + * used in which case a global tenant cache is returned. + * @param env the HBase configuration + * @param tenantId the tenant ID or null if not applicable. + * @return TenantCache + */ + public static TenantCache getTenantCache(RegionCoprocessorEnvironment env, ImmutableBytesWritable tenantId) { + GlobalCache globalCache = GlobalCache.getInstance(env); + TenantCache tenantCache = tenantId == null ? globalCache : globalCache.getChildTenantCache(tenantId); + return tenantCache; + } + + private GlobalCache(Configuration config) { + super(new GlobalMemoryManager(Runtime.getRuntime().totalMemory() * + config.getInt(MAX_MEMORY_PERC_ATTRIB, QueryServicesOptions.DEFAULT_MAX_MEMORY_PERC) / 100, + config.getInt(MAX_MEMORY_WAIT_MS_ATTRIB, QueryServicesOptions.DEFAULT_MAX_MEMORY_WAIT_MS)), + config.getInt(QueryServices.MAX_SERVER_CACHE_TIME_TO_LIVE_MS, QueryServicesOptions.DEFAULT_MAX_SERVER_CACHE_TIME_TO_LIVE_MS)); + this.config = config; + } + + public Configuration getConfig() { + return config; + } + + /** + * Retrieve the tenant cache given an tenantId. + * @param tenantId the ID that identifies the tenant + * @return the existing or newly created TenantCache + */ + public TenantCache getChildTenantCache(ImmutableBytesWritable tenantId) { + TenantCache tenantCache = perTenantCacheMap.get(tenantId); + if (tenantCache == null) { + int maxTenantMemoryPerc = config.getInt(MAX_TENANT_MEMORY_PERC_ATTRIB, QueryServicesOptions.DEFAULT_MAX_TENANT_MEMORY_PERC); + int maxServerCacheTimeToLive = config.getInt(QueryServices.MAX_SERVER_CACHE_TIME_TO_LIVE_MS, QueryServicesOptions.DEFAULT_MAX_SERVER_CACHE_TIME_TO_LIVE_MS); + TenantCacheImpl newTenantCache = new TenantCacheImpl(new ChildMemoryManager(getMemoryManager(), maxTenantMemoryPerc), maxServerCacheTimeToLive); + tenantCache = perTenantCacheMap.putIfAbsent(tenantId, newTenantCache); + if (tenantCache == null) { + tenantCache = newTenantCache; + } + } + return tenantCache; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/cache/HashCache.java b/phoenix-core/src/main/java/org/apache/phoenix/cache/HashCache.java new file mode 100644 index 00000000..4260a50b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/cache/HashCache.java @@ -0,0 +1,40 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.cache; + +import java.io.Closeable; +import java.util.List; + +import org.apache.http.annotation.Immutable; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * Encapsulate deserialized hash cache from bytes into Map. + * The Map uses the row key as the key and the row as the value. + * @author jtaylor + * @since 0.1 + */ +@Immutable +public interface HashCache extends Closeable { + public List get(ImmutableBytesPtr hashKey); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/cache/IndexMetaDataCache.java b/phoenix-core/src/main/java/org/apache/phoenix/cache/IndexMetaDataCache.java new file mode 100644 index 00000000..fac78de9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/cache/IndexMetaDataCache.java @@ -0,0 +1,10 @@ +package org.apache.phoenix.cache; + +import java.io.Closeable; +import java.util.List; + +import org.apache.phoenix.index.IndexMaintainer; + +public interface IndexMetaDataCache extends Closeable { + public List getIndexMaintainers(); +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/ServerCacheClient.java b/phoenix-core/src/main/java/org/apache/phoenix/cache/ServerCacheClient.java similarity index 81% rename from phoenix-core/src/main/java/com/salesforce/phoenix/cache/ServerCacheClient.java rename to phoenix-core/src/main/java/org/apache/phoenix/cache/ServerCacheClient.java index 2d9210f9..8703c7c5 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/ServerCacheClient.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/cache/ServerCacheClient.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.cache; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.cache; import java.io.Closeable; import java.io.IOException; @@ -52,19 +44,19 @@ import org.apache.hadoop.hbase.util.Bytes; import com.google.common.collect.ImmutableSet; -import com.salesforce.phoenix.compile.ScanRanges; -import com.salesforce.phoenix.coprocessor.ServerCachingProtocol; -import com.salesforce.phoenix.coprocessor.ServerCachingProtocol.ServerCacheFactory; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.job.JobManager.JobCallable; -import com.salesforce.phoenix.memory.MemoryManager.MemoryChunk; -import com.salesforce.phoenix.query.ConnectionQueryServices; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.Closeables; -import com.salesforce.phoenix.util.SQLCloseable; -import com.salesforce.phoenix.util.SQLCloseables; +import org.apache.phoenix.compile.ScanRanges; +import org.apache.phoenix.coprocessor.ServerCachingProtocol; +import org.apache.phoenix.coprocessor.ServerCachingProtocol.ServerCacheFactory; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.job.JobManager.JobCallable; +import org.apache.phoenix.memory.MemoryManager.MemoryChunk; +import org.apache.phoenix.query.ConnectionQueryServices; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.Closeables; +import org.apache.phoenix.util.SQLCloseable; +import org.apache.phoenix.util.SQLCloseables; /** * diff --git a/phoenix-core/src/main/java/org/apache/phoenix/cache/TenantCache.java b/phoenix-core/src/main/java/org/apache/phoenix/cache/TenantCache.java new file mode 100644 index 00000000..ad40d8c7 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/cache/TenantCache.java @@ -0,0 +1,44 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.cache; + +import java.io.Closeable; +import java.sql.SQLException; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.coprocessor.ServerCachingProtocol.ServerCacheFactory; +import org.apache.phoenix.memory.MemoryManager; + + +/** + * + * Inteface to set and set cached values for a tenant + * + * @author jtaylor + * @since 0.1 + */ +public interface TenantCache { + MemoryManager getMemoryManager(); + Closeable getServerCache(ImmutableBytesPtr cacheId); + Closeable addServerCache(ImmutableBytesPtr cacheId, ImmutableBytesWritable cachePtr, ServerCacheFactory cacheFactory) throws SQLException; + void removeServerCache(ImmutableBytesPtr cacheId) throws SQLException; +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/cache/TenantCacheImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/cache/TenantCacheImpl.java new file mode 100644 index 00000000..eabeb111 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/cache/TenantCacheImpl.java @@ -0,0 +1,96 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.cache; + +import java.io.Closeable; +import java.sql.SQLException; +import java.util.Collections; +import java.util.concurrent.TimeUnit; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import com.google.common.cache.*; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.coprocessor.ServerCachingProtocol.ServerCacheFactory; +import org.apache.phoenix.memory.MemoryManager; +import org.apache.phoenix.memory.MemoryManager.MemoryChunk; +import org.apache.phoenix.util.Closeables; + +/** + * + * Cache per tenant on server side. Tracks memory usage for each + * tenat as well and rolling up usage to global memory manager. + * + * @author jtaylor + * @since 0.1 + */ +public class TenantCacheImpl implements TenantCache { + private final int maxTimeToLiveMs; + private final MemoryManager memoryManager; + private volatile Cache serverCaches; + + public TenantCacheImpl(MemoryManager memoryManager, int maxTimeToLiveMs) { + this.memoryManager = memoryManager; + this.maxTimeToLiveMs = maxTimeToLiveMs; + } + + @Override + public MemoryManager getMemoryManager() { + return memoryManager; + } + + private Cache getServerCaches() { + /* Delay creation of this map until it's needed */ + if (serverCaches == null) { + synchronized(this) { + if (serverCaches == null) { + serverCaches = CacheBuilder.newBuilder() + .expireAfterAccess(maxTimeToLiveMs, TimeUnit.MILLISECONDS) + .removalListener(new RemovalListener(){ + @Override + public void onRemoval(RemovalNotification notification) { + Closeables.closeAllQuietly(Collections.singletonList(notification.getValue())); + } + }) + .build(); + } + } + } + return serverCaches; + } + + @Override + public Closeable getServerCache(ImmutableBytesPtr cacheId) { + return getServerCaches().getIfPresent(cacheId); + } + + @Override + public Closeable addServerCache(ImmutableBytesPtr cacheId, ImmutableBytesWritable cachePtr, ServerCacheFactory cacheFactory) throws SQLException { + MemoryChunk chunk = this.getMemoryManager().allocate(cachePtr.getLength()); + Closeable element = cacheFactory.newCache(cachePtr, chunk); + getServerCaches().put(cacheId, element); + return element; + } + + @Override + public void removeServerCache(ImmutableBytesPtr cacheId) throws SQLException { + getServerCaches().invalidate(cacheId); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillFile.java b/phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillFile.java similarity index 99% rename from phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillFile.java rename to phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillFile.java index fa3815bb..f2ad9084 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillFile.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillFile.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.cache.aggcache; +package org.apache.phoenix.cache.aggcache; import java.io.Closeable; import java.io.File; diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillManager.java b/phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillManager.java similarity index 80% rename from phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillManager.java rename to phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillManager.java index b2cc17ae..448ffa57 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillManager.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillManager.java @@ -1,22 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source - * and binary forms, with or without modification, are permitted provided that the following - * conditions are met: Redistributions of source code must retain the above copyright notice, this - * list of conditions and the following disclaimer. Redistributions in binary form must reproduce - * the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of - * Salesforce.com nor the names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED - * BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, - * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, - * OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF - * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.cache.aggcache; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.cache.aggcache; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; @@ -36,17 +37,17 @@ import com.google.common.base.Preconditions; import com.google.common.collect.Lists; import com.google.common.io.Closeables; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.ServerAggregators; -import com.salesforce.phoenix.expression.function.SingleAggregateFunction; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.KeyValueSchema; -import com.salesforce.phoenix.schema.ValueBitSet; -import com.salesforce.phoenix.schema.tuple.SingleKeyValueTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.KeyValueUtil; -import com.salesforce.phoenix.util.TupleUtil; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.ServerAggregators; +import org.apache.phoenix.expression.function.SingleAggregateFunction; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.KeyValueSchema; +import org.apache.phoenix.schema.ValueBitSet; +import org.apache.phoenix.schema.tuple.SingleKeyValueTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.KeyValueUtil; +import org.apache.phoenix.util.TupleUtil; /** * Class servers as an adapter between the in-memory LRU cache and the Spill data structures. It diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillMap.java b/phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillMap.java similarity index 99% rename from phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillMap.java rename to phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillMap.java index 5a8237b9..d93d08c6 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillMap.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillMap.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.cache.aggcache; +package org.apache.phoenix.cache.aggcache; import java.io.IOException; import java.nio.BufferOverflowException; @@ -18,7 +18,7 @@ import com.google.common.collect.Maps; import com.google.common.hash.BloomFilter; import com.google.common.hash.Funnels; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; /** * Class implements an active spilled partition serialized tuples are first written into an in-memory data structure diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillableGroupByCache.java b/phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillableGroupByCache.java similarity index 92% rename from phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillableGroupByCache.java rename to phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillableGroupByCache.java index a2f63aef..41a4e659 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/cache/aggcache/SpillableGroupByCache.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/cache/aggcache/SpillableGroupByCache.java @@ -1,12 +1,12 @@ -package com.salesforce.phoenix.cache.aggcache; +package org.apache.phoenix.cache.aggcache; -import static com.salesforce.phoenix.query.QueryConstants.AGG_TIMESTAMP; -import static com.salesforce.phoenix.query.QueryConstants.SINGLE_COLUMN; -import static com.salesforce.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY; -import static com.salesforce.phoenix.query.QueryServices.GROUPBY_MAX_CACHE_SIZE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.GROUPBY_SPILL_FILES_ATTRIB; -import static com.salesforce.phoenix.query.QueryServicesOptions.DEFAULT_GROUPBY_MAX_CACHE_MAX; -import static com.salesforce.phoenix.query.QueryServicesOptions.DEFAULT_GROUPBY_SPILL_FILES; +import static org.apache.phoenix.query.QueryConstants.AGG_TIMESTAMP; +import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN; +import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY; +import static org.apache.phoenix.query.QueryServices.GROUPBY_MAX_CACHE_SIZE_ATTRIB; +import static org.apache.phoenix.query.QueryServices.GROUPBY_SPILL_FILES_ATTRIB; +import static org.apache.phoenix.query.QueryServicesOptions.DEFAULT_GROUPBY_MAX_CACHE_MAX; +import static org.apache.phoenix.query.QueryServicesOptions.DEFAULT_GROUPBY_SPILL_FILES; import java.io.IOException; import java.util.Iterator; @@ -26,18 +26,18 @@ import org.slf4j.LoggerFactory; import com.google.common.io.Closeables; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.cache.GlobalCache; -import com.salesforce.phoenix.cache.TenantCache; -import com.salesforce.phoenix.cache.aggcache.SpillManager.CacheEntry; -import com.salesforce.phoenix.coprocessor.BaseRegionScanner; -import com.salesforce.phoenix.coprocessor.GroupByCache; -import com.salesforce.phoenix.coprocessor.GroupedAggregateRegionObserver; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.ServerAggregators; -import com.salesforce.phoenix.memory.InsufficientMemoryException; -import com.salesforce.phoenix.memory.MemoryManager.MemoryChunk; -import com.salesforce.phoenix.util.KeyValueUtil; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.cache.GlobalCache; +import org.apache.phoenix.cache.TenantCache; +import org.apache.phoenix.cache.aggcache.SpillManager.CacheEntry; +import org.apache.phoenix.coprocessor.BaseRegionScanner; +import org.apache.phoenix.coprocessor.GroupByCache; +import org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.ServerAggregators; +import org.apache.phoenix.memory.InsufficientMemoryException; +import org.apache.phoenix.memory.MemoryManager.MemoryChunk; +import org.apache.phoenix.util.KeyValueUtil; /** * The main entry point is in GroupedAggregateRegionObserver. It instantiates a SpillableGroupByCache and invokes a diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/client/ClientKeyValue.java b/phoenix-core/src/main/java/org/apache/phoenix/client/ClientKeyValue.java similarity index 99% rename from phoenix-core/src/main/java/com/salesforce/phoenix/client/ClientKeyValue.java rename to phoenix-core/src/main/java/org/apache/phoenix/client/ClientKeyValue.java index 39f86e61..3288c4d6 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/client/ClientKeyValue.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/client/ClientKeyValue.java @@ -15,7 +15,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package com.salesforce.phoenix.client; +package org.apache.phoenix.client; import java.io.DataInput; import java.io.DataOutput; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/client/ClientKeyValueBuilder.java b/phoenix-core/src/main/java/org/apache/phoenix/client/ClientKeyValueBuilder.java new file mode 100644 index 00000000..09e295ad --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/client/ClientKeyValueBuilder.java @@ -0,0 +1,67 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.client; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.KeyValue.Type; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; + +/** + * A {@link KeyValueBuilder} that builds {@link ClientKeyValue}, eliminating the extra byte copies + * inherent in the standard {@link KeyValue} implementation. + *

+ * This {@link KeyValueBuilder} is only supported in HBase 0.94.14+ ( + * {@link PhoenixDatabaseMetaData#CLIENT_KEY_VALUE_BUILDER_THRESHOLD}), with the addition of + * HBASE-9834. + */ +public class ClientKeyValueBuilder extends KeyValueBuilder { + + public static final KeyValueBuilder INSTANCE = new ClientKeyValueBuilder(); + + private ClientKeyValueBuilder() { + // private ctor for singleton + } + + @Override + public KeyValue buildPut(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier, long ts, ImmutableBytesWritable value) { + return new ClientKeyValue(row, family, qualifier, ts, Type.Put, value); + } + + @Override + public KeyValue buildDeleteFamily(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier, long ts) { + return new ClientKeyValue(row, family, qualifier, ts, Type.DeleteFamily, null); + } + + @Override + public KeyValue buildDeleteColumns(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier, long ts) { + return new ClientKeyValue(row, family, qualifier, ts, Type.DeleteColumn, null); + } + + @Override + public KeyValue buildDeleteColumn(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier, long ts) { + return new ClientKeyValue(row, family, qualifier, ts, Type.Delete, null); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/client/GenericKeyValueBuilder.java b/phoenix-core/src/main/java/org/apache/phoenix/client/GenericKeyValueBuilder.java new file mode 100644 index 00000000..b3771ca2 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/client/GenericKeyValueBuilder.java @@ -0,0 +1,71 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.client; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.KeyValue.Type; +import org.apache.hadoop.hbase.client.Delete; +import org.apache.hadoop.hbase.client.Put; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import static org.apache.hadoop.hbase.index.util.ImmutableBytesPtr.copyBytesIfNecessary; + +/** + * {@link KeyValueBuilder} that does simple byte[] copies to build the underlying key-value. This is + * exactly the same behavior as currently used in {@link Delete} and {@link Put}. + */ +public class GenericKeyValueBuilder extends KeyValueBuilder { + + public static final KeyValueBuilder INSTANCE = new GenericKeyValueBuilder(); + + private GenericKeyValueBuilder() { + // private ctor for singleton + } + + @Override + public KeyValue buildPut(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier, long ts, ImmutableBytesWritable value) { + return build(row, family, qualifier, ts, Type.Put, value); + } + + @Override + public KeyValue buildDeleteFamily(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier, long ts) { + return build(row, family, qualifier, ts, Type.DeleteFamily, null); + } + + @Override + public KeyValue buildDeleteColumns(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier, long ts) { + return build(row, family, qualifier, ts, Type.DeleteColumn, null); + } + + @Override + public KeyValue buildDeleteColumn(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier, long ts) { + return build(row, family, qualifier, ts, Type.Delete, null); + } + + private KeyValue build(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier, long ts, KeyValue.Type type, ImmutableBytesWritable value) { + return new KeyValue(copyBytesIfNecessary(row), copyBytesIfNecessary(family), + copyBytesIfNecessary(qualifier), ts, type, value == null? null: copyBytesIfNecessary(value)); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/client/KeyValueBuilder.java b/phoenix-core/src/main/java/org/apache/phoenix/client/KeyValueBuilder.java new file mode 100644 index 00000000..48608ff3 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/client/KeyValueBuilder.java @@ -0,0 +1,108 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.client; + +import java.io.IOException; + +import org.apache.hadoop.hbase.HConstants; +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.client.Delete; +import org.apache.hadoop.hbase.client.Put; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.util.MetaDataUtil; + +/** + * Build {@link KeyValue} in an efficient way + */ +public abstract class KeyValueBuilder { + + /** + * Helper method for a {@link KeyValueBuilder} that catches an IOException from a {@link Put} + * when adding a {@link KeyValue} generated by the KeyValueBuilder. + * @throws RuntimeException if there is an IOException thrown from the underlying {@link Put} + */ + @SuppressWarnings("javadoc") + public static void addQuietly(Put put, KeyValueBuilder builder, KeyValue kv) { + try { + put.add(kv); + } catch (IOException e) { + throw new RuntimeException("KeyValue Builder " + builder + " created an invalid kv: " + + kv + "!"); + } + } + + /** + * Helper method for a {@link KeyValueBuilder} that catches an IOException from a {@link Put} + * when adding a {@link KeyValue} generated by the KeyValueBuilder. + * @throws RuntimeException if there is an IOException thrown from the underlying {@link Put} + */ + @SuppressWarnings("javadoc") + public static void deleteQuietly(Delete delete, KeyValueBuilder builder, KeyValue kv) { + try { + delete.addDeleteMarker(kv); + } catch (IOException e) { + throw new RuntimeException("KeyValue Builder " + builder + " created an invalid kv: " + + kv + "!"); + } + } + + private static final int CUSTOM_KEY_VALUE_MIN_VERSION = MetaDataUtil.encodeVersion("0.94.14"); + + public static KeyValueBuilder get(String hbaseVersion) { + int version = MetaDataUtil.encodeVersion(hbaseVersion); + if (version >= CUSTOM_KEY_VALUE_MIN_VERSION) { + return ClientKeyValueBuilder.INSTANCE; + } + return GenericKeyValueBuilder.INSTANCE; + } + + public KeyValue buildPut(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier, ImmutableBytesWritable value) { + return buildPut(row, family, qualifier, HConstants.LATEST_TIMESTAMP, value); + } + + public abstract KeyValue buildPut(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier, long ts, ImmutableBytesWritable value); + + public KeyValue buildDeleteFamily(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier) { + return buildDeleteFamily(row, family, qualifier, HConstants.LATEST_TIMESTAMP); + } + + public abstract KeyValue buildDeleteFamily(ImmutableBytesWritable row, + ImmutableBytesWritable family, ImmutableBytesWritable qualifier, long ts); + + public KeyValue buildDeleteColumns(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier) { + return buildDeleteColumns(row, family, qualifier, HConstants.LATEST_TIMESTAMP); + } + + public abstract KeyValue buildDeleteColumns(ImmutableBytesWritable row, + ImmutableBytesWritable family, ImmutableBytesWritable qualifier, long ts); + + public KeyValue buildDeleteColumn(ImmutableBytesWritable row, ImmutableBytesWritable family, + ImmutableBytesWritable qualifier) { + return buildDeleteColumn(row, family, qualifier, HConstants.LATEST_TIMESTAMP); + } + + public abstract KeyValue buildDeleteColumn(ImmutableBytesWritable row, + ImmutableBytesWritable family, ImmutableBytesWritable qualifier, long ts); +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/AggregationManager.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/AggregationManager.java new file mode 100644 index 00000000..70ea3a2b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/AggregationManager.java @@ -0,0 +1,53 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import org.apache.phoenix.expression.aggregator.ClientAggregators; + +/** + * + * Class that manages aggregations during query compilation + * + * @author jtaylor + * @since 0.1 + */ +public class AggregationManager { + private ClientAggregators aggregators; + private int position = 0; + + public AggregationManager() { + } + + public ClientAggregators getAggregators() { + return aggregators; + } + + /** + * @return allocate the next available zero-based positional index + * for the client-side aggregate function. + */ + protected int nextPosition() { + return position++; + } + + public void setAggregators(ClientAggregators clientAggregator) { + this.aggregators = clientAggregator; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/BindManager.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/BindManager.java new file mode 100644 index 00000000..859d233d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/BindManager.java @@ -0,0 +1,82 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.ParameterMetaData; +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.jdbc.PhoenixParameterMetaData; +import org.apache.phoenix.parse.BindParseNode; +import org.apache.phoenix.schema.PDatum; + + +/** + * + * Class that manages binding parameters and checking type matching. There are + * two main usages: + * + * 1) the standard query case where we have the values for the binds. + * 2) the retrieve param metadata case where we don't have the bind values. + * + * In both cases, during query compilation we figure out what type the bind variable + * "should" be, based on how it's used in the query. For example foo < ? would expect + * that the bind variable type matches or can be coerced to the type of foo. For (1), + * we check that the bind value has the correct type and for (2) we set the param + * metadata type. + * + * @author jtaylor + * @since 0.1 + */ +public class BindManager { + public static final Object UNBOUND_PARAMETER = new Object(); + + private final List binds; + private final PhoenixParameterMetaData bindMetaData; + + public BindManager(List binds) { + this.binds = binds; + this.bindMetaData = new PhoenixParameterMetaData(binds.size()); + } + + public ParameterMetaData getParameterMetaData() { + return bindMetaData; + } + + public Object getBindValue(BindParseNode node) throws SQLException { + int index = node.getIndex(); + if (index < 0 || index >= binds.size()) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.PARAM_INDEX_OUT_OF_BOUND) + .setMessage("binds size: " + binds.size() + "; index: " + index).build().buildException(); + } + Object value = binds.get(index); + if (value == UNBOUND_PARAMETER) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.PARAM_VALUE_UNBOUND) + .setMessage(node.toString()).build().buildException(); + } + return value; + } + + public void addParamMetaData(BindParseNode bind, PDatum column) throws SQLException { + bindMetaData.addParam(bind,column); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/ColumnProjector.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/ColumnProjector.java new file mode 100644 index 00000000..8c1697b7 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/ColumnProjector.java @@ -0,0 +1,70 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.SQLException; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + + +/** + * + * Interface used to access the value of a projected column. + * + * @author jtaylor + * @since 0.1 + */ +public interface ColumnProjector { + /** + * Get the column name as it was referenced in the query + * @return the database column name + */ + String getName(); + + /** + * Get the expression + * @return the expression for the column projector + */ + public Expression getExpression(); + + // TODO: An expression may contain references to multiple tables. + /** + * Get the name of the hbase table containing the column + * @return the hbase table name + */ + String getTableName(); + + /** + * Get the value of the column, coercing it if necessary to the specified type + * @param tuple the row containing the column + * @param type the type to which to coerce the binary value + * @param ptr used to retrieve the value + * @return the object representation of the column value. + * @throws SQLException + */ + Object getValue(Tuple tuple, PDataType type, ImmutableBytesWritable ptr) throws SQLException; + + boolean isCaseSensitive(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/ColumnResolver.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/ColumnResolver.java new file mode 100644 index 00000000..49a19477 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/ColumnResolver.java @@ -0,0 +1,55 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.schema.ColumnRef; +import org.apache.phoenix.schema.TableRef; + + + +/** + * + * Interface used to resolve column references occurring + * in the select statement. + * + * @author jtaylor + * @since 0.1 + */ +public interface ColumnResolver { + + /** + * Returns the collection of resolved tables in the FROM clause. + */ + public List getTables(); + + /** + * Resolves column using name and alias. + * @param schemaName TODO + * @param tableName TODO + * @param colName TODO + * @return the resolved ColumnRef + * @throws ColumnNotFoundException if the column could not be resolved + * @throws AmbiguousColumnException if the column name is ambiguous + */ + public ColumnRef resolveColumn(String schemaName, String tableName, String colName) throws SQLException; +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/CreateIndexCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/CreateIndexCompiler.java new file mode 100644 index 00000000..718d09a3 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/CreateIndexCompiler.java @@ -0,0 +1,88 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.ParameterMetaData; +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +import org.apache.hadoop.hbase.client.Scan; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.parse.CreateIndexStatement; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.schema.MetaDataClient; + +public class CreateIndexCompiler { + private final PhoenixStatement statement; + + public CreateIndexCompiler(PhoenixStatement statement) { + this.statement = statement; + } + + public MutationPlan compile(final CreateIndexStatement create) throws SQLException { + final PhoenixConnection connection = statement.getConnection(); + final ColumnResolver resolver = FromCompiler.getResolver(create, connection); + Scan scan = new Scan(); + final StatementContext context = new StatementContext(statement, resolver, statement.getParameters(), scan); + ExpressionCompiler expressionCompiler = new ExpressionCompiler(context); + List splitNodes = create.getSplitNodes(); + final byte[][] splits = new byte[splitNodes.size()][]; + for (int i = 0; i < splits.length; i++) { + ParseNode node = splitNodes.get(i); + if (!node.isStateless()) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.SPLIT_POINT_NOT_CONSTANT) + .setMessage("Node: " + node).build().buildException(); + } + LiteralExpression expression = (LiteralExpression)node.accept(expressionCompiler); + splits[i] = expression.getBytes(); + } + final MetaDataClient client = new MetaDataClient(connection); + + return new MutationPlan() { + + @Override + public ParameterMetaData getParameterMetaData() { + return context.getBindManager().getParameterMetaData(); + } + + @Override + public PhoenixConnection getConnection() { + return connection; + } + + @Override + public MutationState execute() throws SQLException { + return client.createIndex(create, splits); + } + + @Override + public ExplainPlan getExplainPlan() throws SQLException { + return new ExplainPlan(Collections.singletonList("CREATE INDEX")); + } + }; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/CreateSequenceCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/CreateSequenceCompiler.java new file mode 100644 index 00000000..1a22f110 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/CreateSequenceCompiler.java @@ -0,0 +1,213 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.ParameterMetaData; +import java.sql.SQLException; +import java.util.Collections; + +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.parse.BindParseNode; +import org.apache.phoenix.parse.CreateSequenceStatement; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.MetaDataClient; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDatum; + + +public class CreateSequenceCompiler { + private final PhoenixStatement statement; + + public CreateSequenceCompiler(PhoenixStatement statement) { + this.statement = statement; + } + + private static class LongDatum implements PDatum { + + @Override + public boolean isNullable() { + return false; + } + + @Override + public PDataType getDataType() { + return PDataType.LONG; + } + + @Override + public Integer getByteSize() { + return PDataType.LONG.getByteSize(); + } + + @Override + public Integer getMaxLength() { + return null; + } + + @Override + public Integer getScale() { + return null; + } + + @Override + public ColumnModifier getColumnModifier() { + return null; + } + + } + private static class IntegerDatum implements PDatum { + + @Override + public boolean isNullable() { + return false; + } + + @Override + public PDataType getDataType() { + return PDataType.INTEGER; + } + + @Override + public Integer getByteSize() { + return PDataType.INTEGER.getByteSize(); + } + + @Override + public Integer getMaxLength() { + return null; + } + + @Override + public Integer getScale() { + return null; + } + + @Override + public ColumnModifier getColumnModifier() { + return null; + } + + } + private static final PDatum LONG_DATUM = new LongDatum(); + private static final PDatum INTEGER_DATUM = new IntegerDatum(); + + public MutationPlan compile(final CreateSequenceStatement sequence) throws SQLException { + ParseNode startsWithNode = sequence.getStartWith(); + ParseNode incrementByNode = sequence.getIncrementBy(); + if (!startsWithNode.isStateless()) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.STARTS_WITH_MUST_BE_CONSTANT) + .setSchemaName(sequence.getSequenceName().getSchemaName()) + .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); + } + if (!incrementByNode.isStateless()) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.INCREMENT_BY_MUST_BE_CONSTANT) + .setSchemaName(sequence.getSequenceName().getSchemaName()) + .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); + } + ParseNode cacheNode = sequence.getCacheSize(); + if (cacheNode != null && !cacheNode.isStateless()) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.CACHE_MUST_BE_NON_NEGATIVE_CONSTANT) + .setSchemaName(sequence.getSequenceName().getSchemaName()) + .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); + } + + final PhoenixConnection connection = statement.getConnection(); + final ColumnResolver resolver = FromCompiler.EMPTY_TABLE_RESOLVER; + + final StatementContext context = new StatementContext(statement, resolver, statement.getParameters(), new Scan()); + if (startsWithNode instanceof BindParseNode) { + context.getBindManager().addParamMetaData((BindParseNode)startsWithNode, LONG_DATUM); + } + if (incrementByNode instanceof BindParseNode) { + context.getBindManager().addParamMetaData((BindParseNode)incrementByNode, LONG_DATUM); + } + if (cacheNode instanceof BindParseNode) { + context.getBindManager().addParamMetaData((BindParseNode)cacheNode, INTEGER_DATUM); + } + ExpressionCompiler expressionCompiler = new ExpressionCompiler(context); + Expression startsWithExpr = startsWithNode.accept(expressionCompiler); + ImmutableBytesWritable ptr = context.getTempPtr(); + startsWithExpr.evaluate(null, ptr); + if (ptr.getLength() == 0 || !startsWithExpr.getDataType().isCoercibleTo(PDataType.LONG)) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.STARTS_WITH_MUST_BE_CONSTANT) + .setSchemaName(sequence.getSequenceName().getSchemaName()) + .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); + } + final long startsWith = (Long)PDataType.LONG.toObject(ptr, startsWithExpr.getDataType()); + + Expression incrementByExpr = incrementByNode.accept(expressionCompiler); + incrementByExpr.evaluate(null, ptr); + if (ptr.getLength() == 0 || !incrementByExpr.getDataType().isCoercibleTo(PDataType.LONG)) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.INCREMENT_BY_MUST_BE_CONSTANT) + .setSchemaName(sequence.getSequenceName().getSchemaName()) + .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); + } + final long incrementBy = (Long)PDataType.LONG.toObject(ptr, incrementByExpr.getDataType()); + + int cacheSizeValue = connection.getQueryServices().getProps().getInt(QueryServices.SEQUENCE_CACHE_SIZE_ATTRIB,QueryServicesOptions.DEFAULT_SEQUENCE_CACHE_SIZE); + if (cacheNode != null) { + Expression cacheSizeExpr = cacheNode.accept(expressionCompiler); + cacheSizeExpr.evaluate(null, ptr); + if (ptr.getLength() != 0 && (!cacheSizeExpr.getDataType().isCoercibleTo(PDataType.INTEGER) || (cacheSizeValue = (Integer)PDataType.INTEGER.toObject(ptr)) < 0)) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.CACHE_MUST_BE_NON_NEGATIVE_CONSTANT) + .setSchemaName(sequence.getSequenceName().getSchemaName()) + .setTableName(sequence.getSequenceName().getTableName()).build().buildException(); + } + } + final int cacheSize = Math.max(1, cacheSizeValue); + + + final MetaDataClient client = new MetaDataClient(connection); + return new MutationPlan() { + + @Override + public MutationState execute() throws SQLException { + return client.createSequence(sequence, startsWith, incrementBy, cacheSize); + } + + @Override + public ExplainPlan getExplainPlan() throws SQLException { + return new ExplainPlan(Collections.singletonList("CREATE SEQUENCE")); + } + + @Override + public PhoenixConnection getConnection() { + return connection; + } + + @Override + public ParameterMetaData getParameterMetaData() { + return context.getBindManager().getParameterMetaData(); + } + + }; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/CreateTableCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/CreateTableCompiler.java new file mode 100644 index 00000000..12c60c8a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/CreateTableCompiler.java @@ -0,0 +1,245 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.ParameterMetaData; +import java.sql.SQLException; +import java.util.Collections; +import java.util.Iterator; +import java.util.List; + +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.expression.AndExpression; +import org.apache.phoenix.expression.ComparisonExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.IsNullExpression; +import org.apache.phoenix.expression.KeyValueColumnExpression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.RowKeyColumnExpression; +import org.apache.phoenix.expression.visitor.TraverseNoExpressionVisitor; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.parse.CreateTableStatement; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.SQLParser; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.parse.TableName; +import org.apache.phoenix.parse.WildcardParseNode; +import org.apache.phoenix.query.DelegateConnectionQueryServices; +import org.apache.phoenix.schema.MetaDataClient; +import org.apache.phoenix.schema.PMetaData; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTable.ViewType; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.ByteUtil; + +import com.google.common.collect.Iterators; + + +public class CreateTableCompiler { + private static final String SELECT = "SELECT"; + private static final String FROM = "FROM"; + private static final String WHERE = "WHERE"; + + private final PhoenixStatement statement; + + public CreateTableCompiler(PhoenixStatement statement) { + this.statement = statement; + } + + public MutationPlan compile(final CreateTableStatement create) throws SQLException { + final PhoenixConnection connection = statement.getConnection(); + ColumnResolver resolver = FromCompiler.getResolver(create, connection); + PTableType type = create.getTableType(); + PhoenixConnection connectionToBe = connection; + PTable parentToBe = null; + ViewType viewTypeToBe = null; + Scan scan = new Scan(); + final StatementContext context = new StatementContext(statement, resolver, statement.getParameters(), scan); + ExpressionCompiler expressionCompiler = new ExpressionCompiler(context); + // TODO: support any statement for a VIEW instead of just a WHERE clause + ParseNode whereNode = create.getWhereClause(); + String viewStatementToBe = null; + if (type == PTableType.VIEW) { + TableRef tableRef = resolver.getTables().get(0); + parentToBe = tableRef.getTable(); + viewTypeToBe = parentToBe.getViewType() == ViewType.MAPPED ? ViewType.MAPPED : ViewType.UPDATABLE; + if (whereNode == null) { + viewStatementToBe = parentToBe.getViewStatement(); + } else { + whereNode = StatementNormalizer.normalize(whereNode, resolver); + if (whereNode.isStateless()) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.VIEW_WHERE_IS_CONSTANT) + .build().buildException(); + } + // If our parent has a VIEW statement, combine it with this one + if (parentToBe.getViewStatement() != null) { + SelectStatement select = new SQLParser(parentToBe.getViewStatement()).parseQuery().combine(whereNode); + whereNode = select.getWhere(); + } + Expression where = whereNode.accept(expressionCompiler); + if (where != null && !LiteralExpression.isTrue(where)) { + TableName baseTableName = create.getBaseTableName(); + String schemaName = baseTableName.getSchemaName(); + // Only form we currently support for VIEWs: SELECT * FROM t WHERE ... + viewStatementToBe = SELECT + " " + WildcardParseNode.NAME + " " + FROM + + (schemaName == null ? "" : "\"" + schemaName + "\".") + + (" \"" + baseTableName.getTableName() + "\" ") + + (WHERE + " " + where.toString()); + } + if (viewTypeToBe != ViewType.MAPPED) { + Long scn = connection.getSCN(); + connectionToBe = scn != null ? connection : + // If we haved no SCN on our connection, freeze the SCN at when + // the base table was resolved to prevent any race condition on + // the error checking we do for the base table. The only potential + // issue is if the base table lives on a different region server + // than the new table will, then we're relying here on the system + // clocks being in sync. + new PhoenixConnection( + // When the new table is created, we still want to cache it + // on our connection. + new DelegateConnectionQueryServices(connection.getQueryServices()) { + @Override + public PMetaData addTable(PTable table) throws SQLException { + return connection.addTable(table); + } + }, + connection, tableRef.getTimeStamp()); + ViewWhereExpressionVisitor visitor = new ViewWhereExpressionVisitor(); + where.accept(visitor); + viewTypeToBe = visitor.isUpdatable() ? ViewType.UPDATABLE : ViewType.READ_ONLY; + } + } + } + final ViewType viewType = viewTypeToBe; + final String viewStatement = viewStatementToBe; + List splitNodes = create.getSplitNodes(); + final byte[][] splits = new byte[splitNodes.size()][]; + ImmutableBytesWritable ptr = context.getTempPtr(); + for (int i = 0; i < splits.length; i++) { + ParseNode node = splitNodes.get(i); + if (node.isStateless()) { + Expression expression = node.accept(expressionCompiler); + if (expression.evaluate(null, ptr)) {; + splits[i] = ByteUtil.copyKeyBytesIfNecessary(ptr); + continue; + } + } + throw new SQLExceptionInfo.Builder(SQLExceptionCode.SPLIT_POINT_NOT_CONSTANT) + .setMessage("Node: " + node).build().buildException(); + } + final MetaDataClient client = new MetaDataClient(connectionToBe); + final PTable parent = parentToBe; + + return new MutationPlan() { + + @Override + public ParameterMetaData getParameterMetaData() { + return context.getBindManager().getParameterMetaData(); + } + + @Override + public MutationState execute() throws SQLException { + try { + return client.createTable(create, splits, parent, viewStatement, viewType); + } finally { + if (client.getConnection() != connection) { + client.getConnection().close(); + } + } + } + + @Override + public ExplainPlan getExplainPlan() throws SQLException { + return new ExplainPlan(Collections.singletonList("CREATE TABLE")); + } + + @Override + public PhoenixConnection getConnection() { + return connection; + } + + }; + } + + private static class ViewWhereExpressionVisitor extends TraverseNoExpressionVisitor { + private boolean isUpdatable = true; + + public boolean isUpdatable() { + return isUpdatable; + } + + @Override + public Boolean defaultReturn(Expression node, List l) { + // We only hit this if we're trying to traverse somewhere + // in which we don't have a visitLeave that returns non null + isUpdatable = false; + return null; + } + + @Override + public Iterator visitEnter(AndExpression node) { + return node.getChildren().iterator(); + } + + @Override + public Boolean visitLeave(AndExpression node, List l) { + return l.isEmpty() ? null : Boolean.TRUE; + } + + @Override + public Iterator visitEnter(ComparisonExpression node) { + return node.getFilterOp() == CompareOp.EQUAL && node.getChildren().get(1).isStateless() && node.getChildren().get(1).isDeterministic() ? Iterators.singletonIterator(node.getChildren().get(0)) : super.visitEnter(node); + } + + @Override + public Boolean visitLeave(ComparisonExpression node, List l) { + return l.isEmpty() ? null : Boolean.TRUE; + } + + @Override + public Iterator visitEnter(IsNullExpression node) { + return node.isNegate() ? super.visitEnter(node) : node.getChildren().iterator(); + } + + @Override + public Boolean visitLeave(IsNullExpression node, List l) { + return l.isEmpty() ? null : Boolean.TRUE; + } + + @Override + public Boolean visit(RowKeyColumnExpression node) { + return Boolean.TRUE; + } + + @Override + public Boolean visit(KeyValueColumnExpression node) { + return Boolean.TRUE; + } + + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/DeleteCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/DeleteCompiler.java similarity index 79% rename from phoenix-core/src/main/java/com/salesforce/phoenix/compile/DeleteCompiler.java rename to phoenix-core/src/main/java/org/apache/phoenix/compile/DeleteCompiler.java index 8e94b1f1..a73c111c 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/DeleteCompiler.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/DeleteCompiler.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; import java.sql.ParameterMetaData; import java.sql.ResultSet; @@ -37,47 +29,47 @@ import org.apache.hadoop.hbase.KeyValue; import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.phoenix.cache.ServerCacheClient.ServerCache; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.OrderByCompiler.OrderBy; +import org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.execute.AggregatePlan; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.index.IndexMetaDataCacheClient; +import org.apache.phoenix.index.PhoenixIndexCodec; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixResultSet; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.optimize.QueryOptimizer; +import org.apache.phoenix.parse.AliasedNode; +import org.apache.phoenix.parse.DeleteStatement; +import org.apache.phoenix.parse.HintNode; +import org.apache.phoenix.parse.HintNode.Hint; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.ParseNodeFactory; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.query.ConnectionQueryServices; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PRow; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.ReadOnlyTableException; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.IndexUtil; import com.google.common.collect.Lists; import com.google.common.collect.Maps; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.cache.ServerCacheClient.ServerCache; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.OrderByCompiler.OrderBy; -import com.salesforce.phoenix.coprocessor.UngroupedAggregateRegionObserver; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.execute.AggregatePlan; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.index.IndexMetaDataCacheClient; -import com.salesforce.phoenix.index.PhoenixIndexCodec; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixResultSet; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.optimize.QueryOptimizer; -import com.salesforce.phoenix.parse.AliasedNode; -import com.salesforce.phoenix.parse.DeleteStatement; -import com.salesforce.phoenix.parse.HintNode; -import com.salesforce.phoenix.parse.HintNode.Hint; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.ParseNodeFactory; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.query.ConnectionQueryServices; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PRow; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.ReadOnlyTableException; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.IndexUtil; public class DeleteCompiler { private static ParseNodeFactory FACTORY = new ParseNodeFactory(); @@ -90,6 +82,7 @@ public DeleteCompiler(PhoenixStatement statement) { private static MutationState deleteRows(PhoenixStatement statement, TableRef tableRef, ResultIterator iterator, RowProjector projector) throws SQLException { PhoenixConnection connection = statement.getConnection(); + byte[] tenantId = connection.getTenantId() == null ? null : connection.getTenantId().getBytes(); final boolean isAutoCommit = connection.getAutoCommit(); ConnectionQueryServices services = connection.getQueryServices(); final int maxSize = services.getProps().getInt(QueryServices.MAX_MUTATION_SIZE_ATTRIB,QueryServicesOptions.DEFAULT_MAX_MUTATION_SIZE); @@ -98,8 +91,12 @@ private static MutationState deleteRows(PhoenixStatement statement, TableRef tab try { PTable table = tableRef.getTable(); List pkColumns = table.getPKColumns(); - int offset = table.getBucketNum() == null ? 0 : 1; // Take into account salting + boolean isMultiTenant = table.isMultiTenant() && tenantId != null; + int offset = (table.getBucketNum() == null ? 0 : 1) + (isMultiTenant ? 1 : 0); // Take into account salting and multi-tenant byte[][] values = new byte[pkColumns.size()][]; + if (isMultiTenant) { + values[offset-1] = tenantId; + } ResultSet rs = new PhoenixResultSet(iterator, projector, statement); int rowCount = 0; while (rs.next()) { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/DropSequenceCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/DropSequenceCompiler.java new file mode 100644 index 00000000..be53f6b2 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/DropSequenceCompiler.java @@ -0,0 +1,69 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.ParameterMetaData; +import java.sql.SQLException; +import java.util.Collections; + +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixParameterMetaData; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.parse.DropSequenceStatement; +import org.apache.phoenix.schema.MetaDataClient; + + +public class DropSequenceCompiler { + private final PhoenixStatement statement; + + public DropSequenceCompiler(PhoenixStatement statement) { + this.statement = statement; + } + + + public MutationPlan compile(final DropSequenceStatement sequence) throws SQLException { + final PhoenixConnection connection = statement.getConnection(); + final MetaDataClient client = new MetaDataClient(connection); + return new MutationPlan() { + + @Override + public MutationState execute() throws SQLException { + return client.dropSequence(sequence); + } + + @Override + public ExplainPlan getExplainPlan() throws SQLException { + return new ExplainPlan(Collections.singletonList("DROP SEQUENCE")); + } + + @Override + public PhoenixConnection getConnection() { + return connection; + } + + @Override + public ParameterMetaData getParameterMetaData() { + return PhoenixParameterMetaData.EMPTY_PARAMETER_META_DATA; + } + + }; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/ExplainPlan.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/ExplainPlan.java new file mode 100644 index 00000000..e1049a04 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/ExplainPlan.java @@ -0,0 +1,39 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.util.Collections; +import java.util.List; + +import com.google.common.collect.ImmutableList; + +public class ExplainPlan { + public static final ExplainPlan EMPTY_PLAN = new ExplainPlan(Collections.emptyList()); + + private final List planSteps; + + public ExplainPlan(List planSteps) { + this.planSteps = ImmutableList.copyOf(planSteps); + } + + public List getPlanSteps() { + return planSteps; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExpressionCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/ExpressionCompiler.java similarity index 91% rename from phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExpressionCompiler.java rename to phoenix-core/src/main/java/org/apache/phoenix/compile/ExpressionCompiler.java index 5b729eae..cd2b0e27 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ExpressionCompiler.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/ExpressionCompiler.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; import java.math.BigDecimal; import java.sql.SQLException; @@ -40,79 +32,79 @@ import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.expression.AndExpression; -import com.salesforce.phoenix.expression.ArrayConstructorExpression; -import com.salesforce.phoenix.expression.CaseExpression; -import com.salesforce.phoenix.expression.CoerceExpression; -import com.salesforce.phoenix.expression.ComparisonExpression; -import com.salesforce.phoenix.expression.DateAddExpression; -import com.salesforce.phoenix.expression.DateSubtractExpression; -import com.salesforce.phoenix.expression.DecimalAddExpression; -import com.salesforce.phoenix.expression.DecimalDivideExpression; -import com.salesforce.phoenix.expression.DecimalMultiplyExpression; -import com.salesforce.phoenix.expression.DecimalSubtractExpression; -import com.salesforce.phoenix.expression.DoubleAddExpression; -import com.salesforce.phoenix.expression.DoubleDivideExpression; -import com.salesforce.phoenix.expression.DoubleMultiplyExpression; -import com.salesforce.phoenix.expression.DoubleSubtractExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.InListExpression; -import com.salesforce.phoenix.expression.IsNullExpression; -import com.salesforce.phoenix.expression.LikeExpression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.LongAddExpression; -import com.salesforce.phoenix.expression.LongDivideExpression; -import com.salesforce.phoenix.expression.LongMultiplyExpression; -import com.salesforce.phoenix.expression.LongSubtractExpression; -import com.salesforce.phoenix.expression.NotExpression; -import com.salesforce.phoenix.expression.OrExpression; -import com.salesforce.phoenix.expression.RowKeyColumnExpression; -import com.salesforce.phoenix.expression.RowValueConstructorExpression; -import com.salesforce.phoenix.expression.StringConcatExpression; -import com.salesforce.phoenix.expression.TimestampAddExpression; -import com.salesforce.phoenix.expression.TimestampSubtractExpression; -import com.salesforce.phoenix.parse.AddParseNode; -import com.salesforce.phoenix.parse.AndParseNode; -import com.salesforce.phoenix.parse.ArithmeticParseNode; -import com.salesforce.phoenix.parse.ArrayConstructorNode; -import com.salesforce.phoenix.parse.BindParseNode; -import com.salesforce.phoenix.parse.CaseParseNode; -import com.salesforce.phoenix.parse.CastParseNode; -import com.salesforce.phoenix.parse.ColumnParseNode; -import com.salesforce.phoenix.parse.ComparisonParseNode; -import com.salesforce.phoenix.parse.DivideParseNode; -import com.salesforce.phoenix.parse.FunctionParseNode; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo; -import com.salesforce.phoenix.parse.InListParseNode; -import com.salesforce.phoenix.parse.IsNullParseNode; -import com.salesforce.phoenix.parse.LikeParseNode; -import com.salesforce.phoenix.parse.LiteralParseNode; -import com.salesforce.phoenix.parse.MultiplyParseNode; -import com.salesforce.phoenix.parse.NotParseNode; -import com.salesforce.phoenix.parse.OrParseNode; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.RowValueConstructorParseNode; -import com.salesforce.phoenix.parse.SequenceValueParseNode; -import com.salesforce.phoenix.parse.StringConcatParseNode; -import com.salesforce.phoenix.parse.SubtractParseNode; -import com.salesforce.phoenix.parse.UnsupportedAllParseNodeVisitor; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.ColumnRef; -import com.salesforce.phoenix.schema.DelegateDatum; -import com.salesforce.phoenix.schema.PArrayDataType; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDatum; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.RowKeyValueAccessor; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.schema.TypeMismatchException; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.expression.AndExpression; +import org.apache.phoenix.expression.ArrayConstructorExpression; +import org.apache.phoenix.expression.CaseExpression; +import org.apache.phoenix.expression.CoerceExpression; +import org.apache.phoenix.expression.ComparisonExpression; +import org.apache.phoenix.expression.DateAddExpression; +import org.apache.phoenix.expression.DateSubtractExpression; +import org.apache.phoenix.expression.DecimalAddExpression; +import org.apache.phoenix.expression.DecimalDivideExpression; +import org.apache.phoenix.expression.DecimalMultiplyExpression; +import org.apache.phoenix.expression.DecimalSubtractExpression; +import org.apache.phoenix.expression.DoubleAddExpression; +import org.apache.phoenix.expression.DoubleDivideExpression; +import org.apache.phoenix.expression.DoubleMultiplyExpression; +import org.apache.phoenix.expression.DoubleSubtractExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.InListExpression; +import org.apache.phoenix.expression.IsNullExpression; +import org.apache.phoenix.expression.LikeExpression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.LongAddExpression; +import org.apache.phoenix.expression.LongDivideExpression; +import org.apache.phoenix.expression.LongMultiplyExpression; +import org.apache.phoenix.expression.LongSubtractExpression; +import org.apache.phoenix.expression.NotExpression; +import org.apache.phoenix.expression.OrExpression; +import org.apache.phoenix.expression.RowKeyColumnExpression; +import org.apache.phoenix.expression.RowValueConstructorExpression; +import org.apache.phoenix.expression.StringConcatExpression; +import org.apache.phoenix.expression.TimestampAddExpression; +import org.apache.phoenix.expression.TimestampSubtractExpression; +import org.apache.phoenix.parse.AddParseNode; +import org.apache.phoenix.parse.AndParseNode; +import org.apache.phoenix.parse.ArithmeticParseNode; +import org.apache.phoenix.parse.ArrayConstructorNode; +import org.apache.phoenix.parse.BindParseNode; +import org.apache.phoenix.parse.CaseParseNode; +import org.apache.phoenix.parse.CastParseNode; +import org.apache.phoenix.parse.ColumnParseNode; +import org.apache.phoenix.parse.ComparisonParseNode; +import org.apache.phoenix.parse.DivideParseNode; +import org.apache.phoenix.parse.FunctionParseNode; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo; +import org.apache.phoenix.parse.InListParseNode; +import org.apache.phoenix.parse.IsNullParseNode; +import org.apache.phoenix.parse.LikeParseNode; +import org.apache.phoenix.parse.LiteralParseNode; +import org.apache.phoenix.parse.MultiplyParseNode; +import org.apache.phoenix.parse.NotParseNode; +import org.apache.phoenix.parse.OrParseNode; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.RowValueConstructorParseNode; +import org.apache.phoenix.parse.SequenceValueParseNode; +import org.apache.phoenix.parse.StringConcatParseNode; +import org.apache.phoenix.parse.SubtractParseNode; +import org.apache.phoenix.parse.UnsupportedAllParseNodeVisitor; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.ColumnRef; +import org.apache.phoenix.schema.DelegateDatum; +import org.apache.phoenix.schema.PArrayDataType; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDatum; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.RowKeyValueAccessor; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.schema.TypeMismatchException; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.SchemaUtil; public class ExpressionCompiler extends UnsupportedAllParseNodeVisitor { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/ExpressionManager.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/ExpressionManager.java new file mode 100644 index 00000000..f66e61f8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/ExpressionManager.java @@ -0,0 +1,70 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.util.Iterator; +import java.util.Map; + + +import com.google.common.collect.Maps; +import org.apache.phoenix.expression.Expression; + +/** + * + * Class to manage list of expressions inside of a select statement by + * deduping them. + * + * @author jtaylor + * @since 0.1 + */ +public class ExpressionManager { + // Use a Map instead of a Set because we need to get and return + // the existing Expression + private final Map expressionMap; + + public ExpressionManager() { + expressionMap = Maps.newHashMap(); + } + + /** + * Add the expression to the set of known expressions for the select + * clause. If the expression is already in the set, then the new one + * passed in is ignored. + * @param expression the new expression to add + * @return the new expression if not already present in the set and + * the existing one otherwise. + */ + public Expression addIfAbsent(Expression expression) { + Expression existingExpression = expressionMap.get(expression); + if (existingExpression == null) { + expressionMap.put(expression, expression); + return expression; + } + return existingExpression; + } + + public int getExpressionCount() { + return expressionMap.size(); + } + + public Iterator getExpressions() { + return expressionMap.keySet().iterator(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/ExpressionProjector.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/ExpressionProjector.java new file mode 100644 index 00000000..8b30f0fd --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/ExpressionProjector.java @@ -0,0 +1,93 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + + +import java.sql.SQLException; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + + +/** + * + * Projector for getting value from a select statement for an expression + * + * @author jtaylor + * @since 0.1 + */ +public class ExpressionProjector implements ColumnProjector { + private final String name; + private final Expression expression; + private final String tableName; + private final boolean isCaseSensitive; + + public ExpressionProjector(String name, String tableName, Expression expression, boolean isCaseSensitive) { + this.name = name; + this.expression = expression; + this.tableName = tableName; + this.isCaseSensitive = isCaseSensitive; + } + + @Override + public String getTableName() { + return tableName; + } + + @Override + public Expression getExpression() { + return expression; + } + + @Override + public String getName() { + return name; + } + + @Override + public final Object getValue(Tuple tuple, PDataType type, ImmutableBytesWritable ptr) throws SQLException { + try { + Expression expression = getExpression(); + if (!expression.evaluate(tuple, ptr)) { + return null; + } + if (ptr.getLength() == 0) { + return null; + } + return type.toObject(ptr, expression.getDataType(), expression.getColumnModifier()); + } catch (RuntimeException e) { + // FIXME: Expression.evaluate does not throw SQLException + // so this will unwrap throws from that. + if (e.getCause() instanceof SQLException) { + throw (SQLException) e.getCause(); + } + throw e; + } + } + + @Override + public boolean isCaseSensitive() { + return isCaseSensitive; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/FromCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/FromCompiler.java similarity index 84% rename from phoenix-core/src/main/java/com/salesforce/phoenix/compile/FromCompiler.java rename to phoenix-core/src/main/java/org/apache/phoenix/compile/FromCompiler.java index 288a091d..92d89957 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/FromCompiler.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/FromCompiler.java @@ -1,19 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; import java.sql.SQLException; import java.sql.SQLFeatureNotSupportedException; @@ -24,6 +28,39 @@ import org.apache.hadoop.hbase.HConstants; import org.apache.hadoop.hbase.client.HTableInterface; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.parse.BindTableNode; +import org.apache.phoenix.parse.ColumnDef; +import org.apache.phoenix.parse.CreateTableStatement; +import org.apache.phoenix.parse.DerivedTableNode; +import org.apache.phoenix.parse.JoinTableNode; +import org.apache.phoenix.parse.NamedTableNode; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.parse.SingleTableSQLStatement; +import org.apache.phoenix.parse.TableName; +import org.apache.phoenix.parse.TableNode; +import org.apache.phoenix.parse.TableNodeVisitor; +import org.apache.phoenix.query.ConnectionQueryServices; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.AmbiguousColumnException; +import org.apache.phoenix.schema.AmbiguousTableException; +import org.apache.phoenix.schema.ColumnFamilyNotFoundException; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.ColumnRef; +import org.apache.phoenix.schema.MetaDataClient; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PColumnFamily; +import org.apache.phoenix.schema.PColumnFamilyImpl; +import org.apache.phoenix.schema.PColumnImpl; +import org.apache.phoenix.schema.PName; +import org.apache.phoenix.schema.PNameFactory; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableImpl; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.TableNotFoundException; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.SchemaUtil; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -32,39 +69,6 @@ import com.google.common.collect.ListMultimap; import com.google.common.collect.Lists; import com.google.common.io.Closeables; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.parse.BindTableNode; -import com.salesforce.phoenix.parse.ColumnDef; -import com.salesforce.phoenix.parse.CreateTableStatement; -import com.salesforce.phoenix.parse.DerivedTableNode; -import com.salesforce.phoenix.parse.JoinTableNode; -import com.salesforce.phoenix.parse.NamedTableNode; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.parse.SingleTableSQLStatement; -import com.salesforce.phoenix.parse.TableName; -import com.salesforce.phoenix.parse.TableNode; -import com.salesforce.phoenix.parse.TableNodeVisitor; -import com.salesforce.phoenix.query.ConnectionQueryServices; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.AmbiguousColumnException; -import com.salesforce.phoenix.schema.AmbiguousTableException; -import com.salesforce.phoenix.schema.ColumnFamilyNotFoundException; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.ColumnRef; -import com.salesforce.phoenix.schema.MetaDataClient; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PColumnFamily; -import com.salesforce.phoenix.schema.PColumnFamilyImpl; -import com.salesforce.phoenix.schema.PColumnImpl; -import com.salesforce.phoenix.schema.PName; -import com.salesforce.phoenix.schema.PNameFactory; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableImpl; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.TableNotFoundException; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.SchemaUtil; /** * Validates FROM clause and builds a ColumnResolver for resolving column references @@ -302,7 +306,7 @@ protected PTable addDynamicColumns(List dynColumns, PTable theTable) familyName = PNameFactory.newName(family); } allcolumns.add(new PColumnImpl(name, familyName, dynColumn.getDataType(), dynColumn.getMaxLength(), - dynColumn.getScale(), dynColumn.isNull(), position, dynColumn.getColumnModifier())); + dynColumn.getScale(), dynColumn.isNull(), position, dynColumn.getColumnModifier(), dynColumn.getArraySize())); position++; } theTable = PTableImpl.makePTable(theTable, allcolumns); diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/GroupByCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/GroupByCompiler.java similarity index 80% rename from phoenix-core/src/main/java/com/salesforce/phoenix/compile/GroupByCompiler.java rename to phoenix-core/src/main/java/org/apache/phoenix/compile/GroupByCompiler.java index bebb759d..e9b08f85 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/GroupByCompiler.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/GroupByCompiler.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; import java.sql.SQLException; import java.util.ArrayList; @@ -37,20 +29,20 @@ import com.google.common.collect.ImmutableList; import com.google.common.collect.Lists; -import com.salesforce.phoenix.compile.TrackOrderPreservingExpressionCompiler.Entry; -import com.salesforce.phoenix.compile.TrackOrderPreservingExpressionCompiler.Ordering; -import com.salesforce.phoenix.coprocessor.GroupedAggregateRegionObserver; -import com.salesforce.phoenix.coprocessor.UngroupedAggregateRegionObserver; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.expression.CoerceExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.AliasedNode; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.schema.AmbiguousColumnException; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.PDataType; +import org.apache.phoenix.compile.TrackOrderPreservingExpressionCompiler.Entry; +import org.apache.phoenix.compile.TrackOrderPreservingExpressionCompiler.Ordering; +import org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver; +import org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.expression.CoerceExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.AliasedNode; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.schema.AmbiguousColumnException; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.PDataType; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/HavingCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/HavingCompiler.java new file mode 100644 index 00000000..857d48ab --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/HavingCompiler.java @@ -0,0 +1,241 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.SQLException; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Set; + +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.parse.AddParseNode; +import org.apache.phoenix.parse.AndParseNode; +import org.apache.phoenix.parse.BetweenParseNode; +import org.apache.phoenix.parse.CaseParseNode; +import org.apache.phoenix.parse.ColumnParseNode; +import org.apache.phoenix.parse.ComparisonParseNode; +import org.apache.phoenix.parse.DivideParseNode; +import org.apache.phoenix.parse.FunctionParseNode; +import org.apache.phoenix.parse.IsNullParseNode; +import org.apache.phoenix.parse.MultiplyParseNode; +import org.apache.phoenix.parse.OrParseNode; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.parse.SelectStatementRewriter; +import org.apache.phoenix.parse.SubtractParseNode; +import org.apache.phoenix.parse.TraverseNoParseNodeVisitor; +import org.apache.phoenix.schema.ColumnRef; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.TypeMismatchException; + + +public class HavingCompiler { + + private HavingCompiler() { + } + + public static Expression compile(StatementContext context, SelectStatement statement, GroupBy groupBy) throws SQLException { + ParseNode having = statement.getHaving(); + if (having == null) { + return null; + } + ExpressionCompiler expressionBuilder = new ExpressionCompiler(context, groupBy); + Expression expression = having.accept(expressionBuilder); + if (expression.getDataType() != PDataType.BOOLEAN) { + throw TypeMismatchException.newException(PDataType.BOOLEAN, expression.getDataType(), expression.toString()); + } + if (LiteralExpression.isFalse(expression)) { + context.setScanRanges(ScanRanges.NOTHING); + return null; + } else if (LiteralExpression.isTrue(expression)) { + return null; + } + if (!expressionBuilder.isAggregate()) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.ONLY_AGGREGATE_IN_HAVING_CLAUSE).build().buildException(); + } + return expression; + } + + public static SelectStatement rewrite(StatementContext context, SelectStatement statement, GroupBy groupBy) throws SQLException { + ParseNode having = statement.getHaving(); + if (having == null) { + return statement; + } + HavingClauseVisitor visitor = new HavingClauseVisitor(context, groupBy); + having.accept(visitor); + statement = SelectStatementRewriter.moveFromHavingToWhereClause(statement, visitor.getMoveToWhereClauseExpressions()); + return statement; + } + + /** + * + * Visitor that figures out if an expression can be moved from the HAVING clause to + * the WHERE clause, since it's more optimal to pre-filter instead of post-filter. + * + * The visitor traverses through AND expressions only and into comparison expresssions. + * If a comparison expression uses a GROUP BY column and does not use any aggregate + * functions, then it's moved. For example, these HAVING expressions would be moved: + * + * select count(1) from atable group by a_string having a_string = 'foo' + * select count(1) from atable group by a_date having round(a_date,'hour') > ? + * select count(1) from atable group by a_date,a_string having a_date > ? or a_string = 'a' + * select count(1) from atable group by a_string,b_string having a_string = 'a' and b_string = 'b' + * + * while these would not be moved: + * + * select count(1) from atable having min(a_integer) < 5 + * select count(1) from atable group by a_string having count(a_string) >= 1 + * select count(1) from atable group by a_date,a_string having a_date > ? or min(a_string) = 'a' + * select count(1) from atable group by a_date having round(min(a_date),'hour') < ? + * + * @author jtaylor + * @since 0.1 + */ + private static class HavingClauseVisitor extends TraverseNoParseNodeVisitor { + private ParseNode topNode = null; + private boolean hasNoAggregateFunctions = true; + private Boolean hasOnlyAggregateColumns; + private final StatementContext context; + private final GroupBy groupBy; + private final Set moveToWhereClause = new LinkedHashSet(); + + HavingClauseVisitor(StatementContext context, GroupBy groupBy) { + this.context = context; + this.groupBy = groupBy; + } + + public Set getMoveToWhereClauseExpressions() { + return moveToWhereClause; + } + + @Override + public boolean visitEnter(AndParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(OrParseNode node) throws SQLException { + enterBooleanNode(node); + return true; + } + + @Override + public boolean visitEnter(ComparisonParseNode node) throws SQLException { + enterBooleanNode(node); + return true; + } + + @Override + public boolean visitEnter(IsNullParseNode node) throws SQLException { + enterBooleanNode(node); + return true; + } + + private void enterBooleanNode(ParseNode node) { + if (topNode == null) { + topNode = node; + } + } + + private void leaveBooleanNode(ParseNode node) { + if (topNode == node) { + if ( hasNoAggregateFunctions && !Boolean.FALSE.equals(hasOnlyAggregateColumns)) { + moveToWhereClause.add(node); + } + hasNoAggregateFunctions = true; + hasOnlyAggregateColumns = null; + topNode = null; + } + } + + @Override + public Void visitLeave(OrParseNode node, List l) throws SQLException { + leaveBooleanNode(node); + return null; + } + + @Override + public Void visitLeave(ComparisonParseNode node, List l) throws SQLException { + leaveBooleanNode(node); + return null; + } + + @Override + public Void visitLeave(IsNullParseNode node, List l) throws SQLException { + leaveBooleanNode(node); + return null; + } + + @Override + public boolean visitEnter(FunctionParseNode node) throws SQLException { + boolean isAggregate = node.isAggregate(); + this.hasNoAggregateFunctions = this.hasNoAggregateFunctions && !isAggregate; + return !isAggregate; + } + + @Override + public boolean visitEnter(CaseParseNode node) throws SQLException { + return true; + } + + @Override + public Void visit(ColumnParseNode node) throws SQLException { + ColumnRef ref = context.getResolver().resolveColumn(node.getSchemaName(), node.getTableName(), node.getName()); + boolean isAggregateColumn = groupBy.getExpressions().indexOf(ref.newColumnExpression()) >= 0; + if (hasOnlyAggregateColumns == null) { + hasOnlyAggregateColumns = isAggregateColumn; + } else { + hasOnlyAggregateColumns &= isAggregateColumn; + } + + return null; + } + + @Override + public boolean visitEnter(SubtractParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(AddParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(MultiplyParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(DivideParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(BetweenParseNode node) throws SQLException { + return true; + } + + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/IndexStatementRewriter.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/IndexStatementRewriter.java similarity index 86% rename from phoenix-core/src/main/java/com/salesforce/phoenix/compile/IndexStatementRewriter.java rename to phoenix-core/src/main/java/org/apache/phoenix/compile/IndexStatementRewriter.java index b4383c21..84ac0a11 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/IndexStatementRewriter.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/IndexStatementRewriter.java @@ -1,20 +1,20 @@ -package com.salesforce.phoenix.compile; +package org.apache.phoenix.compile; import java.sql.SQLException; import java.util.Map; -import com.salesforce.phoenix.parse.ColumnParseNode; -import com.salesforce.phoenix.parse.FamilyWildcardParseNode; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.ParseNodeFactory; -import com.salesforce.phoenix.parse.ParseNodeRewriter; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.parse.TableName; -import com.salesforce.phoenix.parse.WildcardParseNode; -import com.salesforce.phoenix.schema.ColumnRef; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.IndexUtil; +import org.apache.phoenix.parse.ColumnParseNode; +import org.apache.phoenix.parse.FamilyWildcardParseNode; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.ParseNodeFactory; +import org.apache.phoenix.parse.ParseNodeRewriter; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.parse.TableName; +import org.apache.phoenix.parse.WildcardParseNode; +import org.apache.phoenix.schema.ColumnRef; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.IndexUtil; public class IndexStatementRewriter extends ParseNodeRewriter { private static final ParseNodeFactory FACTORY = new ParseNodeFactory(); diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/JoinCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/JoinCompiler.java similarity index 90% rename from phoenix-core/src/main/java/com/salesforce/phoenix/compile/JoinCompiler.java rename to phoenix-core/src/main/java/org/apache/phoenix/compile/JoinCompiler.java index 807509f1..4bbb357c 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/JoinCompiler.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/JoinCompiler.java @@ -1,33 +1,25 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; -import static com.salesforce.phoenix.schema.SaltingUtil.SALTING_COLUMN; +import static org.apache.phoenix.schema.SaltingUtil.SALTING_COLUMN; import java.sql.SQLException; import java.sql.SQLFeatureNotSupportedException; @@ -43,61 +35,61 @@ import org.apache.hadoop.hbase.client.Scan; import org.apache.hadoop.hbase.util.Pair; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.expression.AndExpression; +import org.apache.phoenix.expression.CoerceExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.join.ScanProjector; +import org.apache.phoenix.parse.AliasedNode; +import org.apache.phoenix.parse.AndParseNode; +import org.apache.phoenix.parse.BetweenParseNode; +import org.apache.phoenix.parse.BindTableNode; +import org.apache.phoenix.parse.CaseParseNode; +import org.apache.phoenix.parse.CastParseNode; +import org.apache.phoenix.parse.ColumnParseNode; +import org.apache.phoenix.parse.ComparisonParseNode; +import org.apache.phoenix.parse.ConcreteTableNode; +import org.apache.phoenix.parse.DerivedTableNode; +import org.apache.phoenix.parse.EqualParseNode; +import org.apache.phoenix.parse.FunctionParseNode; +import org.apache.phoenix.parse.InListParseNode; +import org.apache.phoenix.parse.IsNullParseNode; +import org.apache.phoenix.parse.JoinTableNode; +import org.apache.phoenix.parse.JoinTableNode.JoinType; +import org.apache.phoenix.parse.LikeParseNode; +import org.apache.phoenix.parse.NamedTableNode; +import org.apache.phoenix.parse.NotParseNode; +import org.apache.phoenix.parse.OrParseNode; +import org.apache.phoenix.parse.OrderByNode; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.ParseNodeFactory; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.parse.StatelessTraverseAllParseNodeVisitor; +import org.apache.phoenix.parse.TableName; +import org.apache.phoenix.parse.TableNode; +import org.apache.phoenix.parse.TableNodeVisitor; +import org.apache.phoenix.parse.TraverseNoParseNodeVisitor; +import org.apache.phoenix.parse.WildcardParseNode; +import org.apache.phoenix.schema.AmbiguousColumnException; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.ColumnRef; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PColumnImpl; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PName; +import org.apache.phoenix.schema.PNameFactory; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableImpl; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.SchemaUtil; import com.google.common.collect.ArrayListMultimap; import com.google.common.collect.ImmutableList; import com.google.common.collect.ListMultimap; import com.google.common.collect.Lists; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.expression.AndExpression; -import com.salesforce.phoenix.expression.CoerceExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.join.ScanProjector; -import com.salesforce.phoenix.parse.AliasedNode; -import com.salesforce.phoenix.parse.AndParseNode; -import com.salesforce.phoenix.parse.BetweenParseNode; -import com.salesforce.phoenix.parse.BindTableNode; -import com.salesforce.phoenix.parse.CaseParseNode; -import com.salesforce.phoenix.parse.CastParseNode; -import com.salesforce.phoenix.parse.ColumnParseNode; -import com.salesforce.phoenix.parse.ComparisonParseNode; -import com.salesforce.phoenix.parse.ConcreteTableNode; -import com.salesforce.phoenix.parse.DerivedTableNode; -import com.salesforce.phoenix.parse.EqualParseNode; -import com.salesforce.phoenix.parse.FunctionParseNode; -import com.salesforce.phoenix.parse.InListParseNode; -import com.salesforce.phoenix.parse.IsNullParseNode; -import com.salesforce.phoenix.parse.JoinTableNode; -import com.salesforce.phoenix.parse.JoinTableNode.JoinType; -import com.salesforce.phoenix.parse.LikeParseNode; -import com.salesforce.phoenix.parse.NamedTableNode; -import com.salesforce.phoenix.parse.NotParseNode; -import com.salesforce.phoenix.parse.OrParseNode; -import com.salesforce.phoenix.parse.OrderByNode; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.ParseNodeFactory; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.parse.StatelessTraverseAllParseNodeVisitor; -import com.salesforce.phoenix.parse.TableName; -import com.salesforce.phoenix.parse.TableNode; -import com.salesforce.phoenix.parse.TableNodeVisitor; -import com.salesforce.phoenix.parse.TraverseNoParseNodeVisitor; -import com.salesforce.phoenix.parse.WildcardParseNode; -import com.salesforce.phoenix.schema.AmbiguousColumnException; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.ColumnRef; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PColumnImpl; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PName; -import com.salesforce.phoenix.schema.PNameFactory; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableImpl; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.SchemaUtil; public class JoinCompiler { @@ -302,7 +294,7 @@ public ProjectedPTableWrapper createProjectedTable(TableRef tableRef, boolean re PTable t = PTableImpl.makePTable(PNameFactory.newName(PROJECTED_TABLE_SCHEMA), table.getName(), PTableType.JOIN, table.getIndexState(), table.getTimeStamp(), table.getSequenceNumber(), table.getPKName(), retainPKColumns ? table.getBucketNum() : null, projectedColumns, table.getParentTableName(), table.getIndexes(), - table.isImmutableRows(), table.getBaseSchemaName(), null, null, null, table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); + table.isImmutableRows(), Collections.emptyList(), null, null, table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); return new ProjectedPTableWrapper(t, columnNameMap, sourceExpressions); } @@ -325,7 +317,7 @@ private static void addProjectedColumn(List projectedColumns, ListemptyList(), null, null, PTable.DEFAULT_DISABLE_WAL, left.isMultiTenant(), left.getViewType()); ListMultimap mergedMap = ArrayListMultimap.create(); mergedMap.putAll(lWrapper.getColumnNameMap()); diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/KeyPart.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/KeyPart.java new file mode 100644 index 00000000..1c8b0eb8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/KeyPart.java @@ -0,0 +1,79 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.util.List; + +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.schema.PColumn; + +/** + * + * Interface that determines how a key part contributes to + * the forming of the key (start/stop of scan and SkipScanFilter) + * for each part of a multi-part primary key. It acts as the glue + * between a built-in function and the setting of the scan key + * during query compilation. + * + * @author jtaylor + * @since 0.12 + */ +public interface KeyPart { + /** + * Calculate the key range given an operator and the key on + * the RHS of an expression. For example, given the expression + * SUBSTR(foo,1,3) = 'bar', the key range would be ['bar','bas'), + * and if foo was fixed length, the upper and lower key range + * bytes would be filled out to the fixed length. + * @param op comparison operator (=, <=, <, >=, >, !=) + * @param rhs the constant on the RHS of an expression. + * @return the key range that encompasses the range for the + * expression for which this keyPart is associated + * + * @see org.apache.phoenix.expression.function.ScalarFunction#newKeyPart(KeyPart) + */ + public KeyRange getKeyRange(CompareOp op, Expression rhs); + + /** + * Determines whether an expression gets extracted from the + * WHERE clause if it contributes toward the building of the + * scan key. For example, the SUBSTR built-in function may + * be extracted, since it may be completely represented + * through a key range. However, the REGEXP_SUBSTR must be + * left in the WHERE clause, since only the constant prefix + * part of the evaluation can be represented through a key + * range (i.e. rows may pass through that will fail when + * the REGEXP_SUBSTR is evaluated). + * + * @return an empty list if the expression should remain in + * the WHEERE clause for post filtering or a singleton list + * containing the expression if it should be removed. + */ + public List getExtractNodes(); + + /** + * Gets the primary key column associated with the start of this key part. + * @return the primary key column for this key part + */ + public PColumn getColumn(); +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/LimitCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/LimitCompiler.java new file mode 100644 index 00000000..5c4cbb21 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/LimitCompiler.java @@ -0,0 +1,110 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.SQLException; + +import org.apache.phoenix.parse.*; +import org.apache.phoenix.schema.*; + + +public class LimitCompiler { + private static final ParseNodeFactory NODE_FACTORY = new ParseNodeFactory(); + + public static final PDatum LIMIT_DATUM = new PDatum() { + @Override + public boolean isNullable() { + return false; + } + @Override + public PDataType getDataType() { + return PDataType.INTEGER; + } + @Override + public Integer getByteSize() { + return getDataType().getByteSize(); + } + @Override + public Integer getMaxLength() { + return null; + } + @Override + public Integer getScale() { + return null; + } + @Override + public ColumnModifier getColumnModifier() { + return null; + } + }; + + private LimitCompiler() { + } + + public static Integer compile(StatementContext context, FilterableStatement statement) throws SQLException { + LimitNode limitNode = statement.getLimit(); + if (limitNode == null) { + return null; + } + LimitParseNodeVisitor visitor = new LimitParseNodeVisitor(context); + limitNode.getLimitParseNode().accept(visitor); + return visitor.getLimit(); + } + + private static class LimitParseNodeVisitor extends TraverseNoParseNodeVisitor { + private final StatementContext context; + private Integer limit; + + private LimitParseNodeVisitor(StatementContext context) { + this.context = context; + } + + public Integer getLimit() { + return limit; + } + + @Override + public Void visit(LiteralParseNode node) throws SQLException { + Object limitValue = node.getValue(); + // If limit is null, leave this.limit set to zero + // This means that we've bound limit to null for the purpose of + // collecting parameter metadata. + if (limitValue != null) { + Integer limit = (Integer)LIMIT_DATUM.getDataType().toObject(limitValue, node.getType()); + if (limit.intValue() >= 0) { // TODO: handle LIMIT 0 + this.limit = limit; + } + } + return null; + } + + @Override + public Void visit(BindParseNode node) throws SQLException { + Object value = context.getBindManager().getBindValue(node); + context.getBindManager().addParamMetaData(node, LIMIT_DATUM); + // Resolve the bind value, create a LiteralParseNode, and call the visit method for it. + // In this way, we can deal with just having a literal on one side of the expression. + visit(NODE_FACTORY.literal(value, LIMIT_DATUM.getDataType())); + return null; + } + + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/MutatingParallelIteratorFactory.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/MutatingParallelIteratorFactory.java new file mode 100644 index 00000000..b19af439 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/MutatingParallelIteratorFactory.java @@ -0,0 +1,115 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import static org.apache.phoenix.query.QueryConstants.AGG_TIMESTAMP; +import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN; +import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY; +import static org.apache.phoenix.query.QueryConstants.UNGROUPED_AGG_ROW_KEY; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.KeyValue; + +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; +import org.apache.phoenix.iterate.PeekingResultIterator; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.query.ConnectionQueryServices; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.schema.tuple.SingleKeyValueTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.KeyValueUtil; + +/** + * Factory class used to instantiate an iterator to handle mutations made during a parallel scan. + */ +public abstract class MutatingParallelIteratorFactory implements ParallelIteratorFactory { + protected final PhoenixConnection connection; + protected final TableRef tableRef; + + protected MutatingParallelIteratorFactory(PhoenixConnection connection, TableRef tableRef) { + this.connection = connection; + this.tableRef = tableRef; + } + + /** + * Method that does the actual mutation work + */ + abstract protected MutationState mutate(PhoenixConnection connection, ResultIterator iterator) throws SQLException; + + @Override + public PeekingResultIterator newIterator(ResultIterator iterator) throws SQLException { + // Clone the connection as it's not thread safe and will be operated on in parallel + final PhoenixConnection connection = new PhoenixConnection(this.connection); + MutationState state = mutate(connection, iterator); + long totalRowCount = state.getUpdateCount(); + if (connection.getAutoCommit()) { + connection.getMutationState().join(state); + connection.commit(); + ConnectionQueryServices services = connection.getQueryServices(); + int maxSize = services.getProps().getInt(QueryServices.MAX_MUTATION_SIZE_ATTRIB,QueryServicesOptions.DEFAULT_MAX_MUTATION_SIZE); + state = new MutationState(maxSize, connection, totalRowCount); + } + final MutationState finalState = state; + byte[] value = PDataType.LONG.toBytes(totalRowCount); + KeyValue keyValue = KeyValueUtil.newKeyValue(UNGROUPED_AGG_ROW_KEY, SINGLE_COLUMN_FAMILY, SINGLE_COLUMN, AGG_TIMESTAMP, value, 0, value.length); + final Tuple tuple = new SingleKeyValueTuple(keyValue); + return new PeekingResultIterator() { + private boolean done = false; + + @Override + public Tuple next() throws SQLException { + if (done) { + return null; + } + done = true; + return tuple; + } + + @Override + public void explain(List planSteps) { + } + + @Override + public void close() throws SQLException { + try { + // Join the child mutation states in close, since this is called in a single threaded manner + // after the parallel results have been processed. + if (!connection.getAutoCommit()) { + MutatingParallelIteratorFactory.this.connection.getMutationState().join(finalState); + } + } finally { + connection.close(); + } + } + + @Override + public Tuple peek() throws SQLException { + return done ? null : tuple; + } + }; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/MutationPlan.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/MutationPlan.java new file mode 100644 index 00000000..11fd2cd6 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/MutationPlan.java @@ -0,0 +1,31 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.SQLException; + +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.jdbc.PhoenixConnection; + + +public interface MutationPlan extends StatementPlan { + public PhoenixConnection getConnection(); + public MutationState execute() throws SQLException; +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/OrderByCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/OrderByCompiler.java new file mode 100644 index 00000000..61c0fcc5 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/OrderByCompiler.java @@ -0,0 +1,138 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + + +import java.sql.SQLException; +import java.util.Collections; +import java.util.LinkedHashSet; +import java.util.List; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Lists; +import com.google.common.collect.Sets; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.TrackOrderPreservingExpressionCompiler.Ordering; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.OrderByExpression; +import org.apache.phoenix.parse.FilterableStatement; +import org.apache.phoenix.parse.OrderByNode; +import org.apache.phoenix.query.ConnectionQueryServices.Feature; +import org.apache.phoenix.schema.ColumnModifier; + +/** + * Validates ORDER BY clause and builds up a list of referenced columns. + * + * @author syyang + * @since 0.1 + */ +public class OrderByCompiler { + public static class OrderBy { + public static final OrderBy EMPTY_ORDER_BY = new OrderBy(Collections.emptyList()); + /** + * Used to indicate that there was an ORDER BY, but it was optimized out because + * rows are already returned in this order. + */ + public static final OrderBy FWD_ROW_KEY_ORDER_BY = new OrderBy(Collections.emptyList()); + public static final OrderBy REV_ROW_KEY_ORDER_BY = new OrderBy(Collections.emptyList()); + + private final List orderByExpressions; + + private OrderBy(List orderByExpressions) { + this.orderByExpressions = ImmutableList.copyOf(orderByExpressions); + } + + public List getOrderByExpressions() { + return orderByExpressions; + } + } + /** + * Gets a list of columns in the ORDER BY clause + * @param context the query context for tracking various states + * associated with the given select statement + * @param statement TODO + * @param groupBy the list of columns in the GROUP BY clause + * @param limit the row limit or null if no limit + * @return the compiled ORDER BY clause + * @throws SQLException + */ + public static OrderBy compile(StatementContext context, + FilterableStatement statement, + GroupBy groupBy, Integer limit) throws SQLException { + List orderByNodes = statement.getOrderBy(); + if (orderByNodes.isEmpty()) { + return OrderBy.EMPTY_ORDER_BY; + } + // accumulate columns in ORDER BY + TrackOrderPreservingExpressionCompiler visitor = + new TrackOrderPreservingExpressionCompiler(context, groupBy, + orderByNodes.size(), Ordering.ORDERED); + LinkedHashSet orderByExpressions = Sets.newLinkedHashSetWithExpectedSize(orderByNodes.size()); + for (OrderByNode node : orderByNodes) { + boolean isAscending = node.isAscending(); + Expression expression = node.getNode().accept(visitor); + if (!expression.isStateless() && visitor.addEntry(expression, isAscending ? null : ColumnModifier.SORT_DESC)) { + // Detect mix of aggregate and non aggregates (i.e. ORDER BY txns, SUM(txns) + if (!visitor.isAggregate()) { + if (statement.isAggregate() || statement.isDistinct()) { + // Detect ORDER BY not in SELECT DISTINCT: SELECT DISTINCT count(*) FROM t ORDER BY x + if (statement.isDistinct()) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.ORDER_BY_NOT_IN_SELECT_DISTINCT) + .setMessage(expression.toString()).build().buildException(); + } + ExpressionCompiler.throwNonAggExpressionInAggException(expression.toString()); + } + } + if (expression.getDataType().isArrayType()) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.ORDER_BY_ARRAY_NOT_SUPPORTED) + .setMessage(expression.toString()).build().buildException(); + } + if (expression.getColumnModifier() == ColumnModifier.SORT_DESC) { + isAscending = !isAscending; + } + OrderByExpression orderByExpression = new OrderByExpression(expression, node.isNullsLast(), isAscending); + orderByExpressions.add(orderByExpression); + } + visitor.reset(); + } + + if (orderByExpressions.isEmpty()) { + return OrderBy.EMPTY_ORDER_BY; + } + // If we're ordering by the order returned by the scan, we don't need an order by + if (visitor.isOrderPreserving()) { + if (visitor.isReverse()) { + if (context.getConnection().getQueryServices().supportsFeature(Feature.REVERSE_SCAN)) { + return OrderBy.REV_ROW_KEY_ORDER_BY; + } + } else { + return OrderBy.FWD_ROW_KEY_ORDER_BY; + } + } + + return new OrderBy(Lists.newArrayList(orderByExpressions.iterator())); + } + + + private OrderByCompiler() { + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/PostDDLCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/PostDDLCompiler.java new file mode 100644 index 00000000..40fe1c38 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/PostDDLCompiler.java @@ -0,0 +1,225 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.ParameterMetaData; +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.phoenix.cache.ServerCacheClient.ServerCache; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.OrderByCompiler.OrderBy; +import org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver; +import org.apache.phoenix.execute.AggregatePlan; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixParameterMetaData; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.ColumnRef; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PColumnFamily; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ScanUtil; + +import com.google.common.collect.Lists; + + +/** + * + * Class that compiles plan to update data values after a DDL command + * executes. + * + * TODO: get rid of this ugly code and just go through the standard APIs. + * The only time we may still need this is to manage updating the empty + * key value, as we sometimes need to "go back through time" to adjust + * this. + * @author jtaylor + * @since 0.1 + */ +public class PostDDLCompiler { + private final PhoenixConnection connection; + + public PostDDLCompiler(PhoenixConnection connection) { + this.connection = connection; + } + + public MutationPlan compile(final List tableRefs, final byte[] emptyCF, final byte[] projectCF, final List deleteList, + final long timestamp) throws SQLException { + + return new MutationPlan() { + + @Override + public PhoenixConnection getConnection() { + return connection; + } + + @Override + public ParameterMetaData getParameterMetaData() { + return PhoenixParameterMetaData.EMPTY_PARAMETER_META_DATA; + } + + @Override + public ExplainPlan getExplainPlan() throws SQLException { + return ExplainPlan.EMPTY_PLAN; + } + + @Override + public MutationState execute() throws SQLException { + if (tableRefs.isEmpty()) { + return null; + } + boolean wasAutoCommit = connection.getAutoCommit(); + try { + connection.setAutoCommit(true); + SQLException sqlE = null; + if (deleteList == null && emptyCF == null) { + return new MutationState(0, connection); + } + /* + * Handles: + * 1) deletion of all rows for a DROP TABLE and subsequently deletion of all rows for a DROP INDEX; + * 2) deletion of all column values for a ALTER TABLE DROP COLUMN + * 3) updating the necessary rows to have an empty KV + */ + long totalMutationCount = 0; + for (final TableRef tableRef : tableRefs) { + Scan scan = new Scan(); + scan.setAttribute(UngroupedAggregateRegionObserver.UNGROUPED_AGG, QueryConstants.TRUE); + SelectStatement select = SelectStatement.COUNT_ONE; + // We need to use this tableRef + ColumnResolver resolver = new ColumnResolver() { + @Override + public List getTables() { + return Collections.singletonList(tableRef); + } + @Override + public ColumnRef resolveColumn(String schemaName, String tableName, String colName) throws SQLException { + PColumn column = tableName != null + ? tableRef.getTable().getColumnFamily(tableName).getColumn(colName) + : tableRef.getTable().getColumn(colName); + return new ColumnRef(tableRef, column.getPosition()); + } + }; + StatementContext context = new StatementContext(new PhoenixStatement(connection), resolver, Collections.emptyList(), scan); + ScanUtil.setTimeRange(scan, timestamp); + if (emptyCF != null) { + scan.setAttribute(UngroupedAggregateRegionObserver.EMPTY_CF, emptyCF); + } + ServerCache cache = null; + try { + if (deleteList != null) { + if (deleteList.isEmpty()) { + scan.setAttribute(UngroupedAggregateRegionObserver.DELETE_AGG, QueryConstants.TRUE); + // In the case of a row deletion, add index metadata so mutable secondary indexing works + /* TODO + ImmutableBytesWritable ptr = context.getTempPtr(); + tableRef.getTable().getIndexMaintainers(ptr); + if (ptr.getLength() > 0) { + IndexMetaDataCacheClient client = new IndexMetaDataCacheClient(connection, tableRef); + cache = client.addIndexMetadataCache(context.getScanRanges(), ptr); + byte[] uuidValue = cache.getId(); + scan.setAttribute(PhoenixIndexCodec.INDEX_UUID, uuidValue); + } + */ + } else { + // In the case of the empty key value column family changing, do not send the index + // metadata, as we're currently managing this from the client. It's possible for the + // data empty column family to stay the same, while the index empty column family + // changes. + PColumn column = deleteList.get(0); + if (emptyCF == null) { + scan.addColumn(column.getFamilyName().getBytes(), column.getName().getBytes()); + } + scan.setAttribute(UngroupedAggregateRegionObserver.DELETE_CF, column.getFamilyName().getBytes()); + scan.setAttribute(UngroupedAggregateRegionObserver.DELETE_CQ, column.getName().getBytes()); + } + } + List columnFamilies = Lists.newArrayListWithExpectedSize(tableRef.getTable().getColumnFamilies().size()); + if (projectCF == null) { + for (PColumnFamily family : tableRef.getTable().getColumnFamilies()) { + columnFamilies.add(family.getName().getBytes()); + } + } else { + columnFamilies.add(projectCF); + } + // Need to project all column families into the scan, since we haven't yet created our empty key value + RowProjector projector = ProjectionCompiler.compile(context, SelectStatement.COUNT_ONE, GroupBy.EMPTY_GROUP_BY); + // Explicitly project these column families and don't project the empty key value, + // since at this point we haven't added the empty key value everywhere. + if (columnFamilies != null) { + scan.getFamilyMap().clear(); + for (byte[] family : columnFamilies) { + scan.addFamily(family); + } + projector = new RowProjector(projector,false); + } + WhereCompiler.compile(context, select); // Push where clause into scan + QueryPlan plan = new AggregatePlan(context, select, tableRef, projector, null, OrderBy.EMPTY_ORDER_BY, null, GroupBy.EMPTY_GROUP_BY, null); + ResultIterator iterator = plan.iterator(); + try { + Tuple row = iterator.next(); + ImmutableBytesWritable ptr = context.getTempPtr(); + totalMutationCount += (Long)projector.getColumnProjector(0).getValue(row, PDataType.LONG, ptr); + } catch (SQLException e) { + sqlE = e; + } finally { + try { + iterator.close(); + } catch (SQLException e) { + if (sqlE == null) { + sqlE = e; + } else { + sqlE.setNextException(e); + } + } finally { + if (sqlE != null) { + throw sqlE; + } + } + } + } finally { + if (cache != null) { // Remove server cache if there is one + cache.close(); + } + } + + } + final long count = totalMutationCount; + return new MutationState(1, connection) { + @Override + public long getUpdateCount() { + return count; + } + }; + } finally { + if (!wasAutoCommit) connection.setAutoCommit(wasAutoCommit); + } + } + }; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/PostIndexDDLCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/PostIndexDDLCompiler.java new file mode 100644 index 00000000..69108d32 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/PostIndexDDLCompiler.java @@ -0,0 +1,119 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.ParameterMetaData; +import java.sql.PreparedStatement; +import java.sql.SQLException; + +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixParameterMetaData; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.IndexUtil; + + +/** + * Class that compiles plan to generate initial data values after a DDL command for + * index table. + */ +public class PostIndexDDLCompiler { + private final PhoenixConnection connection; + private final TableRef dataTableRef; + + public PostIndexDDLCompiler(PhoenixConnection connection, TableRef dataTableRef) { + this.connection = connection; + this.dataTableRef = dataTableRef; + } + + public MutationPlan compile(final PTable indexTable) throws SQLException { + return new MutationPlan() { + + @Override + public PhoenixConnection getConnection() { + return connection; + } + + @Override + public ParameterMetaData getParameterMetaData() { + return PhoenixParameterMetaData.EMPTY_PARAMETER_META_DATA; + } + + @Override + public ExplainPlan getExplainPlan() throws SQLException { + return ExplainPlan.EMPTY_PLAN; + } + + @Override + public MutationState execute() throws SQLException { + boolean wasAutoCommit = connection.getAutoCommit(); + try { + connection.setAutoCommit(true); + /* + * Handles: + * 1) Populate a newly created table with contents. + * 2) Activate the index by setting the INDEX_STATE to + */ + // NOTE: For first version, we would use a upsert/select to populate the new index table and + // returns synchronously. Creating an index on an existing table with large amount of data + // will as a result take a very very long time. + // In the long term, we should change this to an asynchronous process to populate the index + // that would allow the user to easily monitor the process of index creation. + StringBuilder indexColumns = new StringBuilder(); + StringBuilder dataColumns = new StringBuilder(); + for (PColumn col: dataTableRef.getTable().getColumns()) { + String indexColName = IndexUtil.getIndexColumnName(col); + try { + indexTable.getColumn(indexColName); + if (col.getFamilyName() != null) { + dataColumns.append('"').append(col.getFamilyName()).append("\"."); + } + dataColumns.append('"').append(col.getName()).append("\","); + indexColumns.append('"').append(indexColName).append("\","); + } catch (ColumnNotFoundException e) { + // Catch and ignore - means that this data column is not in the index + } + } + dataColumns.setLength(dataColumns.length()-1); + indexColumns.setLength(indexColumns.length()-1); + String schemaName = dataTableRef.getTable().getSchemaName().getString(); + String tableName = indexTable.getTableName().getString(); + + StringBuilder updateStmtStr = new StringBuilder(); + updateStmtStr.append("UPSERT /*+ NO_INDEX */ INTO ").append(schemaName.length() == 0 ? "" : '"' + schemaName + "\".").append('"').append(tableName).append("\"(") + .append(indexColumns).append(") SELECT ").append(dataColumns).append(" FROM ") + .append(schemaName.length() == 0 ? "" : '"' + schemaName + "\".").append('"').append(dataTableRef.getTable().getTableName().getString()).append('"'); + PreparedStatement updateStmt = connection.prepareStatement(updateStmtStr.toString()); + int rowsUpdated = 0; + updateStmt.execute(); + rowsUpdated = updateStmt.getUpdateCount(); + // Return number of rows built for index + return new MutationState(rowsUpdated, connection); + } finally { + if (!wasAutoCommit) connection.setAutoCommit(false); + } + } + }; + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ProjectionCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/ProjectionCompiler.java similarity index 82% rename from phoenix-core/src/main/java/com/salesforce/phoenix/compile/ProjectionCompiler.java rename to phoenix-core/src/main/java/org/apache/phoenix/compile/ProjectionCompiler.java index 624a71b7..13a70a94 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/ProjectionCompiler.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/ProjectionCompiler.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; import java.sql.SQLException; import java.util.ArrayList; @@ -42,39 +34,39 @@ import com.google.common.collect.Iterators; import com.google.common.collect.Lists; import com.google.common.collect.Sets; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.coprocessor.GroupedAggregateRegionObserver; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.expression.CoerceExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.aggregator.ClientAggregators; -import com.salesforce.phoenix.expression.aggregator.ServerAggregators; -import com.salesforce.phoenix.expression.function.SingleAggregateFunction; -import com.salesforce.phoenix.expression.visitor.SingleAggregateFunctionVisitor; -import com.salesforce.phoenix.parse.AliasedNode; -import com.salesforce.phoenix.parse.BindParseNode; -import com.salesforce.phoenix.parse.ColumnParseNode; -import com.salesforce.phoenix.parse.FamilyWildcardParseNode; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.parse.SequenceValueParseNode; -import com.salesforce.phoenix.parse.WildcardParseNode; -import com.salesforce.phoenix.schema.ArgumentTypeMismatchException; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.ColumnRef; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PColumnFamily; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDatum; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTable.ViewType; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.RowKeySchema; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.IndexUtil; -import com.salesforce.phoenix.util.SchemaUtil; -import com.salesforce.phoenix.util.SizedUtil; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.expression.CoerceExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.aggregator.ClientAggregators; +import org.apache.phoenix.expression.aggregator.ServerAggregators; +import org.apache.phoenix.expression.function.SingleAggregateFunction; +import org.apache.phoenix.expression.visitor.SingleAggregateFunctionVisitor; +import org.apache.phoenix.parse.AliasedNode; +import org.apache.phoenix.parse.BindParseNode; +import org.apache.phoenix.parse.ColumnParseNode; +import org.apache.phoenix.parse.FamilyWildcardParseNode; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.parse.SequenceValueParseNode; +import org.apache.phoenix.parse.WildcardParseNode; +import org.apache.phoenix.schema.ArgumentTypeMismatchException; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.ColumnRef; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PColumnFamily; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDatum; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTable.ViewType; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.RowKeySchema; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.IndexUtil; +import org.apache.phoenix.util.SchemaUtil; +import org.apache.phoenix.util.SizedUtil; /** diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/QueryCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/QueryCompiler.java similarity index 78% rename from phoenix-core/src/main/java/com/salesforce/phoenix/compile/QueryCompiler.java rename to phoenix-core/src/main/java/org/apache/phoenix/compile/QueryCompiler.java index 7784abb0..bafacf98 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/QueryCompiler.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/QueryCompiler.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; import java.sql.SQLException; import java.sql.SQLFeatureNotSupportedException; @@ -33,41 +25,40 @@ import java.util.List; import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; import org.apache.hadoop.hbase.util.Pair; - -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.JoinCompiler.JoinSpec; -import com.salesforce.phoenix.compile.JoinCompiler.JoinTable; -import com.salesforce.phoenix.compile.JoinCompiler.JoinedTableColumnResolver; -import com.salesforce.phoenix.compile.JoinCompiler.PTableWrapper; -import com.salesforce.phoenix.compile.JoinCompiler.ProjectedPTableWrapper; -import com.salesforce.phoenix.compile.OrderByCompiler.OrderBy; -import com.salesforce.phoenix.execute.AggregatePlan; -import com.salesforce.phoenix.execute.BasicQueryPlan; -import com.salesforce.phoenix.execute.DegenerateQueryPlan; -import com.salesforce.phoenix.execute.HashJoinPlan; -import com.salesforce.phoenix.execute.ScanPlan; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.join.HashJoinInfo; -import com.salesforce.phoenix.join.ScanProjector; -import com.salesforce.phoenix.parse.JoinTableNode.JoinType; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.SQLParser; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.AmbiguousColumnException; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.PDatum; -import com.salesforce.phoenix.schema.PIndexState; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.ScanUtil; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.JoinCompiler.JoinSpec; +import org.apache.phoenix.compile.JoinCompiler.JoinTable; +import org.apache.phoenix.compile.JoinCompiler.JoinedTableColumnResolver; +import org.apache.phoenix.compile.JoinCompiler.PTableWrapper; +import org.apache.phoenix.compile.JoinCompiler.ProjectedPTableWrapper; +import org.apache.phoenix.compile.OrderByCompiler.OrderBy; +import org.apache.phoenix.execute.AggregatePlan; +import org.apache.phoenix.execute.BasicQueryPlan; +import org.apache.phoenix.execute.DegenerateQueryPlan; +import org.apache.phoenix.execute.HashJoinPlan; +import org.apache.phoenix.execute.ScanPlan; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.join.HashJoinInfo; +import org.apache.phoenix.join.ScanProjector; +import org.apache.phoenix.parse.JoinTableNode.JoinType; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.SQLParser; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.AmbiguousColumnException; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.PDatum; +import org.apache.phoenix.schema.PIndexState; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.ScanUtil; @@ -260,9 +251,11 @@ protected BasicQueryPlan compileSingleQuery(StatementContext context, SelectStat return new DegenerateQueryPlan(context, select, tableRef); } PTable table = tableRef.getTable(); - ParseNode viewNode = SQLParser.parseCondition(table.getViewExpression()); - // Push VIEW expression into select - select = SelectStatement.create(select, viewNode); + if (table.getViewStatement() != null) { + ParseNode viewNode = new SQLParser(table.getViewStatement()).parseQuery().getWhere(); + // Push VIEW expression into select + select = select.combine(viewNode); + } Integer limit = LimitCompiler.compile(context, select); GroupBy groupBy = GroupByCompiler.compile(context, select); diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/QueryPlan.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/QueryPlan.java new file mode 100644 index 00000000..d41fb7f8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/QueryPlan.java @@ -0,0 +1,69 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.OrderByCompiler.OrderBy; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.parse.FilterableStatement; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.schema.TableRef; + + + +/** + * + * Interface for an executable query plan + * + * @author jtaylor + * @since 0.1 + */ +public interface QueryPlan extends StatementPlan { + /** + * Get a result iterator to iterate over the results + * @return result iterator for iterating over the results + * @throws SQLException + */ + public ResultIterator iterator() throws SQLException; + + public long getEstimatedSize(); + + // TODO: change once joins are supported + TableRef getTableRef(); + /** + * Returns projector used to formulate resultSet row + */ + RowProjector getProjector(); + + Integer getLimit(); + + OrderBy getOrderBy(); + + GroupBy getGroupBy(); + + List getSplits(); + + StatementContext getContext(); + + FilterableStatement getStatement(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/RowProjector.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/RowProjector.java new file mode 100644 index 00000000..6b6344c7 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/RowProjector.java @@ -0,0 +1,124 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.SQLException; +import java.util.*; + +import com.google.common.collect.Maps; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.util.SchemaUtil; + + +/** + * + * Class that manages a set of projected columns accessed through the zero-based + * column index for a SELECT clause projection. The column index may be looked up + * via the name using {@link #getColumnIndex(String)}. + * + * @author jtaylor + * @since 0.1 + */ +public class RowProjector { + public static final RowProjector EMPTY_PROJECTOR = new RowProjector(Collections.emptyList(),0, true); + + private final List columnProjectors; + private final Map reverseIndex; + private final boolean allCaseSensitive; + private final boolean someCaseSensitive; + private final int estimatedSize; + private final boolean isProjectEmptyKeyValue; + + public RowProjector(RowProjector projector, boolean isProjectEmptyKeyValue) { + this(projector.getColumnProjectors(), projector.getEstimatedRowByteSize(), isProjectEmptyKeyValue); + } + /** + * Construct RowProjector based on a list of ColumnProjectors. + * @param columnProjectors ordered list of ColumnProjectors corresponding to projected columns in SELECT clause + * aggregating coprocessor. Only required in the case of an aggregate query with a limit clause and otherwise may + * be null. + * @param estimatedRowSize + */ + public RowProjector(List columnProjectors, int estimatedRowSize, boolean isProjectEmptyKeyValue) { + this.columnProjectors = Collections.unmodifiableList(columnProjectors); + int position = columnProjectors.size(); + reverseIndex = Maps.newHashMapWithExpectedSize(position); + boolean allCaseSensitive = true; + boolean someCaseSensitive = false; + for (--position; position >= 0; position--) { + ColumnProjector colProjector = columnProjectors.get(position); + allCaseSensitive &= colProjector.isCaseSensitive(); + someCaseSensitive |= colProjector.isCaseSensitive(); + reverseIndex.put(colProjector.getName(), position); + } + this.allCaseSensitive = allCaseSensitive; + this.someCaseSensitive = someCaseSensitive; + this.estimatedSize = estimatedRowSize; + this.isProjectEmptyKeyValue = isProjectEmptyKeyValue; + } + + public boolean isProjectEmptyKeyValue() { + return isProjectEmptyKeyValue; + } + + public List getColumnProjectors() { + return columnProjectors; + } + + public int getColumnIndex(String name) throws SQLException { + if (!someCaseSensitive) { + name = SchemaUtil.normalizeIdentifier(name); + } + Integer index = reverseIndex.get(name); + if (index == null) { + if (!allCaseSensitive && someCaseSensitive) { + name = SchemaUtil.normalizeIdentifier(name); + index = reverseIndex.get(name); + } + if (index == null) { + throw new ColumnNotFoundException(name); + } + } + return index; + } + + public ColumnProjector getColumnProjector(int index) { + return columnProjectors.get(index); + } + + public int getColumnCount() { + return columnProjectors.size(); + } + + @Override + public String toString() { + StringBuilder buf = new StringBuilder("["); + for (ColumnProjector projector : columnProjectors) { + buf.append(projector.getExpression()); + buf.append(','); + } + buf.setCharAt(buf.length()-1, ']'); + return buf.toString(); + } + + public int getEstimatedRowByteSize() { + return estimatedSize; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/ScanRanges.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/ScanRanges.java new file mode 100644 index 00000000..44f4473e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/ScanRanges.java @@ -0,0 +1,185 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.util.Collections; +import java.util.List; + +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Lists; +import org.apache.phoenix.filter.SkipScanFilter; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.schema.RowKeySchema; +import org.apache.phoenix.util.ScanUtil; + + +public class ScanRanges { + private static final List> EVERYTHING_RANGES = Collections.>emptyList(); + private static final List> NOTHING_RANGES = Collections.>singletonList(Collections.singletonList(KeyRange.EMPTY_RANGE)); + public static final ScanRanges EVERYTHING = new ScanRanges(EVERYTHING_RANGES,null,false); + public static final ScanRanges NOTHING = new ScanRanges(NOTHING_RANGES,null,false); + + public static ScanRanges create(List> ranges, RowKeySchema schema) { + return create(ranges, schema, false); + } + + public static ScanRanges create(List> ranges, RowKeySchema schema, boolean forceRangeScan) { + if (ranges.isEmpty()) { + return EVERYTHING; + } else if (ranges.size() == 1 && ranges.get(0).size() == 1 && ranges.get(0).get(0) == KeyRange.EMPTY_RANGE) { + return NOTHING; + } + return new ScanRanges(ranges, schema, forceRangeScan); + } + + private SkipScanFilter filter; + private final List> ranges; + private final RowKeySchema schema; + private final boolean forceRangeScan; + + private ScanRanges (List> ranges, RowKeySchema schema, boolean forceRangeScan) { + List> sortedRanges = Lists.newArrayListWithExpectedSize(ranges.size()); + for (int i = 0; i < ranges.size(); i++) { + List sorted = Lists.newArrayList(ranges.get(i)); + Collections.sort(sorted, KeyRange.COMPARATOR); + sortedRanges.add(ImmutableList.copyOf(sorted)); + } + this.ranges = ImmutableList.copyOf(sortedRanges); + this.schema = schema; + if (schema != null && !ranges.isEmpty()) { + this.filter = new SkipScanFilter(this.ranges, schema); + } + this.forceRangeScan = forceRangeScan; + } + + public SkipScanFilter getSkipScanFilter() { + return filter; + } + + public List> getRanges() { + return ranges; + } + + public RowKeySchema getSchema() { + return schema; + } + + public boolean isEverything() { + return this == EVERYTHING; + } + + public boolean isDegenerate() { + return this == NOTHING; + } + + /** + * Use SkipScanFilter under two circumstances: + * 1) If we have multiple ranges for a given key slot (use of IN) + * 2) If we have a range (i.e. not a single/point key) that is + * not the last key slot + */ + public boolean useSkipScanFilter() { + if (forceRangeScan) { + return false; + } + boolean hasRangeKey = false, useSkipScan = false; + for (List orRanges : ranges) { + useSkipScan |= orRanges.size() > 1 | hasRangeKey; + if (useSkipScan) { + return true; + } + for (KeyRange range : orRanges) { + hasRangeKey |= !range.isSingleKey(); + } + } + return false; + } + + /** + * @return true if this represents the full key to a single row + */ + public boolean isSingleRowScan() { + if (schema == null || ranges.size() < schema.getMaxFields()) { + return false; + } + boolean isSingleKey = true; + for (List orRanges : ranges) { + if (orRanges.size() > 1) { + return false; + } + isSingleKey &= orRanges.get(0).isSingleKey(); + } + return isSingleKey; + } + + public void setScanStartStopRow(Scan scan) { + if (isEverything()) { + return; + } + if (isDegenerate()) { + scan.setStartRow(KeyRange.EMPTY_RANGE.getLowerRange()); + scan.setStopRow(KeyRange.EMPTY_RANGE.getUpperRange()); + return; + } + + byte[] expectedKey; + expectedKey = ScanUtil.getMinKey(schema, ranges); + if (expectedKey != null) { + scan.setStartRow(expectedKey); + } + expectedKey = ScanUtil.getMaxKey(schema, ranges); + if (expectedKey != null) { + scan.setStopRow(expectedKey); + } + } + + public static final ImmutableBytesWritable UNBOUND = new ImmutableBytesWritable(KeyRange.UNBOUND); + + /** + * Return true if the range formed by the lowerInclusiveKey and upperExclusiveKey + * intersects with any of the scan ranges and false otherwise. We cannot pass in + * a KeyRange here, because the underlying compare functions expect lower inclusive + * and upper exclusive keys. We cannot get their next key because the key must + * conform to the row key schema and if a null byte is added to a lower inclusive + * key, it's no longer a valid, real key. + * @param lowerInclusiveKey lower inclusive key + * @param upperExclusiveKey upper exclusive key + * @return true if the scan range intersects with the specified lower/upper key + * range + */ + public boolean intersect(byte[] lowerInclusiveKey, byte[] upperExclusiveKey) { + if (isEverything()) { + return true; + } + if (isDegenerate()) { + return false; + } + return filter.hasIntersect(lowerInclusiveKey, upperExclusiveKey); + } + + @Override + public String toString() { + return "ScanRanges[" + ranges.toString() + "]"; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/SequenceManager.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/SequenceManager.java new file mode 100644 index 00000000..665c33b3 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/SequenceManager.java @@ -0,0 +1,194 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.SQLException; +import java.util.BitSet; +import java.util.Collections; +import java.util.List; +import java.util.Map; + +import org.apache.hadoop.hbase.HConstants; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import com.google.common.collect.Lists; +import com.google.common.collect.Maps; +import org.apache.phoenix.expression.BaseTerminalExpression; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.parse.SequenceValueParseNode; +import org.apache.phoenix.parse.TableName; +import org.apache.phoenix.query.ConnectionQueryServices; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PName; +import org.apache.phoenix.schema.SequenceKey; +import org.apache.phoenix.schema.tuple.Tuple; + +public class SequenceManager { + private final PhoenixStatement statement; + private int[] sequencePosition; + private long[] srcSequenceValues; + private long[] dstSequenceValues; + private SQLException[] sqlExceptions; + private List nextSequences; + private List currentSequences; + private Map sequenceMap; + private BitSet isNextSequence; + + public SequenceManager(PhoenixStatement statement) { + this.statement = statement; + } + + public int getSequenceCount() { + return sequenceMap == null ? 0 : sequenceMap.size(); + } + + private void setSequenceValues() throws SQLException { + SQLException eTop = null; + for (int i = 0; i < sqlExceptions.length; i++) { + SQLException e = sqlExceptions[i]; + if (e != null) { + if (eTop == null) { + eTop = e; + } else { + e.setNextException(eTop.getNextException()); + eTop.setNextException(e); + } + } else { + dstSequenceValues[sequencePosition[i]] = srcSequenceValues[i]; + } + } + if (eTop != null) { + throw eTop; + } + } + + public void incrementSequenceValues() throws SQLException { + if (sequenceMap == null) { + return; + } + Long scn = statement.getConnection().getSCN(); + long timestamp = scn == null ? HConstants.LATEST_TIMESTAMP : scn; + ConnectionQueryServices services = this.statement.getConnection().getQueryServices(); + services.incrementSequenceValues(nextSequences, timestamp, srcSequenceValues, sqlExceptions); + setSequenceValues(); + int offset = nextSequences.size(); + for (int i = 0; i < currentSequences.size(); i++) { + dstSequenceValues[sequencePosition[offset+i]] = services.getSequenceValue(currentSequences.get(i), timestamp); + } + } + + public SequenceValueExpression newSequenceReference(SequenceValueParseNode node) { + if (sequenceMap == null) { + sequenceMap = Maps.newHashMap(); + isNextSequence = new BitSet(); + } + PName tenantName = statement.getConnection().getTenantId(); + String tenantId = tenantName == null ? null : tenantName.getString(); + TableName tableName = node.getTableName(); + SequenceKey key = new SequenceKey(tenantId, tableName.getSchemaName(), tableName.getTableName()); + SequenceValueExpression expression = sequenceMap.get(key); + if (expression == null) { + int index = sequenceMap.size(); + expression = new SequenceValueExpression(index); + sequenceMap.put(key, expression); + } + // If we see a NEXT and a CURRENT, treat the CURRENT just like a NEXT + if (node.getOp() == SequenceValueParseNode.Op.NEXT_VALUE) { + isNextSequence.set(expression.getIndex()); + } + + return expression; + } + + public void initSequences() throws SQLException { + if (sequenceMap == null) { + return; + } + int maxSize = sequenceMap.size(); + dstSequenceValues = new long[maxSize]; + sequencePosition = new int[maxSize]; + nextSequences = Lists.newArrayListWithExpectedSize(maxSize); + currentSequences = Lists.newArrayListWithExpectedSize(maxSize); + for (Map.Entry entry : sequenceMap.entrySet()) { + if (isNextSequence.get(entry.getValue().getIndex())) { + nextSequences.add(entry.getKey()); + } else { + currentSequences.add(entry.getKey()); + } + } + srcSequenceValues = new long[nextSequences.size()]; + sqlExceptions = new SQLException[nextSequences.size()]; + Collections.sort(nextSequences); + // Create reverse indexes + for (int i = 0; i < nextSequences.size(); i++) { + sequencePosition[i] = sequenceMap.get(nextSequences.get(i)).getIndex(); + } + int offset = nextSequences.size(); + for (int i = 0; i < currentSequences.size(); i++) { + sequencePosition[i+offset] = sequenceMap.get(currentSequences.get(i)).getIndex(); + } + ConnectionQueryServices services = this.statement.getConnection().getQueryServices(); + Long scn = statement.getConnection().getSCN(); + long timestamp = scn == null ? HConstants.LATEST_TIMESTAMP : scn; + services.reserveSequenceValues(nextSequences, timestamp, srcSequenceValues, sqlExceptions); + setSequenceValues(); + } + + private class SequenceValueExpression extends BaseTerminalExpression { + private final int index; + private final byte[] valueBuffer = new byte[PDataType.LONG.getByteSize()]; + + private SequenceValueExpression(int index) { + this.index = index; + } + + public int getIndex() { + return index; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + PDataType.LONG.getCodec().encodeLong(dstSequenceValues[index], valueBuffer, 0); + ptr.set(valueBuffer); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.LONG; + } + + @Override + public boolean isNullable() { + return false; + } + + @Override + public boolean isDeterministic() { + return false; + } + + @Override + public boolean isStateless() { + return true; + } + + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/StatementContext.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/StatementContext.java new file mode 100644 index 00000000..57365363 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/StatementContext.java @@ -0,0 +1,227 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.SQLException; +import java.text.Format; +import java.util.List; + +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.filter.FilterList; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.schema.MetaDataClient; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.DateUtil; +import org.apache.phoenix.util.NumberUtil; +import org.apache.phoenix.util.ScanUtil; + + +/** + * + * Class that keeps common state used across processing the various clauses in a + * top level JDBC statement such as SELECT, UPSERT, DELETE, etc. + * + * @author jtaylor + * @since 0.1 + */ +public class StatementContext { + private ColumnResolver resolver; + private final BindManager binds; + private final Scan scan; + private final ExpressionManager expressions; + private final AggregationManager aggregates; + private final String dateFormat; + private final Format dateFormatter; + private final Format dateParser; + private final String numberFormat; + private final ImmutableBytesWritable tempPtr; + private final PhoenixStatement statement; + + private long currentTime = QueryConstants.UNSET_TIMESTAMP; + private ScanRanges scanRanges = ScanRanges.EVERYTHING; + private KeyRange minMaxRange = null; + private final SequenceManager sequences; + + private TableRef currentTable; + + public StatementContext(PhoenixStatement statement, ColumnResolver resolver, List binds, Scan scan) { + this.statement = statement; + this.resolver = resolver; + this.scan = scan; + this.binds = new BindManager(binds); + this.aggregates = new AggregationManager(); + this.expressions = new ExpressionManager(); + PhoenixConnection connection = statement.getConnection(); + this.dateFormat = connection.getQueryServices().getProps().get(QueryServices.DATE_FORMAT_ATTRIB, DateUtil.DEFAULT_DATE_FORMAT); + this.dateFormatter = DateUtil.getDateFormatter(dateFormat); + this.dateParser = DateUtil.getDateParser(dateFormat); + this.numberFormat = connection.getQueryServices().getProps().get(QueryServices.NUMBER_FORMAT_ATTRIB, NumberUtil.DEFAULT_NUMBER_FORMAT); + this.tempPtr = new ImmutableBytesWritable(); + this.currentTable = resolver != null && !resolver.getTables().isEmpty() ? resolver.getTables().get(0) : null; + this.sequences = new SequenceManager(statement); + } + + public String getDateFormat() { + return dateFormat; + } + + public Format getDateFormatter() { + return dateFormatter; + } + + public Format getDateParser() { + return dateParser; + } + + public String getNumberFormat() { + return numberFormat; + } + + public Scan getScan() { + return scan; + } + + public BindManager getBindManager() { + return binds; + } + + public TableRef getCurrentTable() { + return currentTable; + } + + public void setCurrentTable(TableRef table) { + this.currentTable = table; + } + + public AggregationManager getAggregationManager() { + return aggregates; + } + + public ColumnResolver getResolver() { + return resolver; + } + + public void setResolver(ColumnResolver resolver) { + this.resolver = resolver; + } + + public ExpressionManager getExpressionManager() { + return expressions; + } + + + public ImmutableBytesWritable getTempPtr() { + return tempPtr; + } + + public ScanRanges getScanRanges() { + return this.scanRanges; + } + + public void setScanRanges(ScanRanges scanRanges) { + setScanRanges(scanRanges, null); + } + + public void setScanRanges(ScanRanges scanRanges, KeyRange minMaxRange) { + this.scanRanges = scanRanges; + this.scanRanges.setScanStartStopRow(scan); + PTable table = this.getCurrentTable().getTable(); + if (minMaxRange != null) { + // Ensure minMaxRange is lower inclusive and upper exclusive, as that's + // what we need to intersect against for the HBase scan. + byte[] lowerRange = minMaxRange.getLowerRange(); + if (!minMaxRange.lowerUnbound()) { + if (!minMaxRange.isLowerInclusive()) { + lowerRange = ScanUtil.nextKey(lowerRange, table, tempPtr); + } + } + + byte[] upperRange = minMaxRange.getUpperRange(); + if (!minMaxRange.upperUnbound()) { + if (minMaxRange.isUpperInclusive()) { + upperRange = ScanUtil.nextKey(upperRange, table, tempPtr); + } + } + if (minMaxRange.getLowerRange() != lowerRange || minMaxRange.getUpperRange() != upperRange) { + minMaxRange = KeyRange.getKeyRange(lowerRange, true, upperRange, false); + } + // If we're not salting, we can intersect this now with the scan range. + // Otherwise, we have to wait to do this when we chunk up the scan. + if (table.getBucketNum() == null) { + minMaxRange = minMaxRange.intersect(KeyRange.getKeyRange(scan.getStartRow(), scan.getStopRow())); + scan.setStartRow(minMaxRange.getLowerRange()); + scan.setStopRow(minMaxRange.getUpperRange()); + } + this.minMaxRange = minMaxRange; + } + } + + public PhoenixConnection getConnection() { + return statement.getConnection(); + } + + public PhoenixStatement getStatement() { + return statement; + } + + public long getCurrentTime() throws SQLException { + long ts = this.getCurrentTable().getTimeStamp(); + if (ts != QueryConstants.UNSET_TIMESTAMP) { + return ts; + } + if (currentTime != QueryConstants.UNSET_TIMESTAMP) { + return currentTime; + } + /* + * For an UPSERT VALUES where autocommit off, we won't hit the server until the commit. + * However, if the statement has a CURRENT_DATE() call as a value, we need to know the + * current time at execution time. In that case, we'll call MetaDataClient.updateCache + * purely to bind the current time based on the server time. + */ + PTable table = this.getCurrentTable().getTable(); + PhoenixConnection connection = getConnection(); + MetaDataClient client = new MetaDataClient(connection); + currentTime = client.getCurrentTime(table.getSchemaName().getString(), table.getTableName().getString()); + return currentTime; + } + + /** + * Get the key range derived from row value constructor usage in where clause. These are orthogonal to the ScanRanges + * and form a range for which each scan is intersected against. + */ + public KeyRange getMinMaxRange () { + return minMaxRange; + } + + public boolean isSingleRowScan() { + return this.getScanRanges().isSingleRowScan() && ! (this.getScan().getFilter() instanceof FilterList); + } + + public SequenceManager getSequenceManager(){ + return sequences; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/StatementNormalizer.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/StatementNormalizer.java new file mode 100644 index 00000000..8eb769e9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/StatementNormalizer.java @@ -0,0 +1,102 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.SQLException; +import java.util.List; + +import com.google.common.collect.Lists; +import org.apache.phoenix.parse.BetweenParseNode; +import org.apache.phoenix.parse.ColumnParseNode; +import org.apache.phoenix.parse.ComparisonParseNode; +import org.apache.phoenix.parse.LessThanOrEqualParseNode; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.ParseNodeRewriter; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.util.SchemaUtil; + + +/** + * + * Class that creates a new select statement ensuring that a literal always occurs + * on the RHS (i.e. if literal found on the LHS, then the operator is reversed and + * the literal is put on the RHS) + * + * @author jtaylor + * @since 0.1 + */ +public class StatementNormalizer extends ParseNodeRewriter { + private boolean useFullNameForAlias; + + public StatementNormalizer(ColumnResolver resolver, int expectedAliasCount, boolean useFullNameForAlias) { + super(resolver, expectedAliasCount); + this.useFullNameForAlias = useFullNameForAlias; + } + + public static ParseNode normalize(ParseNode where, ColumnResolver resolver) throws SQLException { + return rewrite(where, new StatementNormalizer(resolver, 0, false)); + } + + /** + * Rewrite the select statement by switching any constants to the right hand side + * of the expression. + * @param statement the select statement + * @param resolver + * @return new select statement + * @throws SQLException + */ + public static SelectStatement normalize(SelectStatement statement, ColumnResolver resolver) throws SQLException { + return rewrite(statement, new StatementNormalizer(resolver, statement.getSelect().size(), statement.getFrom().size() > 1)); + } + + @Override + public ParseNode visitLeave(ComparisonParseNode node, List nodes) throws SQLException { + if (nodes.get(0).isStateless() && !nodes.get(1).isStateless()) { + List normNodes = Lists.newArrayListWithExpectedSize(2); + normNodes.add(nodes.get(1)); + normNodes.add(nodes.get(0)); + nodes = normNodes; + node = NODE_FACTORY.comparison(node.getInvertFilterOp(), nodes.get(0), nodes.get(1)); + } + return super.visitLeave(node, nodes); + } + + @Override + public ParseNode visitLeave(final BetweenParseNode node, List nodes) throws SQLException { + + LessThanOrEqualParseNode lhsNode = NODE_FACTORY.lte(node.getChildren().get(1), node.getChildren().get(0)); + LessThanOrEqualParseNode rhsNode = NODE_FACTORY.lte(node.getChildren().get(0), node.getChildren().get(2)); + List parseNodes = Lists.newArrayListWithExpectedSize(2); + parseNodes.add(this.visitLeave(lhsNode, lhsNode.getChildren())); + parseNodes.add(this.visitLeave(rhsNode, rhsNode.getChildren())); + return super.visitLeave(node, parseNodes); + } + + @Override + public ParseNode visit(ColumnParseNode node) throws SQLException { + if (useFullNameForAlias + && node.getAlias() != null + && node.getTableName() != null + && SchemaUtil.normalizeIdentifier(node.getAlias()).equals(node.getName())) { + node = NODE_FACTORY.column(NODE_FACTORY.table(node.getSchemaName(), node.getTableName()), node.isCaseSensitive() ? '"' + node.getName() + '"' : node.getName(), node.getFullName()); + } + return super.visit(node); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/StatementPlan.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/StatementPlan.java new file mode 100644 index 00000000..e41e86a9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/StatementPlan.java @@ -0,0 +1,47 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.ParameterMetaData; +import java.sql.SQLException; + +import org.apache.phoenix.jdbc.PhoenixParameterMetaData; + + +public interface StatementPlan { + public final StatementPlan EMPTY_PLAN = new StatementPlan() { + @Override + public ParameterMetaData getParameterMetaData() { + return new PhoenixParameterMetaData(0); + } + + @Override + public ExplainPlan getExplainPlan() throws SQLException { + return ExplainPlan.EMPTY_PLAN; + } + }; + + /** + * Returns the ParameterMetaData for the statement + */ + ParameterMetaData getParameterMetaData(); + + ExplainPlan getExplainPlan() throws SQLException; +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/TrackOrderPreservingExpressionCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/TrackOrderPreservingExpressionCompiler.java similarity index 85% rename from phoenix-core/src/main/java/com/salesforce/phoenix/compile/TrackOrderPreservingExpressionCompiler.java rename to phoenix-core/src/main/java/org/apache/phoenix/compile/TrackOrderPreservingExpressionCompiler.java index b119031b..a0c4674f 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/TrackOrderPreservingExpressionCompiler.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/TrackOrderPreservingExpressionCompiler.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.compile; +package org.apache.phoenix.compile; import java.sql.SQLException; import java.util.Collections; @@ -7,19 +7,19 @@ import com.google.common.base.Objects; import com.google.common.collect.Lists; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.function.FunctionExpression; -import com.salesforce.phoenix.expression.function.FunctionExpression.OrderPreserving; -import com.salesforce.phoenix.parse.CaseParseNode; -import com.salesforce.phoenix.parse.ColumnParseNode; -import com.salesforce.phoenix.parse.DivideParseNode; -import com.salesforce.phoenix.parse.MultiplyParseNode; -import com.salesforce.phoenix.parse.SubtractParseNode; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.ColumnRef; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.function.FunctionExpression; +import org.apache.phoenix.expression.function.FunctionExpression.OrderPreserving; +import org.apache.phoenix.parse.CaseParseNode; +import org.apache.phoenix.parse.ColumnParseNode; +import org.apache.phoenix.parse.DivideParseNode; +import org.apache.phoenix.parse.MultiplyParseNode; +import org.apache.phoenix.parse.SubtractParseNode; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.ColumnRef; +import org.apache.phoenix.util.SchemaUtil; /** * Visitor that builds the expressions of a GROUP BY and ORDER BY clause. While traversing @@ -40,6 +40,7 @@ public enum Ordering {ORDERED, UNORDERED}; private OrderPreserving orderPreserving = OrderPreserving.YES; private ColumnRef columnRef; private boolean isOrderPreserving = true; + private Boolean isReverse; TrackOrderPreservingExpressionCompiler(StatementContext context, GroupBy groupBy, int expectedEntrySize, Ordering ordering) { super(context, groupBy); @@ -48,6 +49,9 @@ public enum Ordering {ORDERED, UNORDERED}; this.ordering = ordering; } + public Boolean isReverse() { + return isReverse; + } public boolean isOrderPreserving() { if (!isOrderPreserving) { @@ -158,7 +162,17 @@ public boolean addEntry(Expression expression, ColumnModifier modifier) { // If the expression is sorted in a different order than the specified sort order // then the expressions are not order preserving. if (!Objects.equal(expression.getColumnModifier(), modifier)) { - orderPreserving = OrderPreserving.NO; + if (isReverse == null) { + isReverse = true; + } else if (!isReverse){ + orderPreserving = OrderPreserving.NO; + } + } else { + if (isReverse == null) { + isReverse = false; + } else if (isReverse){ + orderPreserving = OrderPreserving.NO; + } } return addEntry(expression); } diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/UpsertCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/UpsertCompiler.java similarity index 89% rename from phoenix-core/src/main/java/com/salesforce/phoenix/compile/UpsertCompiler.java rename to phoenix-core/src/main/java/org/apache/phoenix/compile/UpsertCompiler.java index e187bdd7..7f121173 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/UpsertCompiler.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/UpsertCompiler.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; import static com.google.common.collect.Lists.newArrayListWithCapacity; @@ -40,63 +32,63 @@ import org.apache.hadoop.hbase.KeyValue; import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.hbase.util.Bytes; +import org.apache.phoenix.cache.ServerCacheClient.ServerCache; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.OrderByCompiler.OrderBy; +import org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.execute.AggregatePlan; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.index.IndexMetaDataCacheClient; +import org.apache.phoenix.index.PhoenixIndexCodec; +import org.apache.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixResultSet; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.optimize.QueryOptimizer; +import org.apache.phoenix.parse.AliasedNode; +import org.apache.phoenix.parse.BindParseNode; +import org.apache.phoenix.parse.ColumnName; +import org.apache.phoenix.parse.ColumnParseNode; +import org.apache.phoenix.parse.ComparisonParseNode; +import org.apache.phoenix.parse.HintNode; +import org.apache.phoenix.parse.HintNode.Hint; +import org.apache.phoenix.parse.IsNullParseNode; +import org.apache.phoenix.parse.LiteralParseNode; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.SQLParser; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.parse.SequenceValueParseNode; +import org.apache.phoenix.parse.UpsertStatement; +import org.apache.phoenix.query.ConnectionQueryServices; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.ColumnRef; +import org.apache.phoenix.schema.ConstraintViolationException; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PColumnImpl; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTable.ViewType; +import org.apache.phoenix.schema.PTableImpl; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.ReadOnlyTableException; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.schema.TypeMismatchException; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.SchemaUtil; import com.google.common.collect.Lists; import com.google.common.collect.Maps; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.cache.ServerCacheClient.ServerCache; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.OrderByCompiler.OrderBy; -import com.salesforce.phoenix.coprocessor.UngroupedAggregateRegionObserver; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.execute.AggregatePlan; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.index.IndexMetaDataCacheClient; -import com.salesforce.phoenix.index.PhoenixIndexCodec; -import com.salesforce.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixResultSet; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.optimize.QueryOptimizer; -import com.salesforce.phoenix.parse.AliasedNode; -import com.salesforce.phoenix.parse.BindParseNode; -import com.salesforce.phoenix.parse.ColumnName; -import com.salesforce.phoenix.parse.ColumnParseNode; -import com.salesforce.phoenix.parse.ComparisonParseNode; -import com.salesforce.phoenix.parse.HintNode; -import com.salesforce.phoenix.parse.HintNode.Hint; -import com.salesforce.phoenix.parse.IsNullParseNode; -import com.salesforce.phoenix.parse.LiteralParseNode; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.SQLParser; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.parse.SequenceValueParseNode; -import com.salesforce.phoenix.parse.UpsertStatement; -import com.salesforce.phoenix.query.ConnectionQueryServices; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.ColumnRef; -import com.salesforce.phoenix.schema.ConstraintViolationException; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PColumnImpl; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTable.ViewType; -import com.salesforce.phoenix.schema.PTableImpl; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.ReadOnlyTableException; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.schema.TypeMismatchException; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.SchemaUtil; public class UpsertCompiler { private static void setValues(byte[][] values, int[] pkSlotIndex, int[] columnIndexes, PTable table, Map> mutation) { @@ -240,7 +232,7 @@ public MutationPlan compile(UpsertStatement upsert) throws SQLException { if (table.getViewType() == ViewType.UPDATABLE) { StatementContext context = new StatementContext(statement, resolver, this.statement.getParameters(), new Scan()); ViewValuesMapBuilder builder = new ViewValuesMapBuilder(context); - ParseNode viewNode = SQLParser.parseCondition(table.getViewExpression()); + ParseNode viewNode = new SQLParser(table.getViewStatement()).parseQuery().getWhere(); viewNode.accept(builder); addViewColumns = builder.getViewColumns(); } diff --git a/phoenix-core/src/main/java/org/apache/phoenix/compile/WhereCompiler.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/WhereCompiler.java new file mode 100644 index 00000000..6f76f0a5 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/WhereCompiler.java @@ -0,0 +1,214 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; + +import java.sql.SQLException; +import java.sql.SQLFeatureNotSupportedException; +import java.util.Iterator; +import java.util.Set; + +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.filter.Filter; + +import com.google.common.collect.Iterators; +import com.google.common.collect.Sets; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.KeyValueColumnExpression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.visitor.KeyValueExpressionVisitor; +import org.apache.phoenix.filter.MultiCFCQKeyValueComparisonFilter; +import org.apache.phoenix.filter.MultiCQKeyValueComparisonFilter; +import org.apache.phoenix.filter.RowKeyComparisonFilter; +import org.apache.phoenix.filter.SingleCFCQKeyValueComparisonFilter; +import org.apache.phoenix.filter.SingleCQKeyValueComparisonFilter; +import org.apache.phoenix.parse.ColumnParseNode; +import org.apache.phoenix.parse.FilterableStatement; +import org.apache.phoenix.parse.HintNode.Hint; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.ParseNodeFactory; +import org.apache.phoenix.schema.AmbiguousColumnException; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.ColumnRef; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.TypeMismatchException; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.ScanUtil; +import org.apache.phoenix.util.SchemaUtil; + + +/** + * + * Class to build the filter of a scan + * + * @author jtaylor + * @since 0.1 + */ +public class WhereCompiler { + protected static final ParseNodeFactory NODE_FACTORY = new ParseNodeFactory(); + + private WhereCompiler() { + } + + /** + * Pushes where clause filter expressions into scan by building and setting a filter. + * @param context the shared context during query compilation + * @param statement TODO + * @throws SQLException if mismatched types are found, bind value do not match binds, + * or invalid function arguments are encountered. + * @throws SQLFeatureNotSupportedException if an unsupported expression is encountered. + * @throws ColumnNotFoundException if column name could not be resolved + * @throws AmbiguousColumnException if an unaliased column name is ambiguous across multiple tables + */ + public static Expression compile(StatementContext context, FilterableStatement statement) throws SQLException { + return compileWhereClause(context, statement, Sets.newHashSet()); + } + + /** + * Used for testing to get access to the expressions that were used to form the start/stop key of the scan + * @param statement TODO + */ + public static Expression compileWhereClause(StatementContext context, FilterableStatement statement, + Set extractedNodes) throws SQLException { + WhereExpressionCompiler whereCompiler = new WhereExpressionCompiler(context); + ParseNode where = statement.getWhere(); + Expression expression = where == null ? LiteralExpression.newConstant(true,PDataType.BOOLEAN,true) : where.accept(whereCompiler); + if (whereCompiler.isAggregate()) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.AGGREGATE_IN_WHERE).build().buildException(); + } + if (expression.getDataType() != PDataType.BOOLEAN) { + throw TypeMismatchException.newException(PDataType.BOOLEAN, expression.getDataType(), expression.toString()); + } + + expression = WhereOptimizer.pushKeyExpressionsToScan(context, statement, expression, extractedNodes); + setScanFilter(context, statement, expression, whereCompiler.disambiguateWithFamily); + + return expression; + } + + private static class WhereExpressionCompiler extends ExpressionCompiler { + private boolean disambiguateWithFamily; + + WhereExpressionCompiler(StatementContext context) { + super(context); + } + + @Override + protected ColumnRef resolveColumn(ColumnParseNode node) throws SQLException { + ColumnRef ref = super.resolveColumn(node); + PTable table = ref.getTable(); + // Track if we need to compare KeyValue during filter evaluation + // using column family. If the column qualifier is enough, we + // just use that. + try { + if (!SchemaUtil.isPKColumn(ref.getColumn())) { + table.getColumn(ref.getColumn().getName().getString()); + } + } catch (AmbiguousColumnException e) { + disambiguateWithFamily = true; + } + return ref; + } + } + + private static final class Counter { + public enum Count {NONE, SINGLE, MULTIPLE}; + private Count count = Count.NONE; + private KeyValueColumnExpression column; + + public void increment(KeyValueColumnExpression column) { + switch (count) { + case NONE: + count = Count.SINGLE; + this.column = column; + break; + case SINGLE: + count = column.equals(this.column) ? Count.SINGLE : Count.MULTIPLE; + break; + case MULTIPLE: + break; + + } + } + public Count getCount() { + return count; + } + } + + /** + * Sets the start/stop key range based on the whereClause expression. + * @param context the shared context during query compilation + * @param whereClause the final where clause expression. + */ + private static void setScanFilter(StatementContext context, FilterableStatement statement, Expression whereClause, boolean disambiguateWithFamily) { + Filter filter = null; + Scan scan = context.getScan(); + assert scan.getFilter() == null; + + if (LiteralExpression.isFalse(whereClause)) { + context.setScanRanges(ScanRanges.NOTHING); + } else if (whereClause != null && !LiteralExpression.isTrue(whereClause)) { + final Counter counter = new Counter(); + whereClause.accept(new KeyValueExpressionVisitor() { + + @Override + public Iterator defaultIterator(Expression node) { + // Stop traversal once we've found multiple KeyValue columns + if (counter.getCount() == Counter.Count.MULTIPLE) { + return Iterators.emptyIterator(); + } + return super.defaultIterator(node); + } + + @Override + public Void visit(KeyValueColumnExpression expression) { + counter.increment(expression); + return null; + } + }); + switch (counter.getCount()) { + case NONE: + PTable table = context.getResolver().getTables().get(0).getTable(); + byte[] essentialCF = table.getType() == PTableType.VIEW + ? ByteUtil.EMPTY_BYTE_ARRAY + : SchemaUtil.getEmptyColumnFamily(table.getColumnFamilies()); + filter = new RowKeyComparisonFilter(whereClause, essentialCF); + break; + case SINGLE: + filter = disambiguateWithFamily ? new SingleCFCQKeyValueComparisonFilter(whereClause) : new SingleCQKeyValueComparisonFilter(whereClause); + break; + case MULTIPLE: + filter = disambiguateWithFamily ? new MultiCFCQKeyValueComparisonFilter(whereClause) : new MultiCQKeyValueComparisonFilter(whereClause); + break; + } + } + + scan.setFilter(filter); + ScanRanges scanRanges = context.getScanRanges(); + boolean forcedSkipScan = statement.getHint().hasHint(Hint.SKIP_SCAN); + boolean forcedRangeScan = statement.getHint().hasHint(Hint.RANGE_SCAN); + if (forcedSkipScan || (scanRanges.useSkipScanFilter() && !forcedRangeScan)) { + ScanUtil.andFilterAtBeginning(scan, scanRanges.getSkipScanFilter()); + } + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/WhereOptimizer.java b/phoenix-core/src/main/java/org/apache/phoenix/compile/WhereOptimizer.java similarity index 93% rename from phoenix-core/src/main/java/com/salesforce/phoenix/compile/WhereOptimizer.java rename to phoenix-core/src/main/java/org/apache/phoenix/compile/WhereOptimizer.java index 9406a198..552ff95c 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/compile/WhereOptimizer.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/compile/WhereOptimizer.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. +/* + * Copyright 2010 The Apache Software Foundation * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. + * http://www.apache.org/licenses/LICENSE-2.0 * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.compile; + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.compile; import static java.util.Collections.singletonList; @@ -44,37 +36,37 @@ import com.google.common.base.Preconditions; import com.google.common.collect.Iterators; import com.google.common.collect.Lists; -import com.salesforce.phoenix.expression.AndExpression; -import com.salesforce.phoenix.expression.BaseTerminalExpression; -import com.salesforce.phoenix.expression.CoerceExpression; -import com.salesforce.phoenix.expression.ComparisonExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.InListExpression; -import com.salesforce.phoenix.expression.IsNullExpression; -import com.salesforce.phoenix.expression.LikeExpression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.OrExpression; -import com.salesforce.phoenix.expression.RowKeyColumnExpression; -import com.salesforce.phoenix.expression.RowValueConstructorExpression; -import com.salesforce.phoenix.expression.RowValueConstructorExpression.ExpressionComparabilityWrapper; -import com.salesforce.phoenix.expression.function.FunctionExpression.OrderPreserving; -import com.salesforce.phoenix.expression.function.ScalarFunction; -import com.salesforce.phoenix.expression.visitor.TraverseNoExpressionVisitor; -import com.salesforce.phoenix.parse.FilterableStatement; -import com.salesforce.phoenix.parse.HintNode.Hint; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PName; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.RowKeySchema; -import com.salesforce.phoenix.schema.SaltingUtil; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.ScanUtil; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.phoenix.expression.AndExpression; +import org.apache.phoenix.expression.BaseTerminalExpression; +import org.apache.phoenix.expression.CoerceExpression; +import org.apache.phoenix.expression.ComparisonExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.InListExpression; +import org.apache.phoenix.expression.IsNullExpression; +import org.apache.phoenix.expression.LikeExpression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.OrExpression; +import org.apache.phoenix.expression.RowKeyColumnExpression; +import org.apache.phoenix.expression.RowValueConstructorExpression; +import org.apache.phoenix.expression.RowValueConstructorExpression.ExpressionComparabilityWrapper; +import org.apache.phoenix.expression.function.FunctionExpression.OrderPreserving; +import org.apache.phoenix.expression.function.ScalarFunction; +import org.apache.phoenix.expression.visitor.TraverseNoExpressionVisitor; +import org.apache.phoenix.parse.FilterableStatement; +import org.apache.phoenix.parse.HintNode.Hint; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PName; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.RowKeySchema; +import org.apache.phoenix.schema.SaltingUtil; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.ScanUtil; +import org.apache.phoenix.util.SchemaUtil; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseRegionScanner.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseRegionScanner.java new file mode 100644 index 00000000..44f4303e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseRegionScanner.java @@ -0,0 +1,69 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; + +import java.io.IOException; +import java.util.List; + +import org.apache.hadoop.hbase.DoNotRetryIOException; +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.regionserver.RegionScanner; + +public abstract class BaseRegionScanner implements RegionScanner { + + @Override + public boolean isFilterDone() { + return false; + } + + @Override + public boolean next(List results, String metric) throws IOException { + return next(results); + } + + @Override + public boolean next(List result, int limit, String metric) throws IOException { + return next(result); + } + @Override + public boolean next(List result, int limit) throws IOException { + return next(result); + } + + @Override + public boolean reseek(byte[] row) throws IOException { + throw new DoNotRetryIOException("Unsupported"); + } + + @Override + public long getMvccReadPoint() { + return Long.MAX_VALUE; + } + + @Override + public boolean nextRaw(List result, String metric) throws IOException { + return next(result, metric); + } + + @Override + public boolean nextRaw(List result, int limit, String metric) throws IOException { + return next(result, limit, metric); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseScannerRegionObserver.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseScannerRegionObserver.java new file mode 100644 index 00000000..6c8306e6 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseScannerRegionObserver.java @@ -0,0 +1,57 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; + +import java.io.IOException; + +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.coprocessor.*; +import org.apache.hadoop.hbase.regionserver.RegionScanner; + +import org.apache.phoenix.util.ServerUtil; + + +abstract public class BaseScannerRegionObserver extends BaseRegionObserver { + + /** + * Used by logger to identify coprocessor + */ + @Override + public String toString() { + return this.getClass().getName(); + } + + abstract protected RegionScanner doPostScannerOpen(final ObserverContext c, final Scan scan, final RegionScanner s) throws Throwable; + + /** + * Wrapper for {@link #postScannerOpen(ObserverContext, Scan, RegionScanner)} that ensures no non IOException is thrown, + * to prevent the coprocessor from becoming blacklisted. + * + */ + @Override + public final RegionScanner postScannerOpen(final ObserverContext c, final Scan scan, final RegionScanner s) throws IOException { + try { + return doPostScannerOpen(c, scan, s); + } catch (Throwable t) { + ServerUtil.throwIOException(c.getEnvironment().getRegion().getRegionNameAsString(), t); + return null; // impossible + } + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/GroupByCache.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/GroupByCache.java new file mode 100644 index 00000000..1fdf272f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/GroupByCache.java @@ -0,0 +1,41 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; + +import java.io.Closeable; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.regionserver.RegionScanner; + +import org.apache.phoenix.expression.aggregator.Aggregator; + +/** + * + * Interface to abstract the way in which distinct group by + * elements are cached + * + * @author jtaylor + * @since 3.0.0 + */ +public interface GroupByCache extends Closeable { + int size(); + Aggregator[] cache(ImmutableBytesWritable key); + RegionScanner getScanner(RegionScanner s); +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/GroupedAggregateRegionObserver.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/GroupedAggregateRegionObserver.java similarity index 87% rename from phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/GroupedAggregateRegionObserver.java rename to phoenix-core/src/main/java/org/apache/phoenix/coprocessor/GroupedAggregateRegionObserver.java index 593ecc35..abdd7bba 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/GroupedAggregateRegionObserver.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/GroupedAggregateRegionObserver.java @@ -1,28 +1,29 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source - * and binary forms, with or without modification, are permitted provided that the following - * conditions are met: Redistributions of source code must retain the above copyright notice, this - * list of conditions and the following disclaimer. Redistributions in binary form must reproduce - * the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of - * Salesforce.com nor the names of its contributors may be used to endorse or promote products - * derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED - * BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, - * INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE - * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, - * OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN - * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF - * THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; - -import static com.salesforce.phoenix.query.QueryConstants.AGG_TIMESTAMP; -import static com.salesforce.phoenix.query.QueryConstants.SINGLE_COLUMN; -import static com.salesforce.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY; -import static com.salesforce.phoenix.query.QueryServices.GROUPBY_SPILLABLE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServicesOptions.DEFAULT_GROUPBY_SPILLABLE; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; + +import static org.apache.phoenix.query.QueryConstants.AGG_TIMESTAMP; +import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN; +import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY; +import static org.apache.phoenix.query.QueryServices.GROUPBY_SPILLABLE_ATTRIB; +import static org.apache.phoenix.query.QueryServicesOptions.DEFAULT_GROUPBY_SPILLABLE; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; @@ -53,23 +54,23 @@ import com.google.common.collect.Maps; import com.google.common.io.Closeables; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.cache.GlobalCache; -import com.salesforce.phoenix.cache.TenantCache; -import com.salesforce.phoenix.cache.aggcache.SpillableGroupByCache; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.ExpressionType; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.ServerAggregators; -import com.salesforce.phoenix.join.HashJoinInfo; -import com.salesforce.phoenix.join.ScanProjector; -import com.salesforce.phoenix.memory.MemoryManager.MemoryChunk; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.tuple.MultiKeyValueTuple; -import com.salesforce.phoenix.util.KeyValueUtil; -import com.salesforce.phoenix.util.ScanUtil; -import com.salesforce.phoenix.util.SizedUtil; -import com.salesforce.phoenix.util.TupleUtil; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.cache.GlobalCache; +import org.apache.phoenix.cache.TenantCache; +import org.apache.phoenix.cache.aggcache.SpillableGroupByCache; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.ExpressionType; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.ServerAggregators; +import org.apache.phoenix.join.HashJoinInfo; +import org.apache.phoenix.join.ScanProjector; +import org.apache.phoenix.memory.MemoryManager.MemoryChunk; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.tuple.MultiKeyValueTuple; +import org.apache.phoenix.util.KeyValueUtil; +import org.apache.phoenix.util.ScanUtil; +import org.apache.phoenix.util.SizedUtil; +import org.apache.phoenix.util.TupleUtil; /** * Region observer that aggregates grouped rows (i.e. SQL query with GROUP BY clause) diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/HashJoinRegionScanner.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/HashJoinRegionScanner.java similarity index 78% rename from phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/HashJoinRegionScanner.java rename to phoenix-core/src/main/java/org/apache/phoenix/coprocessor/HashJoinRegionScanner.java index 22e08f3f..32185395 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/HashJoinRegionScanner.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/HashJoinRegionScanner.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; import java.io.IOException; import java.util.Iterator; @@ -41,21 +33,21 @@ import org.apache.hadoop.hbase.regionserver.RegionScanner; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.cache.GlobalCache; -import com.salesforce.phoenix.cache.HashCache; -import com.salesforce.phoenix.cache.TenantCache; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.join.HashJoinInfo; -import com.salesforce.phoenix.join.ScanProjector; -import com.salesforce.phoenix.join.ScanProjector.ProjectedValueTuple; -import com.salesforce.phoenix.parse.JoinTableNode.JoinType; -import com.salesforce.phoenix.schema.IllegalDataException; -import com.salesforce.phoenix.schema.KeyValueSchema; -import com.salesforce.phoenix.schema.ValueBitSet; -import com.salesforce.phoenix.schema.tuple.ResultTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.TupleUtil; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.cache.GlobalCache; +import org.apache.phoenix.cache.HashCache; +import org.apache.phoenix.cache.TenantCache; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.join.HashJoinInfo; +import org.apache.phoenix.join.ScanProjector; +import org.apache.phoenix.join.ScanProjector.ProjectedValueTuple; +import org.apache.phoenix.parse.JoinTableNode.JoinType; +import org.apache.phoenix.schema.IllegalDataException; +import org.apache.phoenix.schema.KeyValueSchema; +import org.apache.phoenix.schema.ValueBitSet; +import org.apache.phoenix.schema.tuple.ResultTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.TupleUtil; public class HashJoinRegionScanner implements RegionScanner { diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/MetaDataEndpointImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java similarity index 87% rename from phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/MetaDataEndpointImpl.java rename to phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java index 79021a5e..ec0d9f50 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/MetaDataEndpointImpl.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java @@ -1,65 +1,59 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; import static com.google.common.collect.Lists.newArrayList; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.BASE_SCHEMA_NAME_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.BASE_TABLE_NAME_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_COUNT_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_MODIFIER; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_NAME_INDEX; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_SIZE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TABLE_NAME_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TYPE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DECIMAL_DIGITS; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DEFAULT_COLUMN_FAMILY_NAME_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DISABLE_WAL_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.FAMILY_NAME_INDEX; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.IMMUTABLE_ROWS_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.INDEX_STATE_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.MULTI_TENANT_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.NULLABLE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.ORDINAL_POSITION; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.PK_NAME_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SALT_BUCKETS_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SCHEMA_NAME_INDEX; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_FAMILY_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_NAME_INDEX; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_SEQ_NUM_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_TYPE_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TENANT_ID_INDEX; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_EXPRESSION_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_TYPE_BYTES; -import static com.salesforce.phoenix.schema.PTableType.INDEX; -import static com.salesforce.phoenix.util.SchemaUtil.getVarCharLength; -import static com.salesforce.phoenix.util.SchemaUtil.getVarChars; import static org.apache.hadoop.hbase.filter.CompareFilter.CompareOp.EQUAL; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.ARRAY_SIZE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_COUNT_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_MODIFIER; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_NAME_INDEX; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_SIZE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TABLE_NAME_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TYPE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DECIMAL_DIGITS; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DEFAULT_COLUMN_FAMILY_NAME_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DISABLE_WAL_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.FAMILY_NAME_INDEX; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.IMMUTABLE_ROWS_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.INDEX_STATE_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.LINK_TYPE_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.MULTI_TENANT_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.NULLABLE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.ORDINAL_POSITION; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.PK_NAME_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SALT_BUCKETS_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SCHEMA_NAME_INDEX; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_FAMILY_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_NAME_INDEX; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_SEQ_NUM_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_TYPE_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TENANT_ID_INDEX; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_STATEMENT_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_TYPE_BYTES; +import static org.apache.phoenix.schema.PTableType.INDEX; +import static org.apache.phoenix.util.SchemaUtil.getVarCharLength; +import static org.apache.phoenix.util.SchemaUtil.getVarChars; +import java.io.DataInput; +import java.io.DataOutput; import java.io.IOException; import java.sql.ResultSetMetaData; import java.sql.SQLException; @@ -78,48 +72,47 @@ import org.apache.hadoop.hbase.client.Scan; import org.apache.hadoop.hbase.coprocessor.BaseEndpointCoprocessor; import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; -import org.apache.hadoop.hbase.filter.BinaryComparator; -import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; import org.apache.hadoop.hbase.filter.Filter; +import org.apache.hadoop.hbase.filter.FilterBase; import org.apache.hadoop.hbase.filter.FilterList; import org.apache.hadoop.hbase.filter.FirstKeyOnlyFilter; -import org.apache.hadoop.hbase.filter.RowFilter; import org.apache.hadoop.hbase.filter.SingleColumnValueFilter; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.util.IndexManagementUtil; import org.apache.hadoop.hbase.regionserver.HRegion; import org.apache.hadoop.hbase.regionserver.RegionScanner; import org.apache.hadoop.hbase.util.Bytes; import org.apache.hadoop.hbase.util.EnvironmentEdgeManager; +import org.apache.phoenix.cache.GlobalCache; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.AmbiguousColumnException; +import org.apache.phoenix.schema.ColumnFamilyNotFoundException; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PColumnFamily; +import org.apache.phoenix.schema.PColumnImpl; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PIndexState; +import org.apache.phoenix.schema.PName; +import org.apache.phoenix.schema.PNameFactory; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTable.LinkType; +import org.apache.phoenix.schema.PTable.ViewType; +import org.apache.phoenix.schema.PTableImpl; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.TableNotFoundException; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.IndexUtil; +import org.apache.phoenix.util.KeyValueUtil; +import org.apache.phoenix.util.MetaDataUtil; +import org.apache.phoenix.util.SchemaUtil; +import org.apache.phoenix.util.ServerUtil; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import com.google.common.collect.Lists; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.hbase.index.util.IndexManagementUtil; -import com.salesforce.phoenix.cache.GlobalCache; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.AmbiguousColumnException; -import com.salesforce.phoenix.schema.ColumnFamilyNotFoundException; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PColumnFamily; -import com.salesforce.phoenix.schema.PColumnImpl; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PIndexState; -import com.salesforce.phoenix.schema.PName; -import com.salesforce.phoenix.schema.PNameFactory; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTable.ViewType; -import com.salesforce.phoenix.schema.PTableImpl; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.TableNotFoundException; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.IndexUtil; -import com.salesforce.phoenix.util.KeyValueUtil; -import com.salesforce.phoenix.util.MetaDataUtil; -import com.salesforce.phoenix.util.SchemaUtil; -import com.salesforce.phoenix.util.ServerUtil; /** * @@ -148,13 +141,11 @@ public class MetaDataEndpointImpl extends BaseEndpointCoprocessor implements Met private static final KeyValue DATA_TABLE_NAME_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, DATA_TABLE_NAME_BYTES); private static final KeyValue INDEX_STATE_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, INDEX_STATE_BYTES); private static final KeyValue IMMUTABLE_ROWS_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, IMMUTABLE_ROWS_BYTES); - private static final KeyValue VIEW_EXPRESSION_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, VIEW_EXPRESSION_BYTES); + private static final KeyValue VIEW_EXPRESSION_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, VIEW_STATEMENT_BYTES); private static final KeyValue DEFAULT_COLUMN_FAMILY_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, DEFAULT_COLUMN_FAMILY_NAME_BYTES); private static final KeyValue DISABLE_WAL_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, DISABLE_WAL_BYTES); private static final KeyValue MULTI_TENANT_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, MULTI_TENANT_BYTES); private static final KeyValue VIEW_TYPE_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, VIEW_TYPE_BYTES); - private static final KeyValue BASE_SCHEMA_NAME_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, BASE_SCHEMA_NAME_BYTES); - private static final KeyValue BASE_TABLE_NAME_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, BASE_TABLE_NAME_BYTES); private static final List TABLE_KV_COLUMNS = Arrays.asList( TABLE_TYPE_KV, TABLE_SEQ_NUM_KV, @@ -168,9 +159,7 @@ public class MetaDataEndpointImpl extends BaseEndpointCoprocessor implements Met DEFAULT_COLUMN_FAMILY_KV, DISABLE_WAL_KV, MULTI_TENANT_KV, - VIEW_TYPE_KV, - BASE_SCHEMA_NAME_KV, - BASE_TABLE_NAME_KV + VIEW_TYPE_KV ); static { Collections.sort(TABLE_KV_COLUMNS, KeyValue.COMPARATOR); @@ -183,13 +172,11 @@ public class MetaDataEndpointImpl extends BaseEndpointCoprocessor implements Met private static final int DATA_TABLE_NAME_INDEX = TABLE_KV_COLUMNS.indexOf(DATA_TABLE_NAME_KV); private static final int INDEX_STATE_INDEX = TABLE_KV_COLUMNS.indexOf(INDEX_STATE_KV); private static final int IMMUTABLE_ROWS_INDEX = TABLE_KV_COLUMNS.indexOf(IMMUTABLE_ROWS_KV); - private static final int VIEW_EXPRESSION_INDEX = TABLE_KV_COLUMNS.indexOf(VIEW_EXPRESSION_KV); + private static final int VIEW_STATEMENT_INDEX = TABLE_KV_COLUMNS.indexOf(VIEW_EXPRESSION_KV); private static final int DEFAULT_COLUMN_FAMILY_INDEX = TABLE_KV_COLUMNS.indexOf(DEFAULT_COLUMN_FAMILY_KV); private static final int DISABLE_WAL_INDEX = TABLE_KV_COLUMNS.indexOf(DISABLE_WAL_KV); private static final int MULTI_TENANT_INDEX = TABLE_KV_COLUMNS.indexOf(MULTI_TENANT_KV); private static final int VIEW_TYPE_INDEX = TABLE_KV_COLUMNS.indexOf(VIEW_TYPE_KV); - private static final int BASE_SCHEMA_NAME_INDEX = TABLE_KV_COLUMNS.indexOf(BASE_SCHEMA_NAME_KV); - private static final int BASE_TABLE_NAME_INDEX = TABLE_KV_COLUMNS.indexOf(BASE_TABLE_NAME_KV); // KeyValues for Column private static final KeyValue DECIMAL_DIGITS_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, Bytes.toBytes(DECIMAL_DIGITS)); @@ -198,6 +185,7 @@ public class MetaDataEndpointImpl extends BaseEndpointCoprocessor implements Met private static final KeyValue DATA_TYPE_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, Bytes.toBytes(DATA_TYPE)); private static final KeyValue ORDINAL_POSITION_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, Bytes.toBytes(ORDINAL_POSITION)); private static final KeyValue COLUMN_MODIFIER_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, Bytes.toBytes(COLUMN_MODIFIER)); + private static final KeyValue ARRAY_SIZE_KV = KeyValue.createFirstOnRow(ByteUtil.EMPTY_BYTE_ARRAY, TABLE_FAMILY_BYTES, Bytes.toBytes(ARRAY_SIZE)); private static final List COLUMN_KV_COLUMNS = Arrays.asList( DECIMAL_DIGITS_KV, COLUMN_SIZE_KV, @@ -205,7 +193,8 @@ public class MetaDataEndpointImpl extends BaseEndpointCoprocessor implements Met DATA_TYPE_KV, ORDINAL_POSITION_KV, COLUMN_MODIFIER_KV, - DATA_TABLE_NAME_KV // included in both column and table row for metadata APIs + DATA_TABLE_NAME_KV, // included in both column and table row for metadata APIs + ARRAY_SIZE_KV ); static { Collections.sort(COLUMN_KV_COLUMNS, KeyValue.COMPARATOR); @@ -216,6 +205,9 @@ public class MetaDataEndpointImpl extends BaseEndpointCoprocessor implements Met private static final int SQL_DATA_TYPE_INDEX = COLUMN_KV_COLUMNS.indexOf(DATA_TYPE_KV); private static final int ORDINAL_POSITION_INDEX = COLUMN_KV_COLUMNS.indexOf(ORDINAL_POSITION_KV); private static final int COLUMN_MODIFIER_INDEX = COLUMN_KV_COLUMNS.indexOf(COLUMN_MODIFIER_KV); + private static final int ARRAY_SIZE_INDEX = COLUMN_KV_COLUMNS.indexOf(ARRAY_SIZE_KV); + + private static final int LINK_TYPE_INDEX = 0; private static PName newPName(byte[] keyBuffer, int keyOffset, int keyLength) { if (keyLength <= 0) { @@ -319,7 +311,9 @@ private void addColumnToTable(List results, PName colName, PName famNa if (maxLength == null && dataType == PDataType.BINARY) dataType = PDataType.VARBINARY; // For backward compatibility. KeyValue columnModifierKv = colKeyValues[COLUMN_MODIFIER_INDEX]; ColumnModifier sortOrder = columnModifierKv == null ? null : ColumnModifier.fromSystemValue(PDataType.INTEGER.getCodec().decodeInt(columnModifierKv.getBuffer(), columnModifierKv.getValueOffset(), null)); - PColumn column = new PColumnImpl(colName, famName, dataType, maxLength, scale, isNullable, position-1, sortOrder); + KeyValue arraySizeKv = colKeyValues[ARRAY_SIZE_INDEX]; + Integer arraySize = arraySizeKv == null ? null : PDataType.INTEGER.getCodec().decodeInt(arraySizeKv.getBuffer(), arraySizeKv.getValueOffset(), null); + PColumn column = new PColumnImpl(colName, famName, dataType, maxLength, scale, isNullable, position-1, sortOrder, arraySize); columns.add(column); } @@ -400,41 +394,45 @@ private PTable getTable(RegionScanner scanner, long clientTimeStamp, long tableT boolean isImmutableRows = immutableRowsKv == null ? false : (Boolean)PDataType.BOOLEAN.toObject(immutableRowsKv.getBuffer(), immutableRowsKv.getValueOffset(), immutableRowsKv.getValueLength()); KeyValue defaultFamilyNameKv = tableKeyValues[DEFAULT_COLUMN_FAMILY_INDEX]; PName defaultFamilyName = defaultFamilyNameKv != null ? newPName(defaultFamilyNameKv.getBuffer(), defaultFamilyNameKv.getValueOffset(), defaultFamilyNameKv.getValueLength()) : null; - KeyValue viewExpressionKv = tableKeyValues[VIEW_EXPRESSION_INDEX]; - String viewExpression = viewExpressionKv != null ? (String)PDataType.VARCHAR.toObject(viewExpressionKv.getBuffer(), viewExpressionKv.getValueOffset(), viewExpressionKv.getValueLength()) : null; + KeyValue viewStatementKv = tableKeyValues[VIEW_STATEMENT_INDEX]; + String viewStatement = viewStatementKv != null ? (String)PDataType.VARCHAR.toObject(viewStatementKv.getBuffer(), viewStatementKv.getValueOffset(), viewStatementKv.getValueLength()) : null; KeyValue disableWALKv = tableKeyValues[DISABLE_WAL_INDEX]; boolean disableWAL = disableWALKv == null ? PTable.DEFAULT_DISABLE_WAL : Boolean.TRUE.equals(PDataType.BOOLEAN.toObject(disableWALKv.getBuffer(), disableWALKv.getValueOffset(), disableWALKv.getValueLength())); KeyValue multiTenantKv = tableKeyValues[MULTI_TENANT_INDEX]; boolean multiTenant = multiTenantKv == null ? false : Boolean.TRUE.equals(PDataType.BOOLEAN.toObject(multiTenantKv.getBuffer(), multiTenantKv.getValueOffset(), multiTenantKv.getValueLength())); KeyValue viewTypeKv = tableKeyValues[VIEW_TYPE_INDEX]; ViewType viewType = viewTypeKv == null ? null : ViewType.fromSerializedValue(viewTypeKv.getBuffer()[viewTypeKv.getValueOffset()]); - KeyValue baseSchemaNameKv = tableKeyValues[BASE_SCHEMA_NAME_INDEX]; - PName baseSchemaName = baseSchemaNameKv != null ? newPName(baseSchemaNameKv.getBuffer(), baseSchemaNameKv.getValueOffset(), baseSchemaNameKv.getValueLength()) : null; - KeyValue baseTableNameKv = tableKeyValues[BASE_TABLE_NAME_INDEX]; - PName baseTableName = baseTableNameKv != null ? newPName(baseTableNameKv.getBuffer(), baseTableNameKv.getValueOffset(), baseTableNameKv.getValueLength()) : null; List columns = Lists.newArrayListWithExpectedSize(columnCount); List indexes = new ArrayList(); + List physicalTables = new ArrayList(); while (true) { results.clear(); scanner.next(results); if (results.isEmpty()) { break; } - KeyValue colKv = results.get(0); + KeyValue colKv = results.get(LINK_TYPE_INDEX); int colKeyLength = colKv.getRowLength(); PName colName = newPName(colKv.getBuffer(), colKv.getRowOffset() + offset, colKeyLength-offset); int colKeyOffset = offset + colName.getBytes().length + 1; PName famName = newPName(colKv.getBuffer(), colKv.getRowOffset() + colKeyOffset, colKeyLength-colKeyOffset); - if (colName.getString().isEmpty() && famName != null) { - addIndexToTable(tenantId, schemaName, famName, tableName, clientTimeStamp, indexes); + if (colName.getString().isEmpty() && famName != null) { + LinkType linkType = LinkType.fromSerializedValue(colKv.getBuffer()[colKv.getValueOffset()]); + if (linkType == LinkType.INDEX_TABLE) { + addIndexToTable(tenantId, schemaName, famName, tableName, clientTimeStamp, indexes); + } else if (linkType == LinkType.PHYSICAL_TABLE) { + physicalTables.add(famName); + } else { + logger.warn("Unknown link type: " + colKv.getBuffer()[colKv.getValueOffset()] + " for " + SchemaUtil.getTableName(schemaName.getString(), tableName.getString())); + } } else { addColumnToTable(results, colName, famName, colKeyValues, columns); } } return PTableImpl.makePTable(schemaName, tableName, tableType, indexState, timeStamp, tableSeqNum, pkName, saltBucketNum, columns, tableType == INDEX ? dataTableName : null, - indexes, isImmutableRows, baseSchemaName, baseTableName, defaultFamilyName, viewExpression, disableWAL, multiTenant, viewType); + indexes, isImmutableRows, physicalTables, defaultFamilyName, viewStatement, disableWAL, multiTenant, viewType); } private PTable buildDeletedTable(byte[] key, ImmutableBytesPtr cacheKey, HRegion region, long clientTimeStamp) throws IOException { @@ -541,14 +539,6 @@ public MetaDataMutationResult createTable(List tableMetadata) throws I } else { return new MetaDataMutationResult(MutationCode.NEWER_TABLE_FOUND, EnvironmentEdgeManager.currentTimeMillis(), table); } - } else { - // If we're attempting to create the SYSTEM.TABLE, - // then is the first cluster connection to Phoenix v 3.0, in which case we - // need to upgrade from 2.x to 3.0. Since our updates are additive, we do - // not need to delete any rows, but can just allow the mutation to complete. - if (SchemaUtil.isMetaTable(schemaName, tableName)) { - SchemaUtil.upgradeTo3(region, tableMetadata); - } } // TODO: Switch this to HRegion#batchMutate when we want to support indexes on the system @@ -592,6 +582,7 @@ private static void releaseLocks(HRegion region, List lids) { } } + private static final byte[] PHYSICAL_TABLE_BYTES = new byte[] {PTable.LinkType.PHYSICAL_TABLE.getSerializedValue()}; /** * @param tableName parent table's name * @return true if there exist a table that use this table as their base table. @@ -610,14 +601,13 @@ private boolean hasViews(HRegion region, byte[] tenantId, PTable table) throws I scan.setStartRow(startRow); scan.setStopRow(stopRow); } - SingleColumnValueFilter filter1 = new SingleColumnValueFilter(TABLE_FAMILY_BYTES, BASE_SCHEMA_NAME_BYTES, EQUAL, schemaName); - filter1.setFilterIfMissing(schemaName.length > 0); - SingleColumnValueFilter filter2 = new SingleColumnValueFilter(TABLE_FAMILY_BYTES, BASE_TABLE_NAME_BYTES, EQUAL, tableName); - filter2.setFilterIfMissing(true); - BinaryComparator comparator = new BinaryComparator(ByteUtil.concat(tenantId, QueryConstants.SEPARATOR_BYTE_ARRAY, schemaName, QueryConstants.SEPARATOR_BYTE_ARRAY, tableName)); - RowFilter filter3 = new RowFilter(CompareOp.NOT_EQUAL,comparator); - Filter filter = new FilterList(filter1,filter2,filter3); + SingleColumnValueFilter linkFilter = new SingleColumnValueFilter(TABLE_FAMILY_BYTES, LINK_TYPE_BYTES, EQUAL, PHYSICAL_TABLE_BYTES); + linkFilter.setFilterIfMissing(true); + byte[] suffix = ByteUtil.concat(QueryConstants.SEPARATOR_BYTE_ARRAY, SchemaUtil.getTableNameAsBytes(schemaName, tableName)); + SuffixFilter rowFilter = new SuffixFilter(suffix); + Filter filter = new FilterList(linkFilter, rowFilter); scan.setFilter(filter); + scan.addColumn(TABLE_FAMILY_BYTES, LINK_TYPE_BYTES); RegionScanner scanner = region.getScanner(scan); try { List results = newArrayList(); @@ -730,7 +720,7 @@ private MetaDataMutationResult doDropTable(byte[] key, byte[] tenantId, byte[] s } // Don't allow a table with views to be deleted // TODO: support CASCADE with DROP - if (hasViews(region, tenantId, table)) { + if (tableType == PTableType.TABLE && hasViews(region, tenantId, table)) { return new MetaDataMutationResult(MutationCode.UNALLOWED_TABLE_MUTATION, EnvironmentEdgeManager.currentTimeMillis(), null); } if (table.getType() != PTableType.VIEW) { // Add to list of HTables to delete, unless it's a view @@ -741,9 +731,12 @@ private MetaDataMutationResult doDropTable(byte[] key, byte[] tenantId, byte[] s byte[][] rowKeyMetaData = new byte[5][]; byte[] rowKey; do { - rowKey = results.get(0).getRow(); + KeyValue kv = results.get(LINK_TYPE_INDEX); + rowKey = kv.getRow(); int nColumns = getVarChars(rowKey, rowKeyMetaData); - if (nColumns == 5 && rowKeyMetaData[PhoenixDatabaseMetaData.COLUMN_NAME_INDEX].length == 0 && rowKeyMetaData[PhoenixDatabaseMetaData.INDEX_NAME_INDEX].length > 0) { + if (nColumns == 5 && rowKeyMetaData[PhoenixDatabaseMetaData.COLUMN_NAME_INDEX].length == 0 && rowKeyMetaData[PhoenixDatabaseMetaData.INDEX_NAME_INDEX].length > 0 + && Bytes.compareTo(kv.getBuffer(), kv.getQualifierOffset(), kv.getQualifierLength(), LINK_TYPE_BYTES, 0, LINK_TYPE_BYTES.length) == 0 + && LinkType.fromSerializedValue(kv.getBuffer()[kv.getValueOffset()]) == LinkType.INDEX_TABLE) { indexNames.add(rowKeyMetaData[PhoenixDatabaseMetaData.INDEX_NAME_INDEX]); } @SuppressWarnings("deprecation") // FIXME: Remove when unintentionally deprecated method is fixed (HBASE-7870). @@ -1187,4 +1180,41 @@ public MetaDataMutationResult updateIndexState(List tableMetadata) thr return null; // impossible } } + + /** + * + * Matches rows that end with a given byte array suffix + * + * @author jtaylor + * @since 3.0 + */ + private static class SuffixFilter extends FilterBase { + protected byte[] suffix = null; + + public SuffixFilter(final byte[] suffix) { + this.suffix = suffix; + } + + @Override + public boolean filterRowKey(byte[] buffer, int offset, int length) { + if (buffer == null || this.suffix == null) return true; + if (length < suffix.length) return true; + // if they are equal, return false => pass row + // else return true, filter row + // if we are passed the suffix, set flag + int cmp = Bytes.compareTo(buffer, offset + (length - this.suffix.length), + this.suffix.length, this.suffix, 0, this.suffix.length); + return cmp != 0; + } + + @Override + public void readFields(DataInput arg0) throws IOException { + throw new UnsupportedOperationException(); + } + + @Override + public void write(DataOutput arg0) throws IOException { + throw new UnsupportedOperationException(); + } + } } diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/MetaDataProtocol.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataProtocol.java similarity index 81% rename from phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/MetaDataProtocol.java rename to phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataProtocol.java index 8b55139f..9f01fff1 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/MetaDataProtocol.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataProtocol.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; import java.io.DataInput; import java.io.DataOutput; @@ -40,11 +32,11 @@ import org.apache.hadoop.io.WritableUtils; import com.google.common.collect.Lists; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableImpl; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.MetaDataUtil; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableImpl; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.MetaDataUtil; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataRegionObserver.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataRegionObserver.java new file mode 100644 index 00000000..d0bd7c54 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataRegionObserver.java @@ -0,0 +1,40 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; + +import org.apache.hadoop.hbase.coprocessor.BaseRegionObserver; +import org.apache.hadoop.hbase.coprocessor.ObserverContext; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; + +import org.apache.phoenix.cache.GlobalCache; + + +/** + * Coprocessor for metadata related operations. This coprocessor would only be registered + * to SYSTEM.TABLE. + */ +public class MetaDataRegionObserver extends BaseRegionObserver { + + @Override + public void preClose(final ObserverContext c, + boolean abortRequested) { + GlobalCache.getInstance(c.getEnvironment()).getMetaDataCache().clear(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ScanProjector.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ScanProjector.java new file mode 100644 index 00000000..ee9e6e29 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ScanProjector.java @@ -0,0 +1,202 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.util.HashMap; +import java.util.Map; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.hbase.util.Pair; +import org.apache.hadoop.io.WritableUtils; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.KeyValueUtil; + +public class ScanProjector { + + public enum ProjectionType {TABLE, CF, CQ}; + + private static final String SCAN_PROJECTOR = "scanProjector"; + private static final byte[] SEPERATOR = Bytes.toBytes(":"); + + private final ProjectionType type; + private final byte[] tablePrefix; + private final Map cfProjectionMap; + private final Map>> cqProjectionMap; + + private ScanProjector(ProjectionType type, byte[] tablePrefix, + Map cfProjectionMap, Map>> cqProjectionMap) { + this.type = ProjectionType.TABLE; + this.tablePrefix = tablePrefix; + this.cfProjectionMap = cfProjectionMap; + this.cqProjectionMap = cqProjectionMap; + } + + public static void serializeProjectorIntoScan(Scan scan, ScanProjector projector) { + ByteArrayOutputStream stream = new ByteArrayOutputStream(); + try { + DataOutputStream output = new DataOutputStream(stream); + WritableUtils.writeVInt(output, projector.type.ordinal()); + switch (projector.type) { + case TABLE: + WritableUtils.writeCompressedByteArray(output, projector.tablePrefix); + break; + case CF: + WritableUtils.writeVInt(output, projector.cfProjectionMap.size()); + for (Map.Entry entry : projector.cfProjectionMap.entrySet()) { + WritableUtils.writeCompressedByteArray(output, entry.getKey().get()); + WritableUtils.writeCompressedByteArray(output, entry.getValue()); + } + break; + case CQ: + WritableUtils.writeVInt(output, projector.cqProjectionMap.size()); + for (Map.Entry>> entry : + projector.cqProjectionMap.entrySet()) { + WritableUtils.writeCompressedByteArray(output, entry.getKey().get()); + Map> map = entry.getValue(); + WritableUtils.writeVInt(output, map.size()); + for (Map.Entry> e : map.entrySet()) { + WritableUtils.writeCompressedByteArray(output, e.getKey().get()); + WritableUtils.writeCompressedByteArray(output, e.getValue().getFirst()); + WritableUtils.writeCompressedByteArray(output, e.getValue().getSecond()); + } + } + break; + default: + throw new IOException("Unrecognized projection type '" + projector.type + "'"); + } + scan.setAttribute(SCAN_PROJECTOR, stream.toByteArray()); + } catch (IOException e) { + throw new RuntimeException(e); + } finally { + try { + stream.close(); + } catch (IOException e) { + throw new RuntimeException(e); + } + } + + } + + public static ScanProjector deserializeProjectorFromScan(Scan scan) { + byte[] proj = scan.getAttribute(SCAN_PROJECTOR); + if (proj == null) { + return null; + } + ByteArrayInputStream stream = new ByteArrayInputStream(proj); + try { + DataInputStream input = new DataInputStream(stream); + int t = WritableUtils.readVInt(input); + ProjectionType type = ProjectionType.values()[t]; + if (type == ProjectionType.TABLE) { + byte[] tablePrefix = WritableUtils.readCompressedByteArray(input); + return new ScanProjector(type, tablePrefix, null, null); + } + if (type == ProjectionType.CF) { + int count = WritableUtils.readVInt(input); + Map cfMap = new HashMap(); + for (int i = 0; i < count; i++) { + byte[] cf = WritableUtils.readCompressedByteArray(input); + byte[] renamed = WritableUtils.readCompressedByteArray(input); + cfMap.put(new ImmutableBytesPtr(cf), renamed); + } + return new ScanProjector(type, null, cfMap, null); + } + + int count = WritableUtils.readVInt(input); + Map>> cqMap = + new HashMap>>(); + for (int i = 0; i < count; i++) { + byte[] cf = WritableUtils.readCompressedByteArray(input); + int nQuals = WritableUtils.readVInt(input); + Map> map = + new HashMap>(); + for (int j = 0; j < nQuals; j++) { + byte[] cq = WritableUtils.readCompressedByteArray(input); + byte[] renamedCf = WritableUtils.readCompressedByteArray(input); + byte[] renamedCq = WritableUtils.readCompressedByteArray(input); + map.put(new ImmutableBytesPtr(cq), new Pair(renamedCf, renamedCq)); + } + cqMap.put(new ImmutableBytesPtr(cf), map); + } + return new ScanProjector(type, null, null, cqMap); + } catch (IOException e) { + throw new RuntimeException(e); + } finally { + try { + stream.close(); + } catch (IOException e) { + throw new RuntimeException(e); + } + } + } + + public ProjectionType getType() { + return this.type; + } + + public byte[] getTablePrefix() { + return this.tablePrefix; + } + + public Map getCfProjectionMap() { + return this.cfProjectionMap; + } + + public Map>> getCqProjectionMap() { + return this.cqProjectionMap; + } + + public KeyValue getProjectedKeyValue(KeyValue kv) { + if (type == ProjectionType.TABLE) { + byte[] cf = ByteUtil.concat(tablePrefix, SEPERATOR, kv.getFamily()); + return KeyValueUtil.newKeyValue(kv.getBuffer(), kv.getKeyOffset(), kv.getKeyLength(), + cf, kv.getQualifier(), kv.getTimestamp(), kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); + } + + if (type == ProjectionType.CF) { + byte[] cf = cfProjectionMap.get(new ImmutableBytesPtr(kv.getFamily())); + if (cf == null) + return kv; + return KeyValueUtil.newKeyValue(kv.getBuffer(), kv.getKeyOffset(), kv.getKeyLength(), + cf, kv.getQualifier(), kv.getTimestamp(), kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); + } + + Map> map = cqProjectionMap.get(new ImmutableBytesPtr(kv.getFamily())); + if (map == null) + return kv; + + Pair col = map.get(new ImmutableBytesPtr(kv.getQualifier())); + if (col == null) + return kv; + + return KeyValueUtil.newKeyValue(kv.getBuffer(), kv.getKeyOffset(), kv.getKeyLength(), + col.getFirst(), col.getSecond(), kv.getTimestamp(), kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ScanRegionObserver.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ScanRegionObserver.java similarity index 81% rename from phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ScanRegionObserver.java rename to phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ScanRegionObserver.java index ad7964ca..cb76b700 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/ScanRegionObserver.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ScanRegionObserver.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; @@ -46,19 +38,19 @@ import org.apache.hadoop.io.WritableUtils; import com.google.common.collect.Lists; -import com.salesforce.phoenix.cache.GlobalCache; -import com.salesforce.phoenix.cache.TenantCache; -import com.salesforce.phoenix.expression.OrderByExpression; -import com.salesforce.phoenix.iterate.OrderedResultIterator; -import com.salesforce.phoenix.iterate.RegionScannerResultIterator; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.join.HashJoinInfo; -import com.salesforce.phoenix.join.ScanProjector; -import com.salesforce.phoenix.memory.MemoryManager.MemoryChunk; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ScanUtil; -import com.salesforce.phoenix.util.ServerUtil; +import org.apache.phoenix.cache.GlobalCache; +import org.apache.phoenix.cache.TenantCache; +import org.apache.phoenix.expression.OrderByExpression; +import org.apache.phoenix.iterate.OrderedResultIterator; +import org.apache.phoenix.iterate.RegionScannerResultIterator; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.join.HashJoinInfo; +import org.apache.phoenix.join.ScanProjector; +import org.apache.phoenix.memory.MemoryManager.MemoryChunk; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ScanUtil; +import org.apache.phoenix.util.ServerUtil; /** diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/SequenceRegionObserver.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/SequenceRegionObserver.java similarity index 96% rename from phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/SequenceRegionObserver.java rename to phoenix-core/src/main/java/org/apache/phoenix/coprocessor/SequenceRegionObserver.java index 6f43ba5e..ed04cc34 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/SequenceRegionObserver.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/SequenceRegionObserver.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.coprocessor; +package org.apache.phoenix.coprocessor; import java.io.IOException; import java.util.Collections; @@ -24,15 +24,15 @@ import org.apache.hadoop.hbase.util.EnvironmentEdgeManager; import org.apache.hadoop.hbase.util.Pair; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.Sequence; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.KeyValueUtil; -import com.salesforce.phoenix.util.MetaDataUtil; -import com.salesforce.phoenix.util.ServerUtil; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.Sequence; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.KeyValueUtil; +import org.apache.phoenix.util.MetaDataUtil; +import org.apache.phoenix.util.ServerUtil; /** * diff --git a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ServerCachingEndpointImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ServerCachingEndpointImpl.java new file mode 100644 index 00000000..b27c14ce --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ServerCachingEndpointImpl.java @@ -0,0 +1,57 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; + +import java.sql.SQLException; + +import org.apache.hadoop.hbase.coprocessor.BaseEndpointCoprocessor; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.cache.GlobalCache; +import org.apache.phoenix.cache.TenantCache; + + + + +/** + * + * Server-side implementation of {@link ServerCachingProtocol} + * + * @author jtaylor + * @since 0.1 + */ +public class ServerCachingEndpointImpl extends BaseEndpointCoprocessor implements ServerCachingProtocol { + + @Override + public boolean addServerCache(byte[] tenantId, byte[] cacheId, ImmutableBytesWritable cachePtr, ServerCacheFactory cacheFactory) throws SQLException { + TenantCache tenantCache = GlobalCache.getTenantCache((RegionCoprocessorEnvironment)this.getEnvironment(), tenantId == null ? null : new ImmutableBytesPtr(tenantId)); + tenantCache.addServerCache(new ImmutableBytesPtr(cacheId), cachePtr, cacheFactory); + return true; + } + + @Override + public boolean removeServerCache(byte[] tenantId, byte[] cacheId) throws SQLException { + TenantCache tenantCache = GlobalCache.getTenantCache((RegionCoprocessorEnvironment)this.getEnvironment(), tenantId == null ? null : new ImmutableBytesPtr(tenantId)); + tenantCache.removeServerCache(new ImmutableBytesPtr(cacheId)); + return true; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ServerCachingProtocol.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ServerCachingProtocol.java new file mode 100644 index 00000000..abda834c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ServerCachingProtocol.java @@ -0,0 +1,63 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; + +import java.io.Closeable; +import java.sql.SQLException; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.ipc.CoprocessorProtocol; +import org.apache.hadoop.io.Writable; + +import org.apache.phoenix.memory.MemoryManager.MemoryChunk; + +/** + * + * EndPoint coprocessor to send a cache to a region server. + * Used for: + * a) hash joins, to send the smaller side of the join to each region server + * b) secondary indexes, to send the necessary meta data to each region server + * @author jtaylor + * @since 0.1 + */ +public interface ServerCachingProtocol extends CoprocessorProtocol { + public static interface ServerCacheFactory extends Writable { + public Closeable newCache(ImmutableBytesWritable cachePtr, MemoryChunk chunk) throws SQLException; + } + /** + * Add the cache to the region server cache. + * @param tenantId the tenantId or null if not applicable + * @param cacheId unique identifier of the cache + * @param cachePtr pointer to the byte array of the cache + * @param cacheFactory factory that converts from byte array to object representation on the server side + * @return true on success and otherwise throws + * @throws SQLException + */ + public boolean addServerCache(byte[] tenantId, byte[] cacheId, ImmutableBytesWritable cachePtr, ServerCacheFactory cacheFactory) throws SQLException; + /** + * Remove the cache from the region server cache. Called upon completion of + * the operation when cache is no longer needed. + * @param tenantId the tenantId or null if not applicable + * @param cacheId unique identifier of the cache + * @return true on success and otherwise throws + * @throws SQLException + */ + public boolean removeServerCache(byte[] tenantId, byte[] cacheId) throws SQLException; +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/UngroupedAggregateRegionObserver.java b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionObserver.java similarity index 83% rename from phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/UngroupedAggregateRegionObserver.java rename to phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionObserver.java index 0040400b..c7c4575b 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/coprocessor/UngroupedAggregateRegionObserver.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/UngroupedAggregateRegionObserver.java @@ -1,37 +1,29 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.coprocessor; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.coprocessor; -import static com.salesforce.phoenix.query.QueryConstants.AGG_TIMESTAMP; -import static com.salesforce.phoenix.query.QueryConstants.SINGLE_COLUMN; -import static com.salesforce.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY; -import static com.salesforce.phoenix.query.QueryConstants.UNGROUPED_AGG_ROW_KEY; -import static com.salesforce.phoenix.query.QueryServices.MUTATE_BATCH_SIZE_ATTRIB; +import static org.apache.phoenix.query.QueryConstants.AGG_TIMESTAMP; +import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN; +import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY; +import static org.apache.phoenix.query.QueryConstants.UNGROUPED_AGG_ROW_KEY; +import static org.apache.phoenix.query.QueryServices.MUTATE_BATCH_SIZE_ATTRIB; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; @@ -65,30 +57,30 @@ import com.google.common.collect.Lists; import com.google.common.collect.Sets; -import com.salesforce.phoenix.client.KeyValueBuilder; -import com.salesforce.phoenix.exception.ValueTypeIncompatibleException; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.ExpressionType; -import com.salesforce.phoenix.expression.aggregator.Aggregator; -import com.salesforce.phoenix.expression.aggregator.Aggregators; -import com.salesforce.phoenix.expression.aggregator.ServerAggregators; -import com.salesforce.phoenix.index.PhoenixIndexCodec; -import com.salesforce.phoenix.join.HashJoinInfo; -import com.salesforce.phoenix.join.ScanProjector; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.ConstraintViolationException; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PRow; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableImpl; -import com.salesforce.phoenix.schema.tuple.MultiKeyValueTuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.KeyValueUtil; -import com.salesforce.phoenix.util.ScanUtil; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.phoenix.client.KeyValueBuilder; +import org.apache.phoenix.exception.ValueTypeIncompatibleException; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.ExpressionType; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.Aggregators; +import org.apache.phoenix.expression.aggregator.ServerAggregators; +import org.apache.phoenix.index.PhoenixIndexCodec; +import org.apache.phoenix.join.HashJoinInfo; +import org.apache.phoenix.join.ScanProjector; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.ConstraintViolationException; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PRow; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableImpl; +import org.apache.phoenix.schema.tuple.MultiKeyValueTuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.KeyValueUtil; +import org.apache.phoenix.util.ScanUtil; +import org.apache.phoenix.util.SchemaUtil; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/exception/PhoenixIOException.java b/phoenix-core/src/main/java/org/apache/phoenix/exception/PhoenixIOException.java new file mode 100644 index 00000000..d1b0b180 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/exception/PhoenixIOException.java @@ -0,0 +1,33 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.exception; + +import java.sql.SQLException; + + +public class PhoenixIOException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.IO_EXCEPTION; + + public PhoenixIOException(Throwable e) { + super(e.getMessage(), code.getSQLState(), code.getErrorCode(), e); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/exception/PhoenixParserException.java b/phoenix-core/src/main/java/org/apache/phoenix/exception/PhoenixParserException.java new file mode 100644 index 00000000..fc969235 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/exception/PhoenixParserException.java @@ -0,0 +1,112 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.exception; + +import java.sql.SQLSyntaxErrorException; + +import org.antlr.runtime.MismatchedTokenException; +import org.antlr.runtime.MissingTokenException; +import org.antlr.runtime.RecognitionException; +import org.antlr.runtime.Token; +import org.antlr.runtime.UnwantedTokenException; + + +public class PhoenixParserException extends SQLSyntaxErrorException { + private static final long serialVersionUID = 1L; + + public static final PhoenixParserException newException(Throwable cause, String[] tokens) { + return new PhoenixParserException(getErrorMessage(cause, tokens), cause); + } + + public PhoenixParserException(String msg, Throwable throwable) { + super(new SQLExceptionInfo.Builder(getErrorCode(throwable)).setRootCause(throwable) + .setMessage(msg).build().toString(), + getErrorCode(throwable).getSQLState(), getErrorCode(throwable).getErrorCode(), throwable); + } + + public static String getLine(RecognitionException e) { + return Integer.toString(e.token.getLine()); + } + + public static String getColumn(RecognitionException e) { + return Integer.toString(e.token.getCharPositionInLine() + 1); + } + + public static String getTokenLocation(RecognitionException e) { + return "line " + getLine(e) + ", column " + getColumn(e) + "."; + } + + public static String getErrorMessage(Throwable e, String[] tokenNames) { + String msg; + if (e instanceof MissingTokenException) { + MissingTokenException mte = (MissingTokenException)e; + String tokenName; + if (mte.expecting== Token.EOF) { + tokenName = "EOF"; + } else { + tokenName = tokenNames[mte.expecting]; + } + msg = "Missing \""+ tokenName +"\" at "+ getTokenLocation(mte); + } else if (e instanceof UnwantedTokenException) { + UnwantedTokenException ute = (UnwantedTokenException)e; + String tokenName; + if (ute.expecting== Token.EOF) { + tokenName = "EOF"; + } else { + tokenName = tokenNames[ute.expecting]; + } + msg = "Unexpected input. Expecting \"" + tokenName + "\", got \"" + ute.getUnexpectedToken().getText() + + "\" at " + getTokenLocation(ute); + } else if (e instanceof MismatchedTokenException) { + MismatchedTokenException mte = (MismatchedTokenException)e; + String tokenName; + if (mte.expecting== Token.EOF) { + tokenName = "EOF"; + } else { + tokenName = tokenNames[mte.expecting]; + } + msg = "Mismatched input. Expecting \"" + tokenName + "\", got \"" + mte.token.getText() + + "\" at " + getTokenLocation(mte); + } else if (e instanceof RecognitionException){ + RecognitionException re = (RecognitionException) e; + msg = "Encountered \"" + re.token.getText() + "\" at " + getTokenLocation(re); + } else if (e instanceof UnknownFunctionException) { + UnknownFunctionException ufe = (UnknownFunctionException) e; + msg = "Unknown function: \"" + ufe.getFuncName() + "\"."; + } else { + msg = e.getMessage(); + } + return msg; + } + + public static SQLExceptionCode getErrorCode(Throwable e) { + if (e instanceof MissingTokenException) { + return SQLExceptionCode.MISSING_TOKEN; + } else if (e instanceof UnwantedTokenException) { + return SQLExceptionCode.UNWANTED_TOKEN; + } else if (e instanceof MismatchedTokenException) { + return SQLExceptionCode.MISMATCHED_TOKEN; + } else if (e instanceof UnknownFunctionException) { + return SQLExceptionCode.UNKNOWN_FUNCTION; + } else { + return SQLExceptionCode.PARSER_ERROR; + } + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/SQLExceptionCode.java b/phoenix-core/src/main/java/org/apache/phoenix/exception/SQLExceptionCode.java similarity index 84% rename from phoenix-core/src/main/java/com/salesforce/phoenix/exception/SQLExceptionCode.java rename to phoenix-core/src/main/java/org/apache/phoenix/exception/SQLExceptionCode.java index 58862d31..677328eb 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/exception/SQLExceptionCode.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/exception/SQLExceptionCode.java @@ -1,52 +1,45 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.exception; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.exception; import java.sql.SQLException; import java.util.Map; +import org.apache.hadoop.hbase.index.util.IndexManagementUtil; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.schema.AmbiguousColumnException; +import org.apache.phoenix.schema.AmbiguousTableException; +import org.apache.phoenix.schema.ColumnAlreadyExistsException; +import org.apache.phoenix.schema.ColumnFamilyNotFoundException; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.ConcurrentTableMutationException; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.ReadOnlyTableException; +import org.apache.phoenix.schema.SequenceAlreadyExistsException; +import org.apache.phoenix.schema.SequenceNotFoundException; +import org.apache.phoenix.schema.TableAlreadyExistsException; +import org.apache.phoenix.schema.TableNotFoundException; +import org.apache.phoenix.schema.TypeMismatchException; +import org.apache.phoenix.util.MetaDataUtil; + import com.google.common.collect.Maps; -import com.salesforce.hbase.index.util.IndexManagementUtil; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.schema.AmbiguousColumnException; -import com.salesforce.phoenix.schema.AmbiguousTableException; -import com.salesforce.phoenix.schema.ColumnAlreadyExistsException; -import com.salesforce.phoenix.schema.ColumnFamilyNotFoundException; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.ConcurrentTableMutationException; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.ReadOnlyTableException; -import com.salesforce.phoenix.schema.SequenceAlreadyExistsException; -import com.salesforce.phoenix.schema.SequenceNotFoundException; -import com.salesforce.phoenix.schema.TableAlreadyExistsException; -import com.salesforce.phoenix.schema.TableNotFoundException; -import com.salesforce.phoenix.schema.TypeMismatchException; -import com.salesforce.phoenix.util.MetaDataUtil; /** @@ -212,7 +205,7 @@ public SQLException newException(SQLExceptionInfo info) { + IndexManagementUtil.INDEX_WAL_EDIT_CODEC_CLASS_NAME + " in the hbase-sites.xml of every region server"), - CANNOT_CREATE_TENANT_SPECIFIC_BASE_TABLE(1030, "42Y89", "Cannot create table for tenant-specific connection"), + CANNOT_CREATE_TENANT_SPECIFIC_TABLE(1030, "42Y89", "Cannot create table for tenant-specific connection"), CANNOT_DEFINE_PK_FOR_VIEW(1031, "42Y90", "Defining PK columns for a VIEW is not allowed."), DEFAULT_COLUMN_FAMILY_ONLY_ON_CREATE_TABLE(1034, "42Y93", "Default column family may only be specified when creating a table."), INSUFFICIENT_MULTI_TENANT_COLUMNS(1040, "42Y96", "A MULTI_TENANT table must have 2 or more PK columns with the first column being VARCHAR or CHAR."), diff --git a/phoenix-core/src/main/java/org/apache/phoenix/exception/SQLExceptionInfo.java b/phoenix-core/src/main/java/org/apache/phoenix/exception/SQLExceptionInfo.java new file mode 100644 index 00000000..e0250547 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/exception/SQLExceptionInfo.java @@ -0,0 +1,166 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.exception; + +import java.sql.SQLException; + +import org.apache.phoenix.util.SchemaUtil; + + +/** + * Object serves as a closure of all coordinate information for SQLException messages. + * + * @author zhuang + * @since 1.0 + */ +public class SQLExceptionInfo { + + /** + * Constants used in naming exception location. + */ + public static final String SCHEMA_NAME = "schemaName"; + public static final String TABLE_NAME = "tableName"; + public static final String FAMILY_NAME = "familyName"; + public static final String COLUMN_NAME = "columnName"; + + private final Throwable rootCause; + private final SQLExceptionCode code; // Should always have one. + private final String message; + private final String schemaName; + private final String tableName; + private final String familyName; + private final String columnName; + + public static class Builder { + + private Throwable rootCause; + private SQLExceptionCode code; // Should always have one. + private String message; + private String schemaName; + private String tableName; + private String familyName; + private String columnName; + + public Builder(SQLExceptionCode code) { + this.code = code; + } + + public Builder setRootCause(Throwable t) { + this.rootCause = t; + return this; + } + + public Builder setMessage(String message) { + this.message = message; + return this; + } + + public Builder setSchemaName(String schemaName) { + this.schemaName = schemaName; + return this; + } + + public Builder setTableName(String tableName) { + this.tableName = tableName; + return this; + } + + public Builder setFamilyName(String familyName) { + this.familyName = familyName; + return this; + } + + public Builder setColumnName(String columnName) { + this.columnName = columnName; + return this; + } + + public SQLExceptionInfo build() { + return new SQLExceptionInfo(this); + } + + @Override + public String toString() { + return code.toString(); + } + } + + private SQLExceptionInfo(Builder builder) { + code = builder.code; + rootCause = builder.rootCause; + message = builder.message; + schemaName = builder.schemaName; + tableName = builder.tableName; + familyName = builder.familyName; + columnName = builder.columnName; + } + + @Override + public String toString() { + StringBuilder builder = new StringBuilder(code.toString()); + if (message != null) { + builder.append(" ").append(message); + } + String columnDisplayName = SchemaUtil.getMetaDataEntityName(schemaName, tableName, familyName, columnName); + if (columnName != null) { + builder.append(" ").append(COLUMN_NAME).append("=").append(columnDisplayName); + } else if (familyName != null) { + builder.append(" ").append(FAMILY_NAME).append("=").append(columnDisplayName); + } else if (tableName != null) { + builder.append(" ").append(TABLE_NAME).append("=").append(columnDisplayName); + } else if (schemaName != null) { + builder.append(" ").append(SCHEMA_NAME).append("=").append(columnDisplayName); + } + return builder.toString(); + } + + public SQLException buildException() { + return code.getExceptionFactory().newException(this); + } + + public Throwable getRootCause() { + return rootCause; + } + + public String getSchemaName() { + return schemaName; + } + + public String getTableName() { + return tableName; + } + + public String getFamilyName() { + return familyName; + } + + public String getColumnName() { + return columnName; + } + + public SQLExceptionCode getCode() { + return code; + } + + public String getMessage() { + return message; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/exception/UnknownFunctionException.java b/phoenix-core/src/main/java/org/apache/phoenix/exception/UnknownFunctionException.java new file mode 100644 index 00000000..1f669927 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/exception/UnknownFunctionException.java @@ -0,0 +1,37 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.exception; + +/** + * Thrown by ParseNodeFactory when it could not identify a node as a valid function. + */ +public class UnknownFunctionException extends RuntimeException { + private static final long serialVersionUID = 1L; + private final String funcName; + + public UnknownFunctionException(String funcName) { + super(); + this.funcName = funcName; + } + + public String getFuncName() { + return funcName; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/exception/ValueTypeIncompatibleException.java b/phoenix-core/src/main/java/org/apache/phoenix/exception/ValueTypeIncompatibleException.java new file mode 100644 index 00000000..403c0093 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/exception/ValueTypeIncompatibleException.java @@ -0,0 +1,38 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.exception; + +import org.apache.phoenix.schema.IllegalDataException; +import org.apache.phoenix.schema.PDataType; + + +public class ValueTypeIncompatibleException extends IllegalDataException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.DATA_INCOMPATIBLE_WITH_TYPE; + + public ValueTypeIncompatibleException(PDataType type, Integer precision, Integer scale) { + super(new SQLExceptionInfo.Builder(code).setMessage(getTypeDisplayString(type, precision, scale)) + .build().toString()); + } + + private static String getTypeDisplayString(PDataType type, Integer precision, Integer scale) { + return type.toString() + "(" + precision + "," + scale + ")"; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/execute/AggregatePlan.java b/phoenix-core/src/main/java/org/apache/phoenix/execute/AggregatePlan.java new file mode 100644 index 00000000..b16a4e1d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/execute/AggregatePlan.java @@ -0,0 +1,171 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.execute; + + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.OrderByCompiler.OrderBy; +import org.apache.phoenix.compile.RowProjector; +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.OrderByExpression; +import org.apache.phoenix.expression.RowKeyExpression; +import org.apache.phoenix.expression.aggregator.Aggregators; +import org.apache.phoenix.iterate.AggregatingResultIterator; +import org.apache.phoenix.iterate.ConcatResultIterator; +import org.apache.phoenix.iterate.DistinctAggregatingResultIterator; +import org.apache.phoenix.iterate.FilterAggregatingResultIterator; +import org.apache.phoenix.iterate.GroupedAggregatingResultIterator; +import org.apache.phoenix.iterate.LimitingResultIterator; +import org.apache.phoenix.iterate.MergeSortRowKeyResultIterator; +import org.apache.phoenix.iterate.OrderedAggregatingResultIterator; +import org.apache.phoenix.iterate.OrderedResultIterator; +import org.apache.phoenix.iterate.ParallelIterators; +import org.apache.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; +import org.apache.phoenix.iterate.PeekingResultIterator; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.iterate.SequenceResultIterator; +import org.apache.phoenix.iterate.SpoolingResultIterator; +import org.apache.phoenix.iterate.UngroupedAggregatingResultIterator; +import org.apache.phoenix.parse.FilterableStatement; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.TableRef; + + + +/** + * + * Query plan for aggregating queries + * + * @author jtaylor + * @since 0.1 + */ +public class AggregatePlan extends BasicQueryPlan { + private final Aggregators aggregators; + private final Expression having; + private List splits; + + public AggregatePlan( + StatementContext context, FilterableStatement statement, TableRef table, RowProjector projector, + Integer limit, OrderBy orderBy, ParallelIteratorFactory parallelIteratorFactory, GroupBy groupBy, + Expression having) { + super(context, statement, table, projector, context.getBindManager().getParameterMetaData(), limit, orderBy, groupBy, parallelIteratorFactory); + this.having = having; + this.aggregators = context.getAggregationManager().getAggregators(); + } + + @Override + public List getSplits() { + return splits; + } + + private static class OrderingResultIteratorFactory implements ParallelIteratorFactory { + private final QueryServices services; + + public OrderingResultIteratorFactory(QueryServices services) { + this.services = services; + } + @Override + public PeekingResultIterator newIterator(ResultIterator scanner) throws SQLException { + Expression expression = RowKeyExpression.INSTANCE; + OrderByExpression orderByExpression = new OrderByExpression(expression, false, true); + int threshold = services.getProps().getInt(QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB, QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES); + return new OrderedResultIterator(scanner, Collections.singletonList(orderByExpression), threshold); + } + } + + private static class WrappingResultIteratorFactory implements ParallelIteratorFactory { + private final ParallelIteratorFactory innerFactory; + private final ParallelIteratorFactory outerFactory; + + public WrappingResultIteratorFactory(ParallelIteratorFactory innerFactory, ParallelIteratorFactory outerFactory) { + this.innerFactory = innerFactory; + this.outerFactory = outerFactory; + } + @Override + public PeekingResultIterator newIterator(ResultIterator scanner) throws SQLException { + PeekingResultIterator iterator = innerFactory.newIterator(scanner); + return outerFactory.newIterator(iterator); + } + } + + private ParallelIteratorFactory wrapParallelIteratorFactory () { + ParallelIteratorFactory innerFactory; + QueryServices services = context.getConnection().getQueryServices(); + if (groupBy.isEmpty() || groupBy.isOrderPreserving()) { + innerFactory = new SpoolingResultIterator.SpoolingResultIteratorFactory(services); + } else { + innerFactory = new OrderingResultIteratorFactory(services); + } + if (parallelIteratorFactory == null) { + return innerFactory; + } + // wrap any existing parallelIteratorFactory + return new WrappingResultIteratorFactory(innerFactory, parallelIteratorFactory); + } + + @Override + protected ResultIterator newIterator() throws SQLException { + if (groupBy.isEmpty()) { + UngroupedAggregateRegionObserver.serializeIntoScan(context.getScan()); + } + ParallelIterators parallelIterators = new ParallelIterators(context, tableRef, statement, projection, groupBy, null, wrapParallelIteratorFactory()); + splits = parallelIterators.getSplits(); + + AggregatingResultIterator aggResultIterator; + // No need to merge sort for ungrouped aggregation + if (groupBy.isEmpty()) { + aggResultIterator = new UngroupedAggregatingResultIterator(new ConcatResultIterator(parallelIterators), aggregators); + } else { + aggResultIterator = new GroupedAggregatingResultIterator(new MergeSortRowKeyResultIterator(parallelIterators), aggregators); + } + + if (having != null) { + aggResultIterator = new FilterAggregatingResultIterator(aggResultIterator, having); + } + + if (statement.isDistinct() && statement.isAggregate()) { // Dedup on client if select distinct and aggregation + aggResultIterator = new DistinctAggregatingResultIterator(aggResultIterator, getProjector()); + } + + ResultIterator resultScanner = aggResultIterator; + if (orderBy.getOrderByExpressions().isEmpty()) { + if (limit != null) { + resultScanner = new LimitingResultIterator(aggResultIterator, limit); + } + } else { + int thresholdBytes = getConnectionQueryServices(context.getConnection().getQueryServices()).getProps().getInt( + QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB, QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES); + resultScanner = new OrderedAggregatingResultIterator(aggResultIterator, orderBy.getOrderByExpressions(), thresholdBytes, limit); + } + + if (context.getSequenceManager().getSequenceCount() > 0) { + resultScanner = new SequenceResultIterator(resultScanner, context.getSequenceManager()); + } + return resultScanner; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/execute/BasicQueryPlan.java b/phoenix-core/src/main/java/org/apache/phoenix/execute/BasicQueryPlan.java new file mode 100644 index 00000000..2656aaf3 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/execute/BasicQueryPlan.java @@ -0,0 +1,197 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.execute; + +import java.sql.ParameterMetaData; +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import com.google.common.collect.Lists; +import org.apache.phoenix.compile.ExplainPlan; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.OrderByCompiler.OrderBy; +import org.apache.phoenix.compile.QueryPlan; +import org.apache.phoenix.compile.RowProjector; +import org.apache.phoenix.compile.ScanRanges; +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.iterate.DelegateResultIterator; +import org.apache.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.parse.FilterableStatement; +import org.apache.phoenix.query.ConnectionQueryServices; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.SQLCloseable; +import org.apache.phoenix.util.SQLCloseables; +import org.apache.phoenix.util.ScanUtil; + + + +/** + * + * Query plan that has no child plans + * + * @author jtaylor + * @since 0.1 + */ +public abstract class BasicQueryPlan implements QueryPlan { + protected static final long DEFAULT_ESTIMATED_SIZE = 10 * 1024; // 10 K + + protected final TableRef tableRef; + protected final StatementContext context; + protected final FilterableStatement statement; + protected final RowProjector projection; + protected final ParameterMetaData paramMetaData; + protected final Integer limit; + protected final OrderBy orderBy; + protected final GroupBy groupBy; + protected final ParallelIteratorFactory parallelIteratorFactory; + + protected BasicQueryPlan( + StatementContext context, FilterableStatement statement, TableRef table, + RowProjector projection, ParameterMetaData paramMetaData, Integer limit, OrderBy orderBy, + GroupBy groupBy, ParallelIteratorFactory parallelIteratorFactory) { + this.context = context; + this.statement = statement; + this.tableRef = table; + this.projection = projection; + this.paramMetaData = paramMetaData; + this.limit = limit; + this.orderBy = orderBy; + this.groupBy = groupBy; + this.parallelIteratorFactory = parallelIteratorFactory; + } + + @Override + public GroupBy getGroupBy() { + return groupBy; + } + + + @Override + public OrderBy getOrderBy() { + return orderBy; + } + + @Override + public TableRef getTableRef() { + return tableRef; + } + + @Override + public Integer getLimit() { + return limit; + } + + @Override + public RowProjector getProjector() { + return projection; + } + + protected ConnectionQueryServices getConnectionQueryServices(ConnectionQueryServices services) { + // Get child services associated with tenantId of query. + ConnectionQueryServices childServices = context.getConnection().getTenantId() == null ? + services : + services.getChildQueryServices(new ImmutableBytesWritable(context.getConnection().getTenantId().getBytes())); + return childServices; + } + +// /** +// * Sets up an id used to do round robin queue processing on the server +// * @param scan +// */ +// private void setProducer(Scan scan) { +// byte[] producer = Bytes.toBytes(UUID.randomUUID().toString()); +// scan.setAttribute(HBaseServer.CALL_QUEUE_PRODUCER_ATTRIB_NAME, producer); +// } + + @Override + public final ResultIterator iterator() throws SQLException { + return iterator(Collections.emptyList()); + } + + public final ResultIterator iterator(final List dependencies) throws SQLException { + if (context.getScanRanges() == ScanRanges.NOTHING) { + return ResultIterator.EMPTY_ITERATOR; + } + + Scan scan = context.getScan(); + // Set producer on scan so HBase server does round robin processing + //setProducer(scan); + // Set the time range on the scan so we don't get back rows newer than when the statement was compiled + // The time stamp comes from the server at compile time when the meta data + // is resolved. + // TODO: include time range in explain plan? + PhoenixConnection connection = context.getConnection(); + Long scn = connection.getSCN(); + ScanUtil.setTimeRange(scan, scn == null ? context.getCurrentTime() : scn); + ScanUtil.setTenantId(scan, connection.getTenantId() == null ? null : connection.getTenantId().getBytes()); + ResultIterator iterator = newIterator(); + return dependencies.isEmpty() ? + iterator : new DelegateResultIterator(iterator) { + @Override + public void close() throws SQLException { + try { + super.close(); + } finally { + SQLCloseables.closeAll(dependencies); + } + } + }; + } + + abstract protected ResultIterator newIterator() throws SQLException; + + @Override + public long getEstimatedSize() { + return DEFAULT_ESTIMATED_SIZE; + } + + @Override + public ParameterMetaData getParameterMetaData() { + return paramMetaData; + } + + @Override + public FilterableStatement getStatement() { + return statement; + } + + @Override + public StatementContext getContext() { + return context; + } + + @Override + public ExplainPlan getExplainPlan() throws SQLException { + if (context.getScanRanges() == ScanRanges.NOTHING) { + return new ExplainPlan(Collections.singletonList("DEGENERATE SCAN OVER " + tableRef.getTable().getName().getString())); + } + + ResultIterator iterator = iterator(); + List planSteps = Lists.newArrayListWithExpectedSize(5); + iterator.explain(planSteps); + return new ExplainPlan(planSteps); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/execute/CommitException.java b/phoenix-core/src/main/java/org/apache/phoenix/execute/CommitException.java new file mode 100644 index 00000000..4ca47ec8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/execute/CommitException.java @@ -0,0 +1,43 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.execute; + +import java.sql.SQLException; + +public class CommitException extends SQLException { + private static final long serialVersionUID = 1L; + private final MutationState uncommittedState; + private final MutationState committedState; + + public CommitException(Exception e, MutationState uncommittedState, MutationState committedState) { + super(e); + this.uncommittedState = uncommittedState; + this.committedState = committedState; + } + + public MutationState getUncommittedState() { + return uncommittedState; + } + + public MutationState getCommittedState() { + return committedState; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/execute/DegenerateQueryPlan.java b/phoenix-core/src/main/java/org/apache/phoenix/execute/DegenerateQueryPlan.java new file mode 100644 index 00000000..ba38faee --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/execute/DegenerateQueryPlan.java @@ -0,0 +1,52 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.execute; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.OrderByCompiler.OrderBy; +import org.apache.phoenix.compile.*; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.jdbc.PhoenixParameterMetaData; +import org.apache.phoenix.parse.FilterableStatement; +import org.apache.phoenix.query.*; +import org.apache.phoenix.schema.TableRef; + +public class DegenerateQueryPlan extends BasicQueryPlan { + + public DegenerateQueryPlan(StatementContext context, FilterableStatement statement, TableRef table) { + super(context, statement, table, RowProjector.EMPTY_PROJECTOR, PhoenixParameterMetaData.EMPTY_PARAMETER_META_DATA, null, OrderBy.EMPTY_ORDER_BY, GroupBy.EMPTY_GROUP_BY, null); + context.setScanRanges(ScanRanges.NOTHING); + } + + @Override + public List getSplits() { + return Collections.emptyList(); + } + + @Override + protected ResultIterator newIterator() throws SQLException { + return null; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/execute/HashJoinPlan.java b/phoenix-core/src/main/java/org/apache/phoenix/execute/HashJoinPlan.java new file mode 100644 index 00000000..73b456a5 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/execute/HashJoinPlan.java @@ -0,0 +1,186 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.execute; + +import java.sql.ParameterMetaData; +import java.sql.SQLException; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Future; +import org.apache.hadoop.hbase.client.Scan; + +import com.google.common.collect.Lists; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.cache.ServerCacheClient.ServerCache; +import org.apache.phoenix.compile.ExplainPlan; +import org.apache.phoenix.compile.QueryPlan; +import org.apache.phoenix.compile.RowProjector; +import org.apache.phoenix.compile.ScanRanges; +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.OrderByCompiler.OrderBy; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.job.JobManager.JobCallable; +import org.apache.phoenix.join.HashCacheClient; +import org.apache.phoenix.join.HashJoinInfo; +import org.apache.phoenix.parse.FilterableStatement; +import org.apache.phoenix.query.ConnectionQueryServices; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.SQLCloseable; + +public class HashJoinPlan implements QueryPlan { + + private BasicQueryPlan plan; + private HashJoinInfo joinInfo; + private List[] hashExpressions; + private QueryPlan[] hashPlans; + + public HashJoinPlan(BasicQueryPlan plan, HashJoinInfo joinInfo, + List[] hashExpressions, QueryPlan[] hashPlans) { + this.plan = plan; + this.joinInfo = joinInfo; + this.hashExpressions = hashExpressions; + this.hashPlans = hashPlans; + } + + @Override + public Integer getLimit() { + return plan.getLimit(); + } + + @Override + public OrderBy getOrderBy() { + return plan.getOrderBy(); + } + + @Override + public RowProjector getProjector() { + return plan.getProjector(); + } + + @Override + public ResultIterator iterator() throws SQLException { + ImmutableBytesPtr[] joinIds = joinInfo.getJoinIds(); + assert (joinIds.length == hashExpressions.length && joinIds.length == hashPlans.length); + + final HashCacheClient hashClient = new HashCacheClient(plan.getContext().getConnection()); + Scan scan = plan.getContext().getScan(); + final ScanRanges ranges = plan.getContext().getScanRanges(); + + int count = joinIds.length; + ConnectionQueryServices services = getContext().getConnection().getQueryServices(); + ExecutorService executor = services.getExecutor(); + List> futures = new ArrayList>(count); + List dependencies = new ArrayList(count); + for (int i = 0; i < count; i++) { + final int index = i; + futures.add(executor.submit(new JobCallable() { + + @Override + public ServerCache call() throws Exception { + QueryPlan hashPlan = hashPlans[index]; + return hashClient.addHashCache(ranges, hashPlan.iterator(), + hashPlan.getEstimatedSize(), hashExpressions[index], plan.getTableRef()); + } + + @Override + public Object getJobId() { + return HashJoinPlan.this; + } + })); + } + for (int i = 0; i < count; i++) { + try { + ServerCache cache = futures.get(i).get(); + joinIds[i].set(cache.getId()); + dependencies.add(cache); + } catch (InterruptedException e) { + throw new SQLException("Hash join execution interrupted.", e); + } catch (ExecutionException e) { + throw new SQLException("Encountered exception in hash plan execution.", + e.getCause()); + } + } + HashJoinInfo.serializeHashJoinIntoScan(scan, joinInfo); + + return plan.iterator(dependencies); + } + + @Override + public long getEstimatedSize() { + return plan.getEstimatedSize(); + } + + @Override + public List getSplits() { + return plan.getSplits(); + } + + @Override + public ExplainPlan getExplainPlan() throws SQLException { + List mainQuerySteps = plan.getExplainPlan().getPlanSteps(); + List planSteps = Lists.newArrayList(mainQuerySteps); + int count = hashPlans.length; + planSteps.add(" PARALLEL EQUI-JOIN " + count + " HASH TABLES:"); + for (int i = 0; i < count; i++) { + boolean skipMerge = joinInfo.getSchemas()[i].getFieldCount() == 0; + planSteps.add(" BUILD HASH TABLE " + i + (skipMerge ? " (SKIP MERGE)" : "")); + List steps = hashPlans[i].getExplainPlan().getPlanSteps(); + for (String step : steps) { + planSteps.add(" " + step); + } + } + if (joinInfo.getPostJoinFilterExpression() != null) { + planSteps.add(" AFTER-JOIN SERVER FILTER BY " + joinInfo.getPostJoinFilterExpression().toString()); + } + + return new ExplainPlan(planSteps); + } + + @Override + public ParameterMetaData getParameterMetaData() { + return plan.getParameterMetaData(); + } + + @Override + public StatementContext getContext() { + return plan.getContext(); + } + + @Override + public GroupBy getGroupBy() { + return plan.getGroupBy(); + } + + @Override + public TableRef getTableRef() { + return plan.getTableRef(); + } + + @Override + public FilterableStatement getStatement() { + return plan.getStatement(); + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/MutationState.java b/phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java similarity index 87% rename from phoenix-core/src/main/java/com/salesforce/phoenix/execute/MutationState.java rename to phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java index c883295f..298c7c10 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/execute/MutationState.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/execute/MutationState.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.execute; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.execute; import java.io.IOException; import java.sql.SQLException; @@ -46,27 +38,27 @@ import com.google.common.collect.Iterators; import com.google.common.collect.Lists; import com.google.common.collect.Maps; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.cache.ServerCacheClient; -import com.salesforce.phoenix.cache.ServerCacheClient.ServerCache; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.index.IndexMaintainer; -import com.salesforce.phoenix.index.IndexMetaDataCacheClient; -import com.salesforce.phoenix.index.PhoenixIndexCodec; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.IllegalDataException; -import com.salesforce.phoenix.schema.MetaDataClient; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PRow; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.IndexUtil; -import com.salesforce.phoenix.util.PhoenixRuntime; -import com.salesforce.phoenix.util.SQLCloseable; -import com.salesforce.phoenix.util.ServerUtil; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.cache.ServerCacheClient; +import org.apache.phoenix.cache.ServerCacheClient.ServerCache; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.index.IndexMaintainer; +import org.apache.phoenix.index.IndexMetaDataCacheClient; +import org.apache.phoenix.index.PhoenixIndexCodec; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.IllegalDataException; +import org.apache.phoenix.schema.MetaDataClient; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PRow; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.IndexUtil; +import org.apache.phoenix.util.PhoenixRuntime; +import org.apache.phoenix.util.SQLCloseable; +import org.apache.phoenix.util.ServerUtil; /** * diff --git a/phoenix-core/src/main/java/org/apache/phoenix/execute/ScanPlan.java b/phoenix-core/src/main/java/org/apache/phoenix/execute/ScanPlan.java new file mode 100644 index 00000000..01199c98 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/execute/ScanPlan.java @@ -0,0 +1,116 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.execute; + + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.OrderByCompiler.OrderBy; +import org.apache.phoenix.compile.RowProjector; +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.coprocessor.ScanRegionObserver; +import org.apache.phoenix.iterate.ConcatResultIterator; +import org.apache.phoenix.iterate.LimitingResultIterator; +import org.apache.phoenix.iterate.MergeSortRowKeyResultIterator; +import org.apache.phoenix.iterate.MergeSortTopNResultIterator; +import org.apache.phoenix.iterate.ParallelIterators; +import org.apache.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.iterate.SequenceResultIterator; +import org.apache.phoenix.iterate.SpoolingResultIterator; +import org.apache.phoenix.parse.FilterableStatement; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.SaltingUtil; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.ScanUtil; + + + +/** + * + * Query plan for a basic table scan + * + * @author jtaylor + * @since 0.1 + */ +public class ScanPlan extends BasicQueryPlan { + private List splits; + + public ScanPlan(StatementContext context, FilterableStatement statement, TableRef table, RowProjector projector, Integer limit, OrderBy orderBy, ParallelIteratorFactory parallelIteratorFactory) { + super(context, statement, table, projector, context.getBindManager().getParameterMetaData(), limit, orderBy, null, parallelIteratorFactory == null ? new SpoolingResultIterator.SpoolingResultIteratorFactory(context.getConnection().getQueryServices()) : parallelIteratorFactory); + if (!orderBy.getOrderByExpressions().isEmpty()) { // TopN + int thresholdBytes = context.getConnection().getQueryServices().getProps().getInt( + QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB, QueryServicesOptions.DEFAULT_SPOOL_THRESHOLD_BYTES); + ScanRegionObserver.serializeIntoScan(context.getScan(), thresholdBytes, limit == null ? -1 : limit, orderBy.getOrderByExpressions(), projector.getEstimatedRowByteSize()); + } + } + + @Override + public List getSplits() { + return splits; + } + + @Override + protected ResultIterator newIterator() throws SQLException { + // Set any scan attributes before creating the scanner, as it will be too late afterwards + context.getScan().setAttribute(ScanRegionObserver.NON_AGGREGATE_QUERY, QueryConstants.TRUE); + if (OrderBy.REV_ROW_KEY_ORDER_BY.equals(orderBy)) { + ScanUtil.setReversed(context.getScan()); + } + ResultIterator scanner; + TableRef tableRef = this.getTableRef(); + PTable table = tableRef.getTable(); + boolean isSalted = table.getBucketNum() != null; + /* If no limit or topN, use parallel iterator so that we get results faster. Otherwise, if + * limit is provided, run query serially. + */ + boolean isOrdered = !orderBy.getOrderByExpressions().isEmpty(); + ParallelIterators iterators = new ParallelIterators(context, tableRef, statement, projection, GroupBy.EMPTY_GROUP_BY, isOrdered ? null : limit, parallelIteratorFactory); + splits = iterators.getSplits(); + if (isOrdered) { + scanner = new MergeSortTopNResultIterator(iterators, limit, orderBy.getOrderByExpressions()); + } else { + if (isSalted && + (getConnectionQueryServices(context.getConnection().getQueryServices()).getProps().getBoolean( + QueryServices.ROW_KEY_ORDER_SALTED_TABLE_ATTRIB, + QueryServicesOptions.DEFAULT_ROW_KEY_ORDER_SALTED_TABLE) || + orderBy == OrderBy.FWD_ROW_KEY_ORDER_BY || + orderBy == OrderBy.REV_ROW_KEY_ORDER_BY)) { // ORDER BY was optimized out b/c query is in row key order + scanner = new MergeSortRowKeyResultIterator(iterators, SaltingUtil.NUM_SALTING_BYTES, orderBy == OrderBy.REV_ROW_KEY_ORDER_BY); + } else { + scanner = new ConcatResultIterator(iterators); + } + if (limit != null) { + scanner = new LimitingResultIterator(scanner, limit); + } + } + + if (context.getSequenceManager().getSequenceCount() > 0) { + scanner = new SequenceResultIterator(scanner, context.getSequenceManager()); + } + return scanner; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/AddExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/AddExpression.java new file mode 100644 index 00000000..fd906c5f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/AddExpression.java @@ -0,0 +1,56 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; + + +/** + * + * Subtract expression implementation + * + * @author jtaylor + * @since 0.1 + */ +public abstract class AddExpression extends BaseAddSubtractExpression { + public AddExpression() { + } + + public AddExpression(List children) { + super(children); + } + + @Override + public final T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } + + @Override + public String getOperatorString() { + return " + "; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/AndExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/AndExpression.java new file mode 100644 index 00000000..c35eca51 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/AndExpression.java @@ -0,0 +1,79 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; + + +/** + * + * AND expression implementation + * + * @author jtaylor + * @since 0.1 + */ +public class AndExpression extends AndOrExpression { + private static final String AND = "AND"; + + public static String combine(String expression1, String expression2) { + if (expression1 == null) { + return expression2; + } + if (expression2 == null) { + return expression1; + } + return "(" + expression1 + ") " + AND + " (" + expression2 + ")"; + } + + public AndExpression() { + } + + public AndExpression(List children) { + super(children); + } + + @Override + protected boolean getStopValue() { + return Boolean.FALSE; + } + + @Override + public String toString() { + StringBuilder buf = new StringBuilder("("); + for (int i = 0; i < children.size() - 1; i++) { + buf.append(children.get(i) + " " + AND + " "); + } + buf.append(children.get(children.size()-1)); + buf.append(')'); + return buf.toString(); + } + + @Override + public final T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/AndOrExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/AndOrExpression.java new file mode 100644 index 00000000..aebd63ae --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/AndOrExpression.java @@ -0,0 +1,99 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.BitSet; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Abstract expression implementation for compound AND and OR expressions + * + * @author jtaylor + * @since 0.1 + */ +public abstract class AndOrExpression extends BaseCompoundExpression { + // Remember evaluation of child expression for partial evaluation + private BitSet partialEvalState; + + public AndOrExpression() { + } + + public AndOrExpression(List children) { + super(children); + } + + @Override + public int hashCode() { + return 31 * super.hashCode() + Boolean.valueOf(this.getStopValue()).hashCode(); + } + + @Override + public PDataType getDataType() { + return PDataType.BOOLEAN; + } + + @Override + public void reset() { + if (partialEvalState == null) { + partialEvalState = new BitSet(children.size()); + } else { + partialEvalState.clear(); + } + super.reset(); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + boolean isNull = false; + boolean stopValue = getStopValue(); + for (int i = 0; i < children.size(); i++) { + Expression child = children.get(i); + // If partial state is available, then use that to know we've already evaluated this + // child expression and do not need to do so again. + if (partialEvalState == null || !partialEvalState.get(i)) { + // Call through to child evaluate method matching parent call to allow child to optimize + // evaluate versus getValue code path. + if (child.evaluate(tuple, ptr)) { + // Short circuit if we see our stop value + if (Boolean.valueOf(stopValue).equals(PDataType.BOOLEAN.toObject(ptr, child.getDataType()))) { + return true; + } else if (partialEvalState != null) { + partialEvalState.set(i); + } + } else { + isNull = true; + } + } + } + if (isNull) { + return false; + } + return true; + } + + protected abstract boolean getStopValue(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/ArithmeticExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/ArithmeticExpression.java new file mode 100644 index 00000000..622d709c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/ArithmeticExpression.java @@ -0,0 +1,45 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +public abstract class ArithmeticExpression extends BaseCompoundExpression { + + public ArithmeticExpression() { + } + + public ArithmeticExpression(List children) { + super(children); + } + + @Override + public String toString() { + StringBuilder buf = new StringBuilder("("); + for (int i = 0; i < children.size() - 1; i++) { + buf.append(children.get(i) + getOperatorString()); + } + buf.append(children.get(children.size()-1)); + buf.append(')'); + return buf.toString(); + } + + abstract protected String getOperatorString(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/ArrayConstructorExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/ArrayConstructorExpression.java new file mode 100644 index 00000000..6dbf8650 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/ArrayConstructorExpression.java @@ -0,0 +1,101 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; +import java.sql.Types; +import java.util.Arrays; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.io.WritableUtils; + +import org.apache.phoenix.schema.PArrayDataType; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PhoenixArray; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * Creates an expression for Upsert with Values/Select using ARRAY + */ +public class ArrayConstructorExpression extends BaseCompoundExpression { + private PDataType baseType; + private int position = -1; + private Object[] elements; + + + public ArrayConstructorExpression(List children, PDataType baseType) { + super(children); + init(baseType); + } + + private void init(PDataType baseType) { + this.baseType = baseType; + elements = new Object[getChildren().size()]; + } + + @Override + public PDataType getDataType() { + return PDataType.fromTypeId(baseType.getSqlType() + Types.ARRAY); + } + + @Override + public void reset() { + super.reset(); + position = 0; + Arrays.fill(elements, null); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + for (int i = position >= 0 ? position : 0; i < elements.length; i++) { + Expression child = children.get(i); + if (!child.evaluate(tuple, ptr)) { + if (tuple != null && !tuple.isImmutable()) { + if (position >= 0) position = i; + return false; + } + } else { + elements[i] = baseType.toObject(ptr, child.getDataType(), child.getColumnModifier()); + } + } + if (position >= 0) position = elements.length; + PhoenixArray array = PArrayDataType.instantiatePhoenixArray(baseType, elements); + // FIXME: Need to see if this creation of an array and again back to byte[] can be avoided + ptr.set(getDataType().toBytes(array)); + return true; + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + int baseTypeOrdinal = WritableUtils.readVInt(input); + init(PDataType.values()[baseTypeOrdinal]); + } + + @Override + public void write(DataOutput output) throws IOException { + super.write(output); + WritableUtils.writeVInt(output, baseType.ordinal()); + } + +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseAddSubtractExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseAddSubtractExpression.java new file mode 100644 index 00000000..1b9e4e54 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseAddSubtractExpression.java @@ -0,0 +1,53 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.phoenix.schema.PDataType; + + +abstract public class BaseAddSubtractExpression extends ArithmeticExpression { + public BaseAddSubtractExpression() { + } + + public BaseAddSubtractExpression(List children) { + super(children); + } + + protected static Integer getPrecision(Integer lp, Integer rp, Integer ls, Integer rs) { + if (ls == null || rs == null) { + return PDataType.MAX_PRECISION; + } + int val = getScale(lp, rp, ls, rs) + Math.max(lp - ls, rp - rs) + 1; + return Math.min(PDataType.MAX_PRECISION, val); + } + + protected static Integer getScale(Integer lp, Integer rp, Integer ls, Integer rs) { + // If we are adding a decimal with scale and precision to a decimal + // with no precision nor scale, the scale system does not apply. + if (ls == null || rs == null) { + return null; + } + int val = Math.max(ls, rs); + return Math.min(PDataType.MAX_PRECISION, val); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseCompoundExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseCompoundExpression.java new file mode 100644 index 00000000..82a130d3 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseCompoundExpression.java @@ -0,0 +1,145 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; + +import org.apache.hadoop.io.WritableUtils; + +import com.google.common.collect.ImmutableList; +import org.apache.phoenix.expression.visitor.ExpressionVisitor; + + +public abstract class BaseCompoundExpression extends BaseExpression { + protected List children; + private boolean isNullable; + private boolean isStateless; + private boolean isDeterministic; + + public BaseCompoundExpression() { + } + + public BaseCompoundExpression(List children) { + init(children); + } + + private void init(List children) { + this.children = ImmutableList.copyOf(children); + boolean isStateless = true; + boolean isDeterministic = true; + boolean isNullable = false; + for (int i = 0; i < children.size(); i++) { + Expression child = children.get(i); + isNullable |= child.isNullable(); + isStateless &= child.isStateless(); + isDeterministic &= child.isDeterministic(); + } + this.isStateless = isStateless; + this.isDeterministic = isDeterministic; + this.isNullable = isNullable; + } + + @Override + public List getChildren() { + return children; + } + + + @Override + public boolean isDeterministic() { + return isDeterministic; + } + + @Override + public boolean isStateless() { + return isStateless; + } + + @Override + public boolean isNullable() { + return isNullable; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + children.hashCode(); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + BaseCompoundExpression other = (BaseCompoundExpression)obj; + if (!children.equals(other.children)) return false; + return true; + } + + @Override + public void readFields(DataInput input) throws IOException { + int len = WritableUtils.readVInt(input); + Listchildren = new ArrayList(len); + for (int i = 0; i < len; i++) { + Expression child = ExpressionType.values()[WritableUtils.readVInt(input)].newInstance(); + child.readFields(input); + children.add(child); + } + init(children); + } + + @Override + public void write(DataOutput output) throws IOException { + WritableUtils.writeVInt(output, children.size()); + for (int i = 0; i < children.size(); i++) { + Expression child = children.get(i); + WritableUtils.writeVInt(output, ExpressionType.valueOf(child).ordinal()); + child.write(output); + } + } + + @Override + public void reset() { + for (int i = 0; i < children.size(); i++) { + children.get(i).reset(); + } + } + + @Override + public T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } + + @Override + public String toString() { + return this.getClass().getName() + " [children=" + children + "]"; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseDecimalAddSubtractExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseDecimalAddSubtractExpression.java new file mode 100644 index 00000000..4f641fe5 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseDecimalAddSubtractExpression.java @@ -0,0 +1,5 @@ +package org.apache.phoenix.expression; + +public class BaseDecimalAddSubtractExpression { + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseExpression.java new file mode 100644 index 00000000..9bd55813 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseExpression.java @@ -0,0 +1,108 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; +import java.util.ArrayList; +import java.util.Collections; +import java.util.Iterator; +import java.util.List; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.ColumnModifier; + + + +/** + * + * Base class for Expression hierarchy that provides common + * default implementations for most methods + * + * @author jtaylor + * @since 0.1 + */ +public abstract class BaseExpression implements Expression { + @Override + public boolean isNullable() { + return false; + } + + @Override + public Integer getByteSize() { + return getDataType().isFixedWidth() ? getDataType().getByteSize() : null; + } + + @Override + public Integer getMaxLength() { + return null; + } + + @Override + public Integer getScale() { + return null; + } + + @Override + public ColumnModifier getColumnModifier() { + return null; + } + + @Override + public void readFields(DataInput input) throws IOException { + } + + @Override + public void write(DataOutput output) throws IOException { + } + + @Override + public void reset() { + } + + protected final List acceptChildren(ExpressionVisitor visitor, Iterator iterator) { + if (iterator == null) { + iterator = visitor.defaultIterator(this); + } + List l = Collections.emptyList(); + while (iterator.hasNext()) { + Expression child = iterator.next(); + T t = child.accept(visitor); + if (t != null) { + if (l.isEmpty()) { + l = new ArrayList(getChildren().size()); + } + l.add(t); + } + } + return l; + } + + @Override + public boolean isDeterministic() { + return true; + } + + @Override + public boolean isStateless() { + return false; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseSingleExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseSingleExpression.java new file mode 100644 index 00000000..17584387 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseSingleExpression.java @@ -0,0 +1,91 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; +import java.util.List; + +import org.apache.hadoop.io.WritableUtils; + +import com.google.common.collect.ImmutableList; +import org.apache.phoenix.expression.visitor.ExpressionVisitor; + + +/** + * + * Base class for expressions which have a single child expression + * + * @author jtaylor + * @since 0.1 + */ +public abstract class BaseSingleExpression extends BaseExpression { + + protected List children; + + public BaseSingleExpression() { + } + + public BaseSingleExpression(Expression expression) { + this.children = ImmutableList.of(expression); + } + + @Override + public List getChildren() { + return children; + } + + @Override + public void readFields(DataInput input) throws IOException { + Expression expression = ExpressionType.values()[WritableUtils.readVInt(input)].newInstance(); + expression.readFields(input); + children = ImmutableList.of(expression); + } + + @Override + public void write(DataOutput output) throws IOException { + WritableUtils.writeVInt(output, ExpressionType.valueOf(children.get(0)).ordinal()); + children.get(0).write(output); + } + + @Override + public boolean isNullable() { + return children.get(0).isNullable(); + } + + @Override + public void reset() { + children.get(0).reset(); + } + + @Override + public T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, null); + if (l.isEmpty()) { + return visitor.defaultReturn(this, l); + } + return l.get(0); + } + + public Expression getChild() { + return children.get(0); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseTerminalExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseTerminalExpression.java new file mode 100644 index 00000000..aaa43718 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/BaseTerminalExpression.java @@ -0,0 +1,47 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.Collections; +import java.util.List; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; + + + +/** + * + * Grouping class for expression that have no expression children + * + * @author jtaylor + * @since 0.1 + */ +public abstract class BaseTerminalExpression extends BaseExpression { + @Override + public List getChildren() { + return Collections.emptyList(); + } + + @Override + public T accept(ExpressionVisitor visitor) { + return null; + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/CaseExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/CaseExpression.java similarity index 77% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/CaseExpression.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/CaseExpression.java index c10ffe65..f9a524a5 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/CaseExpression.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/CaseExpression.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; import java.io.DataInput; import java.io.DataOutput; @@ -37,11 +29,11 @@ import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.io.WritableUtils; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/CoerceExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/CoerceExpression.java new file mode 100644 index 00000000..827d70e8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/CoerceExpression.java @@ -0,0 +1,153 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.io.WritableUtils; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +public class CoerceExpression extends BaseSingleExpression { + private PDataType toType; + private ColumnModifier toMod; + private Integer byteSize; + + public CoerceExpression() { + } + + public static Expression create(Expression expression, PDataType toType) throws SQLException { + if (toType == expression.getDataType()) { + return expression; + } + return new CoerceExpression(expression, toType); + } + + //Package protected for tests + CoerceExpression(Expression expression, PDataType toType) { + this(expression, toType, null, null); + } + + CoerceExpression(Expression expression, PDataType toType, ColumnModifier toMod, Integer byteSize) { + super(expression); + this.toType = toType; + this.toMod = toMod; + this.byteSize = byteSize; + } + + @Override + public Integer getByteSize() { + return byteSize; + } + + @Override + public Integer getMaxLength() { + return byteSize; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + ((byteSize == null) ? 0 : byteSize.hashCode()); + result = prime * result + ((toMod == null) ? 0 : toMod.hashCode()); + result = prime * result + ((toType == null) ? 0 : toType.hashCode()); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + CoerceExpression other = (CoerceExpression)obj; + if (byteSize == null) { + if (other.byteSize != null) return false; + } else if (!byteSize.equals(other.byteSize)) return false; + if (toMod != other.toMod) return false; + if (toType != other.toType) return false; + return true; + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + toType = PDataType.values()[WritableUtils.readVInt(input)]; + toMod = ColumnModifier.fromSystemValue(WritableUtils.readVInt(input)); + int byteSize = WritableUtils.readVInt(input); + this.byteSize = byteSize == -1 ? null : byteSize; + } + + @Override + public void write(DataOutput output) throws IOException { + super.write(output); + WritableUtils.writeVInt(output, toType.ordinal()); + WritableUtils.writeVInt(output, ColumnModifier.toSystemValue(toMod)); + WritableUtils.writeVInt(output, byteSize == null ? -1 : byteSize); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (getChild().evaluate(tuple, ptr)) { + getDataType().coerceBytes(ptr, getChild().getDataType(), getChild().getColumnModifier(), getColumnModifier()); + return true; + } + return false; + } + + @Override + public PDataType getDataType() { + return toType; + } + + @Override + public ColumnModifier getColumnModifier() { + return toMod; + } + + @Override + public T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } + + @Override + public String toString() { + StringBuilder buf = new StringBuilder("TO_" + toType.toString() + "("); + for (int i = 0; i < children.size() - 1; i++) { + buf.append(children.get(i) + ", "); + } + buf.append(children.get(children.size()-1) + ")"); + return buf.toString(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/ColumnExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/ColumnExpression.java new file mode 100644 index 00000000..bfb0d70a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/ColumnExpression.java @@ -0,0 +1,154 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; + +import org.apache.hadoop.io.WritableUtils; + +import com.google.common.base.Objects; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDatum; + +/** + * + * Common base class for column value accessors + * + * @author jtaylor + * @since 0.1 + */ +abstract public class ColumnExpression extends BaseTerminalExpression { + protected PDataType type; + private Integer byteSize; + private boolean isNullable; + private Integer maxLength; + private Integer scale; + private ColumnModifier columnModifier; + + public ColumnExpression() { + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + (isNullable() ? 1231 : 1237); + Integer maxLength = this.getByteSize(); + result = prime * result + ((maxLength == null) ? 0 : maxLength.hashCode()); + PDataType type = this.getDataType(); + result = prime * result + ((type == null) ? 0 : type.hashCode()); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + ColumnExpression other = (ColumnExpression)obj; + if (this.isNullable() != other.isNullable()) return false; + if (!Objects.equal(this.getByteSize(),other.getByteSize())) return false; + if (this.getDataType() != other.getDataType()) return false; + return true; + } + + public ColumnExpression(PDatum datum) { + this.type = datum.getDataType(); + this.isNullable = datum.isNullable(); + if (type.isFixedWidth() && type.getByteSize() == null) { + this.byteSize = datum.getByteSize(); + } + this.maxLength = datum.getMaxLength(); + this.scale = datum.getScale(); + this.columnModifier = datum.getColumnModifier(); + } + + @Override + public boolean isNullable() { + return isNullable; + } + + @Override + public PDataType getDataType() { + return type; + } + + @Override + public ColumnModifier getColumnModifier() { + return columnModifier; + } + + @Override + public Integer getByteSize() { + if (byteSize != null) { + return byteSize; + } + return super.getByteSize(); + } + + @Override + public Integer getMaxLength() { + return maxLength; + } + + @Override + public Integer getScale() { + return scale; + } + + @Override + public void readFields(DataInput input) throws IOException { + // read/write type ordinal, maxLength presence, scale presence and isNullable bit together to save space + int typeAndFlag = WritableUtils.readVInt(input); + isNullable = (typeAndFlag & 0x01) != 0; + if ((typeAndFlag & 0x02) != 0) { + scale = WritableUtils.readVInt(input); + } + if ((typeAndFlag & 0x04) != 0) { + maxLength = WritableUtils.readVInt(input); + } + type = PDataType.values()[typeAndFlag >>> 3]; + if (type.isFixedWidth() && type.getByteSize() == null) { + byteSize = WritableUtils.readVInt(input); + } + columnModifier = ColumnModifier.fromSystemValue(WritableUtils.readVInt(input)); + } + + @Override + public void write(DataOutput output) throws IOException { + // read/write type ordinal, maxLength presence, scale presence and isNullable bit together to save space + int typeAndFlag = (isNullable ? 1 : 0) | ((scale != null ? 1 : 0) << 1) | ((maxLength != null ? 1 : 0) << 2) + | (type.ordinal() << 3); + WritableUtils.writeVInt(output,typeAndFlag); + if (scale != null) { + WritableUtils.writeVInt(output, scale); + } + if (maxLength != null) { + WritableUtils.writeVInt(output, maxLength); + } + if (type.isFixedWidth() && type.getByteSize() == null) { + WritableUtils.writeVInt(output, byteSize); + } + WritableUtils.writeVInt(output, ColumnModifier.toSystemValue(columnModifier)); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/ComparisonExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/ComparisonExpression.java new file mode 100644 index 00000000..5d3af1b0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/ComparisonExpression.java @@ -0,0 +1,159 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; +import java.util.List; + +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.io.WritableUtils; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.StringUtil; + + +/** + * + * Implementation for <,<=,>,>=,=,!= comparison expressions + * @author jtaylor + * @since 0.1 + */ +public class ComparisonExpression extends BaseCompoundExpression { + private CompareOp op; + private static final String[] CompareOpString = new String[CompareOp.values().length]; + static { + CompareOpString[CompareOp.EQUAL.ordinal()] = " = "; + CompareOpString[CompareOp.NOT_EQUAL.ordinal()] = " != "; + CompareOpString[CompareOp.GREATER.ordinal()] = " > "; + CompareOpString[CompareOp.LESS.ordinal()] = " < "; + CompareOpString[CompareOp.GREATER_OR_EQUAL.ordinal()] = " >= "; + CompareOpString[CompareOp.LESS_OR_EQUAL.ordinal()] = " <= "; + } + + public ComparisonExpression() { + } + + public ComparisonExpression(CompareOp op, List children) { + super(children); + if (op == null) { + throw new NullPointerException(); + } + this.op = op; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = super.hashCode(); + result = prime * result + op.hashCode(); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (!super.equals(obj)) return false; + if (getClass() != obj.getClass()) return false; + ComparisonExpression other = (ComparisonExpression)obj; + if (op != other.op) return false; + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.BOOLEAN; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (!children.get(0).evaluate(tuple, ptr)) { + return false; + } + byte[] lhsBytes = ptr.get(); + int lhsOffset = ptr.getOffset(); + int lhsLength = ptr.getLength(); + PDataType lhsDataType = children.get(0).getDataType(); + ColumnModifier lhsColumnModifier = children.get(0).getColumnModifier(); + + if (!children.get(1).evaluate(tuple, ptr)) { + return false; + } + + byte[] rhsBytes = ptr.get(); + int rhsOffset = ptr.getOffset(); + int rhsLength = ptr.getLength(); + PDataType rhsDataType = children.get(1).getDataType(); + ColumnModifier rhsColumnModifier = children.get(1).getColumnModifier(); + if (rhsDataType == PDataType.CHAR) { + rhsLength = StringUtil.getUnpaddedCharLength(rhsBytes, rhsOffset, rhsLength, rhsColumnModifier); + } + if (lhsDataType == PDataType.CHAR) { + lhsLength = StringUtil.getUnpaddedCharLength(lhsBytes, lhsOffset, lhsLength, lhsColumnModifier); + } + + + int comparisonResult = lhsDataType.compareTo(lhsBytes, lhsOffset, lhsLength, lhsColumnModifier, + rhsBytes, rhsOffset, rhsLength, rhsColumnModifier, rhsDataType); + ptr.set(ByteUtil.compare(op, comparisonResult) ? PDataType.TRUE_BYTES : PDataType.FALSE_BYTES); + return true; + } + + @Override + public void readFields(DataInput input) throws IOException { + op = CompareOp.values()[WritableUtils.readVInt(input)]; + super.readFields(input); + } + + @Override + public void write(DataOutput output) throws IOException { + WritableUtils.writeVInt(output, op.ordinal()); + super.write(output); + } + + @Override + public final T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } + + public CompareOp getFilterOp() { + return op; + } + + public static String toString(CompareOp op, List children) { + return (children.get(0) + CompareOpString[op.ordinal()] + children.get(1)); + } + + @Override + public String toString() { + return toString(getFilterOp(), children); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/CurrentDateTimeFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/CurrentDateTimeFunction.java new file mode 100644 index 00000000..b221c1ca --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/CurrentDateTimeFunction.java @@ -0,0 +1,44 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.phoenix.expression.function.ScalarFunction; + +public abstract class CurrentDateTimeFunction extends ScalarFunction { + + public CurrentDateTimeFunction() { + } + + public CurrentDateTimeFunction(List children) { + super(children); + } + + @Override + public boolean isStateless() { + return true; + } + + @Override + public boolean isDeterministic() { + return false; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/DateAddExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/DateAddExpression.java new file mode 100644 index 00000000..a36bb48f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/DateAddExpression.java @@ -0,0 +1,80 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.math.BigDecimal; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +public class DateAddExpression extends AddExpression { + static private final BigDecimal BD_MILLIS_IN_DAY = BigDecimal.valueOf(QueryConstants.MILLIS_IN_DAY); + + public DateAddExpression() { + } + + public DateAddExpression(List children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + long finalResult=0; + + for(int i=0;i children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + long finalResult=0; + + for(int i=0;i children) { + super(children); + for (int i=0; i children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + BigDecimal result = null; + for (int i=0; i children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + BigDecimal result = null; + for (int i=0; i children) { + super(children); + for (int i=0; i children) { + super(children); + for (int i=0; i T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } + + @Override + public String getOperatorString() { + return " / "; + } + + private static Integer getPrecision(Integer lp, Integer rp, Integer ls, Integer rs) { + if (ls == null || rs == null) { + return PDataType.MAX_PRECISION; + } + int val = getScale(lp, rp, ls, rs) + lp - ls + rp; + return Math.min(PDataType.MAX_PRECISION, val); + } + + private static Integer getScale(Integer lp, Integer rp, Integer ls, Integer rs) { + // If we are adding a decimal with scale and precision to a decimal + // with no precision nor scale, the scale system does not apply. + if (ls == null || rs == null) { + return null; + } + int val = Math.max(PDataType.MAX_PRECISION - lp + ls - rs, 0); + return Math.min(PDataType.MAX_PRECISION, val); + } + + @Override + public Integer getScale() { + return scale; + } + + @Override + public Integer getMaxLength() { + return maxLength; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleAddExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleAddExpression.java new file mode 100644 index 00000000..96fc9fae --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleAddExpression.java @@ -0,0 +1,70 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +public class DoubleAddExpression extends AddExpression { + + public DoubleAddExpression() { + } + + public DoubleAddExpression(List children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + double result = 0.0; + for (int i = 0; i < children.size(); i++) { + Expression child = children.get(i); + if (!child.evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + return true; + } + double childvalue = child.getDataType().getCodec() + .decodeDouble(ptr, child.getColumnModifier()); + if (!Double.isNaN(childvalue) + && childvalue != Double.NEGATIVE_INFINITY + && childvalue != Double.POSITIVE_INFINITY) { + result += childvalue; + } else { + return false; + } + } + byte[] resultPtr = new byte[getDataType().getByteSize()]; + ptr.set(resultPtr); + getDataType().getCodec().encodeDouble(result, ptr); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.DOUBLE; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleDivideExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleDivideExpression.java new file mode 100644 index 00000000..561cf90f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleDivideExpression.java @@ -0,0 +1,74 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +public class DoubleDivideExpression extends DivideExpression { + + public DoubleDivideExpression() { + } + + public DoubleDivideExpression(List children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + double result = 0.0; + for (int i = 0; i < children.size(); i++) { + Expression child = children.get(i); + if (!child.evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + return true; + } + double childvalue = child.getDataType().getCodec() + .decodeDouble(ptr, child.getColumnModifier()); + if (!Double.isNaN(childvalue) + && childvalue != Double.NEGATIVE_INFINITY + && childvalue != Double.POSITIVE_INFINITY) { + if (i == 0) { + result = childvalue; + } else { + result /= childvalue; + } + } else { + return false; + } + } + byte[] resultPtr = new byte[getDataType().getByteSize()]; + ptr.set(resultPtr); + getDataType().getCodec().encodeDouble(result, ptr); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.DOUBLE; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleMultiplyExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleMultiplyExpression.java new file mode 100644 index 00000000..b9e81657 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleMultiplyExpression.java @@ -0,0 +1,70 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +public class DoubleMultiplyExpression extends MultiplyExpression { + + public DoubleMultiplyExpression() { + } + + public DoubleMultiplyExpression(List children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + double result = 1.0; + for (int i = 0; i < children.size(); i++) { + Expression child = children.get(i); + if (!child.evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + return true; + } + double childvalue = child.getDataType().getCodec() + .decodeDouble(ptr, child.getColumnModifier()); + if (!Double.isNaN(childvalue) + && childvalue != Double.NEGATIVE_INFINITY + && childvalue != Double.POSITIVE_INFINITY) { + result *= childvalue; + } else { + return false; + } + } + byte[] resultPtr = new byte[getDataType().getByteSize()]; + ptr.set(resultPtr); + getDataType().getCodec().encodeDouble(result, ptr); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.DOUBLE; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleSubtractExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleSubtractExpression.java new file mode 100644 index 00000000..8752c154 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/DoubleSubtractExpression.java @@ -0,0 +1,74 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +public class DoubleSubtractExpression extends SubtractExpression { + + public DoubleSubtractExpression() { + } + + public DoubleSubtractExpression(List children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + double result = 0.0; + for (int i = 0; i < children.size(); i++) { + Expression child = children.get(i); + if (!child.evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + return true; + } + double childvalue = child.getDataType().getCodec() + .decodeDouble(ptr, child.getColumnModifier()); + if (!Double.isNaN(childvalue) + && childvalue != Double.NEGATIVE_INFINITY + && childvalue != Double.POSITIVE_INFINITY) { + if (i == 0) { + result = childvalue; + } else { + result -= childvalue; + } + } else { + return false; + } + } + byte[] resultPtr = new byte[getDataType().getByteSize()]; + ptr.set(resultPtr); + getDataType().getCodec().encodeDouble(result, ptr); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.DOUBLE; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/Expression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/Expression.java new file mode 100644 index 00000000..ddced6a9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/Expression.java @@ -0,0 +1,82 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.io.Writable; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.PDatum; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Interface for general expression evaluation + * + * @author jtaylor + * @since 0.1 + */ +public interface Expression extends PDatum, Writable { + /** + * Access the value by setting a pointer to it (as opposed to making + * a copy of it which can be expensive) + * @param tuple Single row result during scan iteration + * @param ptr Pointer to byte value being accessed + * @return true if the expression could be evaluated (i.e. ptr was set) + * and false otherwise + */ + boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr); + + /** + * Means of traversing expression tree through visitor. + * @param visitor + */ + T accept(ExpressionVisitor visitor); + + /** + * @return the child expressions + */ + List getChildren(); + + /** + * Resets the state of a expression back to its initial state and + * enables the expession to be evaluated incrementally (which + * occurs during filter evaluation where we see one key value at + * a time; it's possible to evaluate immediately rather than + * wait until all key values have been seen). Note that when + * evaluating incrementally, you must call this method before + * processing a new row. + */ + void reset(); + + /** + * @return true if the expression can be evaluated without + * requiring a row Tuple and false otherwise. + */ + boolean isStateless(); + /** + * @return true if the expression returns the same output every + * time given the same input. + */ + boolean isDeterministic(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/ExpressionType.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/ExpressionType.java new file mode 100644 index 00000000..24bcf21f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/ExpressionType.java @@ -0,0 +1,197 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.Map; + +import com.google.common.collect.Maps; +import org.apache.phoenix.expression.function.ArrayIndexFunction; +import org.apache.phoenix.expression.function.ArrayLengthFunction; +import org.apache.phoenix.expression.function.CeilDateExpression; +import org.apache.phoenix.expression.function.CeilDecimalExpression; +import org.apache.phoenix.expression.function.CeilFunction; +import org.apache.phoenix.expression.function.CeilTimestampExpression; +import org.apache.phoenix.expression.function.CoalesceFunction; +import org.apache.phoenix.expression.function.CountAggregateFunction; +import org.apache.phoenix.expression.function.DistinctCountAggregateFunction; +import org.apache.phoenix.expression.function.FloorDateExpression; +import org.apache.phoenix.expression.function.FloorDecimalExpression; +import org.apache.phoenix.expression.function.FloorFunction; +import org.apache.phoenix.expression.function.IndexStateNameFunction; +import org.apache.phoenix.expression.function.InvertFunction; +import org.apache.phoenix.expression.function.LTrimFunction; +import org.apache.phoenix.expression.function.LengthFunction; +import org.apache.phoenix.expression.function.LowerFunction; +import org.apache.phoenix.expression.function.MD5Function; +import org.apache.phoenix.expression.function.MaxAggregateFunction; +import org.apache.phoenix.expression.function.MinAggregateFunction; +import org.apache.phoenix.expression.function.PercentRankAggregateFunction; +import org.apache.phoenix.expression.function.PercentileContAggregateFunction; +import org.apache.phoenix.expression.function.PercentileDiscAggregateFunction; +import org.apache.phoenix.expression.function.RTrimFunction; +import org.apache.phoenix.expression.function.RegexpReplaceFunction; +import org.apache.phoenix.expression.function.RegexpSubstrFunction; +import org.apache.phoenix.expression.function.ReverseFunction; +import org.apache.phoenix.expression.function.RoundDateExpression; +import org.apache.phoenix.expression.function.RoundDecimalExpression; +import org.apache.phoenix.expression.function.RoundFunction; +import org.apache.phoenix.expression.function.RoundTimestampExpression; +import org.apache.phoenix.expression.function.SQLTableTypeFunction; +import org.apache.phoenix.expression.function.SQLViewTypeFunction; +import org.apache.phoenix.expression.function.SqlTypeNameFunction; +import org.apache.phoenix.expression.function.StddevPopFunction; +import org.apache.phoenix.expression.function.StddevSampFunction; +import org.apache.phoenix.expression.function.SubstrFunction; +import org.apache.phoenix.expression.function.SumAggregateFunction; +import org.apache.phoenix.expression.function.ToCharFunction; +import org.apache.phoenix.expression.function.ToDateFunction; +import org.apache.phoenix.expression.function.ToNumberFunction; +import org.apache.phoenix.expression.function.TrimFunction; +import org.apache.phoenix.expression.function.TruncFunction; +import org.apache.phoenix.expression.function.UpperFunction; + +/** + * + * Enumeration of all Expression types that may be evaluated on the server-side. + * Used during serialization and deserialization to pass Expression between client + * and server. + * + * @author jtaylor + * @since 0.1 + */ +public enum ExpressionType { + ReverseFunction(ReverseFunction.class), + RowKey(RowKeyColumnExpression.class), + KeyValue(KeyValueColumnExpression.class), + LiteralValue(LiteralExpression.class), + RoundFunction(RoundFunction.class), + FloorFunction(FloorFunction.class), + CeilFunction(CeilFunction.class), + RoundDateExpression(RoundDateExpression.class), + FloorDateExpression(FloorDateExpression.class), + CeilDateExpression(CeilDateExpression.class), + RoundTimestampExpression(RoundTimestampExpression.class), + CeilTimestampExpression(CeilTimestampExpression.class), + RoundDecimalExpression(RoundDecimalExpression.class), + FloorDecimalExpression(FloorDecimalExpression.class), + CeilDecimalExpression(CeilDecimalExpression.class), + TruncFunction(TruncFunction.class), + ToDateFunction(ToDateFunction.class), + ToCharFunction(ToCharFunction.class), + ToNumberFunction(ToNumberFunction.class), + CoerceFunction(CoerceExpression.class), + SubstrFunction(SubstrFunction.class), + AndExpression(AndExpression.class), + OrExpression(OrExpression.class), + ComparisonExpression(ComparisonExpression.class), + CountAggregateFunction(CountAggregateFunction.class), + SumAggregateFunction(SumAggregateFunction.class), + MinAggregateFunction(MinAggregateFunction.class), + MaxAggregateFunction(MaxAggregateFunction.class), + LikeExpression(LikeExpression.class), + NotExpression(NotExpression.class), + CaseExpression(CaseExpression.class), + InListExpression(InListExpression.class), + IsNullExpression(IsNullExpression.class), + LongSubtractExpression(LongSubtractExpression.class), + DateSubtractExpression(DateSubtractExpression.class), + DecimalSubtractExpression(DecimalSubtractExpression.class), + LongAddExpression(LongAddExpression.class), + DecimalAddExpression(DecimalAddExpression.class), + DateAddExpression(DateAddExpression.class), + LongMultiplyExpression(LongMultiplyExpression.class), + DecimalMultiplyExpression(DecimalMultiplyExpression.class), + LongDivideExpression(LongDivideExpression.class), + DecimalDivideExpression(DecimalDivideExpression.class), + CoalesceFunction(CoalesceFunction.class), + RegexpReplaceFunction(RegexpReplaceFunction.class), + SQLTypeNameFunction(SqlTypeNameFunction.class), + RegexpSubstrFunction(RegexpSubstrFunction.class), + StringConcatExpression(StringConcatExpression.class), + LengthFunction(LengthFunction.class), + LTrimFunction(LTrimFunction.class), + RTrimFunction(RTrimFunction.class), + UpperFunction(UpperFunction.class), + LowerFunction(LowerFunction.class), + TrimFunction(TrimFunction.class), + DistinctCountAggregateFunction(DistinctCountAggregateFunction.class), + PercentileContAggregateFunction(PercentileContAggregateFunction.class), + PercentRankAggregateFunction(PercentRankAggregateFunction.class), + StddevPopFunction(StddevPopFunction.class), + StddevSampFunction(StddevSampFunction.class), + PercentileDiscAggregateFunction(PercentileDiscAggregateFunction.class), + DoubleAddExpression(DoubleAddExpression.class), + DoubleSubtractExpression(DoubleSubtractExpression.class), + DoubleMultiplyExpression(DoubleMultiplyExpression.class), + DoubleDivideExpression(DoubleDivideExpression.class), + RowValueConstructorExpression(RowValueConstructorExpression.class), + MD5Function(MD5Function.class), + SQLTableTypeFunction(SQLTableTypeFunction.class), + IndexKeyValue(IndexKeyValueColumnExpression.class), + IndexStateName(IndexStateNameFunction.class), + InvertFunction(InvertFunction.class), + ProjectedColumnExpression(ProjectedColumnExpression.class), + TimestampAddExpression(TimestampAddExpression.class), + TimestampSubtractExpression(TimestampSubtractExpression.class), + ArrayIndexFunction(ArrayIndexFunction.class), + ArrayLengthFunction(ArrayLengthFunction.class), + ArrayConstructorExpression(ArrayConstructorExpression.class), + SQLViewTypeFunction(SQLViewTypeFunction.class); + ExpressionType(Class clazz) { + this.clazz = clazz; + } + + public Class getExpressionClass() { + return clazz; + } + + private final Class clazz; + + private static final Map,ExpressionType> classToEnumMap = Maps.newHashMapWithExpectedSize(3); + static { + for (ExpressionType type : ExpressionType.values()) { + classToEnumMap.put(type.clazz, type); + } + } + + /** + * Return the ExpressionType for a given Expression instance + */ + public static ExpressionType valueOf(Expression expression) { + ExpressionType type = classToEnumMap.get(expression.getClass()); + if (type == null) { // FIXME: this exception gets swallowed and retries happen + throw new IllegalArgumentException("No ExpressionType for " + expression.getClass()); + } + return type; + } + + /** + * Instantiates a DataAccessor based on its DataAccessorType + */ + public Expression newInstance() { + try { + return clazz.newInstance(); + } catch (InstantiationException e) { + throw new RuntimeException(e); + } catch (IllegalAccessException e) { + throw new RuntimeException(e); + } + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/InListExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/InListExpression.java similarity index 82% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/InListExpression.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/InListExpression.java index 5dc621c7..f6a397d4 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/InListExpression.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/InListExpression.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. +/* + * Copyright 2010 The Apache Software Foundation * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. + * http://www.apache.org/licenses/LICENSE-2.0 * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; import java.io.DataInput; import java.io.DataOutput; @@ -43,12 +35,12 @@ import com.google.common.collect.Lists; import com.google.common.collect.Sets; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.ConstraintViolationException; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.ConstraintViolationException; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; /* * Implementation of a SQL foo IN (a,b,c) expression. Other than the first diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/IndexKeyValueColumnExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/IndexKeyValueColumnExpression.java similarity index 79% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/IndexKeyValueColumnExpression.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/IndexKeyValueColumnExpression.java index 65a08039..fb066c17 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/IndexKeyValueColumnExpression.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/IndexKeyValueColumnExpression.java @@ -1,10 +1,10 @@ -package com.salesforce.phoenix.expression; +package org.apache.phoenix.expression; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.util.IndexUtil; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.util.IndexUtil; +import org.apache.phoenix.util.SchemaUtil; public class IndexKeyValueColumnExpression extends KeyValueColumnExpression { public IndexKeyValueColumnExpression() { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/IsNullExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/IsNullExpression.java new file mode 100644 index 00000000..e6ef6218 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/IsNullExpression.java @@ -0,0 +1,106 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.io.*; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Implementation of IS NULL and IS NOT NULL expression + * + * @author jtaylor + * @since 0.1 + */ +public class IsNullExpression extends BaseSingleExpression { + private boolean isNegate; + + public IsNullExpression() { + } + + public IsNullExpression(Expression expression, boolean negate) { + super(expression); + this.isNegate = negate; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + boolean evaluated = getChild().evaluate(tuple, ptr); + if (evaluated) { + ptr.set(isNegate ^ ptr.getLength() == 0 ? PDataType.TRUE_BYTES : PDataType.FALSE_BYTES); + return true; + } + if (tuple.isImmutable()) { + ptr.set(isNegate ? PDataType.FALSE_BYTES : PDataType.TRUE_BYTES); + return true; + } + + return false; + } + + public boolean isNegate() { + return isNegate; + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + isNegate = input.readBoolean(); + } + + @Override + public void write(DataOutput output) throws IOException { + super.write(output); + output.writeBoolean(isNegate); + } + + @Override + public PDataType getDataType() { + return PDataType.BOOLEAN; + } + + @Override + public final T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } + + @Override + public String toString() { + StringBuilder buf = new StringBuilder(children.get(0).toString()); + if (isNegate) { + buf.append(" IS NOT NULL"); + } else { + buf.append(" IS NULL"); + } + return buf.toString(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/KeyValueColumnExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/KeyValueColumnExpression.java new file mode 100644 index 00000000..b082abaf --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/KeyValueColumnExpression.java @@ -0,0 +1,113 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; +import java.util.Arrays; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.SchemaUtil; + + +/** + * + * Class to access a column value stored in a KeyValue + * + * @author jtaylor + * @since 0.1 + */ +public class KeyValueColumnExpression extends ColumnExpression { + private byte[] cf; + private byte[] cq; + + public KeyValueColumnExpression() { + } + + public KeyValueColumnExpression(PColumn column) { + super(column); + this.cf = column.getFamilyName().getBytes(); + this.cq = column.getName().getBytes(); + } + + public byte[] getColumnFamily() { + return cf; + } + + public byte[] getColumnName() { + return cq; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + Arrays.hashCode(cf); + result = prime * result + Arrays.hashCode(cq); + return result; + } + + // TODO: assumes single table + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + KeyValueColumnExpression other = (KeyValueColumnExpression)obj; + if (!Arrays.equals(cf, other.cf)) return false; + if (!Arrays.equals(cq, other.cq)) return false; + return true; + } + + @Override + public String toString() { + return SchemaUtil.getColumnDisplayName(cf, cq); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + return tuple.getValue(cf, cq, ptr); + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + cf = Bytes.readByteArray(input); + cq = Bytes.readByteArray(input); + } + + @Override + public void write(DataOutput output) throws IOException { + super.write(output); + Bytes.writeByteArray(output, cf); + Bytes.writeByteArray(output, cq); + } + + @Override + public final T accept(ExpressionVisitor visitor) { + return visitor.visit(this); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/LikeExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/LikeExpression.java similarity index 82% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/LikeExpression.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/LikeExpression.java index fbf0d8b5..038f7050 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/LikeExpression.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/LikeExpression.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; import java.io.DataInput; import java.io.IOException; @@ -36,10 +28,10 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.StringUtil; +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.StringUtil; /** diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/LiteralExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/LiteralExpression.java similarity index 82% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/LiteralExpression.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/LiteralExpression.java index a42ce4a4..202bcb05 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/LiteralExpression.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/LiteralExpression.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; import java.io.DataInput; import java.io.DataOutput; @@ -35,16 +27,16 @@ import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.io.WritableUtils; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.IllegalDataException; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.TypeMismatchException; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.StringUtil; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.IllegalDataException; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.TypeMismatchException; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.StringUtil; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/LongAddExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/LongAddExpression.java new file mode 100644 index 00000000..a14286d6 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/LongAddExpression.java @@ -0,0 +1,62 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +public class LongAddExpression extends AddExpression { + + public LongAddExpression() { + } + + public LongAddExpression(List children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + long finalResult=0; + + for(int i=0;i children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + long finalResult=0; + + for(int i=0;i children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + long finalResult=1; + + for(int i=0;i children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + long finalResult=0; + + for(int i=0;i children) { + super(children); + for (int i=0; i T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } + + @Override + public String getOperatorString() { + return " * "; + } + + private static Integer getPrecision(Integer lp, Integer rp, Integer ls, Integer rs) { + if (ls == null || rs == null) { + return PDataType.MAX_PRECISION; + } + int val = lp + rp; + return Math.min(PDataType.MAX_PRECISION, val); + } + + private static Integer getScale(Integer lp, Integer rp, Integer ls, Integer rs) { + // If we are adding a decimal with scale and precision to a decimal + // with no precision nor scale, the scale system does not apply. + if (ls == null || rs == null) { + return null; + } + int val = ls + rs; + return Math.min(PDataType.MAX_PRECISION, val); + } + + @Override + public Integer getScale() { + return scale; + } + + @Override + public Integer getMaxLength() { + return maxLength; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/NotExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/NotExpression.java new file mode 100644 index 00000000..c3a07c99 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/NotExpression.java @@ -0,0 +1,86 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Implementation of the NOT operator that negates it's + * single boolean child expression. + * + * @author jtaylor + * @since 0.1 + */ +public class NotExpression extends BaseSingleExpression { + + public NotExpression() { + } + + public NotExpression(Expression expression) { + super(expression); + if (expression == null || expression.getDataType() != PDataType.BOOLEAN) { + throw new IllegalArgumentException("NOT expectes a single BOOLEAN expression, but got " + expression); + } + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (!getChild().evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + return true; + } + + ptr.set(Boolean.TRUE.equals(PDataType.BOOLEAN.toObject(ptr)) ? PDataType.FALSE_BYTES : PDataType.TRUE_BYTES); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.BOOLEAN; + } + + @Override + public final T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } + + @Override + public String toString() { + StringBuilder buf = new StringBuilder("NOT ("); + buf.append(children.get(0).toString()); + buf.append(")"); + return buf.toString(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/OrExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/OrExpression.java new file mode 100644 index 00000000..7339852c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/OrExpression.java @@ -0,0 +1,67 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.util.List; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; + + +/** + * + * OR expression implementation + * + * @author jtaylor + * @since 0.1 + */ +public class OrExpression extends AndOrExpression { + public OrExpression() { + } + + public OrExpression(List children) { + super(children); + } + + @Override + protected boolean getStopValue() { + return Boolean.TRUE; + } + + @Override + public String toString() { + StringBuilder buf = new StringBuilder("("); + for (int i = 0; i < children.size() - 1; i++) { + buf.append(children.get(i) + " OR "); + } + buf.append(children.get(children.size()-1)); + buf.append(')'); + return buf.toString(); + } + + @Override + public final T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/OrderByExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/OrderByExpression.java similarity index 98% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/OrderByExpression.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/OrderByExpression.java index fe9d42dd..67d7e994 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/OrderByExpression.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/OrderByExpression.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.expression; +package org.apache.phoenix.expression; import static com.google.common.base.Preconditions.checkNotNull; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/ProjectedColumnExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/ProjectedColumnExpression.java new file mode 100644 index 00000000..22628821 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/ProjectedColumnExpression.java @@ -0,0 +1,144 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.join.ScanProjector; +import org.apache.phoenix.schema.KeyValueSchema; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.ValueBitSet; +import org.apache.phoenix.schema.KeyValueSchema.KeyValueSchemaBuilder; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.SchemaUtil; + +public class ProjectedColumnExpression extends ColumnExpression { + private KeyValueSchema schema; + ValueBitSet bitSet; + private int position; + private byte[] name; // for display purpose only + + public ProjectedColumnExpression() { + } + + public ProjectedColumnExpression(PColumn column, PTable table) { + super(column); + this.schema = buildSchema(table); + this.bitSet = ValueBitSet.newInstance(schema); + this.position = column.getPosition() - table.getPKColumns().size(); + this.name = column.getName().getBytes(); + } + + private static KeyValueSchema buildSchema(PTable table) { + KeyValueSchemaBuilder builder = new KeyValueSchemaBuilder(0); + for (PColumn column : table.getColumns()) { + if (!SchemaUtil.isPKColumn(column)) { + builder.addField(column); + } + } + return builder.build(); + } + + public KeyValueSchema getSchema() { + return schema; + } + + public int getPosition() { + return position; + } + + public byte[] getName() { + return name; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + schema.hashCode(); + result = prime * result + position; + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + ProjectedColumnExpression other = (ProjectedColumnExpression)obj; + if (!schema.equals(other.schema)) return false; + if (position != other.position) return false; + return true; + } + + @Override + public String toString() { + return Bytes.toString(name); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + try { + ScanProjector.decodeProjectedValue(tuple, ptr); + int maxOffset = ptr.getOffset() + ptr.getLength(); + bitSet.clear(); + bitSet.or(ptr); + schema.iterator(ptr, position, bitSet); + Boolean hasValue = schema.next(ptr, position, maxOffset, bitSet); + if (hasValue == null || !hasValue.booleanValue()) + return false; + } catch (IOException e) { + return false; + } + + return true; + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + schema = new KeyValueSchema(); + schema.readFields(input); + bitSet = ValueBitSet.newInstance(schema); + position = input.readInt(); + name = Bytes.readByteArray(input); + } + + @Override + public void write(DataOutput output) throws IOException { + super.write(output); + schema.write(output); + output.writeInt(position); + Bytes.writeByteArray(output, name); + } + + @Override + public final T accept(ExpressionVisitor visitor) { + return visitor.visit(this); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/RowKeyColumnExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/RowKeyColumnExpression.java new file mode 100644 index 00000000..383ef2e9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/RowKeyColumnExpression.java @@ -0,0 +1,150 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDatum; +import org.apache.phoenix.schema.RowKeyValueAccessor; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; + + +/** + * + * Class to access a value stored in the row key + * + * @author jtaylor + * @since 0.1 + */ +public class RowKeyColumnExpression extends ColumnExpression { + private PDataType fromType; + private RowKeyValueAccessor accessor; + protected final String name; + + public RowKeyColumnExpression() { + name = null; // Only on client + } + + private RowKeyColumnExpression(PDatum datum, RowKeyValueAccessor accessor, PDataType fromType, String name) { + super(datum); + this.accessor = accessor; + this.fromType = fromType; + this.name = name; + } + + public RowKeyColumnExpression(PDatum datum, RowKeyValueAccessor accessor) { + this(datum, accessor, datum.getDataType(), datum.toString()); + } + + public RowKeyColumnExpression(PDatum datum, RowKeyValueAccessor accessor, String name) { + this(datum, accessor, datum.getDataType(), name); + } + + public RowKeyColumnExpression(PDatum datum, RowKeyValueAccessor accessor, PDataType fromType) { + this(datum, accessor, fromType, datum.toString()); + } + + public int getPosition() { + return accessor.getIndex(); + } + + @Override + public int hashCode() { + final int prime = 31; + int result = super.hashCode(); + result = prime * result + ((accessor == null) ? 0 : accessor.hashCode()); + return result; + } + + @Override + public String toString() { + return name == null ? "PK[" + accessor.getIndex() + "]" : name; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (!super.equals(obj)) return false; + if (getClass() != obj.getClass()) return false; + RowKeyColumnExpression other = (RowKeyColumnExpression)obj; + return accessor.equals(other.accessor); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + tuple.getKey(ptr); + int offset = accessor.getOffset(ptr.get(), ptr.getOffset()); + // Null is represented in the last expression of a multi-part key + // by the bytes not being present. + int maxOffset = ptr.getOffset() + ptr.getLength(); + if (offset < maxOffset) { + byte[] buffer = ptr.get(); + int fixedByteSize = -1; + // FIXME: fixedByteSize <= maxByteSize ? fixedByteSize : 0 required because HBase passes bogus keys to filter to position scan (HBASE-6562) + if (fromType.isFixedWidth()) { + fixedByteSize = getByteSize(); + fixedByteSize = fixedByteSize <= maxOffset ? fixedByteSize : 0; + } + int length = fixedByteSize >= 0 ? fixedByteSize : accessor.getLength(buffer, offset, maxOffset); + // In the middle of the key, an empty variable length byte array represents null + if (length > 0) { + /* + if (type == fromType) { + ptr.set(buffer,offset,length); + } else { + ptr.set(type.toBytes(type.toObject(buffer, offset, length, fromType))); + } + */ + ptr.set(buffer,offset,length); + type.coerceBytes(ptr, fromType, getColumnModifier(), getColumnModifier()); + } else { + ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); + } + return true; + } + return false; + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + accessor = new RowKeyValueAccessor(); + accessor.readFields(input); + fromType = type; // fromType only needed on client side + } + + @Override + public void write(DataOutput output) throws IOException { + super.write(output); + accessor.write(output); + } + + @Override + public final T accept(ExpressionVisitor visitor) { + return visitor.visit(this); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/RowKeyExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/RowKeyExpression.java similarity index 77% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/RowKeyExpression.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/RowKeyExpression.java index 518bb48a..c3119329 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/RowKeyExpression.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/RowKeyExpression.java @@ -1,9 +1,9 @@ -package com.salesforce.phoenix.expression; +package org.apache.phoenix.expression; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; public class RowKeyExpression extends BaseTerminalExpression { public static final RowKeyExpression INSTANCE = new RowKeyExpression(); diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/RowValueConstructorExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/RowValueConstructorExpression.java similarity index 85% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/RowValueConstructorExpression.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/RowValueConstructorExpression.java index c5ed69b3..3e410cbf 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/RowValueConstructorExpression.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/RowValueConstructorExpression.java @@ -1,30 +1,22 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ /** * Implementation for row value constructor (a,b,c) expression. @@ -32,7 +24,7 @@ * @author samarth.jain * @since 0.1 */ -package com.salesforce.phoenix.expression; +package org.apache.phoenix.expression; import java.io.DataInput; import java.io.DataOutput; @@ -45,18 +37,18 @@ import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import com.google.common.collect.Lists; -import com.salesforce.phoenix.expression.function.CeilDecimalExpression; -import com.salesforce.phoenix.expression.function.CeilTimestampExpression; -import com.salesforce.phoenix.expression.function.FloorDateExpression; -import com.salesforce.phoenix.expression.function.FloorDecimalExpression; -import com.salesforce.phoenix.expression.function.TimeUnit; -import com.salesforce.phoenix.expression.visitor.ExpressionVisitor; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.TypeMismatchException; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.TrustedByteArrayOutputStream; +import org.apache.phoenix.expression.function.CeilDecimalExpression; +import org.apache.phoenix.expression.function.CeilTimestampExpression; +import org.apache.phoenix.expression.function.FloorDateExpression; +import org.apache.phoenix.expression.function.FloorDecimalExpression; +import org.apache.phoenix.expression.function.TimeUnit; +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.TypeMismatchException; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.TrustedByteArrayOutputStream; public class RowValueConstructorExpression extends BaseCompoundExpression { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/StringConcatExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/StringConcatExpression.java new file mode 100644 index 00000000..24e96f7c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/StringConcatExpression.java @@ -0,0 +1,95 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + + +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; + + +/** + * + * Implementation for || string concatenation expression. + * @author kmahadik + * @since 0.1 + */ + +public class StringConcatExpression extends BaseCompoundExpression { + public StringConcatExpression() { + } + + public StringConcatExpression(List children) { + super(children); + } + + @Override + public String toString() { + StringBuilder buf = new StringBuilder("("); + for (int i = 0; i < children.size() - 1; i++) { + buf.append(children.get(i) + " || "); + } + buf.append(children.get(children.size()-1)); + buf.append(')'); + return buf.toString(); + } + + @Override + public final T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + byte[] result = ByteUtil.EMPTY_BYTE_ARRAY; + for (int i=0; i children) { + super(children); + } + + @Override + public final T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } + + @Override + public String getOperatorString() { + return " - "; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/TimestampAddExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/TimestampAddExpression.java new file mode 100644 index 00000000..12067042 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/TimestampAddExpression.java @@ -0,0 +1,89 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression; + +import java.math.BigDecimal; +import java.math.RoundingMode; +import java.sql.Timestamp; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.DateUtil; + +/** + * + * Class to encapsulate addition arithmetic for {@link PDataType#TIMESTAMP}. + * + * @author samarth.jain + * @since 2.1.3 + */ + +public class TimestampAddExpression extends AddExpression { + + public TimestampAddExpression() { + } + + public TimestampAddExpression(List children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + BigDecimal finalResult = BigDecimal.ZERO; + + for(int i=0; i children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + BigDecimal finalResult = BigDecimal.ZERO; + + for(int i=0; i exps, ColumnModifier columnModifier) { + super(columnModifier); + ColumnExpression stdDevColExp = (ColumnExpression)exps.get(0); + this.colPrecision = stdDevColExp.getMaxLength(); + this.colScale = stdDevColExp.getScale(); + } + + @Override + protected int getBufferLength() { + return PDataType.DECIMAL.getByteSize(); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (cachedResult == null) { + BigDecimal ssd = sumSquaredDeviation(); + ssd = ssd.divide(new BigDecimal(getDataPointsCount()), PDataType.DEFAULT_MATH_CONTEXT); + // Calculate the precision for the stddev result. + // There are totalCount #Decimal values for which we are calculating the stddev + // The resultant precision depends on precision and scale of all these values. (See + // BigDecimalUtil.getResultPrecisionScale) + // As of now we are not using the actual precision and scale of individual values but just using the table + // column's max length(precision) and scale for each of the values. + int resultPrecision = colPrecision; + for (int i = 1; i < this.totalCount; i++) { + // Max precision that we can support is 38 See PDataType.MAX_PRECISION + if (resultPrecision >= PDataType.MAX_PRECISION) break; + Pair precisionScale = BigDecimalUtil.getResultPrecisionScale(this.colPrecision, + this.colScale, this.colPrecision, this.colScale, Operation.OTHERS); + resultPrecision = precisionScale.getFirst(); + } + cachedResult = new BigDecimal(Math.sqrt(ssd.doubleValue()), new MathContext(resultPrecision, + RoundingMode.HALF_UP)); + cachedResult.setScale(this.colScale, RoundingMode.HALF_UP); + } + if (buffer == null) { + initBuffer(); + } + buffer = PDataType.DECIMAL.toBytes(cachedResult); + ptr.set(buffer); + return true; + } + + protected abstract long getDataPointsCount(); + + private BigDecimal sumSquaredDeviation() { + BigDecimal m = mean(); + BigDecimal result = BigDecimal.ZERO; + for (Entry entry : valueVsCount.entrySet()) { + BigDecimal colValue = (BigDecimal)PDataType.DECIMAL.toObject(entry.getKey()); + BigDecimal delta = colValue.subtract(m); + result = result.add(delta.multiply(delta).multiply(new BigDecimal(entry.getValue()))); + } + return result; + } + + private BigDecimal mean() { + BigDecimal sum = BigDecimal.ZERO; + for (Entry entry : valueVsCount.entrySet()) { + BigDecimal colValue = (BigDecimal)PDataType.DECIMAL.toObject(entry.getKey()); + sum = sum.add(colValue.multiply(new BigDecimal(entry.getValue()))); + } + return sum.divide(new BigDecimal(totalCount), PDataType.DEFAULT_MATH_CONTEXT); + } + + @Override + public void reset() { + super.reset(); + this.cachedResult = null; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/BaseStddevAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/BaseStddevAggregator.java new file mode 100644 index 00000000..eaafc793 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/BaseStddevAggregator.java @@ -0,0 +1,96 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.math.BigDecimal; +import java.util.List; +import java.util.Map.Entry; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * + * @author anoopsjohn + * @since 1.2.1 + */ +public abstract class BaseStddevAggregator extends DistinctValueWithCountClientAggregator { + + protected Expression stdDevColExp; + private BigDecimal cachedResult = null; + + public BaseStddevAggregator(List exps, ColumnModifier columnModifier) { + super(columnModifier); + this.stdDevColExp = exps.get(0); + } + + @Override + protected int getBufferLength() { + return PDataType.DECIMAL.getByteSize(); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (cachedResult == null) { + double ssd = sumSquaredDeviation(); + double result = Math.sqrt(ssd / getDataPointsCount()); + cachedResult = new BigDecimal(result); + } + if (buffer == null) { + initBuffer(); + } + buffer = PDataType.DECIMAL.toBytes(cachedResult); + ptr.set(buffer); + return true; + } + + protected abstract long getDataPointsCount(); + + private double sumSquaredDeviation() { + double m = mean(); + double result = 0.0; + for (Entry entry : valueVsCount.entrySet()) { + double colValue = (Double)PDataType.DOUBLE.toObject(entry.getKey(), this.stdDevColExp.getDataType()); + double delta = colValue - m; + result += (delta * delta) * entry.getValue(); + } + return result; + } + + private double mean() { + double sum = 0.0; + for (Entry entry : valueVsCount.entrySet()) { + double colValue = (Double)PDataType.DOUBLE.toObject(entry.getKey(), this.stdDevColExp.getDataType()); + sum += colValue * entry.getValue(); + } + return sum / totalCount; + } + + @Override + public void reset() { + super.reset(); + this.cachedResult = null; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/ClientAggregators.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/ClientAggregators.java new file mode 100644 index 00000000..0ac5baf7 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/ClientAggregators.java @@ -0,0 +1,80 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.util.List; + +import org.apache.phoenix.expression.function.SingleAggregateFunction; +import org.apache.phoenix.schema.ValueBitSet; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.TupleUtil; + + + +/** + * + * Aggregators that execute on the client-side + * + * @author jtaylor + * @since 0.1 + */ +public class ClientAggregators extends Aggregators { + private final ValueBitSet tempValueSet; + + private static Aggregator[] getAggregators(List aggFuncs) { + Aggregator[] aggregators = new Aggregator[aggFuncs.size()]; + for (int i = 0; i < aggregators.length; i++) { + aggregators[i] = aggFuncs.get(i).getAggregator(); + } + return aggregators; + } + + public ClientAggregators(List functions, int minNullableIndex) { + super(functions.toArray(new SingleAggregateFunction[functions.size()]), getAggregators(functions), minNullableIndex); + this.tempValueSet = ValueBitSet.newInstance(schema); + } + + @Override + public void aggregate(Aggregator[] aggregators, Tuple result) { + TupleUtil.getAggregateValue(result, ptr); + tempValueSet.clear(); + tempValueSet.or(ptr); + + int i = 0, maxOffset = ptr.getOffset() + ptr.getLength(); + Boolean hasValue; + schema.iterator(ptr); + while ((hasValue=schema.next(ptr, i, maxOffset, tempValueSet)) != null) { + if (hasValue) { + aggregators[i].aggregate(result, ptr); + } + i++; + } + } + + @Override + public Aggregator[] newAggregators() { + Aggregator[] aggregators = new Aggregator[functions.length]; + for (int i = 0; i < functions.length; i++) { + aggregators[i] = functions[i].newClientAggregator(); + } + return aggregators; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/CountAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/CountAggregator.java new file mode 100644 index 00000000..33740c2e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/CountAggregator.java @@ -0,0 +1,92 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.SizedUtil; + +/** + * + * Aggregator for COUNT aggregations + * + * @author jtaylor + * @since 0.1 + */ +public class CountAggregator extends BaseAggregator { + + private long count = 0; + private byte[] buffer = null; + + public CountAggregator() { + super(null); + } + + public CountAggregator(LongSumAggregator clientAgg) { + this(); + count = clientAgg.getSum(); + } + + @Override + public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { + count++; + } + + @Override + public boolean isNullable() { + return false; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (buffer == null) { + buffer = new byte[getDataType().getByteSize()]; + } + getDataType().getCodec().encodeLong(count, buffer, 0); + ptr.set(buffer); + return true; + } + + @Override + public final PDataType getDataType() { + return PDataType.LONG; + } + + @Override + public void reset() { + count = 0; + buffer = null; + super.reset(); + } + + @Override + public String toString() { + return "COUNT [count=" + count + "]"; + } + + @Override + public int getSize() { + return super.getSize() + SizedUtil.LONG_SIZE + SizedUtil.ARRAY_SIZE + + getDataType().getByteSize(); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DecimalStddevPopAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DecimalStddevPopAggregator.java new file mode 100644 index 00000000..c707057e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DecimalStddevPopAggregator.java @@ -0,0 +1,43 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.util.List; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.ColumnModifier; + +/** + * Client side Aggregator for STDDEV_POP aggregations for DECIMAL data type. + * + * @author anoopsjohn + * @since 1.2.1 + */ +public class DecimalStddevPopAggregator extends BaseDecimalStddevAggregator { + + public DecimalStddevPopAggregator(List exps, ColumnModifier columnModifier) { + super(exps, columnModifier); + } + + @Override + protected long getDataPointsCount() { + return totalCount; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DecimalStddevSampAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DecimalStddevSampAggregator.java new file mode 100644 index 00000000..0aa19280 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DecimalStddevSampAggregator.java @@ -0,0 +1,43 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.util.List; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.ColumnModifier; + +/** + * Client side Aggregator for STDDEV_SAMP aggregations for DECIMAL data type. + * + * @author anoopsjohn + * @since 1.2.1 + */ +public class DecimalStddevSampAggregator extends BaseDecimalStddevAggregator { + + public DecimalStddevSampAggregator(List exps, ColumnModifier columnModifier) { + super(exps, columnModifier); + } + + @Override + protected long getDataPointsCount() { + return totalCount - 1; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DecimalSumAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DecimalSumAggregator.java new file mode 100644 index 00000000..5b91ef3b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DecimalSumAggregator.java @@ -0,0 +1,103 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.math.BigDecimal; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.SizedUtil; + + +/** + * + * Aggregator that sums BigDecimal values + * + * @author jtaylor + * @since 0.1 + */ +public class DecimalSumAggregator extends BaseAggregator { + private BigDecimal sum = BigDecimal.ZERO; + private byte[] sumBuffer; + + public DecimalSumAggregator(ColumnModifier columnModifier, ImmutableBytesWritable ptr) { + super(columnModifier); + if (ptr != null) { + initBuffer(); + sum = (BigDecimal)PDataType.DECIMAL.toObject(ptr); + } + } + + private PDataType getInputDataType() { + return PDataType.DECIMAL; + } + + private int getBufferLength() { + return getDataType().getByteSize(); + } + + private void initBuffer() { + sumBuffer = new byte[getBufferLength()]; + } + + @Override + public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { + BigDecimal value = (BigDecimal)getDataType().toObject(ptr, getInputDataType(), columnModifier); + sum = sum.add(value); + if (sumBuffer == null) { + sumBuffer = new byte[getDataType().getByteSize()]; + } + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (sumBuffer == null) { + return false; + } + int len = getDataType().toBytes(sum, sumBuffer, 0); + ptr.set(sumBuffer, 0, len); + return true; + } + + @Override + public final PDataType getDataType() { + return PDataType.DECIMAL; + } + + @Override + public void reset() { + sum = BigDecimal.ZERO; + sumBuffer = null; + super.reset(); + } + + @Override + public String toString() { + return "DECIMAL SUM [sum=" + sum + "]"; + } + + @Override + public int getSize() { + return super.getSize() + SizedUtil.BIG_DECIMAL_SIZE + SizedUtil.ARRAY_SIZE + getDataType().getByteSize(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DistinctCountClientAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DistinctCountClientAggregator.java new file mode 100644 index 00000000..d221e910 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DistinctCountClientAggregator.java @@ -0,0 +1,55 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * Client side Aggregator for DISTINCT COUNT aggregations + * + * @author anoopsjohn + * @since 1.2.1 + */ +public class DistinctCountClientAggregator extends DistinctValueWithCountClientAggregator { + + public DistinctCountClientAggregator(ColumnModifier columnModifier) { + super(columnModifier); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (buffer == null) { + initBuffer(); + } + long value = this.valueVsCount.size(); + buffer = PDataType.LONG.toBytes(value); + ptr.set(buffer); + return true; + } + + @Override + protected int getBufferLength() { + return PDataType.LONG.getByteSize(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DistinctValueWithCountClientAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DistinctValueWithCountClientAggregator.java new file mode 100644 index 00000000..c009a28b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DistinctValueWithCountClientAggregator.java @@ -0,0 +1,132 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.io.ByteArrayInputStream; +import java.io.DataInputStream; +import java.io.IOException; +import java.io.InputStream; +import java.util.Comparator; +import java.util.HashMap; +import java.util.Map; +import java.util.Map.Entry; +import java.util.TreeMap; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.io.WritableUtils; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * Client side Aggregator which will aggregate data and find distinct values with number of occurrences for each. + * + * @author anoopsjohn + * @since 1.2.1 + */ +public abstract class DistinctValueWithCountClientAggregator extends BaseAggregator { + protected Map valueVsCount = new HashMap(); + protected byte[] buffer; + protected long totalCount = 0L; + + public DistinctValueWithCountClientAggregator(ColumnModifier columnModifier) { + super(columnModifier); + } + + @Override + public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { + InputStream is = new ByteArrayInputStream(ptr.get(), ptr.getOffset() + 1, ptr.getLength() - 1); + try { + if (Bytes.equals(ptr.get(), ptr.getOffset(), 1, DistinctValueWithCountServerAggregator.COMPRESS_MARKER, 0, + 1)) { + InputStream decompressionStream = DistinctValueWithCountServerAggregator.COMPRESS_ALGO + .createDecompressionStream(is, + DistinctValueWithCountServerAggregator.COMPRESS_ALGO.getDecompressor(), 0); + is = decompressionStream; + } + DataInputStream in = new DataInputStream(is); + int mapSize = WritableUtils.readVInt(in); + for (int i = 0; i < mapSize; i++) { + int keyLen = WritableUtils.readVInt(in); + byte[] keyBytes = new byte[keyLen]; + in.read(keyBytes, 0, keyLen); + ImmutableBytesPtr key = new ImmutableBytesPtr(keyBytes); + int value = WritableUtils.readVInt(in); + Integer curCount = valueVsCount.get(key); + if (curCount == null) { + valueVsCount.put(key, value); + } else { + valueVsCount.put(key, curCount + value); + } + totalCount += value; + } + } catch (IOException ioe) { + throw new RuntimeException(ioe); // Impossible as we're using a ByteArrayInputStream + } + if (buffer == null) { + initBuffer(); + } + } + + protected abstract int getBufferLength(); + + protected void initBuffer() { + buffer = new byte[getBufferLength()]; + } + + @Override + public boolean isNullable() { + return false; + } + + @Override + public PDataType getDataType() { + return PDataType.VARBINARY; + } + + @Override + public void reset() { + valueVsCount = new HashMap(); + buffer = null; + totalCount = 0L; + super.reset(); + } + + protected Map getSortedValueVsCount(final boolean ascending, final PDataType type) { + // To sort the valueVsCount + Comparator comparator = new Comparator() { + @Override + public int compare(Object o1, Object o2) { + if (ascending) { + return type.compareTo(o1, o2); + } + return type.compareTo(o2, o1); + } + }; + Map sorted = new TreeMap(comparator); + for (Entry entry : valueVsCount.entrySet()) { + sorted.put(type.toObject(entry.getKey(), columnModifier), entry.getValue()); + } + return sorted; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DistinctValueWithCountServerAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DistinctValueWithCountServerAggregator.java similarity index 75% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DistinctValueWithCountServerAggregator.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DistinctValueWithCountServerAggregator.java index 3110f29e..ef0a9c22 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/aggregator/DistinctValueWithCountServerAggregator.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DistinctValueWithCountServerAggregator.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.aggregator; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; import java.io.ByteArrayOutputStream; import java.io.OutputStream; @@ -41,13 +33,13 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.SizedUtil; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.SizedUtil; /** * Server side Aggregator which will aggregate data and find distinct values with number of occurrences for each. diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DoubleSumAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DoubleSumAggregator.java new file mode 100644 index 00000000..fa1dda7b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/DoubleSumAggregator.java @@ -0,0 +1,94 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.SizedUtil; + +public class DoubleSumAggregator extends BaseAggregator { + + private double sum = 0; + private byte[] buffer; + + public DoubleSumAggregator(ColumnModifier columnModifier, ImmutableBytesWritable ptr) { + super(columnModifier); + if (ptr != null) { + initBuffer(); + sum = PDataType.DOUBLE.getCodec().decodeDouble(ptr, columnModifier); + } + } + + protected PDataType getInputDataType() { + return PDataType.DOUBLE; + } + + private void initBuffer() { + buffer = new byte[getDataType().getByteSize()]; + } + + @Override + public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { + double value = getInputDataType().getCodec().decodeDouble(ptr, columnModifier); + sum += value; + if (buffer == null) { + initBuffer(); + } + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (buffer == null) { + if (isNullable()) { + return false; + } + initBuffer(); + } + ptr.set(buffer); + getDataType().getCodec().encodeDouble(sum, ptr); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.DOUBLE; + } + + @Override + public String toString() { + return "SUM [sum=" + sum + "]"; + } + + @Override + public void reset() { + sum = 0; + buffer = null; + super.reset(); + } + + @Override + public int getSize() { + return super.getSize() + SizedUtil.LONG_SIZE + SizedUtil.ARRAY_SIZE + getDataType().getByteSize(); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/IntSumAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/IntSumAggregator.java new file mode 100644 index 00000000..e96a993b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/IntSumAggregator.java @@ -0,0 +1,43 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; + +/** + * + * Aggregator that sums integer values + * + * @author jtaylor + * @since 0.1 + */ +public class IntSumAggregator extends NumberSumAggregator { + + public IntSumAggregator(ColumnModifier columnModifier) { + super(columnModifier); + } + + @Override + protected PDataType getInputDataType() { + return PDataType.INTEGER; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/LongSumAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/LongSumAggregator.java new file mode 100644 index 00000000..bfdadc90 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/LongSumAggregator.java @@ -0,0 +1,42 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; + +/** + * + * Aggregator that sums long values + * + * @author jtaylor + * @since 0.1 + */ +public class LongSumAggregator extends NumberSumAggregator { + + public LongSumAggregator(ColumnModifier columnModifier) { + super(columnModifier); + } + + @Override + protected PDataType getInputDataType() { + return PDataType.LONG; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/MaxAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/MaxAggregator.java new file mode 100644 index 00000000..890e14aa --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/MaxAggregator.java @@ -0,0 +1,50 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.phoenix.schema.ColumnModifier; + + +/** + * Aggregator that finds the max of values. Inverse of {@link MinAggregator}. + * + * @author syyang + * @since 0.1 + */ +abstract public class MaxAggregator extends MinAggregator { + + public MaxAggregator(ColumnModifier columnModifier) { + super(columnModifier); + } + + @Override + protected boolean keepFirst(ImmutableBytesWritable ibw1, ImmutableBytesWritable ibw2) { + return !super.keepFirst(ibw1, ibw2); + } + + @Override + public String toString() { + return "MAX [value=" + Bytes.toStringBinary(value.get(),value.getOffset(),value.getLength()) + "]"; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/MinAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/MinAggregator.java new file mode 100644 index 00000000..8954de28 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/MinAggregator.java @@ -0,0 +1,98 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.SizedUtil; + + +/** + * Aggregator that finds the min of values. Inverse of {@link MaxAggregator}. + * + * @author syyang + * @since 0.1 + */ +abstract public class MinAggregator extends BaseAggregator { + /** Used to store the accumulate the results of the MIN function */ + protected final ImmutableBytesWritable value = new ImmutableBytesWritable(ByteUtil.EMPTY_BYTE_ARRAY); + + public MinAggregator(ColumnModifier columnModifier) { + super(columnModifier); + } + + @Override + public void reset() { + value.set(ByteUtil.EMPTY_BYTE_ARRAY); + super.reset(); + } + + @Override + public int getSize() { + return super.getSize() + /*value*/ SizedUtil.IMMUTABLE_BYTES_WRITABLE_SIZE; + } + + /** + * Compares two bytes writables, and returns true if the first one should be + * kept, and false otherwise. For the MIN function, this method will return + * true if the first bytes writable is less than the second. + * + * @param ibw1 the first bytes writable + * @param ibw2 the second bytes writable + * @return true if the first bytes writable should be kept + */ + protected boolean keepFirst(ImmutableBytesWritable ibw1, ImmutableBytesWritable ibw2) { + return 0 >= getDataType().compareTo(ibw1, columnModifier, ibw2, columnModifier, getDataType()); + } + + private boolean isNull() { + return value.get() == ByteUtil.EMPTY_BYTE_ARRAY; + } + + @Override + public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { + if (isNull()) { + value.set(ptr.get(), ptr.getOffset(), ptr.getLength()); + } else { + if (!keepFirst(value, ptr)) { + // replace the value with the new value + value.set(ptr.get(), ptr.getOffset(), ptr.getLength()); + } + } + } + + @Override + public String toString() { + return "MIN [value=" + Bytes.toStringBinary(value.get(),value.getOffset(),value.getLength()) + "]"; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (isNull()) { + return false; + } + ptr.set(value.get(), value.getOffset(), value.getLength()); + return true; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/NumberSumAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/NumberSumAggregator.java new file mode 100644 index 00000000..a7cfcd19 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/NumberSumAggregator.java @@ -0,0 +1,114 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.SizedUtil; + +/** + * + * Aggregator that sums integral number values + * + * @author jtaylor + * @since 0.1 + */ +abstract public class NumberSumAggregator extends BaseAggregator { + private long sum = 0; + private byte[] buffer; + + public NumberSumAggregator(ColumnModifier columnModifier) { + super(columnModifier); + } + + public NumberSumAggregator(ColumnModifier columnModifier, + ImmutableBytesWritable ptr) { + this(columnModifier); + if (ptr != null) { + initBuffer(); + sum = PDataType.LONG.getCodec().decodeLong(ptr, columnModifier); + } + } + + public long getSum() { + return sum; + } + + abstract protected PDataType getInputDataType(); + + private int getBufferLength() { + return getDataType().getByteSize(); + } + + private void initBuffer() { + buffer = new byte[getBufferLength()]; + } + + @Override + public void aggregate(Tuple tuple, ImmutableBytesWritable ptr) { + // Get either IntNative or LongNative depending on input type + long value = getInputDataType().getCodec().decodeLong(ptr, + columnModifier); + sum += value; + if (buffer == null) { + initBuffer(); + } + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (buffer == null) { + if (isNullable()) { + return false; + } + initBuffer(); + } + ptr.set(buffer); + getDataType().getCodec().encodeLong(sum, ptr); + return true; + } + + @Override + public final PDataType getDataType() { + return PDataType.LONG; + } + + @Override + public void reset() { + sum = 0; + buffer = null; + super.reset(); + } + + @Override + public String toString() { + return "SUM [sum=" + sum + "]"; + } + + @Override + public int getSize() { + return super.getSize() + SizedUtil.LONG_SIZE + SizedUtil.ARRAY_SIZE + + getBufferLength(); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/PercentRankClientAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/PercentRankClientAggregator.java new file mode 100644 index 00000000..42ca2676 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/PercentRankClientAggregator.java @@ -0,0 +1,92 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.math.BigDecimal; +import java.util.*; +import java.util.Map.Entry; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.*; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * Client side Aggregator for PERCENT_RANK aggregations + * + * @author anoopsjohn + * @since 1.2.1 + */ +public class PercentRankClientAggregator extends DistinctValueWithCountClientAggregator { + + private final List exps; + private BigDecimal cachedResult = null; + + public PercentRankClientAggregator(List exps, ColumnModifier columnModifier) { + super(columnModifier); + this.exps = exps; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (cachedResult == null) { + ColumnExpression columnExp = (ColumnExpression)exps.get(0); + // Second exp will be a LiteralExpression of Boolean type indicating whether the ordering to + // be ASC/DESC + LiteralExpression isAscendingExpression = (LiteralExpression)exps.get(1); + boolean isAscending = (Boolean)isAscendingExpression.getValue(); + + // Third expression will be LiteralExpression + LiteralExpression valueExp = (LiteralExpression)exps.get(2); + Map sorted = getSortedValueVsCount(isAscending, columnExp.getDataType()); + long distinctCountsSum = 0; + Object value = valueExp.getValue(); + for (Entry entry : sorted.entrySet()) { + Object colValue = entry.getKey(); + int compareResult = columnExp.getDataType().compareTo(colValue, value, valueExp.getDataType()); + boolean done = isAscending ? compareResult > 0 : compareResult <= 0; + if (done) break; + distinctCountsSum += entry.getValue(); + } + + float result = (float)distinctCountsSum / totalCount; + this.cachedResult = new BigDecimal(result); + } + if (buffer == null) { + initBuffer(); + } + buffer = PDataType.DECIMAL.toBytes(this.cachedResult); + ptr.set(buffer); + return true; + } + + @Override + protected int getBufferLength() { + return PDataType.DECIMAL.getByteSize(); + } + + @Override + public void reset() { + super.reset(); + this.cachedResult = null; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/PercentileClientAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/PercentileClientAggregator.java new file mode 100644 index 00000000..095842ab --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/PercentileClientAggregator.java @@ -0,0 +1,110 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.math.BigDecimal; +import java.util.*; +import java.util.Map.Entry; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.*; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * Client side Aggregator for PERCENTILE_CONT aggregations + * + * @author anoopsjohn + * @since 1.2.1 + */ +public class PercentileClientAggregator extends DistinctValueWithCountClientAggregator { + + private final List exps; + private BigDecimal cachedResult = null; + + public PercentileClientAggregator(List exps, ColumnModifier columnModifier) { + super(columnModifier); + this.exps = exps; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (cachedResult == null) { + ColumnExpression columnExp = (ColumnExpression)exps.get(0); + // Second exp will be a LiteralExpression of Boolean type indicating whether the ordering to + // be ASC/DESC + LiteralExpression isAscendingExpression = (LiteralExpression)exps.get(1); + boolean isAscending = (Boolean)isAscendingExpression.getValue(); + + // Third expression will be LiteralExpression + LiteralExpression percentileExp = (LiteralExpression)exps.get(2); + float p = ((Number)percentileExp.getValue()).floatValue(); + Map sorted = getSortedValueVsCount(isAscending, columnExp.getDataType()); + float i = (p * this.totalCount) + 0.5F; + long k = (long)i; + float f = i - k; + Object o1 = null; + Object o2 = null; + long distinctCountsSum = 0; + for (Entry entry : sorted.entrySet()) { + if (o1 != null) { + o2 = entry.getKey(); + break; + } + distinctCountsSum += entry.getValue(); + if (distinctCountsSum == k) { + o1 = entry.getKey(); + } else if (distinctCountsSum > k) { + o1 = o2 = entry.getKey(); + break; + } + } + + double result = 0.0; + Number n1 = (Number)o1; + if (o2 == null || o1 == o2) { + result = n1.doubleValue(); + } else { + Number n2 = (Number)o2; + result = (n1.doubleValue() * (1.0F - f)) + (n2.doubleValue() * f); + } + this.cachedResult = new BigDecimal(result); + } + if (buffer == null) { + initBuffer(); + } + buffer = PDataType.DECIMAL.toBytes(this.cachedResult); + ptr.set(buffer); + return true; + } + + @Override + protected int getBufferLength() { + return PDataType.DECIMAL.getByteSize(); + } + + @Override + public void reset() { + super.reset(); + this.cachedResult = null; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/PercentileDiscClientAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/PercentileDiscClientAggregator.java new file mode 100644 index 00000000..206ed1d8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/PercentileDiscClientAggregator.java @@ -0,0 +1,106 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.util.*; +import java.util.Map.Entry; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.*; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * + * Built-in function for PERCENTILE_DISC() WITHIN GROUP (ORDER BY ASC/DESC) aggregate function + * + * @author ramkrishna + * @since 1.2.1 + */ +public class PercentileDiscClientAggregator extends + DistinctValueWithCountClientAggregator { + + private final List exps; + private Object cachedResult = null; + ColumnExpression columnExp = null; + + public PercentileDiscClientAggregator(List exps, ColumnModifier columnModifier) { + super(columnModifier); + this.exps = exps; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + // Reset buffer so that it gets initialized with the current datatype of the column + buffer = null; + if (cachedResult == null) { + columnExp = (ColumnExpression)exps.get(0); + // Second exp will be a LiteralExpression of Boolean type indicating + // whether the ordering to be ASC/DESC + LiteralExpression isAscendingExpression = (LiteralExpression) exps + .get(1); + boolean isAscending = (Boolean) isAscendingExpression.getValue(); + + // Third expression will be LiteralExpression + LiteralExpression percentileExp = (LiteralExpression) exps.get(2); + float p = ((Number) percentileExp.getValue()).floatValue(); + Map sorted = getSortedValueVsCount(isAscending, columnExp.getDataType()); + int currValue = 0; + Object result = null; + // Here the Percentile_disc returns the cum_dist() that is greater or equal to the + // Percentile (p) specified in the query. So the result set will be of that of the + // datatype of the column being selected + for (Entry entry : sorted.entrySet()) { + result = entry.getKey(); + Integer value = entry.getValue(); + currValue += value; + float cum_dist = (float) currValue / (float) totalCount; + if (cum_dist >= p) { + break; + } + } + this.cachedResult = result; + } + if (buffer == null) { + // Initialize based on the datatype + // columnExp cannot be null + buffer = new byte[columnExp.getDataType().getByteSize()]; + } + // Copy the result to the buffer. + System.arraycopy(columnExp.getDataType().toBytes(this.cachedResult), 0, buffer, 0, buffer.length); + ptr.set(buffer); + return true; + } + + @Override + public void reset() { + super.reset(); + this.cachedResult = null; + } + + @Override + protected int getBufferLength() { + // Will be used in the aggregate() call + return PDataType.DECIMAL.getByteSize(); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/ServerAggregators.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/ServerAggregators.java new file mode 100644 index 00000000..64577933 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/ServerAggregators.java @@ -0,0 +1,145 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.util.Arrays; +import java.util.List; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.io.WritableUtils; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.ExpressionType; +import org.apache.phoenix.expression.function.SingleAggregateFunction; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Aggregators that execute on the server-side + * + * @author jtaylor + * @since 0.1 + */ +public class ServerAggregators extends Aggregators { + public static final ServerAggregators EMPTY_AGGREGATORS = new ServerAggregators(new SingleAggregateFunction[0], new Aggregator[0], new Expression[0], 0); + private final Expression[] expressions; + + private ServerAggregators(SingleAggregateFunction[] functions, Aggregator[] aggregators, Expression[] expressions, int minNullableIndex) { + super(functions, aggregators, minNullableIndex); + if (aggregators.length != expressions.length) { + throw new IllegalArgumentException("Number of aggregators (" + aggregators.length + + ") must match the number of expressions (" + Arrays.toString(expressions) + ")"); + } + this.expressions = expressions; + } + + @Override + public void aggregate(Aggregator[] aggregators, Tuple result) { + for (int i = 0; i < expressions.length; i++) { + if (expressions[i].evaluate(result, ptr)) { + aggregators[i].aggregate(result, ptr); + } + } + } + + /** + * Serialize an Aggregator into a byte array + * @param aggFuncs list of aggregator to serialize + * @return serialized byte array respresentation of aggregator + */ + public static byte[] serialize(List aggFuncs, int minNullableIndex) { + ByteArrayOutputStream stream = new ByteArrayOutputStream(); + try { + DataOutputStream output = new DataOutputStream(stream); + WritableUtils.writeVInt(output, minNullableIndex); + WritableUtils.writeVInt(output, aggFuncs.size()); + for (int i = 0; i < aggFuncs.size(); i++) { + SingleAggregateFunction aggFunc = aggFuncs.get(i); + WritableUtils.writeVInt(output, ExpressionType.valueOf(aggFunc).ordinal()); + aggFunc.write(output); + } + return stream.toByteArray(); + } catch (IOException e) { + throw new RuntimeException(e); + } finally { + try { + stream.close(); + } catch (IOException e) { + throw new RuntimeException(e); + } + } + } + + @Override + public Aggregator[] newAggregators() { + return newAggregators(null); + } + + public Aggregator[] newAggregators(Configuration conf) { + Aggregator[] aggregators = new Aggregator[functions.length]; + for (int i = 0; i < functions.length; i++) { + aggregators[i] = functions[i].newServerAggregator(conf); + } + return aggregators; + } + + /** + * Deserialize aggregators from the serialized byte array representation + * @param b byte array representation of a list of Aggregators + * @param conf Server side configuration used by HBase + * @return newly instantiated Aggregators instance + */ + public static ServerAggregators deserialize(byte[] b, Configuration conf) { + if (b == null) { + return ServerAggregators.EMPTY_AGGREGATORS; + } + ByteArrayInputStream stream = new ByteArrayInputStream(b); + try { + DataInputStream input = new DataInputStream(stream); + int minNullableIndex = WritableUtils.readVInt(input); + int len = WritableUtils.readVInt(input); + Aggregator[] aggregators = new Aggregator[len]; + Expression[] expressions = new Expression[len]; + SingleAggregateFunction[] functions = new SingleAggregateFunction[len]; + for (int i = 0; i < aggregators.length; i++) { + SingleAggregateFunction aggFunc = (SingleAggregateFunction)ExpressionType.values()[WritableUtils.readVInt(input)].newInstance(); + aggFunc.readFields(input, conf); + functions[i] = aggFunc; + aggregators[i] = aggFunc.getAggregator(); + expressions[i] = aggFunc.getAggregatorExpression(); + } + return new ServerAggregators(functions, aggregators,expressions, minNullableIndex); + } catch (IOException e) { + throw new RuntimeException(e); + } finally { + try { + stream.close(); + } catch (IOException e) { + throw new RuntimeException(e); + } + } + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/StddevPopAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/StddevPopAggregator.java new file mode 100644 index 00000000..30276e11 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/StddevPopAggregator.java @@ -0,0 +1,43 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.util.List; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.ColumnModifier; + +/** + * Client side Aggregator for STDDEV_POP aggregations + * + * @author anoopsjohn + * @since 1.2.1 + */ +public class StddevPopAggregator extends BaseStddevAggregator { + + public StddevPopAggregator(List exps, ColumnModifier columnModifier) { + super(exps, columnModifier); + } + + @Override + protected long getDataPointsCount() { + return totalCount; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/StddevSampAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/StddevSampAggregator.java new file mode 100644 index 00000000..49f52d22 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/StddevSampAggregator.java @@ -0,0 +1,43 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import java.util.List; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.ColumnModifier; + +/** + * Client side Aggregator for STDDEV_SAMP aggregations + * + * @author anoopsjohn + * @since 1.2.1 + */ +public class StddevSampAggregator extends BaseStddevAggregator { + + public StddevSampAggregator(List exps, ColumnModifier columnModifier) { + super(exps, columnModifier); + } + + @Override + protected long getDataPointsCount() { + return totalCount - 1; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/UnsignedIntSumAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/UnsignedIntSumAggregator.java new file mode 100644 index 00000000..c8befcab --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/UnsignedIntSumAggregator.java @@ -0,0 +1,43 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; + +/** + * + * Aggregator that sums unsigned integer values + * + * @author jtaylor + * @since 0.12 + */ +public class UnsignedIntSumAggregator extends NumberSumAggregator { + + public UnsignedIntSumAggregator(ColumnModifier columnModifier) { + super(columnModifier); + } + + @Override + protected PDataType getInputDataType() { + return PDataType.UNSIGNED_INT; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/UnsignedLongSumAggregator.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/UnsignedLongSumAggregator.java new file mode 100644 index 00000000..b91a9346 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/aggregator/UnsignedLongSumAggregator.java @@ -0,0 +1,44 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.aggregator; + +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; + +/** + * + * Aggregator that sums unsigned long values + * TODO: create these classes dynamically based on the type passed through + * + * @author jtaylor + * @since 0.12 + */ +public class UnsignedLongSumAggregator extends NumberSumAggregator { + + public UnsignedLongSumAggregator(ColumnModifier columnModifier) { + super(columnModifier); + } + + @Override + protected PDataType getInputDataType() { + return PDataType.UNSIGNED_LONG; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/AggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/AggregateFunction.java new file mode 100644 index 00000000..72809f51 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/AggregateFunction.java @@ -0,0 +1,54 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.phoenix.expression.Expression; + + + + +/** + * + * Compiled representation of a built-in aggregate function + * + * @author jtaylor + * @since 0.1 + */ +abstract public class AggregateFunction extends FunctionExpression { + + public AggregateFunction() { + } + + public AggregateFunction(List children) { + super(children); + } + + @Override + public boolean isStateless() { + return false; + } + + @Override + public boolean isDeterministic() { + return false; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ArrayIndexFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ArrayIndexFunction.java new file mode 100644 index 00000000..a4185e6a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ArrayIndexFunction.java @@ -0,0 +1,88 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.Types; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.parse.ParseException; +import org.apache.phoenix.schema.PArrayDataType; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +@BuiltInFunction(name = ArrayIndexFunction.NAME, args = { + @Argument(allowedTypes = { PDataType.BINARY_ARRAY, + PDataType.VARBINARY_ARRAY }), + @Argument(allowedTypes = { PDataType.INTEGER }) }) +public class ArrayIndexFunction extends ScalarFunction { + + public static final String NAME = "ARRAY_ELEM"; + + public ArrayIndexFunction() { + } + + public ArrayIndexFunction(List children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + Expression indexExpr = children.get(1); + if (!indexExpr.evaluate(tuple, ptr)) { + return false; + } else if (ptr.getLength() == 0) { + return true; + } + // Use Codec to prevent Integer object allocation + int index = PDataType.INTEGER.getCodec().decodeInt(ptr, indexExpr.getColumnModifier()); + if(index < 0) { + throw new ParseException("Index cannot be negative :" + index); + } + Expression arrayExpr = children.get(0); + if (!arrayExpr.evaluate(tuple, ptr)) { + return false; + } else if (ptr.getLength() == 0) { + return true; + } + + // Given a ptr to the entire array, set ptr to point to a particular element within that array + // given the type of an array element (see comments in PDataTypeForArray) + PArrayDataType.positionAtArrayElement(ptr, index, getDataType()); + return true; + + } + + @Override + public PDataType getDataType() { + return PDataType.fromTypeId(children.get(0).getDataType().getSqlType() + - Types.ARRAY); + } + + @Override + public String getName() { + return NAME; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ArrayLengthFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ArrayLengthFunction.java new file mode 100644 index 00000000..9a94129b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ArrayLengthFunction.java @@ -0,0 +1,75 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.Types; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PArrayDataType; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +@BuiltInFunction(name = ArrayLengthFunction.NAME, args = { @Argument(allowedTypes = { + PDataType.BINARY_ARRAY, PDataType.VARBINARY_ARRAY }) }) +public class ArrayLengthFunction extends ScalarFunction { + public static final String NAME = "ARRAY_LENGTH"; + + public ArrayLengthFunction() { + } + + public ArrayLengthFunction(List children) { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + Expression arrayExpr = children.get(0); + if (!arrayExpr.evaluate(tuple, ptr)) { + return false; + } else if (ptr.getLength() == 0) { + return true; + } + PDataType baseType = PDataType.fromTypeId(children.get(0).getDataType() + .getSqlType() + - Types.ARRAY); + int length = PArrayDataType.getArrayLength(ptr, baseType); + byte[] lengthBuf = new byte[PDataType.INTEGER.getByteSize()]; + PDataType.INTEGER.getCodec().encodeInt(length, lengthBuf, 0); + ptr.set(lengthBuf); + return true; + } + + @Override + public PDataType getDataType() { + // Array length will return an Integer + return PDataType.INTEGER; + } + + @Override + public String getName() { + return NAME; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/AvgAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/AvgAggregateFunction.java new file mode 100644 index 00000000..62bb565e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/AvgAggregateFunction.java @@ -0,0 +1,113 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.math.BigDecimal; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.parse.*; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +@BuiltInFunction(name=AvgAggregateFunction.NAME, nodeClass=AvgAggregateParseNode.class, args= {@Argument(allowedTypes={PDataType.DECIMAL})} ) +public class AvgAggregateFunction extends CompositeAggregateFunction { + public static final String NAME = "AVG"; + private final CountAggregateFunction countFunc; + private final SumAggregateFunction sumFunc; + private Integer scale; + + // TODO: remove when not required at built-in func register time + public AvgAggregateFunction(List children) { + super(children); + this.countFunc = null; + this.sumFunc = null; + setScale(children); + } + + public AvgAggregateFunction(List children, CountAggregateFunction countFunc, SumAggregateFunction sumFunc) { + super(children); + this.countFunc = countFunc; + this.sumFunc = sumFunc; + setScale(children); + } + + private void setScale(List children) { + scale = PDataType.MIN_DECIMAL_AVG_SCALE; // At least 4; + for (Expression child: children) { + if (child.getScale() != null) { + scale = Math.max(scale, child.getScale()); + } + } + } + + @Override + public PDataType getDataType() { + return PDataType.DECIMAL; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (!countFunc.evaluate(tuple, ptr)) { + return false; + } + long count = countFunc.getDataType().getCodec().decodeLong(ptr, null); + if (count == 0) { + return false; + } + + // Normal case where a column reference was used as the argument to AVG + if (!countFunc.isConstantExpression()) { + sumFunc.evaluate(tuple, ptr); + BigDecimal sum = (BigDecimal)PDataType.DECIMAL.toObject(ptr, sumFunc.getDataType()); + // For the final column projection, we divide the sum by the count, both coerced to BigDecimal. + // TODO: base the precision on column metadata instead of constant + BigDecimal avg = sum.divide(BigDecimal.valueOf(count), PDataType.DEFAULT_MATH_CONTEXT); + avg = avg.setScale(scale, BigDecimal.ROUND_DOWN); + ptr.set(PDataType.DECIMAL.toBytes(avg)); + return true; + } + BigDecimal value = (BigDecimal) ((LiteralExpression)countFunc.getChildren().get(0)).getValue(); + value = value.setScale(scale, BigDecimal.ROUND_DOWN); + ptr.set(PDataType.DECIMAL.toBytes(value)); + return true; + } + + @Override + public boolean isNullable() { + return sumFunc != null && sumFunc.isNullable(); + } + + @Override + public String getName() { + return NAME; + } + + @Override + public Integer getScale() { + return scale; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilDateExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilDateExpression.java new file mode 100644 index 00000000..d0951cf8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilDateExpression.java @@ -0,0 +1,77 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.List; + +import com.google.common.collect.Lists; +import org.apache.phoenix.expression.Expression; + +/** + * + * Class encapsulating ceil operation on {@link org.apache.phoenix.schema.PDataType#DATE}. + * + * @author samarth.jain + * @since 3.0.0 + */ +public class CeilDateExpression extends RoundDateExpression { + + public CeilDateExpression() {} + + /** + * @param timeUnit - unit of time to round up to. + * Creates a {@link CeilDateExpression} with default multiplier of 1. + */ + public static Expression create(Expression expr, TimeUnit timeUnit) throws SQLException { + return create(expr, timeUnit, 1); + } + + /** + * @param timeUnit - unit of time to round up to + * @param multiplier - determines the roll up window size. + * Create a {@link CeilDateExpression}. + */ + public static Expression create(Expression expr, TimeUnit timeUnit, int multiplier) throws SQLException { + Expression timeUnitExpr = getTimeUnitExpr(timeUnit); + Expression defaultMultiplierExpr = getMultiplierExpr(multiplier); + List expressions = Lists.newArrayList(expr, timeUnitExpr, defaultMultiplierExpr); + return CeilDateExpression.create(expressions); + } + + public static Expression create(List children) throws SQLException { + return new CeilDateExpression(children); + } + + CeilDateExpression(List children) { + super(children); + } + + @Override + protected long getRoundUpAmount() { + return divBy - 1; + } + + @Override + public String getName() { + return CeilFunction.NAME; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilDecimalExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilDecimalExpression.java new file mode 100644 index 00000000..dd2c22a9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilDecimalExpression.java @@ -0,0 +1,77 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.math.RoundingMode; +import java.sql.SQLException; +import java.util.List; + +import com.google.common.collect.Lists; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.schema.PDataType; + +/** + * + * Class encapsulating the CEIL operation on a {@link org.apache.phoenix.schema.PDataType#DECIMAL} + * + * @author samarth.jain + * @since 3.0.0 + */ +public class CeilDecimalExpression extends RoundDecimalExpression { + + public CeilDecimalExpression() {} + + public CeilDecimalExpression(List children) { + super(children); + } + + /** + * Creates a {@link CeilDecimalExpression} with rounding scale given by @param scale. + * + */ + public static Expression create(Expression expr, int scale) throws SQLException { + if (expr.getDataType().isCoercibleTo(PDataType.LONG)) { + return expr; + } + Expression scaleExpr = LiteralExpression.newConstant(scale, PDataType.INTEGER, true); + List expressions = Lists.newArrayList(expr, scaleExpr); + return new CeilDecimalExpression(expressions); + } + + /** + * Creates a {@link CeilDecimalExpression} with a default scale of 0 used for rounding. + * + */ + public static Expression create(Expression expr) throws SQLException { + return create(expr, 0); + } + + @Override + protected RoundingMode getRoundingMode() { + return RoundingMode.CEILING; + } + + @Override + public String getName() { + return CeilFunction.NAME; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilFunction.java new file mode 100644 index 00000000..60ad329b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilFunction.java @@ -0,0 +1,57 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.CeilParseNode; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +/** + * + * Base class for built-in CEIL function. + * + * @author samarth.jain + * @since 3.0.0 + */ +@BuiltInFunction(name = CeilFunction.NAME, + nodeClass = CeilParseNode.class, + args = { + @Argument(allowedTypes={PDataType.TIMESTAMP, PDataType.DECIMAL}), + @Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValue = "null", isConstant=true), + @Argument(allowedTypes={PDataType.INTEGER}, defaultValue="1", isConstant=true) + } + ) +public abstract class CeilFunction extends ScalarFunction { + + public static final String NAME = "CEIL"; + + public CeilFunction(List children) { + super(children); + } + + @Override + public String getName() { + return NAME; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilTimestampExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilTimestampExpression.java new file mode 100644 index 00000000..7f54cbc0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CeilTimestampExpression.java @@ -0,0 +1,111 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.sql.Timestamp; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import com.google.common.collect.Lists; +import org.apache.phoenix.expression.CoerceExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDataType.PDataCodec; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * + * Class encapsulating the CEIL operation on {@link org.apache.phoenix.schema.PDataType#TIMESTAMP} + * This class only supports CEIL {@link TimeUnit#MILLISECOND}. If you want more options of CEIL like + * using {@link TimeUnit#HOUR} use {@link CeilDateExpression} + * + * @author samarth.jain + * @since 3.0.0 + */ +public class CeilTimestampExpression extends CeilDateExpression { + + public CeilTimestampExpression() {} + + private CeilTimestampExpression(List children) { + super(children); + } + + /** + * Creates a {@link CeilTimestampExpression} that uses {@link TimeUnit#MILLISECOND} + * as the time unit for rounding. + */ + public static CeilTimestampExpression create(Expression expr, int multiplier) throws SQLException { + List childExprs = Lists.newArrayList(expr, getTimeUnitExpr(TimeUnit.MILLISECOND), getMultiplierExpr(multiplier)); + return new CeilTimestampExpression(childExprs); + } + + public static Expression create(List children) throws SQLException { + Expression firstChild = children.get(0); + PDataType firstChildDataType = firstChild.getDataType(); + String timeUnit = (String)((LiteralExpression)children.get(1)).getValue(); + if(TimeUnit.MILLISECOND.toString().equalsIgnoreCase(timeUnit)) { + return new CeilTimestampExpression(children); + } + // Coerce TIMESTAMP to DATE, as the nanos has no affect + List newChildren = Lists.newArrayListWithExpectedSize(children.size()); + newChildren.add(CoerceExpression.create(firstChild, firstChildDataType == PDataType.TIMESTAMP ? PDataType.DATE : PDataType.UNSIGNED_DATE)); + newChildren.addAll(children.subList(1, children.size())); + return CeilDateExpression.create(newChildren); + } + + /** + * Creates a {@link CeilTimestampExpression} that uses {@link TimeUnit#MILLISECOND} + * as the time unit for rounding. + */ + public static CeilTimestampExpression create (Expression expr) throws SQLException { + return create(expr, 1); + } + + @Override + protected PDataCodec getKeyRangeCodec(PDataType columnDataType) { + return columnDataType == PDataType.TIMESTAMP + ? PDataType.DATE.getCodec() + : columnDataType == PDataType.UNSIGNED_TIMESTAMP + ? PDataType.UNSIGNED_DATE.getCodec() + : super.getKeyRangeCodec(columnDataType); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (children.get(0).evaluate(tuple, ptr)) { + ColumnModifier columnModifier = children.get(0).getColumnModifier(); + PDataType dataType = getDataType(); + int nanos = dataType.getNanos(ptr, columnModifier); + if (nanos > 0) { + long millis = dataType.getMillis(ptr, columnModifier); + Timestamp roundedTs = new Timestamp(millis + 1); + byte[] byteValue = dataType.toBytes(roundedTs); + ptr.set(byteValue); + } + return true; // for timestamp we only support rounding up the milliseconds. + } + return false; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CoalesceFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CoalesceFunction.java new file mode 100644 index 00000000..05f6bd8d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CoalesceFunction.java @@ -0,0 +1,100 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Function used to provide an alternative value when the first argument is null. + * Usage: + * COALESCE(expr1,expr2) + * If expr1 is not null, then it is returned, otherwise expr2 is returned. + * + * TODO: better bind parameter type matching, since arg2 must be coercible + * to arg1. consider allowing a common base type? + * @author jtaylor + * @since 0.1 + */ +@BuiltInFunction(name=CoalesceFunction.NAME, args= { + @Argument(), + @Argument()} ) +public class CoalesceFunction extends ScalarFunction { + public static final String NAME = "COALESCE"; + + public CoalesceFunction() { + } + + public CoalesceFunction(List children) throws SQLException { + super(children); + if (!children.get(1).getDataType().isCoercibleTo(children.get(0).getDataType())) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.CANNOT_CONVERT_TYPE) + .setMessage(getName() + " expected " + children.get(0).getDataType() + ", but got " + children.get(1).getDataType()) + .build().buildException(); + } + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (children.get(0).evaluate(tuple, ptr)) { + return true; + } + return children.get(1).evaluate(tuple, ptr); + } + + @Override + public PDataType getDataType() { + return children.get(0).getDataType(); + } + + @Override + public Integer getByteSize() { + Integer maxByteSize1 = children.get(0).getByteSize(); + if (maxByteSize1 != null) { + Integer maxByteSize2 = children.get(1).getByteSize(); + if (maxByteSize2 != null) { + return maxByteSize1 > maxByteSize2 ? maxByteSize1 : maxByteSize2; + } + } + return null; + } + + @Override + public boolean isNullable() { + return children.get(0).isNullable() && children.get(1).isNullable(); + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CompositeAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CompositeAggregateFunction.java new file mode 100644 index 00000000..2c8da08c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CompositeAggregateFunction.java @@ -0,0 +1,47 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.visitor.ExpressionVisitor; + + +/** + * + * Base class for aggregation functions which are composed of other + * aggregation functions (for example, AVG is modeled as a SUM aggregate + * function and a COUNT aggregate function). + * + * @author jtaylor + * @since 0.1 + */ +abstract public class CompositeAggregateFunction extends AggregateFunction { + + public CompositeAggregateFunction(List children) { + super(children); + } + + @Override + public final T accept(ExpressionVisitor visitor) { + return null; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CountAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CountAggregateFunction.java new file mode 100644 index 00000000..5e7dceb4 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CountAggregateFunction.java @@ -0,0 +1,116 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.Arrays; +import java.util.List; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.CountAggregator; +import org.apache.phoenix.expression.aggregator.LongSumAggregator; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.util.SchemaUtil; + + +/** + * + * Built-in function for COUNT() aggregate function, + * for example COUNT(foo), COUNT(1), COUNT(*) + * + * @author jtaylor + * @since 0.1 + */ +@BuiltInFunction(name=CountAggregateFunction.NAME, args= {@Argument()} ) +public class CountAggregateFunction extends SingleAggregateFunction { + public static final String NAME = "COUNT"; + public static final List STAR = Arrays.asList(LiteralExpression.newConstant(1, true)); + public static final String NORMALIZED_NAME = SchemaUtil.normalizeIdentifier(NAME); + + public CountAggregateFunction() { + } + + public CountAggregateFunction(List childExpressions) { + super(childExpressions); + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + CountAggregateFunction other = (CountAggregateFunction)obj; + return (isConstantExpression() && other.isConstantExpression()) || children.equals(other.getChildren()); + } + + @Override + public int hashCode() { + return isConstantExpression() ? 0 : super.hashCode(); + } + + /** + * The COUNT function never returns null + */ + @Override + public boolean isNullable() { + return false; + } + + @Override + public PDataType getDataType() { + return PDataType.LONG; + } + + @Override + public LongSumAggregator newClientAggregator() { + // Since COUNT can never be null, ensure the aggregator is not nullable. + // This allows COUNT(*) to return 0 with the initial state of ClientAggregators + // when no rows are returned. + return new LongSumAggregator(null) { + @Override + public boolean isNullable() { + return false; + } + }; + } + + @Override + public Aggregator newServerAggregator(Configuration conf) { + return new CountAggregator(); + } + + @Override + public String getName() { + return NAME; + } + + @Override + public Aggregator newServerAggregator(Configuration config, ImmutableBytesWritable ptr) { + LongSumAggregator sumAgg = newClientAggregator(); + sumAgg.aggregate(null, ptr); + return new CountAggregator(sumAgg); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CurrentDateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CurrentDateFunction.java new file mode 100644 index 00000000..ad9e989a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CurrentDateFunction.java @@ -0,0 +1,69 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.CurrentDateTimeFunction; +import org.apache.phoenix.parse.CurrentDateParseNode; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Function that returns the current date accurate to the millisecond. Note that this + * function is never evaluated on the server-side, instead the server side date is + * retrieved (piggy-backed on the call to check that the metadata is up-to-date) and + * passed into this function at create time. + * + * @author jtaylor + * @since 0.1 + */ +@BuiltInFunction(name=CurrentDateFunction.NAME, nodeClass=CurrentDateParseNode.class, args= {} ) +public class CurrentDateFunction extends CurrentDateTimeFunction { + public static final String NAME = "CURRENT_DATE"; + private final ImmutableBytesWritable currentDate = new ImmutableBytesWritable(new byte[PDataType.DATE.getByteSize()]); + + public CurrentDateFunction() { + this(System.currentTimeMillis()); + } + + public CurrentDateFunction(long timeStamp) { + getDataType().getCodec().encodeLong(timeStamp, currentDate); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + ptr.set(currentDate.get(), 0, PDataType.DATE.getByteSize()); + return true; + } + + @Override + public final PDataType getDataType() { + return PDataType.DATE; + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CurrentTimeFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CurrentTimeFunction.java new file mode 100644 index 00000000..71a16024 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/CurrentTimeFunction.java @@ -0,0 +1,69 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.CurrentDateTimeFunction; +import org.apache.phoenix.parse.CurrentTimeParseNode; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Function that returns the current date accurate to the millisecond. Note that this + * function is never evaluated on the server-side, instead the server side date is + * retrieved (piggy-backed on the call to check that the metadata is up-to-date) and + * passed into this function at create time. + * + * @author jtaylor + * @since 0.1 + */ +@BuiltInFunction(name=CurrentTimeFunction.NAME, nodeClass=CurrentTimeParseNode.class, args={} ) +public class CurrentTimeFunction extends CurrentDateTimeFunction { + public static final String NAME = "CURRENT_TIME"; + private final ImmutableBytesWritable currentDate = new ImmutableBytesWritable(new byte[PDataType.TIME.getByteSize()]); + + public CurrentTimeFunction() { + this(System.currentTimeMillis()); + } + + public CurrentTimeFunction(long timeStamp) { + getDataType().getCodec().encodeLong(timeStamp, currentDate); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + ptr.set(currentDate.get(), 0, PDataType.TIME.getByteSize()); + return true; + } + + @Override + public final PDataType getDataType() { + return PDataType.TIME; + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/DelegateConstantToCountAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/DelegateConstantToCountAggregateFunction.java new file mode 100644 index 00000000..744e5bbb --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/DelegateConstantToCountAggregateFunction.java @@ -0,0 +1,75 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Base class for non composite aggregation functions that optimize aggregation by + * delegating to {@link CountAggregateFunction} when the child expression is a + * constant. + * + * @author jtaylor + * @since 0.1 + */ +abstract public class DelegateConstantToCountAggregateFunction extends SingleAggregateFunction { + private static final ImmutableBytesWritable ZERO = new ImmutableBytesWritable(PDataType.LONG.toBytes(0L)); + private CountAggregateFunction delegate; + + public DelegateConstantToCountAggregateFunction() { + } + + public DelegateConstantToCountAggregateFunction(List childExpressions, CountAggregateFunction delegate) { + super(childExpressions); + // Using a delegate here causes us to optimize the number of aggregators + // by sharing the CountAggregator across functions. On the server side, + // this will always be null, since if it had not been null on the client, + // the function would not have been transfered over (the delegate would + // have instead). + this.delegate = delegate; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (delegate == null) { + return super.evaluate(tuple, ptr); + } + delegate.evaluate(tuple, ptr); + if (PDataType.LONG.compareTo(ptr,ZERO) == 0) { + return false; + } + return true; + } + + + @Override + protected SingleAggregateFunction getDelegate() { + return delegate != null ? delegate : super.getDelegate(); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/DistinctCountAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/DistinctCountAggregateFunction.java new file mode 100644 index 00000000..b6ea58e0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/DistinctCountAggregateFunction.java @@ -0,0 +1,124 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.DistinctCountClientAggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.SchemaUtil; + + +/** + * + * Built-in function for COUNT(distinct ) aggregate function, + * + * @author anoopsjohn + * @since 1.2.1 + */ +@BuiltInFunction(name=DistinctCountAggregateFunction.NAME, args= {@Argument()} ) +public class DistinctCountAggregateFunction extends DelegateConstantToCountAggregateFunction { + public static final String NAME = "DISTINCT_COUNT"; + public static final String NORMALIZED_NAME = SchemaUtil.normalizeIdentifier(NAME); + public final static byte[] ZERO = PDataType.LONG.toBytes(0L); + public final static byte[] ONE = PDataType.LONG.toBytes(1L); + + public DistinctCountAggregateFunction() { + } + + public DistinctCountAggregateFunction(List childExpressions) { + this(childExpressions, null); + } + + public DistinctCountAggregateFunction(List childExpressions, + CountAggregateFunction delegate) { + super(childExpressions, delegate); + assert childExpressions.size() == 1; + } + + @Override + public int hashCode() { + return isConstantExpression() ? 0 : super.hashCode(); + } + + /** + * The COUNT function never returns null + */ + @Override + public boolean isNullable() { + return false; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + DistinctCountAggregateFunction other = (DistinctCountAggregateFunction)obj; + return (isConstantExpression() && other.isConstantExpression()) || children.equals(other.getChildren()); + } + + @Override + public PDataType getDataType() { + return PDataType.LONG; + } + + @Override + public DistinctCountClientAggregator newClientAggregator() { + return new DistinctCountClientAggregator(getAggregatorExpression().getColumnModifier()); + } + + @Override + public Aggregator newServerAggregator(Configuration conf) { + return new DistinctValueWithCountServerAggregator(conf); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + // TODO: optimize query plan of this to run scan serially for a limit of one row + if (!super.evaluate(tuple, ptr)) { + ptr.set(ZERO); // If evaluate returns false, then no rows were found, so result is 0 + } else if (isConstantExpression()) { + ptr.set(ONE); // Otherwise, we found one or more rows, so a distinct on a constant is 1 + } + return true; // Always evaluates to a LONG value + } + + @Override + public String getName() { + return NAME; + } + + @Override + public Aggregator newServerAggregator(Configuration config, ImmutableBytesWritable ptr) { + DistinctCountClientAggregator clientAgg = newClientAggregator(); + clientAgg.aggregate(null, ptr); + return new DistinctValueWithCountServerAggregator(config, clientAgg); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/DistinctValueWithCountAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/DistinctValueWithCountAggregateFunction.java new file mode 100644 index 00000000..b12c96af --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/DistinctValueWithCountAggregateFunction.java @@ -0,0 +1,50 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; + +public abstract class DistinctValueWithCountAggregateFunction extends SingleAggregateFunction { + + public DistinctValueWithCountAggregateFunction() { + } + + public DistinctValueWithCountAggregateFunction(List children) { + super(children); + } + + @Override + abstract public DistinctValueWithCountClientAggregator newClientAggregator(); + + @Override + public Aggregator newServerAggregator(Configuration config, ImmutableBytesWritable ptr) { + DistinctValueWithCountClientAggregator clientAgg = newClientAggregator(); + clientAgg.aggregate(null, ptr); + return new DistinctValueWithCountServerAggregator(config, clientAgg); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FloorDateExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FloorDateExpression.java new file mode 100644 index 00000000..bfc39693 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FloorDateExpression.java @@ -0,0 +1,88 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.List; + +import com.google.common.collect.Lists; +import org.apache.phoenix.expression.CoerceExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.PDataType; + +/** + * + * Class encapsulating the FLOOR operation on + * a column/literal of type {@link org.apache.phoenix.schema.PDataType#DATE}. + * + * @author samarth.jain + * @since 3.0.0 + */ +public class FloorDateExpression extends RoundDateExpression { + + public FloorDateExpression() {} + + private FloorDateExpression(List children) { + super(children); + } + + public static Expression create(List children) throws SQLException { + Expression firstChild = children.get(0); + PDataType firstChildDataType = firstChild.getDataType(); + if (firstChildDataType == PDataType.TIMESTAMP || firstChildDataType == PDataType.UNSIGNED_TIMESTAMP){ + // Coerce TIMESTAMP to DATE, as the nanos has no affect + List newChildren = Lists.newArrayListWithExpectedSize(children.size()); + newChildren.add(CoerceExpression.create(firstChild, firstChildDataType == PDataType.TIMESTAMP ? PDataType.DATE : PDataType.UNSIGNED_DATE)); + newChildren.addAll(children.subList(1, children.size())); + children = newChildren; + } + return new FloorDateExpression(children); + } + + /** + * @param timeUnit - unit of time to round up to. + * Creates a {@link FloorDateExpression} with default multiplier of 1. + */ + public static Expression create(Expression expr, TimeUnit timeUnit) throws SQLException { + return create(expr, timeUnit, 1); + } + + /** + * @param timeUnit - unit of time to round up to + * @param multiplier - determines the roll up window size. + * Create a {@link FloorDateExpression}. + */ + public static Expression create(Expression expr, TimeUnit timeUnit, int multiplier) throws SQLException { + Expression timeUnitExpr = getTimeUnitExpr(timeUnit); + Expression defaultMultiplierExpr = getMultiplierExpr(multiplier); + List expressions = Lists.newArrayList(expr, timeUnitExpr, defaultMultiplierExpr); + return create(expressions); + } + + @Override + protected long getRoundUpAmount() { + return 0; + } + + @Override + public String getName() { + return FloorFunction.NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FloorDecimalExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FloorDecimalExpression.java new file mode 100644 index 00000000..e8353f21 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FloorDecimalExpression.java @@ -0,0 +1,77 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.math.RoundingMode; +import java.sql.SQLException; +import java.util.List; + +import com.google.common.collect.Lists; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.schema.PDataType; + +/** + * + * Class encapsulating the FLOOR operation on + * a column/literal of type {@link org.apache.phoenix.schema.PDataType#DECIMAL}. + * + * @author samarth.jain + * @since 3.0.0 + */ +public class FloorDecimalExpression extends RoundDecimalExpression { + + public FloorDecimalExpression() {} + + public FloorDecimalExpression(List children) { + super(children); + } + + /** + * Creates a {@link FloorDecimalExpression} with rounding scale given by @param scale. + * + */ + public static Expression create(Expression expr, int scale) throws SQLException { + if (expr.getDataType().isCoercibleTo(PDataType.LONG)) { + return expr; + } + Expression scaleExpr = LiteralExpression.newConstant(scale, PDataType.INTEGER, true); + List expressions = Lists.newArrayList(expr, scaleExpr); + return new FloorDecimalExpression(expressions); + } + + /** + * Creates a {@link FloorDecimalExpression} with a default scale of 0 used for rounding. + * + */ + public static Expression create(Expression expr) throws SQLException { + return create(expr, 0); + } + + @Override + protected RoundingMode getRoundingMode() { + return RoundingMode.FLOOR; + } + + @Override + public String getName() { + return FloorFunction.NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FloorFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FloorFunction.java new file mode 100644 index 00000000..cceb9660 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FloorFunction.java @@ -0,0 +1,56 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FloorParseNode; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +/** + * + * Base class for built-in FLOOR function. + * + * @author samarth.jain + * @since 3.0.0 + */ +@BuiltInFunction(name = FloorFunction.NAME, + nodeClass = FloorParseNode.class, + args = { + @Argument(allowedTypes={PDataType.TIMESTAMP, PDataType.DECIMAL}), + @Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValue = "null", isConstant=true), + @Argument(allowedTypes={PDataType.INTEGER}, defaultValue="1", isConstant=true) + } + ) +public abstract class FloorFunction extends ScalarFunction { + + public static final String NAME = "FLOOR"; + + public FloorFunction(List children) { + super(children); + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FunctionArgumentType.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FunctionArgumentType.java similarity index 89% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FunctionArgumentType.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/function/FunctionArgumentType.java index 160facc8..0284caf9 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/FunctionArgumentType.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FunctionArgumentType.java @@ -1,9 +1,9 @@ -package com.salesforce.phoenix.expression.function; +package org.apache.phoenix.expression.function; import java.text.DecimalFormat; import java.text.Format; -import com.salesforce.phoenix.util.DateUtil; +import org.apache.phoenix.util.DateUtil; public enum FunctionArgumentType { TEMPORAL { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FunctionExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FunctionExpression.java new file mode 100644 index 00000000..db305d3c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/FunctionExpression.java @@ -0,0 +1,71 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.phoenix.expression.BaseCompoundExpression; +import org.apache.phoenix.expression.Expression; + +/** + * + * Compiled representation of a built-in function + * + * @author jtaylor + * @since 0.1 + */ +public abstract class FunctionExpression extends BaseCompoundExpression { + public enum OrderPreserving {NO, YES_IF_LAST, YES}; + public FunctionExpression() { + } + + public FunctionExpression(List children) { + super(children); + } + + /** + * Determines whether or not the result of the function invocation + * will be ordered in the same way as the input to the function. + * Returning YES enables an optimization to occur when a + * GROUP BY contains function invocations using the leading PK + * column(s). + * @return YES if the function invocation will always preserve order for + * the inputs versus the outputs and false otherwise, YES_IF_LAST if the + * function preserves order, but any further column reference would not + * continue to preserve order, and NO if the function does not preserve + * order. + */ + public OrderPreserving preservesOrder() { + return OrderPreserving.NO; + } + + abstract public String getName(); + + @Override + public final String toString() { + StringBuilder buf = new StringBuilder(getName() + "("); + for (int i = 0; i < children.size() - 1; i++) { + buf.append(children.get(i) + ", "); + } + buf.append(children.get(children.size()-1) + ")"); + return buf.toString(); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/IndexStateNameFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/IndexStateNameFunction.java new file mode 100644 index 00000000..41890bc7 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/IndexStateNameFunction.java @@ -0,0 +1,81 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PIndexState; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Function used to get the index state name from the serialized byte value + * Usage: + * IndexStateName('a') + * will return 'ACTIVE' + * + * @author jtaylor + * @since 2.1 + */ +@BuiltInFunction(name=IndexStateNameFunction.NAME, args= { + @Argument(allowedTypes=PDataType.CHAR)} ) +public class IndexStateNameFunction extends ScalarFunction { + public static final String NAME = "IndexStateName"; + + public IndexStateNameFunction() { + } + + public IndexStateNameFunction(List children) throws SQLException { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + Expression child = children.get(0); + if (!child.evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + return true; + } + byte serializedByte = ptr.get()[ptr.getOffset()]; + PIndexState indexState = PIndexState.fromSerializedValue(serializedByte); + ptr.set(indexState.toBytes()); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.VARCHAR; + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/InvertFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/InvertFunction.java new file mode 100644 index 00000000..72ac31aa --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/InvertFunction.java @@ -0,0 +1,129 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.compile.KeyPart; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +@BuiltInFunction(name = InvertFunction.NAME, args = { @Argument() }) +public class InvertFunction extends ScalarFunction { + public static final String NAME = "INVERT"; + + public InvertFunction() throws SQLException {} + + public InvertFunction(List children) throws SQLException { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (!getChildExpression().evaluate(tuple, ptr)) { return false; } + if (ptr.getLength() == 0) { return true; } + byte[] buf = new byte[ptr.getLength()]; + ColumnModifier.SORT_DESC.apply(ptr.get(), ptr.getOffset(), buf, 0, ptr.getLength()); + ptr.set(buf); + return true; + } + + @Override + public ColumnModifier getColumnModifier() { + return getChildExpression().getColumnModifier() == null ? ColumnModifier.SORT_DESC : null; + } + + @Override + public PDataType getDataType() { + return getChildExpression().getDataType(); + } + + @Override + public Integer getMaxLength() { + return getChildExpression().getMaxLength(); + } + + @Override + public Integer getByteSize() { + return getChildExpression().getByteSize(); + } + + @Override + public boolean isNullable() { + return getChildExpression().isNullable(); + } + + @Override + public String getName() { + return NAME; + } + + /** + * INVERT may be optimized through + */ + @Override + public int getKeyFormationTraversalIndex() { + return 0; + } + + /** + * Invert the childPart key range + */ + @Override + public KeyPart newKeyPart(final KeyPart childPart) { + return new KeyPart() { + + @Override + public KeyRange getKeyRange(CompareOp op, Expression rhs) { + KeyRange range = childPart.getKeyRange(op, rhs); + return range.invert(); + } + + @Override + public List getExtractNodes() { + return childPart.getExtractNodes(); + } + + @Override + public PColumn getColumn() { + return childPart.getColumn(); + } + }; + } + + @Override + public OrderPreserving preservesOrder() { + return OrderPreserving.YES; + } + + private Expression getChildExpression() { + return children.get(0); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/LTrimFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/LTrimFunction.java new file mode 100644 index 00000000..1622f55d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/LTrimFunction.java @@ -0,0 +1,115 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.StringUtil; + + +/** + * + * Implementation of the LTrim() build-in function. It removes from the left end of + * space character and other function bytes in single byte utf8 characters + * set. + * + * @author zhuang + * @since 0.1 + */ +@BuiltInFunction(name=LTrimFunction.NAME, args={ + @Argument(allowedTypes={PDataType.VARCHAR})}) +public class LTrimFunction extends ScalarFunction { + public static final String NAME = "LTRIM"; + + private Integer byteSize; + + public LTrimFunction() { } + + public LTrimFunction(List children) throws SQLException { + super(children); + if (getStringExpression().getDataType().isFixedWidth()) { + byteSize = getStringExpression().getByteSize(); + } + } + + private Expression getStringExpression() { + return children.get(0); + } + + @Override + public ColumnModifier getColumnModifier() { + return children.get(0).getColumnModifier(); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + // Starting from the front of the byte, look for all single bytes at the end of the string + // that is below SPACE_UTF8 (space and control characters) or 0x7f (control chars). + if (!getStringExpression().evaluate(tuple, ptr)) { + return false; + } + + if (ptr.getLength() == 0) { + ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); + return true; + } + byte[] string = ptr.get(); + int offset = ptr.getOffset(); + int length = ptr.getLength(); + + ColumnModifier columnModifier = getStringExpression().getColumnModifier(); + // TODO: when we have ColumnModifier.REVERSE, we'll need to trim from the end instead of + // the beginning (just delegate to RTrimFunction or replace from ExpressionCompiler instead?) + int i = StringUtil.getFirstNonBlankCharIdxFromStart(string, offset, length, columnModifier); + if (i == offset + length) { + ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); + return true; + } + + ptr.set(string, i, offset + length - i); + return true; + } + + @Override + public Integer getByteSize() { + return byteSize; + } + + @Override + public PDataType getDataType() { + return PDataType.VARCHAR; + } + + @Override + public String getName() { + return NAME; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/LengthFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/LengthFunction.java new file mode 100644 index 00000000..770f77a6 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/LengthFunction.java @@ -0,0 +1,96 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.io.UnsupportedEncodingException; +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.StringUtil; + + +/** + * + * Implementation of the LENGTH() build-in function. is the string + * of characters we want to find the length of. If is NULL or empty, null + * is returned. + * + * @author zhuang + * @since 0.1 + */ +@BuiltInFunction(name=LengthFunction.NAME, args={ + @Argument(allowedTypes={PDataType.VARCHAR})} ) +public class LengthFunction extends ScalarFunction { + public static final String NAME = "LENGTH"; + + public LengthFunction() { } + + public LengthFunction(List children) throws SQLException { + super(children); + } + + private Expression getStringExpression() { + return children.get(0); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + Expression child = getStringExpression(); + if (!child.evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); + return true; + } + int len; + if (child.getDataType() == PDataType.CHAR) { + // Only single-byte characters allowed in CHAR + len = ptr.getLength(); + } else { + try { + len = StringUtil.calculateUTF8Length(ptr.get(), ptr.getOffset(), ptr.getLength(), child.getColumnModifier()); + } catch (UnsupportedEncodingException e) { + return false; + } + } + ptr.set(PDataType.INTEGER.toBytes(len)); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.INTEGER; + } + + @Override + public String getName() { + return NAME; + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/LowerFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/LowerFunction.java similarity index 83% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/LowerFunction.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/function/LowerFunction.java index 73fd8737..b8e731f1 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/LowerFunction.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/LowerFunction.java @@ -1,14 +1,14 @@ -package com.salesforce.phoenix.expression.function; +package org.apache.phoenix.expression.function; import java.sql.SQLException; import java.util.List; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; @FunctionParseNode.BuiltInFunction(name=LowerFunction.NAME, args={ @FunctionParseNode.Argument(allowedTypes={PDataType.VARCHAR})} ) diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/MD5Function.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/MD5Function.java new file mode 100644 index 00000000..32ef6427 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/MD5Function.java @@ -0,0 +1,100 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +@BuiltInFunction(name = MD5Function.NAME, args={@Argument()}) +public class MD5Function extends ScalarFunction { + public static final String NAME = "MD5"; + public static final Integer LENGTH = 16; + + private final MessageDigest messageDigest; + + public MD5Function() throws SQLException { + try { + messageDigest = MessageDigest.getInstance("MD5"); + } catch (NoSuchAlgorithmException e) { + throw new SQLException(e); + } + } + + public MD5Function(List children) throws SQLException { + super(children); + try { + messageDigest = MessageDigest.getInstance("MD5"); + } catch (NoSuchAlgorithmException e) { + throw new RuntimeException(e); + } + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (!getChildExpression().evaluate(tuple, ptr)) { + return false; + } + + // Update the digest value + messageDigest.update(ptr.get(), ptr.getOffset(), ptr.getLength()); + // Get the digest bytes (note this resets the messageDigest as well) + ptr.set(messageDigest.digest()); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.BINARY; + } + + @Override + public Integer getMaxLength() { + return LENGTH; + } + + @Override + public Integer getByteSize() { + return LENGTH; + } + + @Override + public boolean isNullable() { + return getChildExpression().isNullable(); + } + + @Override + public String getName() { + return NAME; + } + + private Expression getChildExpression() { + return children.get(0); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/MaxAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/MaxAggregateFunction.java new file mode 100644 index 00000000..72e59841 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/MaxAggregateFunction.java @@ -0,0 +1,75 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.hadoop.conf.Configuration; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.MaxAggregator; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.parse.MaxAggregateParseNode; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; + + + +/** + * Built-in function for finding MAX. + * + * @author syyang + * @since 0.1 + */ +@BuiltInFunction(name=MaxAggregateFunction.NAME, nodeClass=MaxAggregateParseNode.class, args= {@Argument()} ) +public class MaxAggregateFunction extends MinAggregateFunction { + public static final String NAME = "MAX"; + + public MaxAggregateFunction() { + } + + public MaxAggregateFunction(List childExpressions, CountAggregateFunction delegate) { + super(childExpressions, delegate); + } + + @Override + public Aggregator newServerAggregator(Configuration conf) { + final PDataType type = getAggregatorExpression().getDataType(); + ColumnModifier columnModifier = getAggregatorExpression().getColumnModifier(); + return new MaxAggregator(columnModifier) { + @Override + public PDataType getDataType() { + return type; + } + }; + } + + @Override + public String getName() { + return NAME; + } + + @Override + public ColumnModifier getColumnModifier() { + return getAggregatorExpression().getColumnModifier(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/MinAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/MinAggregateFunction.java new file mode 100644 index 00000000..26e8fd9f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/MinAggregateFunction.java @@ -0,0 +1,89 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.MinAggregator; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.parse.MinAggregateParseNode; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + + +/** + * Built-in function for finding MIN. + * + * @author syyang + * @since 0.1 + */ +@BuiltInFunction(name=MinAggregateFunction.NAME, nodeClass=MinAggregateParseNode.class, args= {@Argument()} ) +public class MinAggregateFunction extends DelegateConstantToCountAggregateFunction { + public static final String NAME = "MIN"; + + public MinAggregateFunction() { + } + + public MinAggregateFunction(List childExpressions, CountAggregateFunction delegate) { + super(childExpressions, delegate); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + boolean wasEvaluated = super.evaluate(tuple, ptr); + if (!wasEvaluated) { + return false; + } + if (isConstantExpression()) { + getAggregatorExpression().evaluate(tuple, ptr); + } + return true; + } + + @Override + public Aggregator newServerAggregator(Configuration conf) { + final PDataType type = getAggregatorExpression().getDataType(); + ColumnModifier columnModifier = getAggregatorExpression().getColumnModifier(); + return new MinAggregator(columnModifier) { + @Override + public PDataType getDataType() { + return type; + } + }; + } + + @Override + public ColumnModifier getColumnModifier() { + return getAggregatorExpression().getColumnModifier(); + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/PercentRankAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/PercentRankAggregateFunction.java new file mode 100644 index 00000000..bf6c9753 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/PercentRankAggregateFunction.java @@ -0,0 +1,74 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.hadoop.conf.Configuration; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; +import org.apache.phoenix.expression.aggregator.PercentRankClientAggregator; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; + +/** + * + * PERCENT_RANK([,]) WITHIN GROUP (ORDER BY [,] ASC/DESC) aggregate function + * + * @author anoopsjohn + * @since 1.2.1 + */ +@BuiltInFunction(name = PercentRankAggregateFunction.NAME, args = { @Argument(), + @Argument(allowedTypes = { PDataType.BOOLEAN }, isConstant = true), @Argument(isConstant = true) }) +public class PercentRankAggregateFunction extends DistinctValueWithCountAggregateFunction { + public static final String NAME = "PERCENT_RANK"; + + public PercentRankAggregateFunction() { + + } + + public PercentRankAggregateFunction(List childern) { + super(childern); + } + + @Override + public Aggregator newServerAggregator(Configuration conf) { + return new DistinctValueWithCountServerAggregator(conf); + } + + @Override + public DistinctValueWithCountClientAggregator newClientAggregator() { + return new PercentRankClientAggregator(children, getAggregatorExpression().getColumnModifier()); + } + + @Override + public String getName() { + return NAME; + } + + @Override + public PDataType getDataType() { + return PDataType.DECIMAL; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/PercentileContAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/PercentileContAggregateFunction.java new file mode 100644 index 00000000..703d619c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/PercentileContAggregateFunction.java @@ -0,0 +1,75 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.hadoop.conf.Configuration; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; +import org.apache.phoenix.expression.aggregator.PercentileClientAggregator; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; + +/** + * + * Built-in function for PERCENTILE_CONT() WITHIN GROUP (ORDER BY ASC/DESC) aggregate function + * + * @author anoopsjohn + * @since 1.2.1 + */ +@BuiltInFunction(name = PercentileContAggregateFunction.NAME, args = { @Argument(allowedTypes = { PDataType.DECIMAL }), + @Argument(allowedTypes = { PDataType.BOOLEAN }, isConstant = true), + @Argument(allowedTypes = { PDataType.DECIMAL }, isConstant = true, minValue = "0", maxValue = "1") }) +public class PercentileContAggregateFunction extends DistinctValueWithCountAggregateFunction { + public static final String NAME = "PERCENTILE_CONT"; + + public PercentileContAggregateFunction() { + + } + + public PercentileContAggregateFunction(List childern) { + super(childern); + } + + @Override + public Aggregator newServerAggregator(Configuration conf) { + return new DistinctValueWithCountServerAggregator(conf); + } + + @Override + public DistinctValueWithCountClientAggregator newClientAggregator() { + return new PercentileClientAggregator(children, getAggregatorExpression().getColumnModifier()); + } + + @Override + public String getName() { + return NAME; + } + + @Override + public PDataType getDataType() { + return PDataType.DECIMAL; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/PercentileDiscAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/PercentileDiscAggregateFunction.java new file mode 100644 index 00000000..7909c41e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/PercentileDiscAggregateFunction.java @@ -0,0 +1,72 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.hadoop.conf.Configuration; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; +import org.apache.phoenix.expression.aggregator.PercentileDiscClientAggregator; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; + + +/** + * + * Built-in function for PERCENTILE_DISC() WITHIN GROUP (ORDER BY ASC/DESC) aggregate function + * + * @author ramkrishna + * @since 1.2.1 + */ +@BuiltInFunction(name = PercentileDiscAggregateFunction.NAME, args = { @Argument(allowedTypes = { PDataType.DECIMAL }), + @Argument(allowedTypes = { PDataType.BOOLEAN }, isConstant = true), + @Argument(allowedTypes = { PDataType.DECIMAL }, isConstant = true, minValue = "0", maxValue = "1") }) +public class PercentileDiscAggregateFunction extends DistinctValueWithCountAggregateFunction { + + public static final String NAME = "PERCENTILE_DISC"; + + public PercentileDiscAggregateFunction() { + } + + public PercentileDiscAggregateFunction(List childern) { + super(childern); + } + + @Override + public Aggregator newServerAggregator(Configuration conf) { + return new DistinctValueWithCountServerAggregator(conf); + } + + @Override + public DistinctValueWithCountClientAggregator newClientAggregator() { + return new PercentileDiscClientAggregator(children, getAggregatorExpression().getColumnModifier()); + } + + @Override + public String getName() { + return NAME; + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PrefixFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/PrefixFunction.java similarity index 88% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PrefixFunction.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/function/PrefixFunction.java index b3ef87a5..2612d741 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/PrefixFunction.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/PrefixFunction.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.expression.function; +package org.apache.phoenix.expression.function; import java.util.Collections; import java.util.List; @@ -6,12 +6,12 @@ import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import com.salesforce.phoenix.compile.KeyPart; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.util.ByteUtil; +import org.apache.phoenix.compile.KeyPart; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.util.ByteUtil; abstract public class PrefixFunction extends ScalarFunction { public PrefixFunction() { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RTrimFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RTrimFunction.java new file mode 100644 index 00000000..d7702a57 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RTrimFunction.java @@ -0,0 +1,161 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.compile.KeyPart; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.StringUtil; + + +/** + * + * Implementation of the RTrim() build-in function. It removes from the right end of + * space character and other function bytes in single byte utf8 characters set + * + * @author zhuang + * @since 0.1 + */ +@BuiltInFunction(name=RTrimFunction.NAME, args={ + @Argument(allowedTypes={PDataType.VARCHAR})}) +public class RTrimFunction extends ScalarFunction { + public static final String NAME = "RTRIM"; + + private Integer byteSize; + + public RTrimFunction() { } + + public RTrimFunction(List children) throws SQLException { + super(children); + if (getStringExpression().getDataType().isFixedWidth()) { + byteSize = getStringExpression().getByteSize(); + } + } + + private Expression getStringExpression() { + return children.get(0); + } + + @Override + public ColumnModifier getColumnModifier() { + return children.get(0).getColumnModifier(); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + // Starting from the end of the byte, look for all single bytes at the end of the string + // that is below SPACE_UTF8 (space and control characters) or above (control chars). + if (!getStringExpression().evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); + return true; + } + byte[] string = ptr.get(); + int offset = ptr.getOffset(); + int length = ptr.getLength(); + + ColumnModifier columnModifier = getStringExpression().getColumnModifier(); + int i = StringUtil.getFirstNonBlankCharIdxFromEnd(string, offset, length, columnModifier); + if (i == offset - 1) { + ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); + return true; + } + ptr.set(string, offset, i - offset + 1); + return true; + } + + @Override + public OrderPreserving preservesOrder() { + return OrderPreserving.YES_IF_LAST; + } + + @Override + public int getKeyFormationTraversalIndex() { + return 0; + } + + @Override + public KeyPart newKeyPart(final KeyPart childPart) { + return new KeyPart() { + @Override + public KeyRange getKeyRange(CompareOp op, Expression rhs) { + ImmutableBytesWritable ptr = new ImmutableBytesWritable(); + rhs.evaluate(null, ptr); + byte[] key = ByteUtil.copyKeyBytesIfNecessary(ptr); + PDataType type = getColumn().getDataType(); + KeyRange range; + switch (op) { + case EQUAL: + range = type.getKeyRange(key, true, ByteUtil.nextKey(ByteUtil.concat(key, new byte[] {StringUtil.SPACE_UTF8})), false); + break; + case LESS_OR_EQUAL: + range = type.getKeyRange(KeyRange.UNBOUND, false, ByteUtil.nextKey(ByteUtil.concat(key, new byte[] {StringUtil.SPACE_UTF8})), false); + break; + default: + range = childPart.getKeyRange(op, rhs); + break; + } + Integer length = getColumn().getByteSize(); + return length == null ? range : range.fill(length); + } + + @Override + public List getExtractNodes() { + return Collections.emptyList(); + } + + @Override + public PColumn getColumn() { + return childPart.getColumn(); + } + }; + } + + @Override + public Integer getByteSize() { + return byteSize; + } + + @Override + public PDataType getDataType() { + return PDataType.VARCHAR; + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RegexpReplaceFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RegexpReplaceFunction.java new file mode 100644 index 00000000..1f0e28ef --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RegexpReplaceFunction.java @@ -0,0 +1,130 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.io.DataInput; +import java.io.IOException; +import java.util.List; +import java.util.regex.Pattern; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Function similar to the regexp_replace function in Postgres, which is used to pattern + * match a segment of the string. Usage: + * REGEXP_REPLACE(,,) + * source_char is the string in which we want to perform string replacement. pattern is a + * Java compatible regular expression string, and we replace all the matching part with + * replace_string. The first 2 arguments are required and are {@link org.apache.phoenix.schema.PDataType#VARCHAR}, + * the replace_string is default to empty string. + * + * The function returns a {@link org.apache.phoenix.schema.PDataType#VARCHAR} + * + * @author zhuang + * @since 0.1 + */ +@BuiltInFunction(name=RegexpReplaceFunction.NAME, args= { + @Argument(allowedTypes={PDataType.VARCHAR}), + @Argument(allowedTypes={PDataType.VARCHAR}), + @Argument(allowedTypes={PDataType.VARCHAR},defaultValue="null")} ) +public class RegexpReplaceFunction extends ScalarFunction { + public static final String NAME = "REGEXP_REPLACE"; + + private boolean hasReplaceStr; + private Pattern pattern; + + public RegexpReplaceFunction() { } + + // Expect 1 arguments, the pattern. + public RegexpReplaceFunction(List children) { + super(children); + init(); + } + + private void init() { + hasReplaceStr = ((LiteralExpression)getReplaceStrExpression()).getValue() != null; + Object patternString = ((LiteralExpression)children.get(1)).getValue(); + if (patternString != null) { + pattern = Pattern.compile((String)patternString); + } + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + // Can't parse if there is no replacement pattern. + if (pattern == null) { + return false; + } + Expression sourceStrExpression = getSourceStrExpression(); + if (!sourceStrExpression.evaluate(tuple, ptr)) { + return false; + } + String sourceStr = (String)PDataType.VARCHAR.toObject(ptr, sourceStrExpression.getColumnModifier()); + if (sourceStr == null) { + return false; + } + String replaceStr; + if (hasReplaceStr) { + Expression replaceStrExpression = this.getReplaceStrExpression(); + if (!replaceStrExpression.evaluate(tuple, ptr)) { + return false; + } + replaceStr = (String)PDataType.VARCHAR.toObject(ptr, replaceStrExpression.getColumnModifier()); + } else { + replaceStr = ""; + } + String replacedStr = pattern.matcher(sourceStr).replaceAll(replaceStr); + ptr.set(PDataType.VARCHAR.toBytes(replacedStr)); + return true; + } + + private Expression getSourceStrExpression() { + return children.get(0); + } + + private Expression getReplaceStrExpression() { + return children.get(2); + } + + @Override + public PDataType getDataType() { + return PDataType.VARCHAR; + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + init(); + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RegexpSubstrFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RegexpSubstrFunction.java new file mode 100644 index 00000000..2308581f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RegexpSubstrFunction.java @@ -0,0 +1,175 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; + + +/** + * + * Implementation of REGEXP_SUBSTR(, , ) built-in function, + * where is the offset from the start of . Positive offset is treated as 1-based, + * a zero offset is treated as 0-based, and a negative offset starts from the end of the string + * working backwards. The is the pattern we would like to search for in the string. + * The function returns the first occurrence of any substring in the string that matches + * the input as a VARCHAR. + * + * @author zhuang + * @since 0.1 + */ +@BuiltInFunction(name=RegexpSubstrFunction.NAME, args={ + @Argument(allowedTypes={PDataType.VARCHAR}), + @Argument(allowedTypes={PDataType.VARCHAR}), + @Argument(allowedTypes={PDataType.LONG}, defaultValue="1")} ) +public class RegexpSubstrFunction extends PrefixFunction { + public static final String NAME = "REGEXP_SUBSTR"; + + private Pattern pattern; + private boolean isOffsetConstant; + private Integer byteSize; + + public RegexpSubstrFunction() { } + + public RegexpSubstrFunction(List children) { + super(children); + init(); + } + + private void init() { + Object patternString = ((LiteralExpression)children.get(1)).getValue(); + if (patternString != null) { + pattern = Pattern.compile((String)patternString); + } + // If the source string has a fixed width, then the max length would be the length + // of the source string minus the offset, or the absolute value of the offset if + // it's negative. Offset number is a required argument. However, if the source string + // is not fixed width, the maxLength would be null. + isOffsetConstant = getOffsetExpression() instanceof LiteralExpression; + Number offsetNumber = (Number)((LiteralExpression)getOffsetExpression()).getValue(); + if (offsetNumber != null) { + int offset = offsetNumber.intValue(); + if (getSourceStrExpression().getDataType().isFixedWidth()) { + if (offset >= 0) { + byteSize = getSourceStrExpression().getByteSize() - offset - (offset == 0 ? 0 : 1); + } else { + byteSize = -offset; + } + } + } + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (pattern == null) { + return false; + } + if (!getSourceStrExpression().evaluate(tuple, ptr)) { + return false; + } + String sourceStr = (String)PDataType.VARCHAR.toObject(ptr, getSourceStrExpression().getColumnModifier()); + if (sourceStr == null) { + return false; + } + + Expression offsetExpression = getOffsetExpression(); + if (!offsetExpression.evaluate(tuple, ptr)) { + return false; + } + int offset = offsetExpression.getDataType().getCodec().decodeInt(ptr, offsetExpression.getColumnModifier()); + + int strlen = sourceStr.length(); + // Account for 1 versus 0-based offset + offset = offset - (offset <= 0 ? 0 : 1); + if (offset < 0) { // Offset < 0 means get from end + offset = strlen + offset; + } + if (offset < 0 || offset >= strlen) { + return false; + } + + Matcher matcher = pattern.matcher(sourceStr); + boolean hasSubString = matcher.find(offset); + if (!hasSubString) { + ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); + return true; + } + String subString = matcher.group(); + ptr.set(PDataType.VARCHAR.toBytes(subString)); + return true; + } + + @Override + public Integer getByteSize() { + return byteSize; + } + + @Override + public OrderPreserving preservesOrder() { + if (isOffsetConstant) { + LiteralExpression literal = (LiteralExpression) getOffsetExpression(); + Number offsetNumber = (Number) literal.getValue(); + if (offsetNumber != null) { + int offset = offsetNumber.intValue(); + if (offset == 0 || offset == 1) { + return OrderPreserving.YES_IF_LAST; + } + } + } + return OrderPreserving.NO; + } + + @Override + public int getKeyFormationTraversalIndex() { + return preservesOrder() == OrderPreserving.NO ? NO_TRAVERSAL : 0; + } + + private Expression getOffsetExpression() { + return children.get(2); + } + + private Expression getSourceStrExpression() { + return children.get(0); + } + + @Override + public PDataType getDataType() { + // ALways VARCHAR since we do not know in advanced how long the + // matched string will be. + return PDataType.VARCHAR; + } + + @Override + public String getName() { + return NAME; + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ReverseFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ReverseFunction.java similarity index 79% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ReverseFunction.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/function/ReverseFunction.java index 29a4e434..b90fd7c5 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/ReverseFunction.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ReverseFunction.java @@ -1,17 +1,17 @@ -package com.salesforce.phoenix.expression.function; +package org.apache.phoenix.expression.function; import java.sql.SQLException; import java.util.List; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.StringUtil; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.StringUtil; @BuiltInFunction(name=ReverseFunction.NAME, args={ @Argument(allowedTypes={PDataType.VARCHAR})} ) diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundDateExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundDateExpression.java similarity index 78% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundDateExpression.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundDateExpression.java index 51cb5a3f..eab4b281 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/RoundDateExpression.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundDateExpression.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; import java.io.DataInput; import java.io.DataOutput; @@ -40,22 +32,22 @@ import org.apache.hadoop.io.WritableUtils; import com.google.common.collect.Lists; -import com.salesforce.phoenix.compile.KeyPart; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDataType.PDataCodec; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; +import org.apache.phoenix.compile.KeyPart; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDataType.PDataCodec; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; /** * Function used to bucketize date/time values by rounding them to * an even increment. Usage: * ROUND(,<'day'|'hour'|'minute'|'second'|'millisecond'>,) * The integer multiplier is optional and is used to do rollups to a partial time unit (i.e. 10 minute rollup) - * The function returns a {@link com.salesforce.phoenix.schema.PDataType#DATE} + * The function returns a {@link org.apache.phoenix.schema.PDataType#DATE} * @author jtaylor, samarth.jain * @since 0.1 diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundDecimalExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundDecimalExpression.java new file mode 100644 index 00000000..85e7b754 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundDecimalExpression.java @@ -0,0 +1,136 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; +import java.math.BigDecimal; +import java.math.RoundingMode; +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.io.WritableUtils; + +import com.google.common.collect.Lists; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.schema.IllegalDataException; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * + * Class encapsulating the process for rounding off a column/literal of + * type {@link org.apache.phoenix.schema.PDataType#DECIMAL} + * + * @author samarth.jain + * @since 3.0.0 + */ + +public class RoundDecimalExpression extends ScalarFunction { + + private int scale; + + /** + * Creates a {@link RoundDecimalExpression} with rounding scale given by @param scale. + * + */ + public static Expression create(Expression expr, int scale) throws SQLException { + if (expr.getDataType().isCoercibleTo(PDataType.LONG)) { + return expr; + } + Expression scaleExpr = LiteralExpression.newConstant(scale, PDataType.INTEGER, true); + List expressions = Lists.newArrayList(expr, scaleExpr); + return new RoundDecimalExpression(expressions); + } + + /** + * Creates a {@link RoundDecimalExpression} with a default scale of 0 used for rounding. + * + */ + public static Expression create(Expression expr) throws SQLException { + return create(expr, 0); + } + + public RoundDecimalExpression() {} + + public RoundDecimalExpression(List children) { + super(children); + LiteralExpression scaleChild = (LiteralExpression)children.get(1); + PDataType scaleType = scaleChild.getDataType(); + Object scaleValue = scaleChild.getValue(); + if(scaleValue != null) { + if (scaleType.isCoercibleTo(PDataType.INTEGER, scaleValue)) { + int scale = (Integer)PDataType.INTEGER.toObject(scaleValue, scaleType); + if (scale >=0 && scale <= PDataType.MAX_PRECISION) { + this.scale = scale; + return; + } + } + throw new IllegalDataException("Invalid second argument for scale: " + scaleValue + ". The scale must be between 0 and " + PDataType.MAX_PRECISION + " inclusive."); + } + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + Expression childExpr = children.get(0); + if(childExpr.evaluate(tuple, ptr)) { + BigDecimal value = (BigDecimal)PDataType.DECIMAL.toObject(ptr, childExpr.getColumnModifier()); + BigDecimal scaledValue = value.setScale(scale, getRoundingMode()); + ptr.set(getDataType().toBytes(scaledValue)); + return true; + } + return false; + } + + @Override + public PDataType getDataType() { + return children.get(0).getDataType(); + } + + protected RoundingMode getRoundingMode() { + return RoundingMode.HALF_UP; + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + scale = WritableUtils.readVInt(input); + } + + @Override + public void write(DataOutput output) throws IOException { + super.write(output); + WritableUtils.writeVInt(output, scale); + } + + @Override + public String getName() { + return RoundFunction.NAME; + } + + @Override + public OrderPreserving preservesOrder() { + return OrderPreserving.YES; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundFunction.java new file mode 100644 index 00000000..1b0782b0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundFunction.java @@ -0,0 +1,57 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.parse.RoundParseNode; +import org.apache.phoenix.schema.PDataType; + + +/** + * Base class for RoundFunction. + * + * @author jtaylor, samarth.jain + * @since 0.1 + */ +@BuiltInFunction(name = RoundFunction.NAME, + nodeClass = RoundParseNode.class, + args = { + @Argument(allowedTypes={PDataType.TIMESTAMP, PDataType.DECIMAL}), + @Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValue = "null", isConstant=true), + @Argument(allowedTypes={PDataType.INTEGER}, defaultValue="1", isConstant=true) + } + ) +public abstract class RoundFunction extends ScalarFunction { + + public static final String NAME = "ROUND"; + + public RoundFunction(List children) { + super(children); + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundTimestampExpression.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundTimestampExpression.java new file mode 100644 index 00000000..0146b19d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/RoundTimestampExpression.java @@ -0,0 +1,106 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.sql.Timestamp; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import com.google.common.collect.Lists; +import org.apache.phoenix.expression.CoerceExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDataType.PDataCodec; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * + * Class encapsulating the process for rounding off a column/literal of + * type {@link org.apache.phoenix.schema.PDataType#TIMESTAMP} + * This class only supports rounding off the milliseconds that is for + * {@link TimeUnit#MILLISECOND}. If you want more options of rounding like + * using {@link TimeUnit#HOUR} use {@link RoundDateExpression} + * + * @author samarth.jain + * @since 3.0.0 + */ + +public class RoundTimestampExpression extends RoundDateExpression { + + private static final long HALF_OF_NANOS_IN_MILLI = java.util.concurrent.TimeUnit.MILLISECONDS.toNanos(1)/2; + + public RoundTimestampExpression() {} + + private RoundTimestampExpression(List children) { + super(children); + } + + public static Expression create (List children) throws SQLException { + Expression firstChild = children.get(0); + PDataType firstChildDataType = firstChild.getDataType(); + String timeUnit = (String)((LiteralExpression)children.get(1)).getValue(); + LiteralExpression multiplierExpr = (LiteralExpression)children.get(2); + + /* + * When rounding off timestamp to milliseconds, nanos play a part only when the multiplier value + * is equal to 1. This is because for cases when multiplier value is greater than 1, number of nanos/multiplier + * will always be less than half the nanos in a millisecond. + */ + if((timeUnit == null || TimeUnit.MILLISECOND.toString().equalsIgnoreCase(timeUnit)) && ((Number)multiplierExpr.getValue()).intValue() == 1) { + return new RoundTimestampExpression(children); + } + // Coerce TIMESTAMP to DATE, as the nanos has no affect + List newChildren = Lists.newArrayListWithExpectedSize(children.size()); + newChildren.add(CoerceExpression.create(firstChild, firstChildDataType == PDataType.TIMESTAMP ? PDataType.DATE : PDataType.UNSIGNED_DATE)); + newChildren.addAll(children.subList(1, children.size())); + return RoundDateExpression.create(newChildren); + } + + @Override + protected PDataCodec getKeyRangeCodec(PDataType columnDataType) { + return columnDataType == PDataType.TIMESTAMP + ? PDataType.DATE.getCodec() + : columnDataType == PDataType.UNSIGNED_TIMESTAMP + ? PDataType.UNSIGNED_DATE.getCodec() + : super.getKeyRangeCodec(columnDataType); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (children.get(0).evaluate(tuple, ptr)) { + ColumnModifier columnModifier = children.get(0).getColumnModifier(); + PDataType dataType = getDataType(); + int nanos = dataType.getNanos(ptr, columnModifier); + if(nanos >= HALF_OF_NANOS_IN_MILLI) { + long timeMillis = dataType.getMillis(ptr, columnModifier); + Timestamp roundedTs = new Timestamp(timeMillis + 1); + byte[] byteValue = dataType.toBytes(roundedTs); + ptr.set(byteValue); + } + return true; // for timestamp we only support rounding up the milliseconds. + } + return false; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SQLTableTypeFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SQLTableTypeFunction.java new file mode 100644 index 00000000..04465eae --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SQLTableTypeFunction.java @@ -0,0 +1,80 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Function used to get the SQL table type name from the serialized table type. + * Usage: + * SqlTableType('v') will return 'VIEW' based on + * {@link java.sql.DatabaseMetaData#getTableTypes()} + * + * @author jtaylor + * @since 2.2 + */ +@BuiltInFunction(name=SQLTableTypeFunction.NAME, args= { + @Argument(allowedTypes=PDataType.CHAR)} ) +public class SQLTableTypeFunction extends ScalarFunction { + public static final String NAME = "SQLTableType"; + + public SQLTableTypeFunction() { + } + + public SQLTableTypeFunction(List children) throws SQLException { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + Expression child = children.get(0); + if (!child.evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + return true; + } + PTableType tableType = PTableType.fromSerializedValue(ptr.get()[ptr.getOffset()]); + ptr.set(tableType.getValue().getBytes()); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.VARCHAR; + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SQLViewTypeFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SQLViewTypeFunction.java new file mode 100644 index 00000000..e70ec8e0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SQLViewTypeFunction.java @@ -0,0 +1,80 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PTable.ViewType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Function used to get the SQL view type name from the serialized view type. + * Usage: + * SQLViewType('v') will return 'VIEW' based on + * {@link java.sql.DatabaseMetaData#getTableTypes()} + * + * @author jtaylor + * @since 2.2 + */ +@BuiltInFunction(name=SQLViewTypeFunction.NAME, args= { + @Argument(allowedTypes=PDataType.UNSIGNED_TINYINT)} ) +public class SQLViewTypeFunction extends ScalarFunction { + public static final String NAME = "SQLViewType"; + + public SQLViewTypeFunction() { + } + + public SQLViewTypeFunction(List children) throws SQLException { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + Expression child = children.get(0); + if (!child.evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + return true; + } + ViewType viewType = ViewType.fromSerializedValue(ptr.get()[ptr.getOffset()]); + ptr.set(viewType.getBytes()); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.VARCHAR; + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ScalarFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ScalarFunction.java new file mode 100644 index 00000000..1d1059bf --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ScalarFunction.java @@ -0,0 +1,72 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.phoenix.compile.KeyPart; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.visitor.ExpressionVisitor; + + +public abstract class ScalarFunction extends FunctionExpression { + public static final int NO_TRAVERSAL = -1; + + public ScalarFunction() { + } + + public ScalarFunction(List children) { + super(children); + } + + @Override + public final T accept(ExpressionVisitor visitor) { + List l = acceptChildren(visitor, visitor.visitEnter(this)); + T t = visitor.visitLeave(this, l); + if (t == null) { + t = visitor.defaultReturn(this, l); + } + return t; + } + + /** + * Determines whether or not a function may be used to form + * the start/stop key of a scan + * @return the zero-based position of the argument to traverse + * into to look for a primary key column reference, or + * {@value #NO_TRAVERSAL} if the function cannot be used to + * form the scan key. + */ + public int getKeyFormationTraversalIndex() { + return NO_TRAVERSAL; + } + + /** + * Manufactures a KeyPart used to construct the KeyRange given + * a constant and a comparison operator. + * @param childPart the KeyPart formulated for the child expression + * at the {@link #getKeyFormationTraversalIndex()} position. + * @return the KeyPart for constructing the KeyRange for this + * function. + */ + public KeyPart newKeyPart(KeyPart childPart) { + return null; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SingleAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SingleAggregateFunction.java new file mode 100644 index 00000000..f02c4e7d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SingleAggregateFunction.java @@ -0,0 +1,173 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.io.DataInput; +import java.io.IOException; +import java.util.Arrays; +import java.util.Comparator; +import java.util.List; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.visitor.ExpressionVisitor; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Base class for aggregate functions that calculate an aggregation + * using a single {{@link Aggregator} + * + * @author jtaylor + * @since 0.1 + */ +abstract public class SingleAggregateFunction extends AggregateFunction { + private static final List DEFAULT_EXPRESSION_LIST = Arrays.asList(LiteralExpression.newConstant(1, true)); + protected boolean isConstant; + private Aggregator aggregator; + + /** + * Sort aggregate functions with nullable fields last. This allows us not to have to store trailing null values. + * Within non-nullable/nullable groups, put fixed width values first since we can access those more efficiently + * (i.e. we can skip over groups of them in-mass instead of reading the length of each one to skip over as + * required by a variable length value). + */ + public static final Comparator SCHEMA_COMPARATOR = new Comparator() { + + @Override + public int compare(SingleAggregateFunction o1, SingleAggregateFunction o2) { + boolean isNullable1 = o1.isNullable(); + boolean isNullable2 = o2.isNullable(); + if (isNullable1 != isNullable2) { + return isNullable1 ? 1 : -1; + } + isNullable1 = o1.getAggregatorExpression().isNullable(); + isNullable2 = o2.getAggregatorExpression().isNullable(); + if (isNullable1 != isNullable2) { + return isNullable1 ? 1 : -1; + } + // Ensures COUNT(1) sorts first TODO: unit test for this + boolean isConstant1 = o1.isConstantExpression(); + boolean isConstant2 = o2.isConstantExpression(); + if (isConstant1 != isConstant2) { + return isConstant1 ? 1 : -1; + } + PDataType r1 = o1.getAggregator().getDataType(); + PDataType r2 = o2.getAggregator().getDataType(); + if (r1.isFixedWidth() != r2.isFixedWidth()) { + return r1.isFixedWidth() ? -1 : 1; + } + return r1.compareTo(r2); + } + }; + + protected SingleAggregateFunction() { + this(DEFAULT_EXPRESSION_LIST, true); + } + + public SingleAggregateFunction(List children) { + this(children, children.get(0) instanceof LiteralExpression); + } + + private SingleAggregateFunction(List children, boolean isConstant) { + super(children); + this.isConstant = children.get(0) instanceof LiteralExpression; + this.aggregator = newClientAggregator(); + } + + public boolean isConstantExpression() { + return isConstant; + } + + @Override + public PDataType getDataType() { + return children.get(0).getDataType(); + } + + public Expression getAggregatorExpression() { + return children.get(0); + } + + public Aggregator getAggregator() { + return aggregator; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + return getAggregator().evaluate(tuple, ptr); + } + + /** + * Create the aggregator to do server-side aggregation. + * The data type of the returned Aggregator must match + * the data type returned by {@link #newClientAggregator()} + * @param conf HBase configuration. + * @return the aggregator to use on the server-side + */ + abstract public Aggregator newServerAggregator(Configuration conf); + /** + * Create the aggregator to do client-side aggregation + * based on the results returned from the aggregating + * coprocessor. The data type of the returned Aggregator + * must match the data type returned by {@link #newServerAggregator(Configuration)} + * @return the aggregator to use on the client-side + */ + public Aggregator newClientAggregator() { + return newServerAggregator(null); + } + + public Aggregator newServerAggregator(Configuration config, ImmutableBytesWritable ptr) { + Aggregator agg = newServerAggregator(config); + agg.aggregate(null, ptr); + return agg; + } + + public void readFields(DataInput input, Configuration conf) throws IOException { + super.readFields(input); + aggregator = newServerAggregator(conf); + } + + @Override + public boolean isNullable() { + return true; + } + + protected SingleAggregateFunction getDelegate() { + return this; + } + + @Override + public final T accept(ExpressionVisitor visitor) { + SingleAggregateFunction function = getDelegate(); + List l = acceptChildren(visitor, visitor.visitEnter(function)); + T t = visitor.visitLeave(function, l); + if (t == null) { + t = visitor.defaultReturn(function, l); + } + return t; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SqlTypeNameFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SqlTypeNameFunction.java new file mode 100644 index 00000000..5d391234 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SqlTypeNameFunction.java @@ -0,0 +1,86 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.IllegalDataException; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; + + +/** + * + * Function used to get the SQL type name from the SQL type integer. + * Usage: + * SqlTypeName(12) + * will return 'VARCHAR' based on {@link java.sql.Types#VARCHAR} being 12 + * + * @author jtaylor + * @since 0.1 + */ +@BuiltInFunction(name=SqlTypeNameFunction.NAME, args= { + @Argument(allowedTypes=PDataType.INTEGER)} ) +public class SqlTypeNameFunction extends ScalarFunction { + public static final String NAME = "SqlTypeName"; + + public SqlTypeNameFunction() { + } + + public SqlTypeNameFunction(List children) throws SQLException { + super(children); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + Expression child = children.get(0); + if (!child.evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + return true; + } + int sqlType = child.getDataType().getCodec().decodeInt(ptr, child.getColumnModifier()); + try { + byte[] sqlTypeNameBytes = PDataType.fromTypeId(sqlType).getSqlTypeNameBytes(); + ptr.set(sqlTypeNameBytes); + } catch (IllegalDataException e) { + ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); + } + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.VARCHAR; + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/StddevPopFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/StddevPopFunction.java new file mode 100644 index 00000000..4a4e7b82 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/StddevPopFunction.java @@ -0,0 +1,78 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.hadoop.conf.Configuration; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.DecimalStddevPopAggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; +import org.apache.phoenix.expression.aggregator.StddevPopAggregator; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; + +/** + * + * Built-in function for STDDEV_POP() aggregate function + * + * @author anoopsjohn + * @since 1.2.1 + */ +@BuiltInFunction(name = StddevPopFunction.NAME, args = { @Argument(allowedTypes={PDataType.DECIMAL})}) +public class StddevPopFunction extends DistinctValueWithCountAggregateFunction { + public static final String NAME = "STDDEV_POP"; + + public StddevPopFunction() { + + } + + public StddevPopFunction(List childern) { + super(childern); + } + + @Override + public Aggregator newServerAggregator(Configuration conf) { + return new DistinctValueWithCountServerAggregator(conf); + } + + @Override + public DistinctValueWithCountClientAggregator newClientAggregator() { + if (children.get(0).getDataType() == PDataType.DECIMAL) { + // Special Aggregators for DECIMAL datatype for more precision than double + return new DecimalStddevPopAggregator(children, getAggregatorExpression().getColumnModifier()); + } + return new StddevPopAggregator(children, getAggregatorExpression().getColumnModifier()); + } + + @Override + public String getName() { + return NAME; + } + + @Override + public PDataType getDataType() { + return PDataType.DECIMAL; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/StddevSampFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/StddevSampFunction.java new file mode 100644 index 00000000..0e931c60 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/StddevSampFunction.java @@ -0,0 +1,78 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.util.List; + +import org.apache.hadoop.conf.Configuration; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.DecimalStddevSampAggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountClientAggregator; +import org.apache.phoenix.expression.aggregator.DistinctValueWithCountServerAggregator; +import org.apache.phoenix.expression.aggregator.StddevSampAggregator; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; + +/** + * + * Built-in function for STDDEV_SAMP() aggregate function + * + * @author anoopsjohn + * @since 1.2.1 + */ +@BuiltInFunction(name = StddevSampFunction.NAME, args = { @Argument(allowedTypes={PDataType.DECIMAL})}) +public class StddevSampFunction extends DistinctValueWithCountAggregateFunction { + public static final String NAME = "STDDEV_SAMP"; + + public StddevSampFunction() { + + } + + public StddevSampFunction(List childern) { + super(childern); + } + + @Override + public Aggregator newServerAggregator(Configuration conf) { + return new DistinctValueWithCountServerAggregator(conf); + } + + @Override + public DistinctValueWithCountClientAggregator newClientAggregator() { + if (children.get(0).getDataType() == PDataType.DECIMAL) { + // Special Aggregators for DECIMAL datatype for more precision than double + return new DecimalStddevSampAggregator(children, getAggregatorExpression().getColumnModifier()); + } + return new StddevSampAggregator(children, getAggregatorExpression().getColumnModifier()); + } + + @Override + public String getName() { + return NAME; + } + + @Override + public PDataType getDataType() { + return PDataType.DECIMAL; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SubstrFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SubstrFunction.java similarity index 75% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SubstrFunction.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/function/SubstrFunction.java index d52b561b..90aa3c8e 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/SubstrFunction.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SubstrFunction.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.expression.function; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; import java.io.DataInput; import java.io.IOException; @@ -34,14 +26,14 @@ import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.parse.FunctionParseNode.Argument; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.StringUtil; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.StringUtil; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SumAggregateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SumAggregateFunction.java new file mode 100644 index 00000000..ab64ce83 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/SumAggregateFunction.java @@ -0,0 +1,146 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.math.BigDecimal; +import java.util.List; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.DecimalSumAggregator; +import org.apache.phoenix.expression.aggregator.DoubleSumAggregator; +import org.apache.phoenix.expression.aggregator.NumberSumAggregator; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.parse.SumAggregateParseNode; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Built-in function for SUM aggregation function. + * + * @author jtaylor + * @since 0.1 + */ +@BuiltInFunction(name=SumAggregateFunction.NAME, nodeClass=SumAggregateParseNode.class, args= {@Argument(allowedTypes={PDataType.DECIMAL})} ) +public class SumAggregateFunction extends DelegateConstantToCountAggregateFunction { + public static final String NAME = "SUM"; + + public SumAggregateFunction() { + } + + // TODO: remove when not required at built-in func register time + public SumAggregateFunction(List childExpressions){ + super(childExpressions, null); + } + + public SumAggregateFunction(List childExpressions, CountAggregateFunction delegate){ + super(childExpressions, delegate); + } + + private Aggregator newAggregator(final PDataType type, ColumnModifier columnModifier, ImmutableBytesWritable ptr) { + switch( type ) { + case DECIMAL: + return new DecimalSumAggregator(columnModifier, ptr); + case UNSIGNED_DOUBLE: + case UNSIGNED_FLOAT: + case DOUBLE: + case FLOAT: + return new DoubleSumAggregator(columnModifier, ptr) { + @Override + protected PDataType getInputDataType() { + return type; + } + }; + default: + return new NumberSumAggregator(columnModifier, ptr) { + @Override + protected PDataType getInputDataType() { + return type; + } + }; + } + } + + @Override + public Aggregator newClientAggregator() { + return newAggregator(getDataType(), null, null); + } + + @Override + public Aggregator newServerAggregator(Configuration conf) { + Expression child = getAggregatorExpression(); + return newAggregator(child.getDataType(), child.getColumnModifier(), null); + } + + @Override + public Aggregator newServerAggregator(Configuration conf, ImmutableBytesWritable ptr) { + Expression child = getAggregatorExpression(); + return newAggregator(child.getDataType(), child.getColumnModifier(), ptr); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (!super.evaluate(tuple, ptr)) { + return false; + } + if (isConstantExpression()) { + PDataType type = getDataType(); + Object constantValue = ((LiteralExpression)children.get(0)).getValue(); + if (type == PDataType.DECIMAL) { + BigDecimal value = ((BigDecimal)constantValue).multiply((BigDecimal)PDataType.DECIMAL.toObject(ptr, PDataType.LONG)); + ptr.set(PDataType.DECIMAL.toBytes(value)); + } else { + long constantLongValue = ((Number)constantValue).longValue(); + long value = constantLongValue * type.getCodec().decodeLong(ptr, null); + ptr.set(new byte[type.getByteSize()]); + type.getCodec().encodeLong(value, ptr); + } + } + return true; + } + + @Override + public PDataType getDataType() { + switch(super.getDataType()) { + case DECIMAL: + return PDataType.DECIMAL; + case UNSIGNED_FLOAT: + case UNSIGNED_DOUBLE: + case FLOAT: + case DOUBLE: + return PDataType.DOUBLE; + default: + return PDataType.LONG; + } + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/TimeUnit.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/TimeUnit.java new file mode 100644 index 00000000..7ea51612 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/TimeUnit.java @@ -0,0 +1,50 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import com.google.common.base.Joiner; + +public enum TimeUnit { + DAY("day"), + HOUR("hour"), + MINUTE("minute"), + SECOND("second"), + MILLISECOND("millisecond"); + + private String value; + + private TimeUnit(String value) { + this.value = value; + } + + public static final String VALID_VALUES = Joiner.on(", ").join(TimeUnit.values()); + + public static TimeUnit getTimeUnit(String timeUnit) { + if(timeUnit == null) { + throw new IllegalArgumentException("No time unit value specified. Only a time unit value that belongs to one of these : " + VALID_VALUES + " is allowed."); + } + for(TimeUnit tu : values()) { + if(timeUnit.equalsIgnoreCase(tu.value)) { + return tu; + } + } + throw new IllegalArgumentException("Invalid value of time unit " + timeUnit + ". Only a time unit value that belongs to one of these : " + VALID_VALUES + " is allowed."); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ToCharFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ToCharFunction.java new file mode 100644 index 00000000..d6cdc995 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ToCharFunction.java @@ -0,0 +1,135 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.io.*; +import java.sql.SQLException; +import java.text.Format; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.io.WritableUtils; + +import com.google.common.base.Preconditions; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.parse.*; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Implementation of the TO_CHAR(<date>/<number>,[<format-string>] built-in function. + * The first argument must be of type DATE or TIME or TIMESTAMP or DECIMAL or INTEGER, and the second argument must be a constant string. + * + * @author jtaylor + * @since 0.1 + */ +@BuiltInFunction(name=ToCharFunction.NAME, nodeClass=ToCharParseNode.class, args={ + @Argument(allowedTypes={PDataType.TIMESTAMP, PDataType.DECIMAL}), + @Argument(allowedTypes={PDataType.VARCHAR},isConstant=true,defaultValue="null") } ) +public class ToCharFunction extends ScalarFunction { + public static final String NAME = "TO_CHAR"; + private String formatString; + private Format formatter; + private FunctionArgumentType type; + + public ToCharFunction() { + } + + public ToCharFunction(List children, FunctionArgumentType type, String formatString, Format formatter) throws SQLException { + super(children.subList(0, 1)); + Preconditions.checkNotNull(formatString); + Preconditions.checkNotNull(formatter); + Preconditions.checkNotNull(type); + this.type = type; + this.formatString = formatString; + this.formatter = formatter; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + formatString.hashCode(); + result = prime * result + getExpression().hashCode(); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + ToCharFunction other = (ToCharFunction)obj; + if (!getExpression().equals(other.getExpression())) return false; + if (!formatString.equals(other.formatString)) return false; + return true; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + Expression expression = getExpression(); + if (!expression.evaluate(tuple, ptr)) { + return false; + } + PDataType type = expression.getDataType(); + Object value = formatter.format(type.toObject(ptr, expression.getColumnModifier())); + byte[] b = getDataType().toBytes(value); + ptr.set(b); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.VARCHAR; + } + + @Override + public boolean isNullable() { + return getExpression().isNullable(); + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + formatString = WritableUtils.readString(input); + type = WritableUtils.readEnum(input, FunctionArgumentType.class); + formatter = type.getFormatter(formatString); + } + + @Override + public void write(DataOutput output) throws IOException { + super.write(output); + WritableUtils.writeString(output, formatString); + WritableUtils.writeEnum(output, type); + } + + private Expression getExpression() { + return children.get(0); + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ToDateFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ToDateFunction.java new file mode 100644 index 00000000..5ef0034d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ToDateFunction.java @@ -0,0 +1,133 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.io.*; +import java.sql.SQLException; +import java.text.Format; +import java.text.ParseException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.io.WritableUtils; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.*; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.DateUtil; + + +/** + * + * Implementation of the TO_DATE(,[]) built-in function. + * The second argument is optional and defaults to the phoenix.query.dateFormat value + * from the HBase config. If present it must be a constant string. + * + * @author jtaylor + * @since 0.1 + */ +@BuiltInFunction(name=ToDateFunction.NAME, nodeClass=ToDateParseNode.class, args= {@Argument(allowedTypes={PDataType.VARCHAR}),@Argument(allowedTypes={PDataType.VARCHAR},isConstant=true,defaultValue="null")} ) +public class ToDateFunction extends ScalarFunction { + public static final String NAME = "TO_DATE"; + private Format dateParser; + private String dateFormat; + + public ToDateFunction() { + } + + public ToDateFunction(List children, String dateFormat, Format dateParser) throws SQLException { + super(children.subList(0, 1)); + this.dateFormat = dateFormat; + this.dateParser = dateParser; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + dateFormat.hashCode(); + result = prime * result + getExpression().hashCode(); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + ToDateFunction other = (ToDateFunction)obj; + if (!getExpression().equals(other.getExpression())) return false; + if (!dateFormat.equals(other.dateFormat)) return false; + return true; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + Expression expression = getExpression(); + if (!expression.evaluate(tuple, ptr) || ptr.getLength() == 0) { + return false; + } + PDataType type = expression.getDataType(); + String dateStr = (String)type.toObject(ptr, expression.getColumnModifier()); + try { + Object value = dateParser.parseObject(dateStr); + byte[] byteValue = getDataType().toBytes(value); + ptr.set(byteValue); + return true; + } catch (ParseException e) { + throw new IllegalStateException("to_date('" + dateStr + ")' did not match expected date format of '" + dateFormat + "'."); + } + } + + @Override + public PDataType getDataType() { + return PDataType.DATE; + } + + @Override + public boolean isNullable() { + return getExpression().isNullable(); + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + dateFormat = WritableUtils.readString(input); + dateParser = DateUtil.getDateParser(dateFormat); + } + + @Override + public void write(DataOutput output) throws IOException { + super.write(output); + WritableUtils.writeString(output, dateFormat); + } + + private Expression getExpression() { + return children.get(0); + } + + @Override + public String getName() { + return NAME; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ToNumberFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ToNumberFunction.java new file mode 100644 index 00000000..84cc2b4d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/ToNumberFunction.java @@ -0,0 +1,176 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import static org.apache.phoenix.util.ByteUtil.EMPTY_BYTE_ARRAY; + +import java.io.*; +import java.math.BigDecimal; +import java.sql.SQLException; +import java.text.*; +import java.util.Date; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.io.WritableUtils; + +import com.google.common.base.Preconditions; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.parse.*; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * + * Implementation of TO_NUMBER(<string>/<date>/<timestamp>, [<pattern-string>]) built-in function. The format for the optional + * pattern_string param is specified in {@link DecimalFormat}. + * + * @author elevine + * @since 0.1 + */ +@BuiltInFunction(name=ToNumberFunction.NAME, nodeClass=ToNumberParseNode.class, args= { + @Argument(allowedTypes={PDataType.VARCHAR, PDataType.TIMESTAMP}), + @Argument(allowedTypes={PDataType.VARCHAR}, isConstant=true, defaultValue="null")} ) +public class ToNumberFunction extends ScalarFunction { + public static final String NAME = "TO_NUMBER"; + + private String formatString = null; + private Format format = null; + private FunctionArgumentType type; + + public ToNumberFunction() {} + + public ToNumberFunction(List children, FunctionArgumentType type, String formatString, Format formatter) throws SQLException { + super(children.subList(0, 1)); + Preconditions.checkNotNull(type); + this.type = type; + this.formatString = formatString; + this.format = formatter; + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + Expression expression = getExpression(); + if (!expression.evaluate(tuple, ptr)) { + return false; + } else if (ptr.getLength() == 0) { + return true; + } + + PDataType type = expression.getDataType(); + if (type.isCoercibleTo(PDataType.TIMESTAMP)) { + Date date = (Date) type.toObject(ptr, expression.getColumnModifier()); + BigDecimal time = new BigDecimal(date.getTime()); + byte[] byteValue = getDataType().toBytes(time); + ptr.set(byteValue); + return true; + } + + String stringValue = (String)type.toObject(ptr, expression.getColumnModifier()); + if (stringValue == null) { + ptr.set(EMPTY_BYTE_ARRAY); + return true; + } + stringValue = stringValue.trim(); + BigDecimal decimalValue; + if (format == null) { + decimalValue = (BigDecimal) getDataType().toObject(stringValue); + } else { + ParsePosition parsePosition = new ParsePosition(0); + Number number = ((DecimalFormat) format).parse(stringValue, parsePosition); + if (parsePosition.getErrorIndex() > -1) { + ptr.set(EMPTY_BYTE_ARRAY); + return true; + } + + if (number instanceof BigDecimal) { + // since we set DecimalFormat.setParseBigDecimal(true) we are guaranteeing result to be + // of type BigDecimal in most cases. see java.text.DecimalFormat.parse() JavaDoc. + decimalValue = (BigDecimal)number; + } else { + ptr.set(EMPTY_BYTE_ARRAY); + return true; + } + } + byte[] byteValue = getDataType().toBytes(decimalValue); + ptr.set(byteValue); + return true; + } + + @Override + public PDataType getDataType() { + return PDataType.DECIMAL; + } + + @Override + public boolean isNullable() { + return getExpression().isNullable(); + } + + private Expression getExpression() { + return children.get(0); + } + + @Override + public String getName() { + return NAME; + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + formatString = WritableUtils.readString(input); + type = WritableUtils.readEnum(input, FunctionArgumentType.class); + if (formatString != null) { + format = type.getFormatter(formatString); + } + } + + @Override + public void write(DataOutput output) throws IOException { + super.write(output); + WritableUtils.writeString(output, formatString); + WritableUtils.writeEnum(output, type); + } + + @Override + public int hashCode() { + final int prime = 31; + int result = super.hashCode(); + result = prime * result + ((formatString == null) ? 0 : formatString.hashCode()); + result = prime * result + getExpression().hashCode(); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (!super.equals(obj)) return false; + if (getClass() != obj.getClass()) return false; + ToNumberFunction other = (ToNumberFunction)obj; + if (formatString == null) { + if (other.formatString != null) return false; + } else if (!formatString.equals(other.formatString)) return false; + if (!getExpression().equals(other.getExpression())) return false; + return true; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/TrimFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/TrimFunction.java new file mode 100644 index 00000000..b8c06cdb --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/TrimFunction.java @@ -0,0 +1,108 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.StringUtil; + + +/** + * Implementation of the Trim() build-in function. It removes from both end of + * space character and other function bytes in single byte utf8 characters set. + * + * @author zhuang + * @since 0.1 + */ +@BuiltInFunction(name=TrimFunction.NAME, args={ + @Argument(allowedTypes={PDataType.VARCHAR})} ) +public class TrimFunction extends ScalarFunction { + public static final String NAME = "TRIM"; + + private Integer byteSize; + + public TrimFunction() { } + + public TrimFunction(List children) throws SQLException { + super(children); + if (getStringExpression().getDataType().isFixedWidth()) { + byteSize = getStringExpression().getByteSize(); + } + } + + private Expression getStringExpression() { + return children.get(0); + } + + @Override + public ColumnModifier getColumnModifier() { + return children.get(0).getColumnModifier(); + } + + @Override + public boolean evaluate(Tuple tuple, ImmutableBytesWritable ptr) { + if (!getStringExpression().evaluate(tuple, ptr)) { + return false; + } + if (ptr.getLength() == 0) { + ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); + return true; + } + byte[] string = ptr.get(); + int offset = ptr.getOffset(); + int length = ptr.getLength(); + + ColumnModifier columnModifier = getColumnModifier(); + int end = StringUtil.getFirstNonBlankCharIdxFromEnd(string, offset, length, columnModifier); + if (end == offset - 1) { + ptr.set(ByteUtil.EMPTY_BYTE_ARRAY); + return true; + } + int head = StringUtil.getFirstNonBlankCharIdxFromStart(string, offset, length, columnModifier); + ptr.set(string, head, end - head + 1); + return true; + } + + @Override + public Integer getByteSize() { + return byteSize; + } + + @Override + public PDataType getDataType() { + return PDataType.VARCHAR; + } + + @Override + public String getName() { + return NAME; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/function/TruncFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/TruncFunction.java new file mode 100644 index 00000000..26a75eec --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/TruncFunction.java @@ -0,0 +1,65 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.function; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FloorParseNode; +import org.apache.phoenix.parse.FunctionParseNode.Argument; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.schema.PDataType; + + +/** + * + * Function used to bucketize date/time values by truncating them to + * an even increment. Usage: + * TRUNC(,<'day'|'hour'|'minute'|'second'|'millisecond'>,[]) + * The integer multiplier is optional and is used to do rollups to a partial time unit (i.e. 10 minute rollup) + * The function returns a {@link org.apache.phoenix.schema.PDataType#DATE} + * + * @author jtaylor + * @since 0.1 + */ +@BuiltInFunction(name = TruncFunction.NAME, +nodeClass = FloorParseNode.class, +args = { + @Argument(allowedTypes={PDataType.TIMESTAMP, PDataType.DECIMAL}), + @Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValue = "null", isConstant=true), + @Argument(allowedTypes={PDataType.INTEGER}, defaultValue="1", isConstant=true) + } +) +public abstract class TruncFunction extends ScalarFunction { + + public static final String NAME = "TRUNC"; + + public TruncFunction(List children) throws SQLException { + super(children); + } + + @Override + public String getName() { + return NAME; + } + + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/UpperFunction.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/UpperFunction.java similarity index 83% rename from phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/UpperFunction.java rename to phoenix-core/src/main/java/org/apache/phoenix/expression/function/UpperFunction.java index bfc88b50..af4c39c6 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/expression/function/UpperFunction.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/function/UpperFunction.java @@ -1,14 +1,14 @@ -package com.salesforce.phoenix.expression.function; +package org.apache.phoenix.expression.function; import java.sql.SQLException; import java.util.List; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.parse.FunctionParseNode; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.parse.FunctionParseNode; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; @FunctionParseNode.BuiltInFunction(name=UpperFunction.NAME, args={ @FunctionParseNode.Argument(allowedTypes={PDataType.VARCHAR})} ) diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/BaseExpressionVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/BaseExpressionVisitor.java new file mode 100644 index 00000000..a5246176 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/BaseExpressionVisitor.java @@ -0,0 +1,264 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.visitor; + +import java.util.Iterator; +import java.util.List; + +import org.apache.phoenix.expression.AddExpression; +import org.apache.phoenix.expression.AndExpression; +import org.apache.phoenix.expression.ArrayConstructorExpression; +import org.apache.phoenix.expression.CaseExpression; +import org.apache.phoenix.expression.CoerceExpression; +import org.apache.phoenix.expression.ComparisonExpression; +import org.apache.phoenix.expression.DivideExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.InListExpression; +import org.apache.phoenix.expression.IsNullExpression; +import org.apache.phoenix.expression.KeyValueColumnExpression; +import org.apache.phoenix.expression.LikeExpression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.MultiplyExpression; +import org.apache.phoenix.expression.NotExpression; +import org.apache.phoenix.expression.OrExpression; +import org.apache.phoenix.expression.ProjectedColumnExpression; +import org.apache.phoenix.expression.RowKeyColumnExpression; +import org.apache.phoenix.expression.RowValueConstructorExpression; +import org.apache.phoenix.expression.StringConcatExpression; +import org.apache.phoenix.expression.SubtractExpression; +import org.apache.phoenix.expression.function.ScalarFunction; +import org.apache.phoenix.expression.function.SingleAggregateFunction; + + +public abstract class BaseExpressionVisitor implements ExpressionVisitor { + @Override + public E visit(Expression node) { + return null; + } + + @Override + public Iterator visitEnter(Expression node) { + return null; + } + + @Override + public E visitLeave(Expression node, List l) { + return null; + } + + @Override + public E defaultReturn(Expression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(AndExpression node) { + return null; + } + + @Override + public E visitLeave(AndExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(OrExpression node) { + return null; + } + + @Override + public E visitLeave(OrExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(ScalarFunction node) { + return null; + } + + @Override + public E visitLeave(ScalarFunction node, List l) { + return null; + } + + @Override + public Iterator visitEnter(ComparisonExpression node) { + return null; + } + + @Override + public E visitLeave(ComparisonExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(LikeExpression node) { + return null; + } + + @Override + public E visitLeave(LikeExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(SingleAggregateFunction node) { + return null; + } + + @Override + public E visitLeave(SingleAggregateFunction node, List l) { + return null; + } + + @Override + public Iterator visitEnter(CaseExpression node) { + return null; + } + + @Override + public E visitLeave(CaseExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(NotExpression node) { + return null; + } + + @Override + public E visitLeave(NotExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(IsNullExpression node) { + return null; + } + + @Override + public E visitLeave(IsNullExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(InListExpression node) { + return null; + } + + @Override + public E visitLeave(InListExpression node, List l) { + return null; + } + + @Override + public E visit(LiteralExpression node) { + return null; + } + + @Override + public E visit(RowKeyColumnExpression node) { + return null; + } + + @Override + public E visit(KeyValueColumnExpression node) { + return null; + } + + @Override + public E visit(ProjectedColumnExpression node) { + return null; + } + + @Override + public Iterator visitEnter(SubtractExpression node) { + return null; + } + + @Override + public E visitLeave(SubtractExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(AddExpression node) { + return null; + } + @Override + public E visitLeave(AddExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(MultiplyExpression node) { + return null; + } + @Override + public E visitLeave(MultiplyExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(DivideExpression node) { + return null; + } + @Override + public E visitLeave(DivideExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(StringConcatExpression node) { + return null; + } + @Override + public E visitLeave(StringConcatExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(RowValueConstructorExpression node) { + return null; + } + @Override + public E visitLeave(RowValueConstructorExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(CoerceExpression node) { + return null; + } + + @Override + public E visitLeave(CoerceExpression node, List l) { + return null; + } + + @Override + public Iterator visitEnter(ArrayConstructorExpression node) { + return null; + } + @Override + public E visitLeave(ArrayConstructorExpression node, List l) { + return null; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/ExpressionVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/ExpressionVisitor.java new file mode 100644 index 00000000..efde2a4e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/ExpressionVisitor.java @@ -0,0 +1,142 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.visitor; + +import java.util.Iterator; +import java.util.List; + +import org.apache.phoenix.expression.AddExpression; +import org.apache.phoenix.expression.AndExpression; +import org.apache.phoenix.expression.ArrayConstructorExpression; +import org.apache.phoenix.expression.CaseExpression; +import org.apache.phoenix.expression.CoerceExpression; +import org.apache.phoenix.expression.ComparisonExpression; +import org.apache.phoenix.expression.DivideExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.InListExpression; +import org.apache.phoenix.expression.IsNullExpression; +import org.apache.phoenix.expression.KeyValueColumnExpression; +import org.apache.phoenix.expression.LikeExpression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.MultiplyExpression; +import org.apache.phoenix.expression.NotExpression; +import org.apache.phoenix.expression.OrExpression; +import org.apache.phoenix.expression.ProjectedColumnExpression; +import org.apache.phoenix.expression.RowKeyColumnExpression; +import org.apache.phoenix.expression.RowValueConstructorExpression; +import org.apache.phoenix.expression.StringConcatExpression; +import org.apache.phoenix.expression.SubtractExpression; +import org.apache.phoenix.expression.function.ScalarFunction; +import org.apache.phoenix.expression.function.SingleAggregateFunction; + + +/** + * + * Visitor for an expression (which may contain other nested expressions) + * + * @author jtaylor + * @since 0.1 + */ +public interface ExpressionVisitor { + /** + * Default visit method when an expression subclass doesn't + * define an accept method of its own. This will end up calling + * the {@link #defaultIterator(Expression)} to iterate over the + * children calling accept on them + */ + public E visit(Expression node); + /** + * Default visitEnter method when an expression subclass doesn't + * define an accept method of its own. This will end up calling + * the {@link #defaultIterator(Expression)} to iterate over the + * children calling accept on them + */ + public Iterator visitEnter(Expression node); + /** + * Default visitLeave method when an expression subclass doesn't + * define an accept method of its own. This will end up calling + * the {@link #defaultReturn(Expression, List)} with the list from + * the iteration over the children. + */ + public E visitLeave(Expression node, List l); + + public E defaultReturn(Expression node, List l); + public Iterator defaultIterator(Expression node); + + public Iterator visitEnter(AndExpression node); + public E visitLeave(AndExpression node, List l); + + public Iterator visitEnter(OrExpression node); + public E visitLeave(OrExpression node, List l); + + public Iterator visitEnter(ScalarFunction node); + public E visitLeave(ScalarFunction node, List l); + + public Iterator visitEnter(ComparisonExpression node); + public E visitLeave(ComparisonExpression node, List l); + + public Iterator visitEnter(LikeExpression node); + public E visitLeave(LikeExpression node, List l); + + public Iterator visitEnter(SingleAggregateFunction node); + public E visitLeave(SingleAggregateFunction node, List l); + + public Iterator visitEnter(CaseExpression node); + public E visitLeave(CaseExpression node, List l); + + public Iterator visitEnter(NotExpression node); + public E visitLeave(NotExpression node, List l); + + public Iterator visitEnter(InListExpression node); + public E visitLeave(InListExpression node, List l); + + public Iterator visitEnter(IsNullExpression node); + public E visitLeave(IsNullExpression node, List l); + + public Iterator visitEnter(SubtractExpression node); + public E visitLeave(SubtractExpression node, List l); + + public Iterator visitEnter(MultiplyExpression node); + public E visitLeave(MultiplyExpression node, List l); + + public Iterator visitEnter(AddExpression node); + public E visitLeave(AddExpression node, List l); + + public Iterator visitEnter(DivideExpression node); + public E visitLeave(DivideExpression node, List l); + + public Iterator visitEnter(CoerceExpression node); + public E visitLeave(CoerceExpression node, List l); + + public Iterator visitEnter(ArrayConstructorExpression node); + public E visitLeave(ArrayConstructorExpression node, List l); + + public E visit(LiteralExpression node); + public E visit(RowKeyColumnExpression node); + public E visit(KeyValueColumnExpression node); + public E visit(ProjectedColumnExpression node); + + public Iterator visitEnter(StringConcatExpression node); + public E visitLeave(StringConcatExpression node, List l); + + public Iterator visitEnter(RowValueConstructorExpression node); + public E visitLeave(RowValueConstructorExpression node, List l); + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/KeyValueExpressionVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/KeyValueExpressionVisitor.java new file mode 100644 index 00000000..838bb0b1 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/KeyValueExpressionVisitor.java @@ -0,0 +1,38 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.visitor; + +import org.apache.phoenix.expression.KeyValueColumnExpression; + + + + +/** + * + * Implementation of ExpressionVisitor where only KeyValueDataAccessor + * is being visited + * + * @author jtaylor + * @since 0.1 + */ +public abstract class KeyValueExpressionVisitor extends TraverseAllExpressionVisitor { + @Override + abstract public Void visit(KeyValueColumnExpression node); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/SingleAggregateFunctionVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/SingleAggregateFunctionVisitor.java new file mode 100644 index 00000000..63313a7a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/SingleAggregateFunctionVisitor.java @@ -0,0 +1,40 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.visitor; + +import java.util.Iterator; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.SingleAggregateFunction; + + + +/** + * + * Implementation of ExpressionVisitor where only SingleAggregateFunction + * instances are visited + * + * @author jtaylor + * @since 0.1 + */ +public abstract class SingleAggregateFunctionVisitor extends TraverseAllExpressionVisitor { + @Override + abstract public Iterator visitEnter(SingleAggregateFunction node); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/TraverseAllExpressionVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/TraverseAllExpressionVisitor.java new file mode 100644 index 00000000..39e7a3db --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/TraverseAllExpressionVisitor.java @@ -0,0 +1,57 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.visitor; + +import java.util.*; + +import org.apache.phoenix.expression.Expression; + + + + +public class TraverseAllExpressionVisitor extends BaseExpressionVisitor { + + @Override + public Iterator defaultIterator(Expression node) { + final List children = node.getChildren(); + return new Iterator() { + private int position; + + @Override + public final boolean hasNext() { + return position < children.size(); + } + + @Override + public final Expression next() { + if (!hasNext()) { + throw new NoSuchElementException(); + } + return children.get(position++); + } + + @Override + public final void remove() { + throw new UnsupportedOperationException(); + } + }; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/TraverseNoExpressionVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/TraverseNoExpressionVisitor.java new file mode 100644 index 00000000..fe1d1f5e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/expression/visitor/TraverseNoExpressionVisitor.java @@ -0,0 +1,35 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.expression.visitor; + +import java.util.Iterator; + + +import com.google.common.collect.Iterators; +import org.apache.phoenix.expression.Expression; + +public class TraverseNoExpressionVisitor extends BaseExpressionVisitor { + + @Override + public Iterator defaultIterator(Expression node) { + return Iterators.emptyIterator(); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/filter/BooleanExpressionFilter.java b/phoenix-core/src/main/java/org/apache/phoenix/filter/BooleanExpressionFilter.java new file mode 100644 index 00000000..d7b22438 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/filter/BooleanExpressionFilter.java @@ -0,0 +1,125 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.filter; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; + +import org.apache.hadoop.hbase.filter.FilterBase; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.io.WritableUtils; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.ExpressionType; +import org.apache.phoenix.schema.IllegalDataException; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ServerUtil; + + +/** + * + * Base class for filter that evaluates a WHERE clause expression. + * + * @author jtaylor + * @since 0.1 + */ +abstract public class BooleanExpressionFilter extends FilterBase { + + protected Expression expression; + protected boolean evaluateOnCompletion; + private ImmutableBytesWritable tempPtr = new ImmutableBytesWritable(); + + public BooleanExpressionFilter() { + } + + public BooleanExpressionFilter(Expression expression) { + this.expression = expression; + } + + protected void setEvaluateOnCompletion(boolean evaluateOnCompletion) { + this.evaluateOnCompletion = evaluateOnCompletion; + } + + protected boolean evaluateOnCompletion() { + return evaluateOnCompletion; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + expression.hashCode(); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + BooleanExpressionFilter other = (BooleanExpressionFilter)obj; + if (!expression.equals(other.expression)) return false; + return true; + } + + @Override + public String toString() { + return expression.toString(); + } + + @edu.umd.cs.findbugs.annotations.SuppressWarnings( + value="NP_BOOLEAN_RETURN_NULL", + justification="Returns null by design.") + protected Boolean evaluate(Tuple input) { + try { + if (!expression.evaluate(input, tempPtr)) { + return null; + } + } catch (IllegalDataException e) { + return Boolean.FALSE; + } + return (Boolean)expression.getDataType().toObject(tempPtr); + } + + @Override + public void readFields(DataInput input) throws IOException { + try { + expression = ExpressionType.values()[WritableUtils.readVInt(input)].newInstance(); + expression.readFields(input); + } catch (Throwable t) { // Catches incompatibilities during reading/writing and doesn't retry + ServerUtil.throwIOException("BooleanExpressionFilter failed during reading", t); + } + } + + @Override + public void write(DataOutput output) throws IOException { + try { + WritableUtils.writeVInt(output, ExpressionType.valueOf(expression).ordinal()); + expression.write(output); + } catch (Throwable t) { // Catches incompatibilities during reading/writing and doesn't retry + ServerUtil.throwIOException("BooleanExpressionFilter failed during writing", t); + } + } + + @Override + public void reset() { + expression.reset(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/filter/EvaluateOnCompletionVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/filter/EvaluateOnCompletionVisitor.java new file mode 100644 index 00000000..7dcc9549 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/filter/EvaluateOnCompletionVisitor.java @@ -0,0 +1,83 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.filter; + +import java.util.Iterator; + +import org.apache.phoenix.expression.ArrayConstructorExpression; +import org.apache.phoenix.expression.CaseExpression; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.IsNullExpression; +import org.apache.phoenix.expression.RowKeyColumnExpression; +import org.apache.phoenix.expression.RowValueConstructorExpression; +import org.apache.phoenix.expression.visitor.TraverseAllExpressionVisitor; + + +/** + * + * Implementation of ExpressionVisitor for the expression used by the + * BooleanExpressionFilter that looks for expressions that need to be + * evaluated upon completion. Examples include: + * - CaseExpression with an else clause, since upon completion, the + * else clause would apply if the when clauses could not be evaluated + * due to the absense of a value. + * - IsNullExpression that's not negated, since upon completion, we + * know definitively that a column value was not found. + * - row key columns are used, since we may never have encountered a + * key value column of interest, but the expression may evaluate to true + * just based on the row key columns. + * @author jtaylor + * @since 0.1 + */ +public class EvaluateOnCompletionVisitor extends TraverseAllExpressionVisitor { + private boolean evaluateOnCompletion = false; + + public boolean evaluateOnCompletion() { + return evaluateOnCompletion; + } + + @Override + public Iterator visitEnter(IsNullExpression node) { + evaluateOnCompletion |= !node.isNegate(); + return null; + } + @Override + public Iterator visitEnter(CaseExpression node) { + evaluateOnCompletion |= node.hasElse(); + return null; + } + @Override + public Void visit(RowKeyColumnExpression node) { + evaluateOnCompletion = true; + return null; + } + @Override + public Iterator visitEnter(RowValueConstructorExpression node) { + evaluateOnCompletion = true; + return null; + } + + @Override + public Iterator visitEnter(ArrayConstructorExpression node) { + evaluateOnCompletion = true; + return null; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/filter/MultiCFCQKeyValueComparisonFilter.java b/phoenix-core/src/main/java/org/apache/phoenix/filter/MultiCFCQKeyValueComparisonFilter.java new file mode 100644 index 00000000..27a356b4 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/filter/MultiCFCQKeyValueComparisonFilter.java @@ -0,0 +1,134 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.filter; + +import java.util.TreeSet; + +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.phoenix.expression.Expression; + + +/** + * + * Filter that evaluates WHERE clause expression, used in the case where there + * are references to multiple column qualifiers over multiple column families. + * + * @author jtaylor + * @since 0.1 + */ +public class MultiCFCQKeyValueComparisonFilter extends MultiKeyValueComparisonFilter { + private final ImmutablePairBytesPtr ptr = new ImmutablePairBytesPtr(); + private TreeSet cfSet; + + public MultiCFCQKeyValueComparisonFilter() { + } + + public MultiCFCQKeyValueComparisonFilter(Expression expression) { + super(expression); + } + + @Override + protected void init() { + cfSet = new TreeSet(Bytes.BYTES_COMPARATOR); + super.init(); + } + + @Override + protected Object setColumnKey(byte[] cf, int cfOffset, int cfLength, + byte[] cq, int cqOffset, int cqLength) { + ptr.set(cf, cfOffset, cfLength, cq, cqOffset, cqLength); + return ptr; + } + + @Override + protected Object newColumnKey(byte[] cf, int cfOffset, int cfLength, + byte[] cq, int cqOffset, int cqLength) { + + byte[] cfKey; + if (cfOffset == 0 && cf.length == cfLength) { + cfKey = cf; + } else { + // Copy bytes here, but figure cf names are typically a few bytes at most, + // so this will be better than creating an ImmutableBytesPtr + cfKey = new byte[cfLength]; + System.arraycopy(cf, cfOffset, cfKey, 0, cfLength); + } + cfSet.add(cfKey); + return new ImmutablePairBytesPtr(cf, cfOffset, cfLength, cq, cqOffset, cqLength); + } + + private static class ImmutablePairBytesPtr { + private byte[] bytes1; + private int offset1; + private int length1; + private byte[] bytes2; + private int offset2; + private int length2; + private int hashCode; + + private ImmutablePairBytesPtr() { + } + + private ImmutablePairBytesPtr(byte[] bytes1, int offset1, int length1, byte[] bytes2, int offset2, int length2) { + set(bytes1, offset1, length1, bytes2, offset2, length2); + } + + @Override + public int hashCode() { + return hashCode; + } + + public void set(byte[] bytes1, int offset1, int length1, byte[] bytes2, int offset2, int length2) { + this.bytes1 = bytes1; + this.offset1 = offset1; + this.length1 = length1; + this.bytes2 = bytes2; + this.offset2 = offset2; + this.length2 = length2; + int hash = 1; + for (int i = offset1; i < offset1 + length1; i++) + hash = (31 * hash) + bytes1[i]; + for (int i = offset2; i < offset2 + length2; i++) + hash = (31 * hash) + bytes2[i]; + hashCode = hash; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + ImmutablePairBytesPtr that = (ImmutablePairBytesPtr)obj; + if (this.hashCode != that.hashCode) return false; + if (Bytes.compareTo(this.bytes2, this.offset2, this.length2, that.bytes2, that.offset2, that.length2) != 0) return false; + if (Bytes.compareTo(this.bytes1, this.offset1, this.length1, that.bytes1, that.offset1, that.length1) != 0) return false; + return true; + } + } + + + @SuppressWarnings("all") // suppressing missing @Override since this doesn't exist for HBase 0.94.4 + public boolean isFamilyEssential(byte[] name) { + // Only the column families involved in the expression are essential. + // The others are for columns projected in the select expression. + return cfSet.contains(name); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/filter/MultiCQKeyValueComparisonFilter.java b/phoenix-core/src/main/java/org/apache/phoenix/filter/MultiCQKeyValueComparisonFilter.java new file mode 100644 index 00000000..c66b7889 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/filter/MultiCQKeyValueComparisonFilter.java @@ -0,0 +1,70 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.filter; + +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.expression.Expression; + +/** + * + * Filter that evaluates WHERE clause expression, used in the case where there + * are references to multiple column qualifiers over a single column family. + * + * @author jtaylor + * @since 0.1 + */ +public class MultiCQKeyValueComparisonFilter extends MultiKeyValueComparisonFilter { + private ImmutableBytesPtr ptr = new ImmutableBytesPtr(); + private byte[] cf; + + public MultiCQKeyValueComparisonFilter() { + } + + public MultiCQKeyValueComparisonFilter(Expression expression) { + super(expression); + } + + @Override + protected Object setColumnKey(byte[] cf, int cfOffset, int cfLength, byte[] cq, int cqOffset, + int cqLength) { + ptr.set(cq, cqOffset, cqLength); + return ptr; + } + + @Override + protected Object newColumnKey(byte[] cf, int cfOffset, int cfLength, byte[] cq, int cqOffset, + int cqLength) { + if (cfOffset == 0 && cf.length == cfLength) { + this.cf = cf; + } else { + this.cf = new byte[cfLength]; + System.arraycopy(cf, cfOffset, this.cf, 0, cfLength); + } + return new ImmutableBytesPtr(cq, cqOffset, cqLength); + } + + + @SuppressWarnings("all") // suppressing missing @Override since this doesn't exist for HBase 0.94.4 + public boolean isFamilyEssential(byte[] name) { + return Bytes.compareTo(cf, name) == 0; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/MultiKeyValueComparisonFilter.java b/phoenix-core/src/main/java/org/apache/phoenix/filter/MultiKeyValueComparisonFilter.java similarity index 80% rename from phoenix-core/src/main/java/com/salesforce/phoenix/filter/MultiKeyValueComparisonFilter.java rename to phoenix-core/src/main/java/org/apache/phoenix/filter/MultiKeyValueComparisonFilter.java index d69110f8..ab8e1c0b 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/MultiKeyValueComparisonFilter.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/filter/MultiKeyValueComparisonFilter.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.filter; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.filter; import java.io.DataInput; import java.io.IOException; @@ -36,9 +28,9 @@ import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.KeyValueColumnExpression; -import com.salesforce.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.KeyValueColumnExpression; +import org.apache.phoenix.schema.tuple.Tuple; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/filter/RowKeyComparisonFilter.java b/phoenix-core/src/main/java/org/apache/phoenix/filter/RowKeyComparisonFilter.java new file mode 100644 index 00000000..bd47a8c8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/filter/RowKeyComparisonFilter.java @@ -0,0 +1,156 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.filter; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.io.WritableUtils; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Filter for use when expressions only reference row key columns + * + * @author jtaylor + * @since 0.1 + */ +public class RowKeyComparisonFilter extends BooleanExpressionFilter { + private static final Logger logger = LoggerFactory.getLogger(RowKeyComparisonFilter.class); + + private boolean evaluate = true; + private boolean keepRow = false; + private RowKeyTuple inputTuple = new RowKeyTuple(); + private byte[] essentialCF; + + public RowKeyComparisonFilter() { + } + + public RowKeyComparisonFilter(Expression expression, byte[] essentialCF) { + super(expression); + this.essentialCF = essentialCF; + } + + @Override + public void reset() { + this.keepRow = false; + this.evaluate = true; + super.reset(); + } + + /** + * Evaluate in filterKeyValue instead of filterRowKey, because HBASE-6562 causes filterRowKey + * to be called with deleted or partial row keys. + */ + @Override + public ReturnCode filterKeyValue(KeyValue v) { + if (evaluate) { + inputTuple.setKey(v.getBuffer(), v.getRowOffset(), v.getRowLength()); + this.keepRow = Boolean.TRUE.equals(evaluate(inputTuple)); + if (logger.isDebugEnabled()) { + logger.debug("RowKeyComparisonFilter: " + (this.keepRow ? "KEEP" : "FILTER") + " row " + inputTuple); + } + evaluate = false; + } + return keepRow ? ReturnCode.INCLUDE : ReturnCode.NEXT_ROW; + } + + private final class RowKeyTuple implements Tuple { + private byte[] buf; + private int offset; + private int length; + + public void setKey(byte[] buf, int offset, int length) { + this.buf = buf; + this.offset = offset; + this.length = length; + } + + @Override + public void getKey(ImmutableBytesWritable ptr) { + ptr.set(buf, offset, length); + } + + @Override + public KeyValue getValue(byte[] cf, byte[] cq) { + return null; + } + + @Override + public boolean isImmutable() { + return true; + } + + @Override + public String toString() { + return Bytes.toStringBinary(buf, offset, length); + } + + @Override + public int size() { + return 0; + } + + @Override + public KeyValue getValue(int index) { + throw new IndexOutOfBoundsException(Integer.toString(index)); + } + + @Override + public boolean getValue(byte[] family, byte[] qualifier, + ImmutableBytesWritable ptr) { + return false; + } + } + + @Override + public boolean filterRow() { + return !this.keepRow; + } + + @SuppressWarnings("all") // suppressing missing @Override since this doesn't exist for HBase 0.94.4 + public boolean isFamilyEssential(byte[] name) { + // We only need our "guaranteed to have a key value" column family, + // which we pass in and serialize through. In the case of a VIEW where + // we don't have this, we have to say that all families are essential. + return this.essentialCF.length == 0 ? true : Bytes.compareTo(this.essentialCF, name) == 0; + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + this.essentialCF = WritableUtils.readCompressedByteArray(input); + } + + @Override + public void write(DataOutput output) throws IOException { + super.write(output); + WritableUtils.writeCompressedByteArray(output, this.essentialCF); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/filter/SingleCFCQKeyValueComparisonFilter.java b/phoenix-core/src/main/java/org/apache/phoenix/filter/SingleCFCQKeyValueComparisonFilter.java new file mode 100644 index 00000000..5b8a5f06 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/filter/SingleCFCQKeyValueComparisonFilter.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.filter; + +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.phoenix.expression.Expression; + + +/** + * + * SingleKeyValueComparisonFilter that needs to compare both the column family and + * column qualifier parts of the key value to disambiguate with another similarly + * named column qualifier in a different column family. + * + * @author jtaylor + * @since 0.1 + */ +public class SingleCFCQKeyValueComparisonFilter extends SingleKeyValueComparisonFilter { + public SingleCFCQKeyValueComparisonFilter() { + } + + public SingleCFCQKeyValueComparisonFilter(Expression expression) { + super(expression); + } + + @Override + protected final int compare(byte[] cfBuf, int cfOffset, int cfLength, byte[] cqBuf, int cqOffset, int cqLength) { + int c = Bytes.compareTo(cf, 0, cf.length, cfBuf, cfOffset, cfLength); + if (c != 0) return c; + return Bytes.compareTo(cq, 0, cq.length, cqBuf, cqOffset, cqLength); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/filter/SingleCQKeyValueComparisonFilter.java b/phoenix-core/src/main/java/org/apache/phoenix/filter/SingleCQKeyValueComparisonFilter.java new file mode 100644 index 00000000..425839ae --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/filter/SingleCQKeyValueComparisonFilter.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.filter; + +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.phoenix.expression.Expression; + + +/** + * + * SingleKeyValueComparisonFilter that needs to only compare the column qualifier + * part of the key value since the column qualifier is unique across all column + * families. + * + * @author jtaylor + * @since 0.1 + */ +public class SingleCQKeyValueComparisonFilter extends SingleKeyValueComparisonFilter { + public SingleCQKeyValueComparisonFilter() { + } + + public SingleCQKeyValueComparisonFilter(Expression expression) { + super(expression); + } + + @Override + protected final int compare(byte[] cfBuf, int cfOffset, int cfLength, byte[] cqBuf, int cqOffset, int cqLength) { + return Bytes.compareTo(cq, 0, cq.length, cqBuf, cqOffset, cqLength); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/filter/SingleKeyValueComparisonFilter.java b/phoenix-core/src/main/java/org/apache/phoenix/filter/SingleKeyValueComparisonFilter.java new file mode 100644 index 00000000..1caa3325 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/filter/SingleKeyValueComparisonFilter.java @@ -0,0 +1,147 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.filter; + +import java.io.DataInput; +import java.io.IOException; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.KeyValueColumnExpression; +import org.apache.phoenix.schema.tuple.SingleKeyValueTuple; + + + +/** + * + * Modeled after {@link org.apache.hadoop.hbase.filter.SingleColumnValueFilter}, + * but for general expression evaluation in the case where only a single KeyValue + * column is referenced in the expression. + * + * @author jtaylor + * @since 0.1 + */ +public abstract class SingleKeyValueComparisonFilter extends BooleanExpressionFilter { + private final SingleKeyValueTuple inputTuple = new SingleKeyValueTuple(); + private boolean matchedColumn; + protected byte[] cf; + protected byte[] cq; + + public SingleKeyValueComparisonFilter() { + } + + public SingleKeyValueComparisonFilter(Expression expression) { + super(expression); + init(); + } + + protected abstract int compare(byte[] cfBuf, int cfOffset, int cfLength, byte[] cqBuf, int cqOffset, int cqLength); + + private void init() { + EvaluateOnCompletionVisitor visitor = new EvaluateOnCompletionVisitor() { + @Override + public Void visit(KeyValueColumnExpression expression) { + cf = expression.getColumnFamily(); + cq = expression.getColumnName(); + return null; + } + }; + expression.accept(visitor); + this.evaluateOnCompletion = visitor.evaluateOnCompletion(); + } + + private boolean foundColumn() { + return inputTuple.size() > 0; + } + + @Override + public ReturnCode filterKeyValue(KeyValue keyValue) { + if (this.matchedColumn) { + // We already found and matched the single column, all keys now pass + // TODO: why won't this cause earlier versions of a kv to be included? + return ReturnCode.INCLUDE; + } + if (this.foundColumn()) { + // We found all the columns, but did not match the expression, so skip to next row + return ReturnCode.NEXT_ROW; + } + byte[] buf = keyValue.getBuffer(); + if (compare(buf, keyValue.getFamilyOffset(), keyValue.getFamilyLength(), buf, keyValue.getQualifierOffset(), keyValue.getQualifierLength()) != 0) { + // Remember the key in case this is the only key value we see. + // We'll need it if we have row key columns too. + inputTuple.setKey(keyValue); + // This is a key value we're not interested in + // TODO: use NEXT_COL when bug fix comes through that includes the row still + return ReturnCode.INCLUDE; + } + inputTuple.setKeyValue(keyValue); + + // We have the columns, so evaluate here + if (!Boolean.TRUE.equals(evaluate(inputTuple))) { + return ReturnCode.NEXT_ROW; + } + this.matchedColumn = true; + return ReturnCode.INCLUDE; + } + + @Override + public boolean filterRow() { + // If column was found, return false if it was matched, true if it was not. + if (foundColumn()) { + return !this.matchedColumn; + } + // If column was not found, evaluate the expression here upon completion. + // This is required with certain expressions, for example, with IS NULL + // expressions where they'll evaluate to TRUE when the column being + // tested wasn't found. + // Since the filter is called also to position the scan initially, we have + // to guard against this by checking whether or not we've filtered in + // the key value (i.e. filterKeyValue was called and we found the keyValue + // for which we're looking). + if (inputTuple.hasKey() && evaluateOnCompletion()) { + return !Boolean.TRUE.equals(evaluate(inputTuple)); + } + // Finally, if we have no values, and we're not required to re-evaluate it + // just filter the row + return true; + } + + @Override + public void reset() { + inputTuple.reset(); + matchedColumn = false; + super.reset(); + } + + @Override + public void readFields(DataInput input) throws IOException { + super.readFields(input); + init(); + } + + @SuppressWarnings("all") // suppressing missing @Override since this doesn't exist for HBase 0.94.4 + public boolean isFamilyEssential(byte[] name) { + // Only the column families involved in the expression are essential. + // The others are for columns projected in the select expression + return Bytes.compareTo(cf, name) == 0; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/SkipScanFilter.java b/phoenix-core/src/main/java/org/apache/phoenix/filter/SkipScanFilter.java similarity index 90% rename from phoenix-core/src/main/java/com/salesforce/phoenix/filter/SkipScanFilter.java rename to phoenix-core/src/main/java/org/apache/phoenix/filter/SkipScanFilter.java index 4ac68213..ac8f174d 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/filter/SkipScanFilter.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/filter/SkipScanFilter.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. +/* + * Copyright 2010 The Apache Software Foundation * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. + * http://www.apache.org/licenses/LICENSE-2.0 * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.filter; + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.filter; import java.io.DataInput; import java.io.DataOutput; @@ -45,13 +37,13 @@ import com.google.common.hash.HashFunction; import com.google.common.hash.Hasher; import com.google.common.hash.Hashing; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.query.KeyRange.Bound; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.RowKeySchema; -import com.salesforce.phoenix.schema.ValueSchema.Field; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.ScanUtil; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.query.KeyRange.Bound; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.RowKeySchema; +import org.apache.phoenix.schema.ValueSchema.Field; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.ScanUtil; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/index/BaseIndexCodec.java b/phoenix-core/src/main/java/org/apache/phoenix/index/BaseIndexCodec.java new file mode 100644 index 00000000..c06beccb --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/index/BaseIndexCodec.java @@ -0,0 +1,61 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.index; + +import java.io.IOException; + +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; + +import org.apache.hadoop.hbase.index.covered.IndexCodec; + +/** + * + */ +public abstract class BaseIndexCodec implements IndexCodec { + + @Override + public void initialize(RegionCoprocessorEnvironment env) throws IOException { + // noop + } + + /** + * {@inheritDoc} + *

+ * By default, the codec is always enabled. Subclasses should override this method if they want do + * decide to index on a per-mutation basis. + * @throws IOException + */ + @Override + public boolean isEnabled(Mutation m) throws IOException { + return true; + } + + /** + * {@inheritDoc} + *

+ * Assumes each mutation is not in a batch. Subclasses that have different batching behavior + * should override this. + */ + @Override + public byte[] getBatchId(Mutation m) { + return null; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/index/IndexMaintainer.java b/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java similarity index 92% rename from phoenix-core/src/main/java/com/salesforce/phoenix/index/IndexMaintainer.java rename to phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java index 89f8413d..99d0b207 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/index/IndexMaintainer.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMaintainer.java @@ -1,19 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.index; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.index; import java.io.ByteArrayInputStream; import java.io.DataInput; @@ -43,27 +47,27 @@ import com.google.common.collect.Lists; import com.google.common.collect.Maps; import com.google.common.collect.Sets; -import com.salesforce.hbase.index.ValueGetter; -import com.salesforce.hbase.index.covered.update.ColumnReference; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.client.KeyValueBuilder; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PColumnFamily; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PIndexState; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.RowKeySchema; -import com.salesforce.phoenix.schema.SaltingUtil; -import com.salesforce.phoenix.schema.ValueSchema; -import com.salesforce.phoenix.schema.ValueSchema.Field; -import com.salesforce.phoenix.util.BitSet; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.IndexUtil; -import com.salesforce.phoenix.util.SchemaUtil; -import com.salesforce.phoenix.util.TrustedByteArrayOutputStream; +import org.apache.hadoop.hbase.index.ValueGetter; +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.client.KeyValueBuilder; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PColumnFamily; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PIndexState; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.RowKeySchema; +import org.apache.phoenix.schema.SaltingUtil; +import org.apache.phoenix.schema.ValueSchema; +import org.apache.phoenix.schema.ValueSchema.Field; +import org.apache.phoenix.util.BitSet; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.IndexUtil; +import org.apache.phoenix.util.SchemaUtil; +import org.apache.phoenix.util.TrustedByteArrayOutputStream; /** * @@ -71,9 +75,9 @@ * row and caches any covered columns. Client-side serializes into byte array using * @link #serialize(PTable, ImmutableBytesWritable)} * and transmits to server-side through either the - * {@link com.salesforce.phoenix.index.PhoenixIndexCodec#INDEX_MD} + * {@link org.apache.phoenix.index.PhoenixIndexCodec#INDEX_MD} * Mutation attribute or as a separate RPC call using - * {@link com.salesforce.phoenix.cache.ServerCacheClient}) + * {@link org.apache.phoenix.cache.ServerCacheClient}) * * @author jtaylor * @since 2.1.0 diff --git a/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMetaDataCacheClient.java b/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMetaDataCacheClient.java new file mode 100644 index 00000000..97ba55b1 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMetaDataCacheClient.java @@ -0,0 +1,98 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.index; + +import static org.apache.phoenix.query.QueryServices.INDEX_MUTATE_BATCH_SIZE_THRESHOLD_ATTRIB; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.cache.ServerCacheClient; +import org.apache.phoenix.cache.ServerCacheClient.ServerCache; +import org.apache.phoenix.compile.ScanRanges; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.join.MaxServerCacheSizeExceededException; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.ReadOnlyProps; +import org.apache.phoenix.util.ScanUtil; + +public class IndexMetaDataCacheClient { + + private final ServerCacheClient serverCache; + private TableRef cacheUsingTableRef; + + /** + * Construct client used to send index metadata to each region server + * for caching during batched put for secondary index maintenance. + * @param connection the client connection + * @param cacheUsingTableRef table ref to table that will use the cache during its scan + */ + public IndexMetaDataCacheClient(PhoenixConnection connection, TableRef cacheUsingTableRef) { + serverCache = new ServerCacheClient(connection); + this.cacheUsingTableRef = cacheUsingTableRef; + } + + /** + * Determines whether or not to use the IndexMetaDataCache to send the index metadata + * to the region servers. The alternative is to just set the index metadata as an attribute on + * the mutations. + * @param connection + * @param mutations the list of mutations that will be sent in a batch to server + * @param indexMetaDataByteLength length in bytes of the index metadata cache + */ + public static boolean useIndexMetadataCache(PhoenixConnection connection, List mutations, int indexMetaDataByteLength) { + ReadOnlyProps props = connection.getQueryServices().getProps(); + int threshold = props.getInt(INDEX_MUTATE_BATCH_SIZE_THRESHOLD_ATTRIB, QueryServicesOptions.DEFAULT_INDEX_MUTATE_BATCH_SIZE_THRESHOLD); + return (indexMetaDataByteLength > ServerCacheClient.UUID_LENGTH && mutations.size() > threshold); + } + + /** + * Send the index metadata cahce to all region servers for regions that will handle the mutations. + * @return client-side {@link ServerCache} representing the added index metadata cache + * @throws SQLException + * @throws MaxServerCacheSizeExceededException if size of hash cache exceeds max allowed + * size + */ + public ServerCache addIndexMetadataCache(List mutations, ImmutableBytesWritable ptr) throws SQLException { + /** + * Serialize and compress hashCacheTable + */ + return serverCache.addServerCache(ScanUtil.newScanRanges(mutations), ptr, new IndexMetaDataCacheFactory(), cacheUsingTableRef); + } + + + /** + * Send the index metadata cahce to all region servers for regions that will handle the mutations. + * @return client-side {@link ServerCache} representing the added index metadata cache + * @throws SQLException + * @throws MaxServerCacheSizeExceededException if size of hash cache exceeds max allowed + * size + */ + public ServerCache addIndexMetadataCache(ScanRanges ranges, ImmutableBytesWritable ptr) throws SQLException { + /** + * Serialize and compress hashCacheTable + */ + return serverCache.addServerCache(ranges, ptr, new IndexMetaDataCacheFactory(), cacheUsingTableRef); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMetaDataCacheFactory.java b/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMetaDataCacheFactory.java new file mode 100644 index 00000000..3be88332 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/index/IndexMetaDataCacheFactory.java @@ -0,0 +1,66 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.index; + +import java.io.Closeable; +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.cache.IndexMetaDataCache; +import org.apache.phoenix.client.GenericKeyValueBuilder; +import org.apache.phoenix.coprocessor.ServerCachingProtocol.ServerCacheFactory; +import org.apache.phoenix.memory.MemoryManager.MemoryChunk; + +public class IndexMetaDataCacheFactory implements ServerCacheFactory { + public IndexMetaDataCacheFactory() { + } + + @Override + public void readFields(DataInput arg0) throws IOException { + } + + @Override + public void write(DataOutput arg0) throws IOException { + } + + @Override + public Closeable newCache (ImmutableBytesWritable cachePtr, final MemoryChunk chunk) throws SQLException { + // just use the standard keyvalue builder - this doesn't really need to be fast + final List maintainers = + IndexMaintainer.deserialize(cachePtr, GenericKeyValueBuilder.INSTANCE); + return new IndexMetaDataCache() { + + @Override + public void close() throws IOException { + chunk.close(); + } + + @Override + public List getIndexMaintainers() { + return maintainers; + } + }; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexBuilder.java b/phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexBuilder.java new file mode 100644 index 00000000..e278b24e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexBuilder.java @@ -0,0 +1,96 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.index; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.regionserver.HRegion; +import org.apache.hadoop.hbase.regionserver.MiniBatchOperationInProgress; +import org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl; +import org.apache.hadoop.hbase.regionserver.RegionScanner; +import org.apache.hadoop.hbase.util.Pair; + +import com.google.common.collect.Lists; +import org.apache.hadoop.hbase.index.covered.CoveredColumnsIndexBuilder; +import org.apache.hadoop.hbase.index.util.IndexManagementUtil; +import org.apache.phoenix.compile.ScanRanges; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.util.SchemaUtil; + +/** + * Index builder for covered-columns index that ties into phoenix for faster use. + */ +public class PhoenixIndexBuilder extends CoveredColumnsIndexBuilder { + + @Override + public void batchStarted(MiniBatchOperationInProgress> miniBatchOp) throws IOException { + // The entire purpose of this method impl is to get the existing rows for the + // table rows being indexed into the block cache, as the index maintenance code + // does a point scan per row + List keys = Lists.newArrayListWithExpectedSize(miniBatchOp.size()); + List maintainers = new ArrayList(); + for (int i = 0; i < miniBatchOp.size(); i++) { + Mutation m = miniBatchOp.getOperation(i).getFirst(); + keys.add(PDataType.VARBINARY.getKeyRange(m.getRow())); + maintainers.addAll(getCodec().getIndexMaintainers(m.getAttributesMap())); + } + Scan scan = IndexManagementUtil.newLocalStateScan(maintainers); + ScanRanges scanRanges = ScanRanges.create(Collections.singletonList(keys), SchemaUtil.VAR_BINARY_SCHEMA); + scanRanges.setScanStartStopRow(scan); + scan.setFilter(scanRanges.getSkipScanFilter()); + HRegion region = this.env.getRegion(); + RegionScanner scanner = region.getScanner(scan); + // Run through the scanner using internal nextRaw method + MultiVersionConsistencyControl.setThreadReadPoint(scanner.getMvccReadPoint()); + region.startRegionOperation(); + try { + boolean hasMore; + do { + List results = Lists.newArrayList(); + // Results are potentially returned even when the return value of s.next is false + // since this is an indication of whether or not there are more values after the + // ones returned + hasMore = scanner.nextRaw(results, null); + } while (hasMore); + } finally { + try { + scanner.close(); + } finally { + region.closeRegionOperation(); + } + } + } + + private PhoenixIndexCodec getCodec() { + return (PhoenixIndexCodec)this.codec; + } + + @Override + public byte[] getBatchId(Mutation m){ + return this.codec.getBatchId(m); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexCodec.java b/phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexCodec.java new file mode 100644 index 00000000..4f98a32b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexCodec.java @@ -0,0 +1,182 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.index; + +import java.io.IOException; +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; +import java.util.Map; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.client.Delete; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.client.Put; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Pair; + +import com.google.common.collect.Lists; +import org.apache.hadoop.hbase.index.ValueGetter; +import org.apache.hadoop.hbase.index.covered.IndexCodec; +import org.apache.hadoop.hbase.index.covered.IndexUpdate; +import org.apache.hadoop.hbase.index.covered.TableState; +import org.apache.hadoop.hbase.index.scanner.Scanner; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.util.IndexManagementUtil; +import org.apache.hadoop.hbase.index.write.IndexWriter; +import org.apache.phoenix.cache.GlobalCache; +import org.apache.phoenix.cache.IndexMetaDataCache; +import org.apache.phoenix.cache.ServerCacheClient; +import org.apache.phoenix.cache.TenantCache; +import org.apache.phoenix.client.KeyValueBuilder; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.util.PhoenixRuntime; +import org.apache.phoenix.util.ServerUtil; + +/** + * Phoenix-based {@link IndexCodec}. Manages all the logic of how to cleanup an index ( + * {@link #getIndexDeletes(TableState)}) as well as what the new index state should be ( + * {@link #getIndexUpserts(TableState)}). + */ +public class PhoenixIndexCodec extends BaseIndexCodec { + public static final String INDEX_MD = "IdxMD"; + public static final String INDEX_UUID = "IdxUUID"; + + private RegionCoprocessorEnvironment env; + private KeyValueBuilder builder; + + @Override + public void initialize(RegionCoprocessorEnvironment env) { + this.env = env; + Configuration conf = env.getConfiguration(); + // Install handler that will attempt to disable the index first before killing the region + // server + conf.setIfUnset(IndexWriter.INDEX_FAILURE_POLICY_CONF_KEY, + PhoenixIndexFailurePolicy.class.getName()); + this.builder = KeyValueBuilder.get(env.getHBaseVersion()); + } + + List getIndexMaintainers(Map attributes) throws IOException{ + if (attributes == null) { + return Collections.emptyList(); + } + byte[] uuid = attributes.get(INDEX_UUID); + if (uuid == null) { + return Collections.emptyList(); + } + byte[] md = attributes.get(INDEX_MD); + List indexMaintainers; + if (md != null) { + indexMaintainers = IndexMaintainer.deserialize(md, builder); + } else { + byte[] tenantIdBytes = attributes.get(PhoenixRuntime.TENANT_ID_ATTRIB); + ImmutableBytesWritable tenantId = + tenantIdBytes == null ? null : new ImmutableBytesWritable(tenantIdBytes); + TenantCache cache = GlobalCache.getTenantCache(env, tenantId); + IndexMetaDataCache indexCache = + (IndexMetaDataCache) cache.getServerCache(new ImmutableBytesPtr(uuid)); + if (indexCache == null) { + String msg = "key="+ServerCacheClient.idToString(uuid) + " region=" + env.getRegion(); + SQLException e = new SQLExceptionInfo.Builder(SQLExceptionCode.INDEX_METADATA_NOT_FOUND) + .setMessage(msg).build().buildException(); + ServerUtil.throwIOException("Index update failed", e); // will not return + } + indexMaintainers = indexCache.getIndexMaintainers(); + } + + return indexMaintainers; + } + + @Override + public Iterable getIndexUpserts(TableState state) throws IOException { + List indexMaintainers = getIndexMaintainers(state.getUpdateAttributes()); + if (indexMaintainers.isEmpty()) { + return Collections.emptyList(); + } + ImmutableBytesWritable ptr = new ImmutableBytesWritable(); + List indexUpdates = Lists.newArrayList(); + // TODO: state.getCurrentRowKey() should take an ImmutableBytesWritable arg to prevent byte copy + byte[] dataRowKey = state.getCurrentRowKey(); + for (IndexMaintainer maintainer : indexMaintainers) { + // Short-circuit building state when we know it's a row deletion + if (maintainer.isRowDeleted(state.getPendingUpdate())) { + continue; + } + + // Get a scanner over the columns this maintainer would like to look at + // Any updates that we would make for those columns are then added to the index update + Pair statePair = state.getIndexedColumnsTableState(maintainer.getAllColumns()); + IndexUpdate indexUpdate = statePair.getSecond(); + Scanner scanner = statePair.getFirst(); + + // get the values from the scanner so we can actually use them + ValueGetter valueGetter = IndexManagementUtil.createGetterFromScanner(scanner, dataRowKey); + ptr.set(dataRowKey); + Put put = maintainer.buildUpdateMutation(valueGetter, ptr, state.getCurrentTimestamp()); + indexUpdate.setTable(maintainer.getIndexTableName()); + indexUpdate.setUpdate(put); + //make sure we close the scanner when we are done + scanner.close(); + indexUpdates.add(indexUpdate); + } + return indexUpdates; + } + + @Override + public Iterable getIndexDeletes(TableState state) throws IOException { + List indexMaintainers = getIndexMaintainers(state.getUpdateAttributes()); + if (indexMaintainers.isEmpty()) { + return Collections.emptyList(); + } + List indexUpdates = Lists.newArrayList(); + ImmutableBytesWritable ptr = new ImmutableBytesWritable(); + // TODO: state.getCurrentRowKey() should take an ImmutableBytesWritable arg to prevent byte copy + byte[] dataRowKey = state.getCurrentRowKey(); + for (IndexMaintainer maintainer : indexMaintainers) { + // TODO: if more efficient, I could do this just once with all columns in all indexes + Pair statePair = state.getIndexedColumnsTableState(maintainer.getAllColumns()); + Scanner scanner = statePair.getFirst(); + IndexUpdate indexUpdate = statePair.getSecond(); + indexUpdate.setTable(maintainer.getIndexTableName()); + ValueGetter valueGetter = IndexManagementUtil.createGetterFromScanner(scanner, dataRowKey); + ptr.set(dataRowKey); + Delete delete = + maintainer.buildDeleteMutation(valueGetter, ptr, state.getPendingUpdate(), + state.getCurrentTimestamp()); + scanner.close(); + indexUpdate.setUpdate(delete); + indexUpdates.add(indexUpdate); + } + return indexUpdates; + } + + @Override + public boolean isEnabled(Mutation m) throws IOException { + return !getIndexMaintainers(m.getAttributesMap()).isEmpty(); + } + + @Override + public byte[] getBatchId(Mutation m) { + Map attributes = m.getAttributesMap(); + return attributes.get(INDEX_UUID); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexFailurePolicy.java b/phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexFailurePolicy.java new file mode 100644 index 00000000..3f56bb8b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexFailurePolicy.java @@ -0,0 +1,102 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.index; + +import java.io.IOException; +import java.util.Collections; +import java.util.List; +import java.util.Set; + +import org.apache.commons.logging.Log; +import org.apache.commons.logging.LogFactory; +import org.apache.hadoop.hbase.DoNotRetryIOException; +import org.apache.hadoop.hbase.Stoppable; +import org.apache.hadoop.hbase.client.HTableInterface; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.client.Put; +import org.apache.hadoop.hbase.coprocessor.RegionCoprocessorEnvironment; + +import com.google.common.collect.Multimap; +import org.apache.hadoop.hbase.index.table.HTableInterfaceReference; +import org.apache.hadoop.hbase.index.write.KillServerOnFailurePolicy; +import org.apache.phoenix.coprocessor.MetaDataProtocol; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MutationCode; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.schema.PIndexState; +import org.apache.phoenix.util.SchemaUtil; + +/** + * + * Handler called in the event that index updates cannot be written to their + * region server. First attempts to disable the index and failing that falls + * back to the default behavior of killing the region server. + * + * TODO: use delegate pattern instead + * + * @author jtaylor + * @since 2.1 + */ +public class PhoenixIndexFailurePolicy extends KillServerOnFailurePolicy { + private static final Log LOG = LogFactory.getLog(PhoenixIndexFailurePolicy.class); + private RegionCoprocessorEnvironment env; + + public PhoenixIndexFailurePolicy() { + } + + @Override + public void setup(Stoppable parent, RegionCoprocessorEnvironment env) { + super.setup(parent, env); + this.env = env; + } + + @Override + public void handleFailure(Multimap attempted, Exception cause) throws IOException { + Set refs = attempted.asMap().keySet(); + StringBuilder buf = new StringBuilder("Disabled index" + (refs.size() > 1 ? "es " : " ")); + try { + for (HTableInterfaceReference ref : refs) { + // Disable the index by using the updateIndexState method of MetaDataProtocol end point coprocessor. + String indexTableName = ref.getTableName(); + byte[] indexTableKey = SchemaUtil.getTableKeyFromFullName(indexTableName); + HTableInterface systemTable = env.getTable(PhoenixDatabaseMetaData.TYPE_TABLE_NAME_BYTES); + MetaDataProtocol mdProxy = systemTable.coprocessorProxy(MetaDataProtocol.class, indexTableKey); + // Mimic the Put that gets generated by the client on an update of the index state + Put put = new Put(indexTableKey); + put.add(PhoenixDatabaseMetaData.TABLE_FAMILY_BYTES, PhoenixDatabaseMetaData.INDEX_STATE_BYTES, PIndexState.DISABLE.getSerializedBytes()); + List tableMetadata = Collections.singletonList(put); + MetaDataMutationResult result = mdProxy.updateIndexState(tableMetadata); + if (result.getMutationCode() != MutationCode.TABLE_ALREADY_EXISTS) { + LOG.warn("Attempt to disable index " + indexTableName + " failed with code = " + result.getMutationCode() + ". Will use default failure policy instead."); + super.handleFailure(attempted, cause); + } + LOG.info("Successfully disabled index " + indexTableName); + buf.append(indexTableName); + buf.append(','); + } + buf.setLength(buf.length()-1); + buf.append(" due to an exception while writing updates"); + } catch (Throwable t) { + super.handleFailure(attempted, cause); + } + throw new DoNotRetryIOException(buf.toString(), cause); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/AggregatingResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/AggregatingResultIterator.java new file mode 100644 index 00000000..237a89ad --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/AggregatingResultIterator.java @@ -0,0 +1,40 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Interface for scanners that either do aggregation + * or delegate to scanners that do aggregation. + * + * @author jtaylor + * @since 0.1 + */ +public interface AggregatingResultIterator extends ResultIterator { + /** + * Provides a means of re-aggregating a result row. For + * scanners that need to look ahead (i.e. {@link org.apache.phoenix.iterate.OrderedAggregatingResultIterator} + * @param result the row to re-aggregate + */ + void aggregate(Tuple result); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterator.java new file mode 100644 index 00000000..ed6c3600 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/BaseResultIterator.java @@ -0,0 +1,43 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +/** + * + * Abstract base class for ResultIterator implementations that + * do nothing on close and have no explain plan steps + * + * @author jtaylor + * @since 1.2 + */ +public abstract class BaseResultIterator implements ResultIterator { + + @Override + public void close() throws SQLException { + } + + @Override + public void explain(List planSteps) { + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/ConcatResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ConcatResultIterator.java new file mode 100644 index 00000000..08b4639a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ConcatResultIterator.java @@ -0,0 +1,91 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Result iterator that concatenates a list of other iterators. + * + * @author jtaylor + * @since 0.1 + */ +public class ConcatResultIterator implements PeekingResultIterator { + private final ResultIterators resultIterators; + private List iterators; + private int index; + + public ConcatResultIterator(ResultIterators iterators) { + this.resultIterators = iterators; + } + + private List getIterators() throws SQLException { + if (iterators == null) { + iterators = resultIterators.getIterators(); + } + return iterators; + } + + @Override + public void close() throws SQLException { + if (iterators != null) { + for (;index < iterators.size(); index++) { + PeekingResultIterator iterator = iterators.get(index); + iterator.close(); + } + } + } + + + @Override + public void explain(List planSteps) { + resultIterators.explain(planSteps); + } + + private PeekingResultIterator currentIterator() throws SQLException { + List iterators = getIterators(); + while (index < iterators.size()) { + PeekingResultIterator iterator = iterators.get(index); + Tuple r = iterator.peek(); + if (r != null) { + return iterator; + } + iterator.close(); + index++; + } + return EMPTY_ITERATOR; + } + + @Override + public Tuple peek() throws SQLException { + return currentIterator().peek(); + } + + @Override + public Tuple next() throws SQLException { + return currentIterator().next(); + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/DefaultParallelIteratorRegionSplitter.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/DefaultParallelIteratorRegionSplitter.java similarity index 81% rename from phoenix-core/src/main/java/com/salesforce/phoenix/iterate/DefaultParallelIteratorRegionSplitter.java rename to phoenix-core/src/main/java/org/apache/phoenix/iterate/DefaultParallelIteratorRegionSplitter.java index 003bd74f..ab194f6d 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/DefaultParallelIteratorRegionSplitter.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/DefaultParallelIteratorRegionSplitter.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; import java.sql.SQLException; import java.util.Collection; @@ -43,16 +35,16 @@ import com.google.common.collect.Iterables; import com.google.common.collect.ListMultimap; import com.google.common.collect.Lists; -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.parse.HintNode; -import com.salesforce.phoenix.parse.HintNode.Hint; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.query.StatsManager; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.ReadOnlyProps; +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.parse.HintNode; +import org.apache.phoenix.parse.HintNode.Hint; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.query.StatsManager; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.ReadOnlyProps; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/DelegateResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/DelegateResultIterator.java new file mode 100644 index 00000000..25df5ada --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/DelegateResultIterator.java @@ -0,0 +1,50 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.schema.tuple.Tuple; + + +public class DelegateResultIterator implements ResultIterator { + private final ResultIterator delegate; + + public DelegateResultIterator(ResultIterator delegate) { + this.delegate = delegate; + } + + @Override + public void close() throws SQLException { + delegate.close(); + } + + @Override + public Tuple next() throws SQLException { + return delegate.next(); + } + + @Override + public void explain(List planSteps) { + delegate.explain(planSteps); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/DistinctAggregatingResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/DistinctAggregatingResultIterator.java new file mode 100644 index 00000000..f7af26cc --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/DistinctAggregatingResultIterator.java @@ -0,0 +1,163 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.*; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import com.google.common.collect.Iterators; +import com.google.common.collect.Sets; +import org.apache.phoenix.compile.ColumnProjector; +import org.apache.phoenix.compile.RowProjector; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * Result scanner that dedups the incoming tuples to make them distinct. + *

+ * Note that the results are held in memory + * + * @author jtaylor + * @since 1.2 + */ +public class DistinctAggregatingResultIterator implements AggregatingResultIterator { + private final AggregatingResultIterator delegate; + private final RowProjector rowProjector; + private Iterator resultIterator; + private final ImmutableBytesWritable ptr1 = new ImmutableBytesWritable(); + private final ImmutableBytesWritable ptr2 = new ImmutableBytesWritable(); + + private class ResultEntry { + private final int hashCode; + private final Tuple result; + + ResultEntry(Tuple result) { + final int prime = 31; + this.result = result; + int hashCode = 0; + for (ColumnProjector column : rowProjector.getColumnProjectors()) { + Expression e = column.getExpression(); + if (e.evaluate(this.result, ptr1)) { + hashCode = prime * hashCode + ptr1.hashCode(); + } + } + this.hashCode = hashCode; + } + + @Override + public boolean equals(Object o) { + if (o == this) { + return true; + } + if (o == null) { + return false; + } + if (o.getClass() != this.getClass()) { + return false; + } + ResultEntry that = (ResultEntry) o; + for (ColumnProjector column : rowProjector.getColumnProjectors()) { + Expression e = column.getExpression(); + boolean isNull1 = !e.evaluate(this.result, ptr1); + boolean isNull2 = !e.evaluate(that.result, ptr2); + if (isNull1 && isNull2) { + return true; + } + if (isNull1 || isNull2) { + return false; + } + if (ptr1.compareTo(ptr2) != 0) { + return false; + } + } + return true; + } + + @Override + public int hashCode() { + return hashCode; + } + + Tuple getResult() { + return result; + } + } + + protected ResultIterator getDelegate() { + return delegate; + } + + public DistinctAggregatingResultIterator(AggregatingResultIterator delegate, + RowProjector rowProjector) { + this.delegate = delegate; + this.rowProjector = rowProjector; + } + + @Override + public Tuple next() throws SQLException { + Iterator iterator = getResultIterator(); + if (iterator.hasNext()) { + ResultEntry entry = iterator.next(); + Tuple tuple = entry.getResult(); + aggregate(tuple); + return tuple; + } + resultIterator = Iterators.emptyIterator(); + return null; + } + + private Iterator getResultIterator() throws SQLException { + if (resultIterator != null) { + return resultIterator; + } + + Set entries = Sets.newHashSet(); // TODO: size? + try { + for (Tuple result = delegate.next(); result != null; result = delegate.next()) { + ResultEntry entry = new ResultEntry(result); + entries.add(entry); + } + } finally { + delegate.close(); + } + + resultIterator = entries.iterator(); + return resultIterator; + } + + @Override + public void close() { + resultIterator = Iterators.emptyIterator(); + } + + + @Override + public void explain(List planSteps) { + delegate.explain(planSteps); + planSteps.add("CLIENT DISTINCT ON " + rowProjector.toString()); + } + + @Override + public void aggregate(Tuple result) { + delegate.aggregate(result); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ExplainTable.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ExplainTable.java similarity index 80% rename from phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ExplainTable.java rename to phoenix-core/src/main/java/org/apache/phoenix/iterate/ExplainTable.java index 7dfd1db7..f32b3776 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ExplainTable.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ExplainTable.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; import java.text.Format; import java.util.Collections; @@ -42,16 +34,16 @@ import org.apache.hadoop.hbase.util.Bytes; import com.google.common.collect.Iterators; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.ScanRanges; -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.query.KeyRange.Bound; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.RowKeySchema; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.StringUtil; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.ScanRanges; +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.query.KeyRange.Bound; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.RowKeySchema; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.StringUtil; public abstract class ExplainTable { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/FilterAggregatingResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/FilterAggregatingResultIterator.java new file mode 100644 index 00000000..d6874804 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/FilterAggregatingResultIterator.java @@ -0,0 +1,80 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Post aggregation filter for HAVING clause. Due to the way we cache aggregation values + * we cannot have a look ahead for this Iterator, because the expressions in the SELECT + * clause would return values for the peeked row instead of the current row. If we only + * use the Result argument in {@link org.apache.phoenix.expression.Expression} + * instead of our cached value in Aggregators, we could have a look ahead. + * + * @author jtaylor + * @since 0.1 + */ +public class FilterAggregatingResultIterator implements AggregatingResultIterator { + private final AggregatingResultIterator delegate; + private final Expression expression; + private final ImmutableBytesWritable ptr = new ImmutableBytesWritable(); + + public FilterAggregatingResultIterator(AggregatingResultIterator delegate, Expression expression) { + this.delegate = delegate; + this.expression = expression; + if (expression.getDataType() != PDataType.BOOLEAN) { + throw new IllegalArgumentException("FilterResultIterator requires a boolean expression, but got " + expression); + } + } + + @Override + public Tuple next() throws SQLException { + Tuple next; + do { + next = delegate.next(); + } while (next != null && expression.evaluate(next, ptr) && Boolean.FALSE.equals(expression.getDataType().toObject(ptr))); + return next; + } + + @Override + public void close() throws SQLException { + delegate.close(); + } + + @Override + public void aggregate(Tuple result) { + delegate.aggregate(result); + } + + @Override + public void explain(List planSteps) { + delegate.explain(planSteps); + planSteps.add("CLIENT FILTER BY " + expression.toString()); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/FilterResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/FilterResultIterator.java new file mode 100644 index 00000000..5518643d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/FilterResultIterator.java @@ -0,0 +1,78 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Result scanner that filters out rows based on the results of a boolean + * expression (i.e. filters out if {@link org.apache.phoenix.expression.Expression#evaluate(Tuple, ImmutableBytesWritable)} + * returns false or the ptr contains a FALSE value}). May not be used where + * the delegate provided is an {@link org.apache.phoenix.iterate.AggregatingResultIterator}. + * For these, the {@link org.apache.phoenix.iterate.FilterAggregatingResultIterator} should be used. + * + * @author jtaylor + * @since 0.1 + */ +public class FilterResultIterator extends LookAheadResultIterator { + private final ResultIterator delegate; + private final Expression expression; + private final ImmutableBytesWritable ptr = new ImmutableBytesWritable(); + + public FilterResultIterator(ResultIterator delegate, Expression expression) { + if (delegate instanceof AggregatingResultIterator) { + throw new IllegalArgumentException("FilterResultScanner may not be used with an aggregate delegate. Use phoenix.iterate.FilterAggregateResultScanner instead"); + } + this.delegate = delegate; + this.expression = expression; + if (expression.getDataType() != PDataType.BOOLEAN) { + throw new IllegalArgumentException("FilterResultIterator requires a boolean expression, but got " + expression); + } + } + + @Override + protected Tuple advance() throws SQLException { + Tuple next; + do { + next = delegate.next(); + } while (next != null && expression.evaluate(next, ptr) && Boolean.FALSE.equals(expression.getDataType().toObject(ptr))); + return next; + } + + @Override + public void close() throws SQLException { + delegate.close(); + } + + @Override + public void explain(List planSteps) { + delegate.explain(planSteps); + planSteps.add("CLIENT FILTER BY " + expression.toString()); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/GroupedAggregatingResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/GroupedAggregatingResultIterator.java new file mode 100644 index 00000000..79856998 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/GroupedAggregatingResultIterator.java @@ -0,0 +1,106 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import static org.apache.phoenix.query.QueryConstants.*; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.aggregator.Aggregator; +import org.apache.phoenix.expression.aggregator.Aggregators; +import org.apache.phoenix.schema.tuple.SingleKeyValueTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.KeyValueUtil; +import org.apache.phoenix.util.TupleUtil; + + + +/** + * + * Result scanner that aggregates the row count value for rows with duplicate keys. + * The rows from the backing result iterator must be in key sorted order. For example, + * given the following input: + * a 1 + * a 2 + * b 1 + * b 3 + * c 1 + * the following will be output: + * a 3 + * b 4 + * c 1 + * + * @author jtaylor + * @since 0.1 + */ +public class GroupedAggregatingResultIterator implements AggregatingResultIterator { + private final ImmutableBytesWritable tempPtr = new ImmutableBytesWritable(); + private final PeekingResultIterator resultIterator; + protected final Aggregators aggregators; + + public GroupedAggregatingResultIterator( PeekingResultIterator resultIterator, Aggregators aggregators) { + if (resultIterator == null) throw new NullPointerException(); + if (aggregators == null) throw new NullPointerException(); + this.resultIterator = resultIterator; + this.aggregators = aggregators; + } + + @Override + public Tuple next() throws SQLException { + Tuple result = resultIterator.next(); + if (result == null) { + return null; + } + Aggregator[] rowAggregators = aggregators.getAggregators(); + aggregators.reset(rowAggregators); + while (true) { + aggregators.aggregate(rowAggregators, result); + Tuple nextResult = resultIterator.peek(); + if (nextResult == null || !TupleUtil.equals(result, nextResult, tempPtr)) { + break; + } + result = resultIterator.next(); + } + + byte[] value = aggregators.toBytes(rowAggregators); + result.getKey(tempPtr); + return new SingleKeyValueTuple(KeyValueUtil.newKeyValue(tempPtr, SINGLE_COLUMN_FAMILY, SINGLE_COLUMN, AGG_TIMESTAMP, value, 0, value.length)); + } + + @Override + public void close() throws SQLException { + resultIterator.close(); + } + + @Override + public void aggregate(Tuple result) { + Aggregator[] rowAggregators = aggregators.getAggregators(); + aggregators.reset(rowAggregators); + aggregators.aggregate(rowAggregators, result); + } + + @Override + public void explain(List planSteps) { + resultIterator.explain(planSteps); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/LimitingResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/LimitingResultIterator.java new file mode 100644 index 00000000..ecea92d0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/LimitingResultIterator.java @@ -0,0 +1,57 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * + * Iterates through tuples up to a limit + * + * @author jtaylor + * @since 1.2 + */ +public class LimitingResultIterator extends DelegateResultIterator { + private int rowCount; + private final int limit; + + public LimitingResultIterator(ResultIterator delegate, int limit) { + super(delegate); + this.limit = limit; + } + + @Override + public Tuple next() throws SQLException { + if (rowCount++ >= limit) { + return null; + } + return super.next(); + } + + @Override + public void explain(List planSteps) { + super.explain(planSteps); + planSteps.add("CLIENT " + limit + " ROW LIMIT"); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/LookAheadResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/LookAheadResultIterator.java new file mode 100644 index 00000000..0e4b78b2 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/LookAheadResultIterator.java @@ -0,0 +1,53 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; + +import org.apache.phoenix.schema.tuple.ResultTuple; +import org.apache.phoenix.schema.tuple.Tuple; + + +abstract public class LookAheadResultIterator implements PeekingResultIterator { + private final static Tuple UNINITIALIZED = new ResultTuple(); + private Tuple next = UNINITIALIZED; + + abstract protected Tuple advance() throws SQLException; + + private void init() throws SQLException { + if (next == UNINITIALIZED) { + next = advance(); + } + } + + @Override + public Tuple next() throws SQLException { + init(); + Tuple next = this.next; + this.next = advance(); + return next; + } + + @Override + public Tuple peek() throws SQLException { + init(); + return next; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MappedByteBufferSortedQueue.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/MappedByteBufferSortedQueue.java similarity index 87% rename from phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MappedByteBufferSortedQueue.java rename to phoenix-core/src/main/java/org/apache/phoenix/iterate/MappedByteBufferSortedQueue.java index 8be164e2..1606ae6a 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/MappedByteBufferSortedQueue.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/MappedByteBufferSortedQueue.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; import java.io.File; import java.io.IOException; @@ -46,9 +38,9 @@ import org.apache.hadoop.hbase.util.Bytes; import com.google.common.collect.MinMaxPriorityQueue; -import com.salesforce.phoenix.iterate.OrderedResultIterator.ResultEntry; -import com.salesforce.phoenix.schema.tuple.ResultTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.iterate.OrderedResultIterator.ResultEntry; +import org.apache.phoenix.schema.tuple.ResultTuple; +import org.apache.phoenix.schema.tuple.Tuple; public class MappedByteBufferSortedQueue extends AbstractQueue { private Comparator comparator; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/MaterializedResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/MaterializedResultIterator.java new file mode 100644 index 00000000..c509b2ca --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/MaterializedResultIterator.java @@ -0,0 +1,112 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.*; + +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Fully materialized result iterator backed by the result list provided. + * No copy is made of the backing results collection. + * + * @author jtaylor + * @since 0.1 + */ +public class MaterializedResultIterator implements PeekingResultIterator { + private final PeekingCollectionIterator iterator; + + public MaterializedResultIterator(Collection results) { + iterator = new PeekingCollectionIterator(results); + } + + @Override + public void close() { + } + + @Override + public Tuple next() throws SQLException { + return iterator.nextOrNull(); + } + + @Override + public Tuple peek() throws SQLException { + return iterator.peek(); + } + + private static class PeekingCollectionIterator implements Iterator { + private final Iterator iterator; + private Tuple current; + + private PeekingCollectionIterator(Collection results) { + iterator = results.iterator(); + advance(); + } + + private Tuple advance() { + if (iterator.hasNext()) { + current = iterator.next(); + } else { + current = null; + } + return current; + } + + @Override + public boolean hasNext() { + return current != null; + } + + @Override + public Tuple next() { + Tuple next = nextOrNull(); + if (next == null) { + throw new NoSuchElementException(); + } + return next; + } + + public Tuple nextOrNull() { + if (current == null) { + return null; + } + Tuple next = current; + advance(); + return next; + } + + @Override + public void remove() { + throw new UnsupportedOperationException(); + } + + public Tuple peek() { + return current; + } + + } + + @Override + public void explain(List planSteps) { + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/MergeSortResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/MergeSortResultIterator.java new file mode 100644 index 00000000..58b7e40f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/MergeSortResultIterator.java @@ -0,0 +1,95 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.SQLCloseables; + + +/** + * + * Base class for a ResultIterator that does a merge sort on the list of iterators + * provided. + * + * @author jtaylor + * @since 1.2 + */ +public abstract class MergeSortResultIterator implements PeekingResultIterator { + protected final ResultIterators resultIterators; + protected final ImmutableBytesWritable tempPtr = new ImmutableBytesWritable(); + private List iterators; + + public MergeSortResultIterator(ResultIterators iterators) { + this.resultIterators = iterators; + } + + private List getIterators() throws SQLException { + if (iterators == null) { + iterators = resultIterators.getIterators(); + } + return iterators; + } + + @Override + public void close() throws SQLException { + if (iterators != null) { + SQLCloseables.closeAll(iterators); + } + } + + abstract protected int compare(Tuple t1, Tuple t2); + + private PeekingResultIterator minIterator() throws SQLException { + List iterators = getIterators(); + Tuple minResult = null; + PeekingResultIterator minIterator = EMPTY_ITERATOR; + for (int i = iterators.size()-1; i >= 0; i--) { + PeekingResultIterator iterator = iterators.get(i); + Tuple r = iterator.peek(); + if (r != null) { + if (minResult == null || compare(r, minResult) < 0) { + minResult = r; + minIterator = iterator; + } + continue; + } + iterator.close(); + iterators.remove(i); + } + return minIterator; + } + + @Override + public Tuple peek() throws SQLException { + PeekingResultIterator iterator = minIterator(); + return iterator.peek(); + } + + @Override + public Tuple next() throws SQLException { + PeekingResultIterator iterator = minIterator(); + return iterator.next(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/MergeSortRowKeyResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/MergeSortRowKeyResultIterator.java new file mode 100644 index 00000000..a411bab2 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/MergeSortRowKeyResultIterator.java @@ -0,0 +1,61 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.util.List; + +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.TupleUtil; + + +/** + * + * ResultIterator that does a merge sort on the list of iterators provided, + * returning the rows in row key ascending order. The iterators provided + * must be in row key ascending order. + * + * @author jtaylor + * @since 0.1 + */ +public class MergeSortRowKeyResultIterator extends MergeSortResultIterator { + private final int keyOffset; + private final int factor; + + public MergeSortRowKeyResultIterator(ResultIterators iterators) { + this(iterators, 0, false); + } + + public MergeSortRowKeyResultIterator(ResultIterators iterators, int keyOffset, boolean isReverse) { + super(iterators); + this.keyOffset = keyOffset; + this.factor = isReverse ? -1 : 1; + } + + @Override + protected int compare(Tuple t1, Tuple t2) { + return factor * TupleUtil.compare(t1, t2, tempPtr, keyOffset); + } + + @Override + public void explain(List planSteps) { + resultIterators.explain(planSteps); + planSteps.add("CLIENT MERGE SORT"); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/MergeSortTopNResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/MergeSortTopNResultIterator.java new file mode 100644 index 00000000..77d1c62a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/MergeSortTopNResultIterator.java @@ -0,0 +1,100 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.OrderByExpression; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * + * ResultIterator that does a merge sort on the list of iterators provided, + * returning the rows ordered by the OrderByExpression. The input + * iterators must be ordered by the OrderByExpression. + * + * @author jtaylor + * @since 0.1 + */ +public class MergeSortTopNResultIterator extends MergeSortResultIterator { + + private final int limit; + private int count = 0; + private final List orderByColumns; + private final ImmutableBytesWritable ptr1 = new ImmutableBytesWritable(); + private final ImmutableBytesWritable ptr2 = new ImmutableBytesWritable(); + + public MergeSortTopNResultIterator(ResultIterators iterators, Integer limit, List orderByColumns) { + super(iterators); + this.limit = limit == null ? -1 : limit; + this.orderByColumns = orderByColumns; + } + + @Override + protected int compare(Tuple t1, Tuple t2) { + for (int i = 0; i < orderByColumns.size(); i++) { + OrderByExpression order = orderByColumns.get(i); + Expression orderExpr = order.getExpression(); + boolean isNull1 = !orderExpr.evaluate(t1, ptr1) || ptr1.getLength() == 0; + boolean isNull2 = !orderExpr.evaluate(t2, ptr2) || ptr2.getLength() == 0; + if (isNull1 && isNull2) { + continue; + } else if (isNull1) { + return order.isNullsLast() ? 1 : -1; + } else if (isNull2) { + return order.isNullsLast() ? -1 : 1; + } + int cmp = ptr1.compareTo(ptr2); + if (cmp == 0) { + continue; + } + return order.isAscending() ? cmp : -cmp; + } + return 0; + } + + @Override + public Tuple peek() throws SQLException { + if (limit >= 0 && count >= limit) { + return null; + } + return super.peek(); + } + + @Override + public Tuple next() throws SQLException { + if (limit >= 0 && count++ >= limit) { + return null; + } + return super.next(); + } + + + @Override + public void explain(List planSteps) { + resultIterators.explain(planSteps); + planSteps.add(" SERVER TOP " + limit + " ROW" + (limit == 1 ? "" : "S") + " SORTED BY " + orderByColumns.toString()); + planSteps.add("CLIENT MERGE SORT"); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/OrderedAggregatingResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/OrderedAggregatingResultIterator.java new file mode 100644 index 00000000..2b7011ad --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/OrderedAggregatingResultIterator.java @@ -0,0 +1,63 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.expression.OrderByExpression; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * Result scanner that sorts aggregated rows by columns specified in the ORDER BY clause. + *

+ * Note that currently the sort is entirely done in memory. + * + * @author syyang + * @since 0.1 + */ +public class OrderedAggregatingResultIterator extends OrderedResultIterator implements AggregatingResultIterator { + + public OrderedAggregatingResultIterator(AggregatingResultIterator delegate, + List orderByExpressions, + int thresholdBytes, Integer limit) throws SQLException { + super (delegate, orderByExpressions, thresholdBytes, limit); + } + + @Override + protected AggregatingResultIterator getDelegate() { + return (AggregatingResultIterator)super.getDelegate(); + } + + @Override + public Tuple next() throws SQLException { + Tuple tuple = super.next(); + if (tuple != null) { + aggregate(tuple); + } + return tuple; + } + + @Override + public void aggregate(Tuple result) { + getDelegate().aggregate(result); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/OrderedResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/OrderedResultIterator.java similarity index 81% rename from phoenix-core/src/main/java/com/salesforce/phoenix/iterate/OrderedResultIterator.java rename to phoenix-core/src/main/java/org/apache/phoenix/iterate/OrderedResultIterator.java index 7aa56416..288c4659 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/OrderedResultIterator.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/OrderedResultIterator.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkPositionIndex; @@ -42,10 +34,10 @@ import com.google.common.collect.Collections2; import com.google.common.collect.Lists; import com.google.common.collect.Ordering; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.OrderByExpression; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.SizedUtil; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.OrderByExpression; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.SizedUtil; /** * Result scanner that sorts aggregated rows by columns specified in the ORDER BY clause. diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/ParallelIteratorRegionSplitter.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ParallelIteratorRegionSplitter.java new file mode 100644 index 00000000..a14b1e88 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ParallelIteratorRegionSplitter.java @@ -0,0 +1,36 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.query.KeyRange; + + +/** + * Interface for strategies determining how to split regions in ParallelIterators. + * + * @author zhuang + */ +public interface ParallelIteratorRegionSplitter { + + public List getSplits() throws SQLException; +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/ParallelIteratorRegionSplitterFactory.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ParallelIteratorRegionSplitterFactory.java new file mode 100644 index 00000000..88efc44f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ParallelIteratorRegionSplitterFactory.java @@ -0,0 +1,40 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.parse.HintNode; +import org.apache.phoenix.schema.TableRef; + + +/** + * Factory class for the Region Splitter used by the project. + */ +public class ParallelIteratorRegionSplitterFactory { + + public static ParallelIteratorRegionSplitter getSplitter(StatementContext context, TableRef table, HintNode hintNode) throws SQLException { + if (context.getScanRanges().useSkipScanFilter()) { + return SkipRangeParallelIteratorRegionSplitter.getInstance(context, table, hintNode); + } + return DefaultParallelIteratorRegionSplitter.getInstance(context, table, hintNode); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ParallelIterators.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ParallelIterators.java similarity index 76% rename from phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ParallelIterators.java rename to phoenix-core/src/main/java/org/apache/phoenix/iterate/ParallelIterators.java index 2805fde0..8ea2d3df 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/ParallelIterators.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ParallelIterators.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; import java.sql.SQLException; import java.util.ArrayList; @@ -49,24 +41,24 @@ import org.slf4j.LoggerFactory; import com.google.common.base.Function; -import com.salesforce.phoenix.compile.GroupByCompiler.GroupBy; -import com.salesforce.phoenix.compile.RowProjector; -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.job.JobManager.JobCallable; -import com.salesforce.phoenix.parse.FilterableStatement; -import com.salesforce.phoenix.parse.HintNode; -import com.salesforce.phoenix.query.ConnectionQueryServices; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.SaltingUtil; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.ReadOnlyProps; -import com.salesforce.phoenix.util.SQLCloseables; -import com.salesforce.phoenix.util.ScanUtil; -import com.salesforce.phoenix.util.SchemaUtil; -import com.salesforce.phoenix.util.ServerUtil; +import org.apache.phoenix.compile.GroupByCompiler.GroupBy; +import org.apache.phoenix.compile.RowProjector; +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.job.JobManager.JobCallable; +import org.apache.phoenix.parse.FilterableStatement; +import org.apache.phoenix.parse.HintNode; +import org.apache.phoenix.query.ConnectionQueryServices; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.SaltingUtil; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.ReadOnlyProps; +import org.apache.phoenix.util.SQLCloseables; +import org.apache.phoenix.util.ScanUtil; +import org.apache.phoenix.util.SchemaUtil; +import org.apache.phoenix.util.ServerUtil; /** @@ -170,6 +162,8 @@ public List getIterators() throws SQLException { } } if (ScanUtil.intersectScanRange(splitScan, split.getLowerRange(), split.getUpperRange(), this.context.getScanRanges().useSkipScanFilter())) { + // Delay the swapping of start/stop row until row so we don't muck with the intersect logic + ScanUtil.swapStartStopRowIfReversed(splitScan); Future future = executor.submit(new JobCallable() { @@ -199,12 +193,13 @@ public Object getJobId() { } int timeoutMs = props.getInt(QueryServices.THREAD_TIMEOUT_MS_ATTRIB, DEFAULT_THREAD_TIMEOUT_MS); + final int factor = ScanUtil.isReversed(this.context.getScan()) ? -1 : 1; // Sort futures by row key so that we have a predicatble order we're getting rows back for scans. // We're going to wait here until they're finished anyway and this makes testing much easier. Collections.sort(futures, new Comparator>>() { @Override public int compare(Pair> o1, Pair> o2) { - return Bytes.compareTo(o1.getFirst(), o2.getFirst()); + return factor * Bytes.compareTo(o1.getFirst(), o2.getFirst()); } }); for (Pair> future : futures) { @@ -237,4 +232,4 @@ public void explain(List planSteps) { buf.append("CLIENT PARALLEL " + size() + "-WAY "); explain(buf.toString(),planSteps); } -} +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/PeekingResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/PeekingResultIterator.java new file mode 100644 index 00000000..994f343f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/PeekingResultIterator.java @@ -0,0 +1,63 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Interface for iterating through results returned from a scan, adding the + * ability to peek at the next result. + * + * @author jtaylor + * @since 0.1 + */ +public interface PeekingResultIterator extends ResultIterator { + public static final PeekingResultIterator EMPTY_ITERATOR = new PeekingResultIterator() { + + @Override + public Tuple next() throws SQLException { + return null; + } + + @Override + public Tuple peek() { + return null; + } + + @Override + public void close() throws SQLException { + } + + @Override + public void explain(List planSteps) { + } + }; + + /** + * Returns the next result without advancing the iterator + * @throws SQLException + */ + public Tuple peek() throws SQLException; +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/RegionScannerResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/RegionScannerResultIterator.java new file mode 100644 index 00000000..41601742 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/RegionScannerResultIterator.java @@ -0,0 +1,65 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.io.IOException; +import java.sql.SQLException; +import java.util.ArrayList; +import java.util.List; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.regionserver.MultiVersionConsistencyControl; +import org.apache.hadoop.hbase.regionserver.RegionScanner; + +import org.apache.phoenix.schema.tuple.MultiKeyValueTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ServerUtil; + + +public class RegionScannerResultIterator extends BaseResultIterator { + private final RegionScanner scanner; + + public RegionScannerResultIterator(RegionScanner scanner) { + this.scanner = scanner; + MultiVersionConsistencyControl.setThreadReadPoint(scanner.getMvccReadPoint()); + } + + @Override + public Tuple next() throws SQLException { + try { + // TODO: size + List results = new ArrayList(); + // Results are potentially returned even when the return value of s.next is false + // since this is an indication of whether or not there are more values after the + // ones returned + boolean hasMore = scanner.nextRaw(results, null); + if (!hasMore && results.isEmpty()) { + return null; + } + // We instantiate a new tuple because in all cases currently we hang on to it (i.e. + // to compute and hold onto the TopN). + MultiKeyValueTuple tuple = new MultiKeyValueTuple(); + tuple.setKeyValues(results); + return tuple; + } catch (IOException e) { + throw ServerUtil.parseServerException(e); + } + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/ResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ResultIterator.java new file mode 100644 index 00000000..43e4758f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ResultIterator.java @@ -0,0 +1,54 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.SQLCloseable; + + +public interface ResultIterator extends SQLCloseable { + public static final ResultIterator EMPTY_ITERATOR = new ResultIterator() { + @Override + public void close() throws SQLException { + } + + @Override + public Tuple next() throws SQLException { + return null; + } + + @Override + public void explain(List planSteps) { + } + }; + + /** + * Grab the next row's worth of values. The iterator will return a Tuple. + * @return Tuple object if there is another row, null if the scanner is + * exhausted. + * @throws SQLException e + */ + public Tuple next() throws SQLException; + + public void explain(List planSteps); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/ResultIterators.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ResultIterators.java new file mode 100644 index 00000000..0ee95628 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ResultIterators.java @@ -0,0 +1,29 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +public interface ResultIterators { + public List getIterators() throws SQLException; + public int size(); + public void explain(List planSteps); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/ScanningResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ScanningResultIterator.java new file mode 100644 index 00000000..0c981214 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/ScanningResultIterator.java @@ -0,0 +1,61 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.io.IOException; +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.client.Result; +import org.apache.hadoop.hbase.client.ResultScanner; + +import org.apache.phoenix.schema.tuple.ResultTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ServerUtil; + + +public class ScanningResultIterator implements ResultIterator { + private final ResultScanner scanner; + + public ScanningResultIterator(ResultScanner scanner) { + this.scanner = scanner; + } + + @Override + public void close() throws SQLException { + scanner.close(); + } + + @Override + public Tuple next() throws SQLException { + try { + Result result = scanner.next(); + // TODO: use ResultTuple.setResult(result) + // Need to create a new one if holding on to it (i.e. OrderedResultIterator) + return result == null ? null : new ResultTuple(result); + } catch (IOException e) { + throw ServerUtil.parseServerException(e); + } + } + + @Override + public void explain(List planSteps) { + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/SequenceResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/SequenceResultIterator.java new file mode 100644 index 00000000..e3aeb7f8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/SequenceResultIterator.java @@ -0,0 +1,59 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.SequenceManager; +import org.apache.phoenix.schema.tuple.Tuple; + +/** + * + * Iterates through tuples retrieving sequences from the server as needed + * + * @author jtaylor + * @since 3.0 + */ +public class SequenceResultIterator extends DelegateResultIterator { + private final SequenceManager sequenceManager; + + public SequenceResultIterator(ResultIterator delegate, SequenceManager sequenceManager) throws SQLException { + super(delegate); + sequenceManager.initSequences(); + this.sequenceManager = sequenceManager; + } + + @Override + public Tuple next() throws SQLException { + Tuple next = super.next(); + if (next == null) { + return null; + } + sequenceManager.incrementSequenceValues(); + return next; + } + + @Override + public void explain(List planSteps) { + super.explain(planSteps); + planSteps.add("CLIENT RESERVE " + sequenceManager.getSequenceCount() + " SEQUENCES"); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/SkipRangeParallelIteratorRegionSplitter.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/SkipRangeParallelIteratorRegionSplitter.java new file mode 100644 index 00000000..2f6e0fae --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/SkipRangeParallelIteratorRegionSplitter.java @@ -0,0 +1,85 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.HRegionLocation; + +import com.google.common.base.Predicate; +import com.google.common.collect.Iterables; +import com.google.common.collect.Lists; +import org.apache.phoenix.compile.ScanRanges; +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.parse.HintNode; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.schema.SaltingUtil; +import org.apache.phoenix.schema.TableRef; + + +/** + * Split the region according to the information contained in the scan's SkipScanFilter. + */ +public class SkipRangeParallelIteratorRegionSplitter extends DefaultParallelIteratorRegionSplitter { + + public static SkipRangeParallelIteratorRegionSplitter getInstance(StatementContext context, TableRef table, HintNode hintNode) { + return new SkipRangeParallelIteratorRegionSplitter(context, table, hintNode); + } + + protected SkipRangeParallelIteratorRegionSplitter(StatementContext context, TableRef table, HintNode hintNode) { + super(context, table, hintNode); + } + + @Override + protected List getAllRegions() throws SQLException { + List allTableRegions = context.getConnection().getQueryServices().getAllTableRegions(tableRef.getTable().getPhysicalName().getBytes()); + return filterRegions(allTableRegions, context.getScanRanges()); + } + + public List filterRegions(List allTableRegions, final ScanRanges ranges) { + Iterable regions; + if (ranges == ScanRanges.EVERYTHING) { + return allTableRegions; + } else if (ranges == ScanRanges.NOTHING) { // TODO: why not emptyList? + return Lists.newArrayList(); + } else { + regions = Iterables.filter(allTableRegions, + new Predicate() { + @Override + public boolean apply(HRegionLocation region) { + KeyRange minMaxRange = context.getMinMaxRange(); + if (minMaxRange != null) { + KeyRange range = KeyRange.getKeyRange(region.getRegionInfo().getStartKey(), region.getRegionInfo().getEndKey()); + if (tableRef.getTable().getBucketNum() != null) { + // Add salt byte, as minMaxRange won't have it + minMaxRange = SaltingUtil.addSaltByte(region.getRegionInfo().getStartKey(), minMaxRange); + } + range = range.intersect(minMaxRange); + return ranges.intersect(range.getLowerRange(), range.getUpperRange()); + } + return ranges.intersect(region.getRegionInfo().getStartKey(), region.getRegionInfo().getEndKey()); + } + }); + } + return Lists.newArrayList(regions); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/SpoolTooBigToDiskException.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/SpoolTooBigToDiskException.java new file mode 100644 index 00000000..4dffee00 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/SpoolTooBigToDiskException.java @@ -0,0 +1,17 @@ +package org.apache.phoenix.iterate; + +/** + * Thrown by {@link org.apache.phoenix.iterate.SpoolingResultIterator } when + * result is too big to fit into memory and too big to spool to disk. + * + * @author haitaoyao + * + */ +public class SpoolTooBigToDiskException extends RuntimeException { + + private static final long serialVersionUID = 1L; + + public SpoolTooBigToDiskException(String msg) { + super(msg); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SpoolingResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/SpoolingResultIterator.java similarity index 81% rename from phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SpoolingResultIterator.java rename to phoenix-core/src/main/java/org/apache/phoenix/iterate/SpoolingResultIterator.java index 70f9112e..5acf71c5 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/iterate/SpoolingResultIterator.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/SpoolingResultIterator.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.iterate; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; import java.io.*; import java.sql.SQLException; @@ -36,14 +28,14 @@ import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.io.WritableUtils; -import com.salesforce.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; -import com.salesforce.phoenix.memory.MemoryManager; -import com.salesforce.phoenix.memory.MemoryManager.MemoryChunk; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.tuple.ResultTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.*; +import org.apache.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; +import org.apache.phoenix.memory.MemoryManager; +import org.apache.phoenix.memory.MemoryManager.MemoryChunk; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.tuple.ResultTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.*; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/TableResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/TableResultIterator.java new file mode 100644 index 00000000..c9354a86 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/TableResultIterator.java @@ -0,0 +1,86 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import java.io.IOException; +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.client.HTableInterface; +import org.apache.hadoop.hbase.client.Scan; + +import com.google.common.io.Closeables; +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ServerUtil; + + +/** + * + * Wrapper for ResultScanner creation that closes HTableInterface + * when ResultScanner is closed. + * + * @author jtaylor + * @since 0.1 + */ +public class TableResultIterator extends ExplainTable implements ResultIterator { + private final HTableInterface htable; + private final ResultIterator delegate; + + public TableResultIterator(StatementContext context, TableRef tableRef) throws SQLException { + this(context, tableRef, context.getScan()); + } + + public TableResultIterator(StatementContext context, TableRef tableRef, Scan scan) throws SQLException { + super(context, tableRef); + htable = context.getConnection().getQueryServices().getTable(tableRef.getTable().getPhysicalName().getBytes()); + try { + delegate = new ScanningResultIterator(htable.getScanner(scan)); + } catch (IOException e) { + Closeables.closeQuietly(htable); + throw ServerUtil.parseServerException(e); + } + } + + @Override + public void close() throws SQLException { + try { + delegate.close(); + } finally { + try { + htable.close(); + } catch (IOException e) { + throw ServerUtil.parseServerException(e); + } + } + } + + @Override + public Tuple next() throws SQLException { + return delegate.next(); + } + + @Override + public void explain(List planSteps) { + StringBuilder buf = new StringBuilder(); + explain(buf.toString(),planSteps); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/iterate/UngroupedAggregatingResultIterator.java b/phoenix-core/src/main/java/org/apache/phoenix/iterate/UngroupedAggregatingResultIterator.java new file mode 100644 index 00000000..347e8eb0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/iterate/UngroupedAggregatingResultIterator.java @@ -0,0 +1,56 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.iterate; + +import static org.apache.phoenix.query.QueryConstants.*; + +import java.sql.SQLException; + +import org.apache.phoenix.expression.aggregator.Aggregators; +import org.apache.phoenix.schema.tuple.SingleKeyValueTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.KeyValueUtil; + + +public class UngroupedAggregatingResultIterator extends GroupedAggregatingResultIterator { + private boolean hasRows = false; + + public UngroupedAggregatingResultIterator( PeekingResultIterator resultIterator, Aggregators aggregators) { + super(resultIterator, aggregators); + } + + @Override + public Tuple next() throws SQLException { + Tuple result = super.next(); + // Ensure ungrouped aggregregation always returns a row, even if the underlying iterator doesn't. + if (result == null && !hasRows) { + // Generate value using unused ClientAggregators + byte[] value = aggregators.toBytes(aggregators.getAggregators()); + result = new SingleKeyValueTuple( + KeyValueUtil.newKeyValue(UNGROUPED_AGG_ROW_KEY, + SINGLE_COLUMN_FAMILY, + SINGLE_COLUMN, + AGG_TIMESTAMP, + value)); + } + hasRows = true; + return result; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/jdbc/Jdbc7Shim.java b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/Jdbc7Shim.java new file mode 100644 index 00000000..7a73a4ea --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/Jdbc7Shim.java @@ -0,0 +1,66 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.jdbc; + +import java.sql.SQLException; +import java.sql.SQLFeatureNotSupportedException; +import java.util.concurrent.Executor; +import java.util.logging.Logger; + +/** + * Interfaces to be implemented by classes that need to be "JDK7" compliant, + * but also run in JDK6 + */ +public final class Jdbc7Shim { + + public interface Statement { // Note: do not extend "regular" statement or else eclipse 3.7 complains + void closeOnCompletion() throws SQLException; + boolean isCloseOnCompletion() throws SQLException; + } + + public interface CallableStatement extends Statement { + public T getObject(int columnIndex, Class type) throws SQLException; + public T getObject(String columnLabel, Class type) throws SQLException; + } + + public interface Connection { + void setSchema(String schema) throws SQLException; + String getSchema() throws SQLException; + void abort(Executor executor) throws SQLException; + void setNetworkTimeout(Executor executor, int milliseconds) throws SQLException; + int getNetworkTimeout() throws SQLException; + } + + public interface ResultSet { + public T getObject(int columnIndex, Class type) throws SQLException; + public T getObject(String columnLabel, Class type) throws SQLException; + } + + public interface DatabaseMetaData { + java.sql.ResultSet getPseudoColumns(String catalog, String schemaPattern, + String tableNamePattern, String columnNamePattern) + throws SQLException; + boolean generatedKeyAlwaysReturned() throws SQLException; + } + + public interface Driver { + public Logger getParentLogger() throws SQLFeatureNotSupportedException; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixConnection.java b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixConnection.java similarity index 87% rename from phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixConnection.java rename to phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixConnection.java index ad74281a..ac92570b 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixConnection.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixConnection.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.jdbc; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.jdbc; import java.io.EOFException; import java.io.IOException; @@ -63,30 +55,30 @@ import com.google.common.base.Strings; import com.google.common.collect.Lists; import com.google.common.collect.Maps; -import com.salesforce.phoenix.client.KeyValueBuilder; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.jdbc.PhoenixStatement.PhoenixStatementParser; -import com.salesforce.phoenix.query.ConnectionQueryServices; -import com.salesforce.phoenix.query.DelegateConnectionQueryServices; -import com.salesforce.phoenix.query.MetaDataMutated; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.PArrayDataType; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PMetaData; -import com.salesforce.phoenix.schema.PMetaDataImpl; -import com.salesforce.phoenix.schema.PName; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.util.DateUtil; -import com.salesforce.phoenix.util.JDBCUtil; -import com.salesforce.phoenix.util.PhoenixRuntime; -import com.salesforce.phoenix.util.ReadOnlyProps; -import com.salesforce.phoenix.util.SQLCloseable; -import com.salesforce.phoenix.util.SQLCloseables; +import org.apache.phoenix.client.KeyValueBuilder; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.jdbc.PhoenixStatement.PhoenixStatementParser; +import org.apache.phoenix.query.ConnectionQueryServices; +import org.apache.phoenix.query.DelegateConnectionQueryServices; +import org.apache.phoenix.query.MetaDataMutated; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.PArrayDataType; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PMetaData; +import org.apache.phoenix.schema.PMetaDataImpl; +import org.apache.phoenix.schema.PName; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.util.DateUtil; +import org.apache.phoenix.util.JDBCUtil; +import org.apache.phoenix.util.PhoenixRuntime; +import org.apache.phoenix.util.ReadOnlyProps; +import org.apache.phoenix.util.SQLCloseable; +import org.apache.phoenix.util.SQLCloseables; /** @@ -102,7 +94,7 @@ * @author jtaylor * @since 0.1 */ -public class PhoenixConnection implements Connection, com.salesforce.phoenix.jdbc.Jdbc7Shim.Connection, MetaDataMutated { +public class PhoenixConnection implements Connection, org.apache.phoenix.jdbc.Jdbc7Shim.Connection, MetaDataMutated { private final String url; private final ConnectionQueryServices services; private final Properties info; diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixDatabaseMetaData.java b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixDatabaseMetaData.java similarity index 91% rename from phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixDatabaseMetaData.java rename to phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixDatabaseMetaData.java index 098b506a..a46ab5fc 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixDatabaseMetaData.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixDatabaseMetaData.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.jdbc; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.jdbc; import java.sql.Connection; import java.sql.DatabaseMetaData; @@ -42,37 +34,37 @@ import org.apache.commons.lang.StringEscapeUtils; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.hbase.util.Bytes; +import org.apache.phoenix.compile.ColumnProjector; +import org.apache.phoenix.compile.ExpressionProjector; +import org.apache.phoenix.compile.RowProjector; +import org.apache.phoenix.coprocessor.MetaDataProtocol; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.expression.BaseTerminalExpression; +import org.apache.phoenix.expression.RowKeyColumnExpression; +import org.apache.phoenix.expression.function.IndexStateNameFunction; +import org.apache.phoenix.expression.function.SQLTableTypeFunction; +import org.apache.phoenix.expression.function.SQLViewTypeFunction; +import org.apache.phoenix.expression.function.SqlTypeNameFunction; +import org.apache.phoenix.iterate.DelegateResultIterator; +import org.apache.phoenix.iterate.MaterializedResultIterator; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.parse.HintNode.Hint; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDatum; +import org.apache.phoenix.schema.PName; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.RowKeyValueAccessor; +import org.apache.phoenix.schema.tuple.SingleKeyValueTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.KeyValueUtil; +import org.apache.phoenix.util.MetaDataUtil; +import org.apache.phoenix.util.SchemaUtil; import com.google.common.collect.Lists; -import com.salesforce.phoenix.compile.ColumnProjector; -import com.salesforce.phoenix.compile.ExpressionProjector; -import com.salesforce.phoenix.compile.RowProjector; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.expression.BaseTerminalExpression; -import com.salesforce.phoenix.expression.RowKeyColumnExpression; -import com.salesforce.phoenix.expression.function.IndexStateNameFunction; -import com.salesforce.phoenix.expression.function.SQLTableTypeFunction; -import com.salesforce.phoenix.expression.function.SQLViewTypeFunction; -import com.salesforce.phoenix.expression.function.SqlTypeNameFunction; -import com.salesforce.phoenix.iterate.DelegateResultIterator; -import com.salesforce.phoenix.iterate.MaterializedResultIterator; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.parse.HintNode.Hint; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDatum; -import com.salesforce.phoenix.schema.PName; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.RowKeyValueAccessor; -import com.salesforce.phoenix.schema.tuple.SingleKeyValueTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.KeyValueUtil; -import com.salesforce.phoenix.util.MetaDataUtil; -import com.salesforce.phoenix.util.SchemaUtil; /** @@ -100,7 +92,7 @@ * @author jtaylor * @since 0.1 */ -public class PhoenixDatabaseMetaData implements DatabaseMetaData, com.salesforce.phoenix.jdbc.Jdbc7Shim.DatabaseMetaData { +public class PhoenixDatabaseMetaData implements DatabaseMetaData, org.apache.phoenix.jdbc.Jdbc7Shim.DatabaseMetaData { public static final int INDEX_NAME_INDEX = 4; // Shared with FAMILY_NAME_INDEX public static final int FAMILY_NAME_INDEX = 4; public static final int COLUMN_NAME_INDEX = 3; @@ -109,13 +101,17 @@ public class PhoenixDatabaseMetaData implements DatabaseMetaData, com.salesforce public static final int TENANT_ID_INDEX = 0; public static final String TYPE_SCHEMA = "SYSTEM"; - public static final String TYPE_TABLE = "TABLE"; + public static final String TYPE_TABLE = "CATALOG"; public static final String TYPE_SCHEMA_AND_TABLE = TYPE_SCHEMA + ".\"" + TYPE_TABLE + "\""; public static final byte[] TYPE_TABLE_BYTES = TYPE_TABLE.getBytes(); public static final byte[] TYPE_SCHEMA_BYTES = TYPE_SCHEMA.getBytes(); public static final String TYPE_TABLE_NAME = SchemaUtil.getTableName(TYPE_SCHEMA, TYPE_TABLE); public static final byte[] TYPE_TABLE_NAME_BYTES = SchemaUtil.getTableNameAsBytes(TYPE_SCHEMA_BYTES, TYPE_TABLE_BYTES); + public static final String TYPE_SCHEMA_ALIAS = "SYSTEM"; + public static final String TYPE_TABLE_ALIAS = "TABLE"; + public static final String TYPE_SCHEMA_AND_TABLE_ALIAS = "\"" + TYPE_SCHEMA_ALIAS + "." + TYPE_TABLE_ALIAS + "\""; + public static final String TABLE_NAME_NAME = "TABLE_NAME"; public static final String TABLE_TYPE_NAME = "TABLE_TYPE"; public static final byte[] TABLE_TYPE_BYTES = Bytes.toBytes(TABLE_TYPE_NAME); @@ -171,18 +167,18 @@ public class PhoenixDatabaseMetaData implements DatabaseMetaData, com.salesforce public static final byte[] IMMUTABLE_ROWS_BYTES = Bytes.toBytes(IMMUTABLE_ROWS); public static final String DEFAULT_COLUMN_FAMILY_NAME = "DEFAULT_COLUMN_FAMILY"; public static final byte[] DEFAULT_COLUMN_FAMILY_NAME_BYTES = Bytes.toBytes(DEFAULT_COLUMN_FAMILY_NAME); - public static final String VIEW_EXPRESSION = "VIEW_EXPRESSION"; - public static final byte[] VIEW_EXPRESSION_BYTES = Bytes.toBytes(VIEW_EXPRESSION); + public static final String VIEW_STATEMENT = "VIEW_STATEMENT"; + public static final byte[] VIEW_STATEMENT_BYTES = Bytes.toBytes(VIEW_STATEMENT); public static final String DISABLE_WAL = "DISABLE_WAL"; public static final byte[] DISABLE_WAL_BYTES = Bytes.toBytes(DISABLE_WAL); public static final String MULTI_TENANT = "MULTI_TENANT"; public static final byte[] MULTI_TENANT_BYTES = Bytes.toBytes(MULTI_TENANT); public static final String VIEW_TYPE = "VIEW_TYPE"; public static final byte[] VIEW_TYPE_BYTES = Bytes.toBytes(VIEW_TYPE); - public static final String BASE_SCHEMA_NAME = "BASE_SCHEMA_NAME"; - public static final byte[] BASE_SCHEMA_NAME_BYTES = Bytes.toBytes(BASE_SCHEMA_NAME); - public static final String BASE_TABLE_NAME = "BASE_TABLE_NAME"; - public static final byte[] BASE_TABLE_NAME_BYTES = Bytes.toBytes(BASE_TABLE_NAME); + public static final String LINK_TYPE = "LINK_TYPE"; + public static final byte[] LINK_TYPE_BYTES = Bytes.toBytes(LINK_TYPE); + public static final String ARRAY_SIZE = "ARRAY_SIZE"; + public static final byte[] ARRAY_SIZE_BYTES = Bytes.toBytes(ARRAY_SIZE); public static final String TABLE_FAMILY = QueryConstants.DEFAULT_COLUMN_FAMILY; public static final byte[] TABLE_FAMILY_BYTES = QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES; @@ -323,8 +319,9 @@ public ResultSet getColumns(String catalog, String schemaPattern, String tableNa SCOPE_SCHEMA + "," + SCOPE_TABLE + "," + SOURCE_DATA_TYPE + "," + - IS_AUTOINCREMENT + - " from " + TYPE_SCHEMA_AND_TABLE + + IS_AUTOINCREMENT + "," + + ARRAY_SIZE + + " from " + TYPE_SCHEMA_AND_TABLE + " " + TYPE_SCHEMA_AND_TABLE_ALIAS + " where "); buf.append(getTenantIdWhereClause()); if (schemaPattern != null) { @@ -611,7 +608,7 @@ public ResultSet getPrimaryKeys(String catalog, String schema, String table) thr "CASE WHEN " + COLUMN_MODIFIER + " = " + ColumnModifier.toSystemValue(ColumnModifier.SORT_DESC) + " THEN 'D' ELSE 'A' END ASC_OR_DESC," + DATA_TYPE + "," + // include type info, though not in spec SqlTypeNameFunction.NAME + "(" + DATA_TYPE + ") AS " + TYPE_NAME + - " from " + TYPE_SCHEMA_AND_TABLE + + " from " + TYPE_SCHEMA_AND_TABLE + " " + TYPE_SCHEMA_AND_TABLE_ALIAS + " where "); buf.append(getTenantIdWhereClause()); buf.append(" and " + TABLE_SCHEM_NAME + (schema == null || schema.length() == 0 ? " is null" : " = '" + SchemaUtil.normalizeIdentifier(schema) + "'" )); @@ -750,7 +747,7 @@ public ResultSet getSchemas(String catalog, String schemaPattern) throws SQLExce StringBuilder buf = new StringBuilder("select /*+" + Hint.NO_INTRA_REGION_PARALLELIZATION + "*/ distinct " + "null " + TABLE_CATALOG_NAME + "," + // no catalog for tables TABLE_SCHEM_NAME + - " from " + TYPE_SCHEMA_AND_TABLE + + " from " + TYPE_SCHEMA_AND_TABLE + " " + TYPE_SCHEMA_AND_TABLE_ALIAS + " where " + COLUMN_NAME + " is null"); if (schemaPattern != null) { buf.append(" and " + TABLE_SCHEM_NAME + " like '" + SchemaUtil.normalizeIdentifier(schemaPattern) + "'"); @@ -787,11 +784,10 @@ private ResultSet getSuperTables(String catalog, String schemaPattern, String ta IndexStateNameFunction.NAME + "(" + INDEX_STATE + ") AS " + INDEX_STATE + "," + IMMUTABLE_ROWS + "," + SALT_BUCKETS + "," + - BASE_SCHEMA_NAME + "," + - BASE_TABLE_NAME + "," + - VIEW_EXPRESSION + "," + - TENANT_ID + - " from " + TYPE_SCHEMA_AND_TABLE + + TENANT_ID + "," + + VIEW_STATEMENT + "," + + SQLViewTypeFunction.NAME + "(" + VIEW_TYPE + ") AS " + VIEW_TYPE + + " from " + TYPE_SCHEMA_AND_TABLE + " " + TYPE_SCHEMA_AND_TABLE_ALIAS + " where " + COLUMN_NAME + " is null" + " and " + TABLE_CAT_NAME + " is null"); buf.append(" and " + TENANT_ID + " IS NOT NULL "); @@ -904,12 +900,10 @@ public ResultSet getTables(String catalog, String schemaPattern, String tableNam IndexStateNameFunction.NAME + "(" + INDEX_STATE + ") AS " + INDEX_STATE + "," + IMMUTABLE_ROWS + "," + SALT_BUCKETS + "," + - BASE_SCHEMA_NAME + "," + - BASE_TABLE_NAME + "," + - VIEW_EXPRESSION + "," + MULTI_TENANT + "," + + VIEW_STATEMENT + "," + SQLViewTypeFunction.NAME + "(" + VIEW_TYPE + ") AS " + VIEW_TYPE + - " from " + TYPE_SCHEMA_AND_TABLE + + " from " + TYPE_SCHEMA_AND_TABLE + " " + TYPE_SCHEMA_AND_TABLE_ALIAS + " where " + COLUMN_NAME + " is null" + " and " + TABLE_CAT_NAME + " is null"); buf.append(" and " + getTenantIdWhereClause()); diff --git a/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixDriver.java b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixDriver.java new file mode 100644 index 00000000..590b29f9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixDriver.java @@ -0,0 +1,96 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.jdbc; + +import java.sql.DriverManager; +import java.sql.SQLException; +import java.util.Properties; +import java.util.concurrent.ConcurrentHashMap; +import java.util.concurrent.ConcurrentMap; + +import org.apache.phoenix.query.*; +import org.apache.phoenix.util.SQLCloseables; + + +/** + * + * JDBC Driver implementation of Phoenix for production. + * To use this driver, specify the following URL: + * jdbc:phoenix:; + * Only an embedded driver is currently supported (Phoenix client + * runs in the same JVM as the driver). Connections are lightweight + * and are not pooled. The last part of the URL, the hbase zookeeper + * quorum server name, determines the hbase cluster to which queries + * will be routed. + * + * @author jtaylor + * @since 0.1 + */ +public final class PhoenixDriver extends PhoenixEmbeddedDriver { + public static final PhoenixDriver INSTANCE; + static { + try { + DriverManager.registerDriver( INSTANCE = new PhoenixDriver() ); + } catch (SQLException e) { + throw new IllegalStateException("Untable to register " + PhoenixDriver.class.getName() + ": "+ e.getMessage()); + } + } + private final ConcurrentMap connectionQueryServicesMap = new ConcurrentHashMap(3); + + public PhoenixDriver() { // for Squirrel + // Use production services implementation + super(new QueryServicesImpl()); + } + + @Override + public boolean acceptsURL(String url) throws SQLException { + // Accept the url only if test=true attribute not set + return super.acceptsURL(url) && !isTestUrl(url); + } + + @Override + protected ConnectionQueryServices getConnectionQueryServices(String url, Properties info) throws SQLException { + ConnectionInfo connInfo = ConnectionInfo.create(url); + ConnectionInfo normalizedConnInfo = connInfo.normalize(getQueryServices().getProps()); + ConnectionQueryServices connectionQueryServices = connectionQueryServicesMap.get(normalizedConnInfo); + if (connectionQueryServices == null) { + if (normalizedConnInfo.isConnectionless()) { + connectionQueryServices = new ConnectionlessQueryServicesImpl(getQueryServices()); + } else { + connectionQueryServices = new ConnectionQueryServicesImpl(getQueryServices(), normalizedConnInfo); + } + connectionQueryServices.init(url, info); + ConnectionQueryServices prevValue = connectionQueryServicesMap.putIfAbsent(normalizedConnInfo, connectionQueryServices); + if (prevValue != null) { + connectionQueryServices = prevValue; + } + } + return connectionQueryServices; + } + + @Override + public void close() throws SQLException { + try { + SQLCloseables.closeAll(connectionQueryServicesMap.values()); + } finally { + connectionQueryServicesMap.clear(); + } + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixEmbeddedDriver.java b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java similarity index 84% rename from phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixEmbeddedDriver.java rename to phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java index 7422697a..3550584b 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixEmbeddedDriver.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixEmbeddedDriver.java @@ -1,43 +1,35 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.jdbc; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.jdbc; import java.sql.*; import java.util.*; import java.util.logging.Logger; import com.google.common.collect.Maps; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.query.ConnectionQueryServices; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.util.*; +import org.apache.phoenix.coprocessor.MetaDataProtocol; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.query.ConnectionQueryServices; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.util.*; @@ -48,7 +40,7 @@ * @author jtaylor * @since 0.1 */ -public abstract class PhoenixEmbeddedDriver implements Driver, com.salesforce.phoenix.jdbc.Jdbc7Shim.Driver, SQLCloseable { +public abstract class PhoenixEmbeddedDriver implements Driver, org.apache.phoenix.jdbc.Jdbc7Shim.Driver, SQLCloseable { /** * The protocol for Phoenix Network Client */ diff --git a/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixParameterMetaData.java b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixParameterMetaData.java new file mode 100644 index 00000000..830e0547 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixParameterMetaData.java @@ -0,0 +1,172 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.jdbc; + +import java.sql.ParameterMetaData; +import java.sql.ResultSetMetaData; +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.parse.BindParseNode; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDatum; +import org.apache.phoenix.schema.TypeMismatchException; + + + +/** + * + * Implementation of ParameterMetaData for Phoenix + * + * @author jtaylor + * @since 0.1 + */ +public class PhoenixParameterMetaData implements ParameterMetaData { + private final PDatum[] params; + private static final PDatum EMPTY_DATUM = new PDatum() { + @Override + public boolean isNullable() { + return false; + } + + @Override + public Integer getScale() { + return null; + } + + @Override + public Integer getMaxLength() { + return null; + } + + @Override + public PDataType getDataType() { + return null; + } + + @Override + public ColumnModifier getColumnModifier() { + return null; + } + + @Override + public Integer getByteSize() { + return null; + } + }; + public static final PhoenixParameterMetaData EMPTY_PARAMETER_META_DATA = new PhoenixParameterMetaData(0); + public PhoenixParameterMetaData(int paramCount) { + params = new PDatum[paramCount]; + //initialize the params array with the empty_datum marker value. + for(int i = 0; i < paramCount; i++) { + params[i] = EMPTY_DATUM; + } + } + + private PDatum getParam(int index) throws SQLException { + if (index <= 0 || index > params.length) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.PARAM_INDEX_OUT_OF_BOUND) + .setMessage("The index is " + index + ". Must be between 1 and " + params.length) + .build().buildException(); + } + PDatum param = params[index-1]; + + if (param == EMPTY_DATUM) { + //value at params[index-1] was never set. + throw new SQLExceptionInfo.Builder(SQLExceptionCode.PARAM_VALUE_UNBOUND) + .setMessage("Parameter at index " + index + " is unbound").build().buildException(); + } + return param; + } + @Override + public String getParameterClassName(int index) throws SQLException { + PDatum datum = getParam(index); + PDataType type = datum == null ? null : datum.getDataType(); + return type == null ? null : type.getJavaClassName(); + } + + @Override + public int getParameterCount() throws SQLException { + return params.length; + } + + @Override + public int getParameterMode(int index) throws SQLException { + return ParameterMetaData.parameterModeIn; + } + + @Override + public int getParameterType(int index) throws SQLException { + return getParam(index).getDataType().getSqlType(); + } + + @Override + public String getParameterTypeName(int index) throws SQLException { + return getParam(index).getDataType().getSqlTypeName(); + } + + @Override + public int getPrecision(int index) throws SQLException { + return 0; + } + + @Override + public int getScale(int index) throws SQLException { + return 0; + } + + @Override + public int isNullable(int index) throws SQLException { + return getParam(index).isNullable() ? ResultSetMetaData.columnNullable : ResultSetMetaData.columnNoNulls; + } + + @Override + public boolean isSigned(int index) throws SQLException { + @SuppressWarnings("rawtypes") + Class clazz = getParam(index).getDataType().getJavaClass(); + return Number.class.isInstance(clazz); + } + + @Override + public boolean isWrapperFor(Class iface) throws SQLException { + return iface.isInstance(this); + } + + @SuppressWarnings("unchecked") + @Override + public T unwrap(Class iface) throws SQLException { + if (!iface.isInstance(this)) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.CLASS_NOT_UNWRAPPABLE) + .setMessage(this.getClass().getName() + " not unwrappable from " + iface.getName()) + .build().buildException(); + } + return (T)this; + } + + public void addParam(BindParseNode bind, PDatum datum) throws SQLException { + PDatum bindDatum = params[bind.getIndex()]; + if (bindDatum != null && bindDatum.getDataType() != null && !datum.getDataType().isCoercibleTo(bindDatum.getDataType())) { + throw TypeMismatchException.newException(datum.getDataType(), bindDatum.getDataType()); + } + params[bind.getIndex()] = datum; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixPreparedStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixPreparedStatement.java similarity index 87% rename from phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixPreparedStatement.java rename to phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixPreparedStatement.java index da6a7ece..8582e749 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixPreparedStatement.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixPreparedStatement.java @@ -1,19 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. All rights reserved. Redistribution and use in source and binary forms, with - * or without modification, are permitted provided that the following conditions are met: Redistributions of source code - * must retain the above copyright notice, this list of conditions and the following disclaimer. Redistributions in - * binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. Neither the name of Salesforce.com nor the names - * of its contributors may be used to endorse or promote products derived from this software without specific prior - * written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR - * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A - * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, - * INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF - * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN - * ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.jdbc; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.jdbc; import java.io.EOFException; import java.io.IOException; @@ -43,14 +47,14 @@ import java.util.Collections; import java.util.List; -import com.salesforce.phoenix.compile.BindManager; -import com.salesforce.phoenix.compile.QueryPlan; -import com.salesforce.phoenix.compile.StatementPlan; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.util.DateUtil; -import com.salesforce.phoenix.util.SQLCloseable; +import org.apache.phoenix.compile.BindManager; +import org.apache.phoenix.compile.QueryPlan; +import org.apache.phoenix.compile.StatementPlan; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.util.DateUtil; +import org.apache.phoenix.util.SQLCloseable; /** * JDBC PreparedStatement implementation of Phoenix. Currently only the following methods (in addition to the ones diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixResultSet.java b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixResultSet.java similarity index 94% rename from phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixResultSet.java rename to phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixResultSet.java index 0c79d7be..a5807aff 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixResultSet.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixResultSet.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.jdbc; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.jdbc; import java.io.InputStream; import java.io.Reader; @@ -54,16 +46,16 @@ import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import com.salesforce.phoenix.compile.ColumnProjector; -import com.salesforce.phoenix.compile.RowProjector; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.tuple.ResultTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.DateUtil; -import com.salesforce.phoenix.util.SQLCloseable; +import org.apache.phoenix.compile.ColumnProjector; +import org.apache.phoenix.compile.RowProjector; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.tuple.ResultTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.DateUtil; +import org.apache.phoenix.util.SQLCloseable; @@ -93,7 +85,7 @@ * @author jtaylor * @since 0.1 */ -public class PhoenixResultSet implements ResultSet, SQLCloseable, com.salesforce.phoenix.jdbc.Jdbc7Shim.ResultSet { +public class PhoenixResultSet implements ResultSet, SQLCloseable, org.apache.phoenix.jdbc.Jdbc7Shim.ResultSet { private final static String STRING_FALSE = "0"; private final static BigDecimal BIG_DECIMAL_FALSE = BigDecimal.valueOf(0); private final static Integer INTEGER_FALSE = Integer.valueOf(0); diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixResultSetMetaData.java b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixResultSetMetaData.java similarity index 75% rename from phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixResultSetMetaData.java rename to phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixResultSetMetaData.java index 68eeed3a..55d558a7 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixResultSetMetaData.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixResultSetMetaData.java @@ -1,40 +1,32 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.jdbc; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.jdbc; import java.sql.*; -import com.salesforce.phoenix.compile.ColumnProjector; -import com.salesforce.phoenix.compile.RowProjector; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.PDataType; +import org.apache.phoenix.compile.ColumnProjector; +import org.apache.phoenix.compile.RowProjector; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.PDataType; /** diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java similarity index 88% rename from phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixStatement.java rename to phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java index 361912f4..8592f498 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/jdbc/PhoenixStatement.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.jdbc; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.jdbc; import java.io.IOException; import java.io.Reader; @@ -47,74 +39,74 @@ import com.google.common.collect.ListMultimap; import com.google.common.collect.Lists; -import com.salesforce.phoenix.compile.ColumnProjector; -import com.salesforce.phoenix.compile.CreateIndexCompiler; -import com.salesforce.phoenix.compile.CreateSequenceCompiler; -import com.salesforce.phoenix.compile.CreateTableCompiler; -import com.salesforce.phoenix.compile.DeleteCompiler; -import com.salesforce.phoenix.compile.DropSequenceCompiler; -import com.salesforce.phoenix.compile.ExplainPlan; -import com.salesforce.phoenix.compile.ExpressionProjector; -import com.salesforce.phoenix.compile.MutationPlan; -import com.salesforce.phoenix.compile.QueryCompiler; -import com.salesforce.phoenix.compile.QueryPlan; -import com.salesforce.phoenix.compile.RowProjector; -import com.salesforce.phoenix.compile.StatementPlan; -import com.salesforce.phoenix.compile.UpsertCompiler; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.expression.RowKeyColumnExpression; -import com.salesforce.phoenix.iterate.MaterializedResultIterator; -import com.salesforce.phoenix.iterate.ResultIterator; -import com.salesforce.phoenix.parse.AddColumnStatement; -import com.salesforce.phoenix.parse.AliasedNode; -import com.salesforce.phoenix.parse.AlterIndexStatement; -import com.salesforce.phoenix.parse.BindableStatement; -import com.salesforce.phoenix.parse.ColumnDef; -import com.salesforce.phoenix.parse.ColumnName; -import com.salesforce.phoenix.parse.CreateIndexStatement; -import com.salesforce.phoenix.parse.CreateSequenceStatement; -import com.salesforce.phoenix.parse.CreateTableStatement; -import com.salesforce.phoenix.parse.DeleteStatement; -import com.salesforce.phoenix.parse.DropColumnStatement; -import com.salesforce.phoenix.parse.DropIndexStatement; -import com.salesforce.phoenix.parse.DropSequenceStatement; -import com.salesforce.phoenix.parse.DropTableStatement; -import com.salesforce.phoenix.parse.ExplainStatement; -import com.salesforce.phoenix.parse.HintNode; -import com.salesforce.phoenix.parse.LimitNode; -import com.salesforce.phoenix.parse.NamedNode; -import com.salesforce.phoenix.parse.NamedTableNode; -import com.salesforce.phoenix.parse.OrderByNode; -import com.salesforce.phoenix.parse.ParseNode; -import com.salesforce.phoenix.parse.ParseNodeFactory; -import com.salesforce.phoenix.parse.PrimaryKeyConstraint; -import com.salesforce.phoenix.parse.SQLParser; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.parse.TableName; -import com.salesforce.phoenix.parse.TableNode; -import com.salesforce.phoenix.parse.UpsertStatement; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.ExecuteQueryNotApplicableException; -import com.salesforce.phoenix.schema.ExecuteUpdateNotApplicableException; -import com.salesforce.phoenix.schema.MetaDataClient; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PDatum; -import com.salesforce.phoenix.schema.PIndexState; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.RowKeyValueAccessor; -import com.salesforce.phoenix.schema.tuple.SingleKeyValueTuple; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.KeyValueUtil; -import com.salesforce.phoenix.util.SQLCloseable; -import com.salesforce.phoenix.util.SQLCloseables; -import com.salesforce.phoenix.util.ServerUtil; +import org.apache.phoenix.compile.ColumnProjector; +import org.apache.phoenix.compile.CreateIndexCompiler; +import org.apache.phoenix.compile.CreateSequenceCompiler; +import org.apache.phoenix.compile.CreateTableCompiler; +import org.apache.phoenix.compile.DeleteCompiler; +import org.apache.phoenix.compile.DropSequenceCompiler; +import org.apache.phoenix.compile.ExplainPlan; +import org.apache.phoenix.compile.ExpressionProjector; +import org.apache.phoenix.compile.MutationPlan; +import org.apache.phoenix.compile.QueryCompiler; +import org.apache.phoenix.compile.QueryPlan; +import org.apache.phoenix.compile.RowProjector; +import org.apache.phoenix.compile.StatementPlan; +import org.apache.phoenix.compile.UpsertCompiler; +import org.apache.phoenix.coprocessor.MetaDataProtocol; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.expression.RowKeyColumnExpression; +import org.apache.phoenix.iterate.MaterializedResultIterator; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.parse.AddColumnStatement; +import org.apache.phoenix.parse.AliasedNode; +import org.apache.phoenix.parse.AlterIndexStatement; +import org.apache.phoenix.parse.BindableStatement; +import org.apache.phoenix.parse.ColumnDef; +import org.apache.phoenix.parse.ColumnName; +import org.apache.phoenix.parse.CreateIndexStatement; +import org.apache.phoenix.parse.CreateSequenceStatement; +import org.apache.phoenix.parse.CreateTableStatement; +import org.apache.phoenix.parse.DeleteStatement; +import org.apache.phoenix.parse.DropColumnStatement; +import org.apache.phoenix.parse.DropIndexStatement; +import org.apache.phoenix.parse.DropSequenceStatement; +import org.apache.phoenix.parse.DropTableStatement; +import org.apache.phoenix.parse.ExplainStatement; +import org.apache.phoenix.parse.HintNode; +import org.apache.phoenix.parse.LimitNode; +import org.apache.phoenix.parse.NamedNode; +import org.apache.phoenix.parse.NamedTableNode; +import org.apache.phoenix.parse.OrderByNode; +import org.apache.phoenix.parse.ParseNode; +import org.apache.phoenix.parse.ParseNodeFactory; +import org.apache.phoenix.parse.PrimaryKeyConstraint; +import org.apache.phoenix.parse.SQLParser; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.parse.TableName; +import org.apache.phoenix.parse.TableNode; +import org.apache.phoenix.parse.UpsertStatement; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.ExecuteQueryNotApplicableException; +import org.apache.phoenix.schema.ExecuteUpdateNotApplicableException; +import org.apache.phoenix.schema.MetaDataClient; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDatum; +import org.apache.phoenix.schema.PIndexState; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.RowKeyValueAccessor; +import org.apache.phoenix.schema.tuple.SingleKeyValueTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.KeyValueUtil; +import org.apache.phoenix.util.SQLCloseable; +import org.apache.phoenix.util.SQLCloseables; +import org.apache.phoenix.util.ServerUtil; /** @@ -135,7 +127,7 @@ * @author jtaylor * @since 0.1 */ -public class PhoenixStatement implements Statement, SQLCloseable, com.salesforce.phoenix.jdbc.Jdbc7Shim.Statement { +public class PhoenixStatement implements Statement, SQLCloseable, org.apache.phoenix.jdbc.Jdbc7Shim.Statement { public enum UpdateOperation { DELETED("deleted"), UPSERTED("upserted"); diff --git a/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatementFactory.java b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatementFactory.java new file mode 100644 index 00000000..50810496 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatementFactory.java @@ -0,0 +1,24 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.jdbc; + +public interface PhoenixStatementFactory { + public PhoenixStatement newStatement(PhoenixConnection connection); +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/job/AbstractRoundRobinQueue.java b/phoenix-core/src/main/java/org/apache/phoenix/job/AbstractRoundRobinQueue.java similarity index 84% rename from phoenix-core/src/main/java/com/salesforce/phoenix/job/AbstractRoundRobinQueue.java rename to phoenix-core/src/main/java/org/apache/phoenix/job/AbstractRoundRobinQueue.java index 3f70622c..e77e4cac 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/job/AbstractRoundRobinQueue.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/job/AbstractRoundRobinQueue.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.job; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.job; import java.util.*; import java.util.concurrent.BlockingQueue; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/job/JobManager.java b/phoenix-core/src/main/java/org/apache/phoenix/job/JobManager.java new file mode 100644 index 00000000..7d17a6d9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/job/JobManager.java @@ -0,0 +1,115 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.job; + +import java.util.concurrent.*; +import java.util.concurrent.atomic.AtomicLong; + +import com.google.common.util.concurrent.ThreadFactoryBuilder; + +/** + * + * Thread pool executor that executes scans in parallel + * + * @author jtaylor + * @since 0.1 + */ +@SuppressWarnings("rawtypes") +public class JobManager extends AbstractRoundRobinQueue { + + private static final AtomicLong PHOENIX_POOL_INDEX = new AtomicLong(1); + + public JobManager(int maxSize) { + super(maxSize, true); // true -> new producers move to front of queue; this reduces latency. + } + + @Override + protected Object extractProducer(T o) { + return ((JobFutureTask)o).getJobId(); + } + + public static interface JobRunnable extends Runnable { + public Object getJobId(); + } + + public static ThreadPoolExecutor createThreadPoolExec(int keepAliveMs, int size, int queueSize) { + BlockingQueue queue; + if (queueSize == 0) { + queue = new SynchronousQueue(); // Specialized for 0 length. + } else { + queue = new JobManager(queueSize); + } + ThreadFactory threadFactory = new ThreadFactoryBuilder().setNameFormat( + "phoenix-" + PHOENIX_POOL_INDEX.getAndIncrement() + + "-thread-%s").build(); + // For thread pool, set core threads = max threads -- we don't ever want to exceed core threads, but want to go up to core threads *before* using the queue. + ThreadPoolExecutor exec = new ThreadPoolExecutor(size, size, keepAliveMs, TimeUnit.MILLISECONDS, queue, threadFactory) { + @Override + protected RunnableFuture newTaskFor(Callable call) { + // Override this so we can create a JobFutureTask so we can extract out the parentJobId (otherwise, in the default FutureTask, it is private). + return new JobFutureTask(call); + } + + @Override + protected RunnableFuture newTaskFor(Runnable runnable, T value) { + return new JobFutureTask((JobRunnable)runnable, value); + } + + }; + + exec.allowCoreThreadTimeOut(true); // ... and allow core threads to time out. This just keeps things clean when idle, and is nice for ftests modes, etc., where we'd especially like these not to linger. + return exec; + } + + /** + * Subclasses FutureTask for the sole purpose of providing {@link #getCallable()}, which is used to extract the producer in the {@link JobBasedRoundRobinQueue} + */ + static class JobFutureTask extends FutureTask { + private final Object jobId; + + public JobFutureTask(JobRunnable r, T t) { + super(r, t); + this.jobId = r.getJobId(); + } + + public JobFutureTask(Callable c) { + super(c); + // FIXME: this fails when executor used by hbase + if (c instanceof JobCallable) { + this.jobId = ((JobCallable) c).getJobId(); + } else { + this.jobId = this; + } + } + + public Object getJobId() { + return jobId; + } + } + + + /** + * Delegating callable implementation that preserves the parentJobId and sets up thread tracker stuff before delegating to the actual command. + */ + public static interface JobCallable extends Callable { + public Object getJobId(); + } +} + diff --git a/phoenix-core/src/main/java/org/apache/phoenix/join/HashCacheClient.java b/phoenix-core/src/main/java/org/apache/phoenix/join/HashCacheClient.java new file mode 100644 index 00000000..c3746183 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/join/HashCacheClient.java @@ -0,0 +1,133 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.join; + +import java.io.DataOutputStream; +import java.io.IOException; +import java.sql.SQLException; +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.io.WritableUtils; +import org.xerial.snappy.Snappy; + +import org.apache.phoenix.cache.ServerCacheClient; +import org.apache.phoenix.cache.ServerCacheClient.ServerCache; +import org.apache.phoenix.compile.ScanRanges; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.ExpressionType; +import org.apache.phoenix.iterate.ResultIterator; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ServerUtil; +import org.apache.phoenix.util.TrustedByteArrayOutputStream; +import org.apache.phoenix.util.TupleUtil; + +/** + * + * Client for adding cache of one side of a join to region servers + * + * @author jtaylor + * @since 0.1 + */ +public class HashCacheClient { + private final ServerCacheClient serverCache; + /** + * Construct client used to create a serialized cached snapshot of a table and send it to each region server + * for caching during hash join processing. + * @param connection the client connection + */ + public HashCacheClient(PhoenixConnection connection) { + serverCache = new ServerCacheClient(connection); + } + + /** + * Send the results of scanning through the scanner to all + * region servers for regions of the table that will use the cache + * that intersect with the minMaxKeyRange. + * @param scanner scanner for the table or intermediate results being cached + * @return client-side {@link ServerCache} representing the added hash cache + * @throws SQLException + * @throws MaxServerCacheSizeExceededException if size of hash cache exceeds max allowed + * size + */ + public ServerCache addHashCache(ScanRanges keyRanges, ResultIterator iterator, long estimatedSize, List onExpressions, TableRef cacheUsingTableRef) throws SQLException { + /** + * Serialize and compress hashCacheTable + */ + ImmutableBytesWritable ptr = new ImmutableBytesWritable(); + serialize(ptr, iterator, estimatedSize, onExpressions); + return serverCache.addServerCache(keyRanges, ptr, new HashCacheFactory(), cacheUsingTableRef); + } + + private void serialize(ImmutableBytesWritable ptr, ResultIterator iterator, long estimatedSize, List onExpressions) throws SQLException { + long maxSize = serverCache.getConnection().getQueryServices().getProps().getLong(QueryServices.MAX_SERVER_CACHE_SIZE_ATTRIB, QueryServicesOptions.DEFAULT_MAX_SERVER_CACHE_SIZE); + estimatedSize = Math.min(estimatedSize, maxSize); + if (estimatedSize > Integer.MAX_VALUE) { + throw new IllegalStateException("Estimated size(" + estimatedSize + ") must not be greater than Integer.MAX_VALUE(" + Integer.MAX_VALUE + ")"); + } + try { + TrustedByteArrayOutputStream baOut = new TrustedByteArrayOutputStream((int)estimatedSize); + DataOutputStream out = new DataOutputStream(baOut); + // Write onExpressions first, for hash key evaluation along with deserialization + out.writeInt(onExpressions.size()); + for (Expression expression : onExpressions) { + WritableUtils.writeVInt(out, ExpressionType.valueOf(expression).ordinal()); + expression.write(out); + } + int exprSize = baOut.size() + Bytes.SIZEOF_INT; + out.writeInt(exprSize); + int nRows = 0; + out.writeInt(nRows); // In the end will be replaced with total number of rows + for (Tuple result = iterator.next(); result != null; result = iterator.next()) { + TupleUtil.write(result, out); + if (baOut.size() > maxSize) { + throw new MaxServerCacheSizeExceededException("Size of hash cache (" + baOut.size() + " bytes) exceeds the maximum allowed size (" + maxSize + " bytes)"); + } + nRows++; + } + TrustedByteArrayOutputStream sizeOut = new TrustedByteArrayOutputStream(Bytes.SIZEOF_INT); + DataOutputStream dataOut = new DataOutputStream(sizeOut); + try { + dataOut.writeInt(nRows); + dataOut.flush(); + byte[] cache = baOut.getBuffer(); + // Replace number of rows written above with the correct value. + System.arraycopy(sizeOut.getBuffer(), 0, cache, exprSize, sizeOut.size()); + // Reallocate to actual size plus compressed buffer size (which is allocated below) + int maxCompressedSize = Snappy.maxCompressedLength(baOut.size()); + byte[] compressed = new byte[maxCompressedSize]; // size for worst case + int compressedSize = Snappy.compress(baOut.getBuffer(), 0, baOut.size(), compressed, 0); + // Last realloc to size of compressed buffer. + ptr.set(compressed,0,compressedSize); + } finally { + dataOut.close(); + } + } catch (IOException e) { + throw ServerUtil.parseServerException(e); + } finally { + iterator.close(); + } + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/join/HashCacheFactory.java b/phoenix-core/src/main/java/org/apache/phoenix/join/HashCacheFactory.java new file mode 100644 index 00000000..6c3f69b2 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/join/HashCacheFactory.java @@ -0,0 +1,127 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.join; + +import java.io.*; +import java.sql.SQLException; +import java.util.*; + +import net.jcip.annotations.Immutable; + +import org.apache.hadoop.hbase.client.Result; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.io.WritableUtils; +import org.xerial.snappy.Snappy; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.cache.HashCache; +import org.apache.phoenix.coprocessor.ServerCachingProtocol.ServerCacheFactory; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.ExpressionType; +import org.apache.phoenix.memory.MemoryManager.MemoryChunk; +import org.apache.phoenix.schema.tuple.ResultTuple; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.*; + +public class HashCacheFactory implements ServerCacheFactory { + + public HashCacheFactory() { + } + + @Override + public void readFields(DataInput input) throws IOException { + } + + @Override + public void write(DataOutput output) throws IOException { + } + + @Override + public Closeable newCache(ImmutableBytesWritable cachePtr, MemoryChunk chunk) throws SQLException { + try { + int size = Snappy.uncompressedLength(cachePtr.get()); + byte[] uncompressed = new byte[size]; + Snappy.uncompress(cachePtr.get(), 0, cachePtr.getLength(), uncompressed, 0); + return new HashCacheImpl(uncompressed, chunk); + } catch (IOException e) { + throw ServerUtil.parseServerException(e); + } + } + + @Immutable + private class HashCacheImpl implements HashCache { + private final Map> hashCache; + private final MemoryChunk memoryChunk; + + private HashCacheImpl(byte[] hashCacheBytes, MemoryChunk memoryChunk) { + try { + this.memoryChunk = memoryChunk; + byte[] hashCacheByteArray = hashCacheBytes; + int offset = 0; + ByteArrayInputStream input = new ByteArrayInputStream(hashCacheByteArray, offset, hashCacheBytes.length); + DataInputStream dataInput = new DataInputStream(input); + int nExprs = dataInput.readInt(); + List onExpressions = new ArrayList(nExprs); + for (int i = 0; i < nExprs; i++) { + int expressionOrdinal = WritableUtils.readVInt(dataInput); + Expression expression = ExpressionType.values()[expressionOrdinal].newInstance(); + expression.readFields(dataInput); + onExpressions.add(expression); + } + int exprSize = dataInput.readInt(); + offset += exprSize; + int nRows = dataInput.readInt(); + int estimatedSize = SizedUtil.sizeOfMap(nRows, SizedUtil.IMMUTABLE_BYTES_WRITABLE_SIZE, SizedUtil.RESULT_SIZE) + hashCacheBytes.length; + this.memoryChunk.resize(estimatedSize); + HashMap> hashCacheMap = new HashMap>(nRows * 5 / 4); + offset += Bytes.SIZEOF_INT; + // Build Map with evaluated hash key as key and row as value + for (int i = 0; i < nRows; i++) { + int resultSize = (int)Bytes.readVLong(hashCacheByteArray, offset); + offset += WritableUtils.decodeVIntSize(hashCacheByteArray[offset]); + ImmutableBytesWritable value = new ImmutableBytesWritable(hashCacheByteArray,offset,resultSize); + Tuple result = new ResultTuple(new Result(value)); + ImmutableBytesPtr key = TupleUtil.getConcatenatedValue(result, onExpressions); + List tuples = hashCacheMap.get(key); + if (tuples == null) { + tuples = new ArrayList(1); + hashCacheMap.put(key, tuples); + } + tuples.add(result); + offset += resultSize; + } + this.hashCache = Collections.unmodifiableMap(hashCacheMap); + } catch (IOException e) { // Not possible with ByteArrayInputStream + throw new RuntimeException(e); + } + } + + @Override + public void close() { + memoryChunk.close(); + } + + @Override + public List get(ImmutableBytesPtr hashKey) { + return hashCache.get(hashKey); + } + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/join/HashJoinInfo.java b/phoenix-core/src/main/java/org/apache/phoenix/join/HashJoinInfo.java similarity index 76% rename from phoenix-core/src/main/java/com/salesforce/phoenix/join/HashJoinInfo.java rename to phoenix-core/src/main/java/org/apache/phoenix/join/HashJoinInfo.java index 47a8c2f7..efeb7175 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/join/HashJoinInfo.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/join/HashJoinInfo.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.join; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.join; import java.io.*; import java.util.ArrayList; @@ -34,15 +26,15 @@ import org.apache.hadoop.hbase.client.Scan; import org.apache.hadoop.io.WritableUtils; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.ExpressionType; -import com.salesforce.phoenix.parse.JoinTableNode.JoinType; -import com.salesforce.phoenix.schema.KeyValueSchema; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.KeyValueSchema.KeyValueSchemaBuilder; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.ExpressionType; +import org.apache.phoenix.parse.JoinTableNode.JoinType; +import org.apache.phoenix.schema.KeyValueSchema; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.KeyValueSchema.KeyValueSchemaBuilder; +import org.apache.phoenix.util.SchemaUtil; public class HashJoinInfo { private static final String HASH_JOIN = "HashJoin"; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/join/MaxServerCacheSizeExceededException.java b/phoenix-core/src/main/java/org/apache/phoenix/join/MaxServerCacheSizeExceededException.java new file mode 100644 index 00000000..e72d6284 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/join/MaxServerCacheSizeExceededException.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.join; + +/** + * + * Exception thrown when the size of the hash cache exceeds the + * maximum size as specified by the phoenix.query.maxHashCacheBytes + * parameter in the {@link org.apache.hadoop.conf.Configuration} + * + * @author jtaylor + * @since 0.1 + */ +public class MaxServerCacheSizeExceededException extends RuntimeException { + private static final long serialVersionUID = 1L; + + public MaxServerCacheSizeExceededException() { + } + + public MaxServerCacheSizeExceededException(String message) { + super(message); + } + + public MaxServerCacheSizeExceededException(Throwable cause) { + super(cause); + } + + public MaxServerCacheSizeExceededException(String message, Throwable cause) { + super(message, cause); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/join/ScanProjector.java b/phoenix-core/src/main/java/org/apache/phoenix/join/ScanProjector.java new file mode 100644 index 00000000..9f8777da --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/join/ScanProjector.java @@ -0,0 +1,242 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.join; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.DataInputStream; +import java.io.DataOutputStream; +import java.io.IOException; +import java.util.List; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.client.Scan; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.io.WritableUtils; + +import org.apache.phoenix.compile.JoinCompiler.ProjectedPTableWrapper; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.ExpressionType; +import org.apache.phoenix.schema.KeyValueSchema; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.ValueBitSet; +import org.apache.phoenix.schema.KeyValueSchema.KeyValueSchemaBuilder; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.KeyValueUtil; +import org.apache.phoenix.util.SchemaUtil; + +public class ScanProjector { + public static final byte[] VALUE_COLUMN_FAMILY = Bytes.toBytes("_v"); + public static final byte[] VALUE_COLUMN_QUALIFIER = new byte[0]; + + private static final String SCAN_PROJECTOR = "scanProjector"; + + private final KeyValueSchema schema; + private final Expression[] expressions; + private ValueBitSet valueSet; + private final ImmutableBytesWritable ptr = new ImmutableBytesWritable(); + + public ScanProjector(ProjectedPTableWrapper projected) { + List columns = projected.getTable().getColumns(); + expressions = new Expression[columns.size() - projected.getTable().getPKColumns().size()]; + // we do not count minNullableIndex for we might do later merge. + KeyValueSchemaBuilder builder = new KeyValueSchemaBuilder(0); + int i = 0; + for (PColumn column : projected.getTable().getColumns()) { + if (!SchemaUtil.isPKColumn(column)) { + builder.addField(column); + expressions[i++] = projected.getSourceExpression(column); + } + } + schema = builder.build(); + valueSet = ValueBitSet.newInstance(schema); + } + + private ScanProjector(KeyValueSchema schema, Expression[] expressions) { + this.schema = schema; + this.expressions = expressions; + this.valueSet = ValueBitSet.newInstance(schema); + } + + public void setValueBitSet(ValueBitSet bitSet) { + this.valueSet = bitSet; + } + + public static void serializeProjectorIntoScan(Scan scan, ScanProjector projector) { + ByteArrayOutputStream stream = new ByteArrayOutputStream(); + try { + DataOutputStream output = new DataOutputStream(stream); + projector.schema.write(output); + int count = projector.expressions.length; + WritableUtils.writeVInt(output, count); + for (int i = 0; i < count; i++) { + WritableUtils.writeVInt(output, ExpressionType.valueOf(projector.expressions[i]).ordinal()); + projector.expressions[i].write(output); + } + scan.setAttribute(SCAN_PROJECTOR, stream.toByteArray()); + } catch (IOException e) { + throw new RuntimeException(e); + } finally { + try { + stream.close(); + } catch (IOException e) { + throw new RuntimeException(e); + } + } + + } + + public static ScanProjector deserializeProjectorFromScan(Scan scan) { + byte[] proj = scan.getAttribute(SCAN_PROJECTOR); + if (proj == null) { + return null; + } + ByteArrayInputStream stream = new ByteArrayInputStream(proj); + try { + DataInputStream input = new DataInputStream(stream); + KeyValueSchema schema = new KeyValueSchema(); + schema.readFields(input); + int count = WritableUtils.readVInt(input); + Expression[] expressions = new Expression[count]; + for (int i = 0; i < count; i++) { + int ordinal = WritableUtils.readVInt(input); + expressions[i] = ExpressionType.values()[ordinal].newInstance(); + expressions[i].readFields(input); + } + return new ScanProjector(schema, expressions); + } catch (IOException e) { + throw new RuntimeException(e); + } finally { + try { + stream.close(); + } catch (IOException e) { + throw new RuntimeException(e); + } + } + } + + public static class ProjectedValueTuple implements Tuple { + private ImmutableBytesWritable keyPtr = new ImmutableBytesWritable(); + private long timestamp; + private byte[] projectedValue; + private int bitSetLen; + private KeyValue keyValue; + + private ProjectedValueTuple(byte[] keyBuffer, int keyOffset, int keyLength, long timestamp, byte[] projectedValue, int bitSetLen) { + this.keyPtr.set(keyBuffer, keyOffset, keyLength); + this.timestamp = timestamp; + this.projectedValue = projectedValue; + this.bitSetLen = bitSetLen; + } + + public ImmutableBytesWritable getKeyPtr() { + return keyPtr; + } + + public long getTimestamp() { + return timestamp; + } + + public byte[] getProjectedValue() { + return projectedValue; + } + + public int getBitSetLength() { + return bitSetLen; + } + + @Override + public void getKey(ImmutableBytesWritable ptr) { + ptr.set(keyPtr.get(), keyPtr.getOffset(), keyPtr.getLength()); + } + + @Override + public KeyValue getValue(int index) { + if (index != 0) { + throw new IndexOutOfBoundsException(Integer.toString(index)); + } + return getValue(VALUE_COLUMN_FAMILY, VALUE_COLUMN_QUALIFIER); + } + + @Override + public KeyValue getValue(byte[] family, byte[] qualifier) { + if (keyValue == null) { + keyValue = KeyValueUtil.newKeyValue(keyPtr.get(), keyPtr.getOffset(), keyPtr.getLength(), + VALUE_COLUMN_FAMILY, VALUE_COLUMN_QUALIFIER, timestamp, projectedValue, 0, projectedValue.length); + } + return keyValue; + } + + @Override + public boolean getValue(byte[] family, byte[] qualifier, + ImmutableBytesWritable ptr) { + ptr.set(projectedValue); + return true; + } + + @Override + public boolean isImmutable() { + return true; + } + + @Override + public int size() { + return 1; + } + } + + public ProjectedValueTuple projectResults(Tuple tuple) { + byte[] bytesValue = schema.toBytes(tuple, expressions, valueSet, ptr); + KeyValue base = tuple.getValue(0); + return new ProjectedValueTuple(base.getBuffer(), base.getRowOffset(), base.getRowLength(), base.getTimestamp(), bytesValue, valueSet.getEstimatedLength()); + } + + public static void decodeProjectedValue(Tuple tuple, ImmutableBytesWritable ptr) throws IOException { + boolean b = tuple.getValue(VALUE_COLUMN_FAMILY, VALUE_COLUMN_QUALIFIER, ptr); + if (!b) + throw new IOException("Trying to decode a non-projected value."); + } + + public static ProjectedValueTuple mergeProjectedValue(ProjectedValueTuple dest, KeyValueSchema destSchema, ValueBitSet destBitSet, + Tuple src, KeyValueSchema srcSchema, ValueBitSet srcBitSet, int offset) throws IOException { + ImmutableBytesWritable destValue = new ImmutableBytesWritable(dest.getProjectedValue()); + destBitSet.clear(); + destBitSet.or(destValue); + int origDestBitSetLen = dest.getBitSetLength(); + ImmutableBytesWritable srcValue = new ImmutableBytesWritable(); + decodeProjectedValue(src, srcValue); + srcBitSet.clear(); + srcBitSet.or(srcValue); + int origSrcBitSetLen = srcBitSet.getEstimatedLength(); + for (int i = 0; i < srcBitSet.getMaxSetBit(); i++) { + if (srcBitSet.get(i)) { + destBitSet.set(offset + i); + } + } + int destBitSetLen = destBitSet.getEstimatedLength(); + byte[] merged = new byte[destValue.getLength() - origDestBitSetLen + srcValue.getLength() - origSrcBitSetLen + destBitSetLen]; + int o = Bytes.putBytes(merged, 0, destValue.get(), destValue.getOffset(), destValue.getLength() - origDestBitSetLen); + o = Bytes.putBytes(merged, o, srcValue.get(), srcValue.getOffset(), srcValue.getLength() - origSrcBitSetLen); + destBitSet.toBytes(merged, o); + ImmutableBytesWritable keyPtr = dest.getKeyPtr(); + return new ProjectedValueTuple(keyPtr.get(), keyPtr.getOffset(), keyPtr.getLength(), dest.getTimestamp(), merged, destBitSetLen); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/map/reduce/CSVBulkLoader.java b/phoenix-core/src/main/java/org/apache/phoenix/map/reduce/CSVBulkLoader.java similarity index 86% rename from phoenix-core/src/main/java/com/salesforce/phoenix/map/reduce/CSVBulkLoader.java rename to phoenix-core/src/main/java/org/apache/phoenix/map/reduce/CSVBulkLoader.java index d7461a4b..e7c61596 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/map/reduce/CSVBulkLoader.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/map/reduce/CSVBulkLoader.java @@ -1,31 +1,23 @@ -/******************************************************************************* -* Copyright (c) 2013, Salesforce.com, Inc. -* All rights reserved. -* -* Redistribution and use in source and binary forms, with or without -* modification, are permitted provided that the following conditions are met: -* -* Redistributions of source code must retain the above copyright notice, -* this list of conditions and the following disclaimer. -* Redistributions in binary form must reproduce the above copyright notice, -* this list of conditions and the following disclaimer in the documentation -* and/or other materials provided with the distribution. -* Neither the name of Salesforce.com nor the names of its contributors may -* be used to endorse or promote products derived from this software without -* specific prior written permission. -* -* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -* DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -* OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -******************************************************************************/ -package com.salesforce.phoenix.map.reduce; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.map.reduce; import java.io.BufferedReader; import java.io.BufferedWriter; @@ -60,11 +52,11 @@ import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; import com.google.common.io.Closeables; -import com.salesforce.phoenix.map.reduce.util.ConfigReader; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.util.PhoenixRuntime; -import com.salesforce.phoenix.util.SchemaUtil; -import com.salesforce.phoenix.util.StringUtil; +import org.apache.phoenix.map.reduce.util.ConfigReader; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.util.PhoenixRuntime; +import org.apache.phoenix.util.SchemaUtil; +import org.apache.phoenix.util.StringUtil; public class CSVBulkLoader { private static final String UNDERSCORE = "_"; diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/map/reduce/MapReduceJob.java b/phoenix-core/src/main/java/org/apache/phoenix/map/reduce/MapReduceJob.java similarity index 76% rename from phoenix-core/src/main/java/com/salesforce/phoenix/map/reduce/MapReduceJob.java rename to phoenix-core/src/main/java/org/apache/phoenix/map/reduce/MapReduceJob.java index 89e4ceaa..6dd9910e 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/map/reduce/MapReduceJob.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/map/reduce/MapReduceJob.java @@ -1,31 +1,23 @@ -/******************************************************************************* -* Copyright (c) 2013, Salesforce.com, Inc. -* All rights reserved. -* -* Redistribution and use in source and binary forms, with or without -* modification, are permitted provided that the following conditions are met: -* -* Redistributions of source code must retain the above copyright notice, -* this list of conditions and the following disclaimer. -* Redistributions in binary form must reproduce the above copyright notice, -* this list of conditions and the following disclaimer in the documentation -* and/or other materials provided with the distribution. -* Neither the name of Salesforce.com nor the names of its contributors may -* be used to endorse or promote products derived from this software without -* specific prior written permission. -* -* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND -* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED -* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE -* DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE -* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL -* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR -* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER -* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, -* OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE -* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -******************************************************************************/ -package com.salesforce.phoenix.map.reduce; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.map.reduce; import java.io.ByteArrayInputStream; import java.io.IOException; @@ -52,9 +44,9 @@ import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Mapper; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.util.PhoenixRuntime; -import com.salesforce.phoenix.util.QueryUtil; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.util.PhoenixRuntime; +import org.apache.phoenix.util.QueryUtil; public class MapReduceJob { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/map/reduce/util/ConfigReader.java b/phoenix-core/src/main/java/org/apache/phoenix/map/reduce/util/ConfigReader.java new file mode 100644 index 00000000..8f739f5c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/map/reduce/util/ConfigReader.java @@ -0,0 +1,114 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.map.reduce.util; + +import java.util.Enumeration; +import java.util.HashMap; +import java.util.Map; +import java.util.ResourceBundle; + +/** + * Class to read configs. + * + */ + +public class ConfigReader + { + + private String propertyFile = null; + private boolean loaded = false; + private static final Object _synObj = new Object(); + private Map properties = new HashMap(); + private Exception loadException = null; + + /** + * Retrieves singleton config objects from a hashmap of stored objects, + * creates these objects if they aren't in the hashmap. + */ + + public ConfigReader(String propertyFile) { + this.propertyFile = propertyFile; + } + + public void load() throws Exception { + if (loaded) { + if (loadException != null) { + throw new Exception(loadException); + } + return; + } + synchronized (_synObj) { + if (!loaded) { + try { + String tmpFile = propertyFile.trim(); + if (tmpFile.endsWith(".properties")) { + tmpFile = tmpFile + .substring(0, tmpFile.lastIndexOf(".")); + } + ResourceBundle resource = ResourceBundle.getBundle(tmpFile); + Enumeration enm = resource.getKeys(); + + while (enm.hasMoreElements()) { + String key = enm.nextElement(); + String value = resource.getString(key); + properties.put(key, value); + } + } catch (Exception e) { + System.err + .println("Exception while loading the config.properties file :: " + + e.getMessage()); + loadException = e; + loaded = true; + throw e; + } + loaded = true; + } + } + } + + public void addConfig(String key, String value) { + try { + load(); + } catch (Exception e) { + System.err.println("ERROR :: " + e.getMessage()); + } + properties.put(key, value); + } + + public boolean hasConfig(String key) { + try { + load(); + } catch (Exception e) { + System.err.println("ERROR :: " + e.getMessage()); + } + return properties.containsKey(key); + } + + public String getConfig(String key) throws Exception { + load(); + return properties.get(key); + } + + public Map getAllConfigMap() throws Exception { + load(); + return properties; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/memory/ChildMemoryManager.java b/phoenix-core/src/main/java/org/apache/phoenix/memory/ChildMemoryManager.java new file mode 100644 index 00000000..925cdfd3 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/memory/ChildMemoryManager.java @@ -0,0 +1,118 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.memory; + +import org.apache.http.annotation.GuardedBy; +import org.apache.http.annotation.ThreadSafe; + +/** + * + * Child memory manager that delegates through to global memory manager, + * but enforces that at most a threshold percentage is used by this + * memory manager. No blocking is done if the threshold is exceeded, + * but the standard blocking will be done by the global memory manager. + * + * @author jtaylor + * @since 0.1 + */ +@ThreadSafe +public class ChildMemoryManager extends DelegatingMemoryManager { + private final Object sync = new Object(); + private final int maxPercOfTotal; + @GuardedBy("sync") + private long allocatedBytes; + + public ChildMemoryManager(MemoryManager mm, int maxPercOfTotal) { + super(mm); + if (mm instanceof ChildMemoryManager) { + throw new IllegalStateException("ChildMemoryManager cannot delegate to another ChildMemoryManager"); + } + this.maxPercOfTotal = maxPercOfTotal; + if (maxPercOfTotal <= 0 || maxPercOfTotal > 100) { + throw new IllegalArgumentException("Max percentage of total memory (" + maxPercOfTotal + "%) must be greater than zero and less than or equal to 100"); + } + } + + + private long adjustAllocation(long minBytes, long reqBytes) { + assert(reqBytes >= minBytes); + long availBytes = getAvailableMemory(); + // Check if this memory managers percentage of allocated bytes exceeds its allowed maximum + if (minBytes > availBytes) { + throw new InsufficientMemoryException("Attempt to allocate more memory than the max allowed of " + maxPercOfTotal + "%"); + } + // Revise reqBytes down to available memory if necessary + return Math.min(reqBytes,availBytes); + } + + @Override + public MemoryChunk allocate(long minBytes, long nBytes) { + synchronized (sync) { + nBytes = adjustAllocation(minBytes, nBytes); + final MemoryChunk chunk = super.allocate(minBytes, nBytes); + allocatedBytes += chunk.getSize(); + // Instantiate delegate chunk to track allocatedBytes correctly + return new MemoryChunk() { + @Override + public void close() { + synchronized (sync) { + allocatedBytes -= chunk.getSize(); + chunk.close(); + } + } + + @Override + public long getSize() { + return chunk.getSize(); + } + + @Override + public void resize(long nBytes) { + synchronized (sync) { + long size = getSize(); + long deltaBytes = nBytes - size; + if (deltaBytes > 0) { + adjustAllocation(deltaBytes,deltaBytes); // Throw if too much memory + } + chunk.resize(nBytes); + allocatedBytes += deltaBytes; + } + } + }; + } + } + + @Override + public long getAvailableMemory() { + synchronized (sync) { + long availBytes = getMaxMemory() - allocatedBytes; + // Sanity check (should never happen) + if (availBytes < 0) { + throw new IllegalStateException("Available memory has become negative: " + availBytes + " bytes. Allocated memory: " + allocatedBytes + " bytes."); + } + return availBytes; + } + } + + @Override + public long getMaxMemory() { + return maxPercOfTotal * super.getMaxMemory() / 100; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/memory/DelegatingMemoryManager.java b/phoenix-core/src/main/java/org/apache/phoenix/memory/DelegatingMemoryManager.java new file mode 100644 index 00000000..f50c43d9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/memory/DelegatingMemoryManager.java @@ -0,0 +1,61 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.memory; + +/** + * + * Memory manager that delegates through to another memory manager. + * + * @author jtaylor + * @since 0.1 + */ +public class DelegatingMemoryManager implements MemoryManager { + private final MemoryManager parent; + + public DelegatingMemoryManager(MemoryManager globalMemoryManager){ + this.parent = globalMemoryManager; + } + + @Override + public long getAvailableMemory() { + return parent.getAvailableMemory(); + } + + @Override + public long getMaxMemory() { + return parent.getMaxMemory(); + } + + @Override + public MemoryChunk allocate(long minBytes, long reqBytes) { + return parent.allocate(minBytes, reqBytes); + } + + + @Override + public MemoryChunk allocate(long nBytes) { + return allocate(nBytes, nBytes); + } + + public MemoryManager getParent() { + return parent; + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/memory/GlobalMemoryManager.java b/phoenix-core/src/main/java/org/apache/phoenix/memory/GlobalMemoryManager.java similarity index 76% rename from phoenix-core/src/main/java/com/salesforce/phoenix/memory/GlobalMemoryManager.java rename to phoenix-core/src/main/java/org/apache/phoenix/memory/GlobalMemoryManager.java index fc79e823..38a3cd98 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/memory/GlobalMemoryManager.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/memory/GlobalMemoryManager.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.memory; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.memory; import org.apache.http.annotation.GuardedBy; import org.slf4j.Logger; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/memory/InsufficientMemoryException.java b/phoenix-core/src/main/java/org/apache/phoenix/memory/InsufficientMemoryException.java new file mode 100644 index 00000000..8e7eaff6 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/memory/InsufficientMemoryException.java @@ -0,0 +1,47 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.memory; + +/** + * + * Exception thrown by MemoryManager when insufficient memory is available + * + * @author jtaylor + * @since 0.1 + */ +public class InsufficientMemoryException extends RuntimeException { + private static final long serialVersionUID = 1L; + + public InsufficientMemoryException() { + } + + public InsufficientMemoryException(String message) { + super(message); + } + + public InsufficientMemoryException(Throwable cause) { + super(cause); + } + + public InsufficientMemoryException(String message, Throwable cause) { + super(message, cause); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/memory/MemoryManager.java b/phoenix-core/src/main/java/org/apache/phoenix/memory/MemoryManager.java new file mode 100644 index 00000000..e9b93557 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/memory/MemoryManager.java @@ -0,0 +1,97 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.memory; + +import java.io.Closeable; + +/** + * + * Memory manager used to track memory usage. Either throttles + * memory usage by blocking when the max memory is reached or + * allocates up to a maximum without blocking. + * + * @author jtaylor + * @since 0.1 + */ +public interface MemoryManager { + /** + * Get the total amount of memory (in bytes) that may be allocated. + */ + long getMaxMemory(); + + /** + * Get the amount of available memory (in bytes) not yet allocated. + */ + long getAvailableMemory(); + + /** + * Allocate up to reqBytes of memory, dialing the amount down to + * minBytes if full amount is not available. If minBytes is not + * available, then this call will block for a configurable amount + * of time and throw if minBytes does not become available. + * @param minBytes minimum number of bytes required + * @param reqBytes requested number of bytes. Must be greater + * than or equal to minBytes + * @return MemoryChunk that was allocated + * @throws InsufficientMemoryException if unable to allocate minBytes + * during configured amount of time + */ + MemoryChunk allocate(long minBytes, long reqBytes); + + /** + * Equivalent to calling {@link #allocate(long, long)} where + * minBytes and reqBytes being the same. + */ + MemoryChunk allocate(long nBytes); + + /** + * + * Chunk of allocated memory. To reclaim the memory, call {@link #close()} + * + * @author jtaylor + * @since 0.1 + */ + public static interface MemoryChunk extends Closeable { + /** + * Get the size in bytes of the allocated chunk. + */ + long getSize(); + + /** + * Free up the memory associated with this chunk + */ + @Override + void close(); + + /** + * Resize an already allocated memory chunk up or down to a + * new amount. If decreasing allocation, this call will not block. + * If increasing allocation, and nBytes is not available, then + * this call will block for a configurable amount of time and + * throw if nBytes does not become available. Most commonly + * used to adjust the allocation of a memory buffer that was + * originally sized for the worst case scenario. + * @param nBytes new number of bytes required for this chunk + * @throws InsufficientMemoryException if unable to allocate minBytes + * during configured amount of time + */ + void resize(long nBytes); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/optimize/QueryOptimizer.java b/phoenix-core/src/main/java/org/apache/phoenix/optimize/QueryOptimizer.java similarity index 91% rename from phoenix-core/src/main/java/com/salesforce/phoenix/optimize/QueryOptimizer.java rename to phoenix-core/src/main/java/org/apache/phoenix/optimize/QueryOptimizer.java index 329da991..b60f742c 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/optimize/QueryOptimizer.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/optimize/QueryOptimizer.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.optimize; +package org.apache.phoenix.optimize; import java.sql.SQLException; import java.util.Collections; @@ -6,25 +6,25 @@ import java.util.List; import com.google.common.collect.Lists; -import com.salesforce.phoenix.compile.ColumnProjector; -import com.salesforce.phoenix.compile.IndexStatementRewriter; -import com.salesforce.phoenix.compile.QueryCompiler; -import com.salesforce.phoenix.compile.QueryPlan; -import com.salesforce.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; -import com.salesforce.phoenix.jdbc.PhoenixStatement; -import com.salesforce.phoenix.parse.HintNode; -import com.salesforce.phoenix.parse.HintNode.Hint; -import com.salesforce.phoenix.parse.ParseNodeFactory; -import com.salesforce.phoenix.parse.SelectStatement; -import com.salesforce.phoenix.parse.TableNode; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PDatum; -import com.salesforce.phoenix.schema.PIndexState; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableType; +import org.apache.phoenix.compile.ColumnProjector; +import org.apache.phoenix.compile.IndexStatementRewriter; +import org.apache.phoenix.compile.QueryCompiler; +import org.apache.phoenix.compile.QueryPlan; +import org.apache.phoenix.iterate.ParallelIterators.ParallelIteratorFactory; +import org.apache.phoenix.jdbc.PhoenixStatement; +import org.apache.phoenix.parse.HintNode; +import org.apache.phoenix.parse.HintNode.Hint; +import org.apache.phoenix.parse.ParseNodeFactory; +import org.apache.phoenix.parse.SelectStatement; +import org.apache.phoenix.parse.TableNode; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PDatum; +import org.apache.phoenix.schema.PIndexState; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableType; public class QueryOptimizer { private static final ParseNodeFactory FACTORY = new ParseNodeFactory(); diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/AddColumnStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/AddColumnStatement.java new file mode 100644 index 00000000..1059fce7 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/AddColumnStatement.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.Collections; +import java.util.List; +import java.util.Map; + +import org.apache.phoenix.schema.PTableType; + +public class AddColumnStatement extends AlterTableStatement { + private final List columnDefs; + private final boolean ifNotExists; + private final Map props; + + protected AddColumnStatement(NamedTableNode table, PTableType tableType, List columnDefs, boolean ifNotExists, Map props) { + super(table, tableType); + this.columnDefs = columnDefs; + this.props = props == null ? Collections.emptyMap() : props; + this.ifNotExists = ifNotExists; + } + + public List getColumnDefs() { + return columnDefs; + } + + public boolean ifNotExists() { + return ifNotExists; + } + + public Map getProps() { + return props; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/AddParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/AddParseNode.java new file mode 100644 index 00000000..f4ae4910 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/AddParseNode.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing addition in a SQL expression + * + * @author jtaylor + * @since 0.1 + */ +public class AddParseNode extends ArithmeticParseNode { + + AddParseNode(List children) { + super(children); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/AggregateFunctionParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/AggregateFunctionParseNode.java new file mode 100644 index 00000000..a715dd75 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/AggregateFunctionParseNode.java @@ -0,0 +1,39 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.List; + + +public class AggregateFunctionParseNode extends FunctionParseNode { + + public AggregateFunctionParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + /** + * Aggregate function are not stateless, even though all the args may be stateless, + * for example, COUNT(1) + */ + @Override + public boolean isStateless() { + return false; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/AliasedNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/AliasedNode.java new file mode 100644 index 00000000..d2302fde --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/AliasedNode.java @@ -0,0 +1,53 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.phoenix.util.SchemaUtil; + +/** + * + * Node representing an aliased parse node in a SQL select clause + * + * @author jtaylor + * @since 0.1 + */ +public class AliasedNode { + private final String alias; + private final ParseNode node; + private final boolean isCaseSensitve; + + public AliasedNode(String alias, ParseNode node) { + this.isCaseSensitve = alias != null && SchemaUtil.isCaseSensitive(alias); + this.alias = alias == null ? null : SchemaUtil.normalizeIdentifier(alias); + this.node = node; + } + + public String getAlias() { + return alias; + } + + public ParseNode getNode() { + return node; + } + + public boolean isCaseSensitve() { + return isCaseSensitve; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/AlterIndexStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/AlterIndexStatement.java new file mode 100644 index 00000000..bee74989 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/AlterIndexStatement.java @@ -0,0 +1,53 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.phoenix.schema.PIndexState; + +public class AlterIndexStatement extends SingleTableSQLStatement { + private final String dataTableName; + private final boolean ifExists; + private final PIndexState indexState; + + public AlterIndexStatement(NamedTableNode indexTableNode, String dataTableName, boolean ifExists, PIndexState indexState) { + super(indexTableNode,0); + this.dataTableName = dataTableName; + this.ifExists = ifExists; + this.indexState = indexState; + } + + public String getTableName() { + return dataTableName; + } + + @Override + public int getBindCount() { + return 0; + } + + public boolean ifExists() { + return ifExists; + } + + public PIndexState getIndexState() { + return indexState; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/AlterTableStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/AlterTableStatement.java new file mode 100644 index 00000000..e6d4c807 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/AlterTableStatement.java @@ -0,0 +1,35 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.phoenix.schema.PTableType; + +public abstract class AlterTableStatement extends SingleTableSQLStatement { + private final PTableType tableType; + + AlterTableStatement(NamedTableNode table, PTableType tableType) { + super(table, 0); + this.tableType = tableType; + } + + public PTableType getTableType() { + return tableType; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/AndParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/AndParseNode.java new file mode 100644 index 00000000..452d8935 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/AndParseNode.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + + +/** + * + * Node representing AND in a SQL expression + * + * @author jtaylor + * @since 0.1 + */ +public class AndParseNode extends CompoundParseNode { + + AndParseNode(List children) { + super(children); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ArithmeticParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ArithmeticParseNode.java new file mode 100644 index 00000000..4e909608 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ArithmeticParseNode.java @@ -0,0 +1,30 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.List; + +public abstract class ArithmeticParseNode extends CompoundParseNode { + + public ArithmeticParseNode(List children) { + super(children); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ArrayConstructorNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ArrayConstructorNode.java new file mode 100644 index 00000000..7d62b32c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ArrayConstructorNode.java @@ -0,0 +1,44 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +/** + * Holds the list of array elements that will be used by the upsert stmt with ARRAY column + * + */ +public class ArrayConstructorNode extends CompoundParseNode { + + public ArrayConstructorNode(List children) { + super(children); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/AvgAggregateParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/AvgAggregateParseNode.java new file mode 100644 index 00000000..0b944c0d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/AvgAggregateParseNode.java @@ -0,0 +1,48 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.*; + + +public class AvgAggregateParseNode extends AggregateFunctionParseNode { + + public AvgAggregateParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public FunctionExpression create(List children, StatementContext context) throws SQLException { + SumAggregateFunction sumFunc; + CountAggregateFunction countFunc = (CountAggregateFunction)context.getExpressionManager().addIfAbsent(new CountAggregateFunction(children)); + if (!countFunc.isConstantExpression()) { + sumFunc = (SumAggregateFunction)context.getExpressionManager().addIfAbsent(new SumAggregateFunction(countFunc.getChildren(),null)); + } else { + sumFunc = null; + } + + return new AvgAggregateFunction(children, countFunc, sumFunc); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/BaseParseNodeVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/BaseParseNodeVisitor.java new file mode 100644 index 00000000..233d5ec8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/BaseParseNodeVisitor.java @@ -0,0 +1,70 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.sql.SQLFeatureNotSupportedException; +import java.util.List; + + + +/** + * + * Base class for parse node visitors. + * + * @author jtaylor + * @since 0.1 + */ +public abstract class BaseParseNodeVisitor implements ParseNodeVisitor { + + /** + * Fall through visitEnter method. Anything coming through + * here means that a more specific method wasn't found + * and thus this CompoundNode is not yet supported. + */ + @Override + public boolean visitEnter(CompoundParseNode expressionNode) throws SQLException { + throw new SQLFeatureNotSupportedException(expressionNode.toString()); + } + + @Override + public E visitLeave(CompoundParseNode expressionNode, List l) throws SQLException { + throw new SQLFeatureNotSupportedException(expressionNode.toString()); + } + + /** + * Fall through visit method. Anything coming through + * here means that a more specific method wasn't found + * and thus this Node is not yet supported. + */ + @Override + public E visit(ParseNode expressionNode) throws SQLException { + throw new SQLFeatureNotSupportedException(expressionNode.toString()); + } + + @Override + public List newElementList(int size) { + return null; + } + + @Override + public void addElement(List l, E element) { + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/BetweenParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/BetweenParseNode.java new file mode 100644 index 00000000..6306d67e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/BetweenParseNode.java @@ -0,0 +1,54 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.*; + + + +/** + * + * Node representing BETWEEN in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class BetweenParseNode extends CompoundParseNode { + private final boolean negate; + + BetweenParseNode(ParseNode l, ParseNode r1, ParseNode r2, boolean negate) { + super(Arrays.asList(l, r1, r2)); + this.negate = negate; + } + + public boolean isNegate() { + return negate; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/BinaryParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/BinaryParseNode.java new file mode 100644 index 00000000..82695f52 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/BinaryParseNode.java @@ -0,0 +1,44 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.Arrays; + +/** + * + * Abstract class for operators that operate on exactly two nodes + * + * @author jtaylor + * @since 0.1 + */ +public abstract class BinaryParseNode extends CompoundParseNode { + + BinaryParseNode(ParseNode lhs, ParseNode rhs) { + super(Arrays.asList(lhs, rhs)); + } + + public ParseNode getLHS() { + return getChildren().get(0); + } + + public ParseNode getRHS() { + return getChildren().get(1); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/BindParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/BindParseNode.java new file mode 100644 index 00000000..5d4f1734 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/BindParseNode.java @@ -0,0 +1,61 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; + + + +/** + * + * Node representing a bind variable in a SQL expression + * + * @author jtaylor + * @since 0.1 + */ +public class BindParseNode extends NamedParseNode { + private final int index; + + BindParseNode(String name) { + super(name); + index = Integer.parseInt(name); + } + + public int getIndex() { + return index-1; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + return visitor.visit(this); + } + + + @Override + public boolean isStateless() { + return true; + } + + @Override + public String toString() { + return ":" + index; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/BindTableNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/BindTableNode.java new file mode 100644 index 00000000..7aca261e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/BindTableNode.java @@ -0,0 +1,44 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; + + + +/** + * + * Node representing a TABLE bound using an ARRAY variable + * TODO: modify grammar to support this + * @author jtaylor + * @since 0.1 + */ +public class BindTableNode extends ConcreteTableNode { + + BindTableNode(String alias, TableName name) { + super(alias, name); + } + + @Override + public void accept(TableNodeVisitor visitor) throws SQLException { + visitor.visit(this); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/BindableStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/BindableStatement.java new file mode 100644 index 00000000..7ce9162f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/BindableStatement.java @@ -0,0 +1,25 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + + +public interface BindableStatement { + public int getBindCount(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/CaseParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/CaseParseNode.java new file mode 100644 index 00000000..bd044900 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/CaseParseNode.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing a CASE in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class CaseParseNode extends CompoundParseNode { + + CaseParseNode(List children) { + super(children); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/CastParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/CastParseNode.java new file mode 100644 index 00000000..24cae703 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/CastParseNode.java @@ -0,0 +1,80 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.RoundDecimalExpression; +import org.apache.phoenix.expression.function.RoundTimestampExpression; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.TypeMismatchException; + +/** + * + * Node representing the CAST operator in SQL. + * + * @author samarth.jain + * @since 0.1 + * + */ +public class CastParseNode extends UnaryParseNode { + + private final PDataType dt; + + CastParseNode(ParseNode expr, String dataType) { + super(expr); + dt = PDataType.fromSqlTypeName(dataType); + } + + CastParseNode(ParseNode expr, PDataType dataType) { + super(expr); + dt = dataType; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } + + public PDataType getDataType() { + return dt; + } + + public static Expression convertToRoundExpressionIfNeeded(PDataType fromDataType, PDataType targetDataType, List expressions) throws SQLException { + Expression firstChildExpr = expressions.get(0); + if(fromDataType == targetDataType) { + return firstChildExpr; + } else if(fromDataType == PDataType.DECIMAL && targetDataType.isCoercibleTo(PDataType.LONG)) { + return new RoundDecimalExpression(expressions); + } else if((fromDataType == PDataType.TIMESTAMP || fromDataType == PDataType.UNSIGNED_TIMESTAMP) && targetDataType.isCoercibleTo(PDataType.DATE)) { + return RoundTimestampExpression.create(expressions); + } else if(!fromDataType.isCoercibleTo(targetDataType)) { + throw TypeMismatchException.newException(fromDataType, targetDataType, firstChildExpr.toString()); + } + return firstChildExpr; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/CeilParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/CeilParseNode.java new file mode 100644 index 00000000..b041c843 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/CeilParseNode.java @@ -0,0 +1,82 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.CeilDateExpression; +import org.apache.phoenix.expression.function.CeilDecimalExpression; +import org.apache.phoenix.expression.function.CeilFunction; +import org.apache.phoenix.expression.function.CeilTimestampExpression; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.TypeMismatchException; + +/** + * Parse node corresponding to {@link CeilFunction}. + * It also acts as a factory for creating the right kind of + * ceil expression according to the data type of the + * first child. + * + * @author samarth.jain + * @since 3.0.0 + */ +public class CeilParseNode extends FunctionParseNode { + + CeilParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public Expression create(List children, StatementContext context) throws SQLException { + return getCeilExpression(children); + } + + public static Expression getCeilExpression(List children) throws SQLException { + final Expression firstChild = children.get(0); + final PDataType firstChildDataType = firstChild.getDataType(); + if(firstChildDataType.isCoercibleTo(PDataType.DATE)) { + return CeilDateExpression.create(children); + } else if (firstChildDataType == PDataType.TIMESTAMP || firstChildDataType == PDataType.UNSIGNED_TIMESTAMP) { + return CeilTimestampExpression.create(children); + } else if(firstChildDataType.isCoercibleTo(PDataType.DECIMAL)) { + return new CeilDecimalExpression(children); + } else { + throw TypeMismatchException.newException(firstChildDataType, "1"); + } + } + + /** + * When ceiling off decimals, user need not specify the scale. In such cases, + * we need to prevent the function from getting evaluated as null. This is really + * a hack. A better way would have been if {@link org.apache.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo} provided a + * way of associating default values for each permissible data type. + * Something like: @ Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValues = {"null", "1"} isConstant=true) + * Till then, this will have to do. + */ + @Override + public boolean evalToNullIfParamIsNull(StatementContext context, int index) throws SQLException { + return index == 0; + } + + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnDef.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnDef.java new file mode 100644 index 00000000..6e8d87fd --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnDef.java @@ -0,0 +1,200 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.util.SchemaUtil; + + +/** + * + * Represents a column definition during DDL + * + * @author jtaylor + * @since 0.1 + */ +public class ColumnDef { + private final ColumnName columnDefName; + private PDataType dataType; + private final boolean isNull; + private final Integer maxLength; + private final Integer scale; + private final boolean isPK; + private final ColumnModifier columnModifier; + private final boolean isArray; + private final Integer arrSize; + + ColumnDef(ColumnName columnDefName, String sqlTypeName, boolean isArray, Integer arrSize, boolean isNull, Integer maxLength, + Integer scale, boolean isPK, ColumnModifier columnModifier) { + try { + PDataType localType = null; + this.columnDefName = columnDefName; + this.isArray = isArray; + // TODO : Add correctness check for arrSize. Should this be ignored as in postgres + // Also add what is the limit that we would support. Are we going to support a + // fixed size or like postgres allow infinite. May be the data types max limit can + // be used for the array size (May be too big) + if(this.isArray) { + localType = sqlTypeName == null ? null : PDataType.fromTypeId(PDataType.sqlArrayType(SchemaUtil.normalizeIdentifier(sqlTypeName))); + this.dataType = sqlTypeName == null ? null : PDataType.fromSqlTypeName(SchemaUtil.normalizeIdentifier(sqlTypeName)); + this.arrSize = arrSize; // Can only be non negative based on parsing + } else { + this.dataType = sqlTypeName == null ? null : PDataType.fromSqlTypeName(SchemaUtil.normalizeIdentifier(sqlTypeName)); + this.arrSize = null; + } + + this.isNull = isNull; + if (!this.isArray) { + if (this.dataType == PDataType.CHAR) { + if (maxLength == null) { + throw new SQLExceptionInfo.Builder( + SQLExceptionCode.MISSING_CHAR_LENGTH) + .setColumnName(columnDefName.getColumnName()) + .build().buildException(); + } + if (maxLength < 1) { + throw new SQLExceptionInfo.Builder( + SQLExceptionCode.NONPOSITIVE_CHAR_LENGTH) + .setColumnName(columnDefName.getColumnName()) + .build().buildException(); + } + scale = null; + } else if (this.dataType == PDataType.VARCHAR) { + if (maxLength != null && maxLength < 1) { + throw new SQLExceptionInfo.Builder( + SQLExceptionCode.NONPOSITIVE_CHAR_LENGTH) + .setColumnName(columnDefName.getColumnName()) + .build().buildException(); + } + scale = null; + } else if (this.dataType == PDataType.DECIMAL) { + Integer origMaxLength = maxLength; + maxLength = maxLength == null ? PDataType.MAX_PRECISION + : maxLength; + // for deciaml, 1 <= maxLength <= PDataType.MAX_PRECISION; + if (maxLength < 1 || maxLength > PDataType.MAX_PRECISION) { + throw new SQLExceptionInfo.Builder( + SQLExceptionCode.DECIMAL_PRECISION_OUT_OF_RANGE) + .setColumnName(columnDefName.getColumnName()) + .build().buildException(); + } + // When a precision is specified and a scale is not + // specified, it is set to 0. + // + // This is the standard as specified in + // http://docs.oracle.com/cd/B28359_01/server.111/b28318/datatype.htm#CNCPT1832 + // and + // http://docs.oracle.com/javadb/10.6.2.1/ref/rrefsqlj15260.html. + // Otherwise, if scale is bigger than maxLength, just set it + // to the maxLength; + // + // When neither a precision nor a scale is specified, the + // precision and scale is + // ignored. All decimal are stored with as much decimal + // points as possible. + scale = scale == null ? origMaxLength == null ? null + : PDataType.DEFAULT_SCALE + : scale > maxLength ? maxLength : scale; + } else if (this.dataType == PDataType.BINARY) { + if (maxLength == null) { + throw new SQLExceptionInfo.Builder( + SQLExceptionCode.MISSING_BINARY_LENGTH) + .setColumnName(columnDefName.getColumnName()) + .build().buildException(); + } + if (maxLength < 1) { + throw new SQLExceptionInfo.Builder( + SQLExceptionCode.NONPOSITIVE_BINARY_LENGTH) + .setColumnName(columnDefName.getColumnName()) + .build().buildException(); + } + scale = null; + } else if (this.dataType == PDataType.INTEGER) { + maxLength = PDataType.INT_PRECISION; + scale = PDataType.ZERO; + } else if (this.dataType == PDataType.LONG) { + maxLength = PDataType.LONG_PRECISION; + scale = PDataType.ZERO; + } else { + // ignore maxLength and scale for other types. + maxLength = null; + scale = null; + } + } else { + maxLength = null; + scale = null; + } + this.maxLength = maxLength; + this.scale = scale; + this.isPK = isPK; + this.columnModifier = columnModifier; + if(this.isArray) { + this.dataType = localType; + } + } catch (SQLException e) { + throw new ParseException(e); + } + } + ColumnDef(ColumnName columnDefName, String sqlTypeName, boolean isNull, Integer maxLength, + Integer scale, boolean isPK, ColumnModifier columnModifier) { + this(columnDefName, sqlTypeName, false, 0, isNull, maxLength, scale, isPK, columnModifier); + } + + public ColumnName getColumnDefName() { + return columnDefName; + } + + public PDataType getDataType() { + return dataType; + } + + public boolean isNull() { + return isNull; + } + + public Integer getMaxLength() { + return maxLength; + } + + public Integer getScale() { + return scale; + } + + public boolean isPK() { + return isPK; + } + + public ColumnModifier getColumnModifier() { + return columnModifier; + } + + public boolean isArray() { + return isArray; + } + + public Integer getArraySize() { + return arrSize; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnFamilyDef.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnFamilyDef.java new file mode 100644 index 00000000..53230664 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnFamilyDef.java @@ -0,0 +1,57 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.*; + + +import com.google.common.collect.ImmutableList; +import org.apache.phoenix.util.SchemaUtil; + +/** + * + * Definition of a Column Family at DDL time + * + * @author jtaylor + * @since 0.1 + */ +public class ColumnFamilyDef { + private final String name; + private final List columnDefs; + private final Map props; + + ColumnFamilyDef(String name, List columnDefs, Map props) { + this.name = SchemaUtil.normalizeIdentifier(name); + this.columnDefs = ImmutableList.copyOf(columnDefs); + this.props = props == null ? Collections.emptyMap() : props; + } + + public String getName() { + return name; + } + + public List getColumnDefs() { + return columnDefs; + } + + public Map getProps() { + return props; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnName.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnName.java new file mode 100644 index 00000000..34511c7e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnName.java @@ -0,0 +1,102 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.phoenix.util.SchemaUtil; + + +public class ColumnName { + private final NamedNode familyNode; + private final NamedNode columnNode; + + public static ColumnName caseSensitiveColumnName(String familyName, String columnName) { + return new ColumnName(NamedNode.caseSensitiveNamedNode(familyName), NamedNode.caseSensitiveNamedNode(columnName)); + } + + public static ColumnName caseSensitiveColumnName(String columnName) { + return new ColumnName(null, NamedNode.caseSensitiveNamedNode(columnName)); + } + + public static ColumnName newColumnName(NamedNode columnName) { + return new ColumnName(null, columnName); + } + + public static ColumnName newColumnName(NamedNode familyName, NamedNode columnName) { + return new ColumnName(familyName, columnName); + } + + private ColumnName(NamedNode familyNode, NamedNode columnNode) { + this.familyNode = familyNode; + this.columnNode = columnNode; + } + + + ColumnName(String familyName, String columnName) { + this.familyNode = familyName == null ? null : new NamedNode(familyName); + this.columnNode = new NamedNode(columnName); + } + + ColumnName(String columnName) { + this(null, columnName); + } + + public String getFamilyName() { + return familyNode == null ? null : familyNode.getName(); + } + + public String getColumnName() { + return columnNode.getName(); + } + + public NamedNode getFamilyNode() { + return familyNode; + } + + public NamedNode getColumnNode() { + return columnNode; + } + + @Override + public String toString() { + return SchemaUtil.getColumnName(getFamilyName(),getColumnName()); + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + columnNode.hashCode(); + result = prime * result + ((familyNode == null) ? 0 : familyNode.hashCode()); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + ColumnName other = (ColumnName)obj; + if (!columnNode.equals(other.columnNode)) return false; + if (familyNode == null) { + if (other.familyNode != null) return false; + } else if (!familyNode.equals(other.familyNode)) return false; + return true; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnParseNode.java new file mode 100644 index 00000000..33058e99 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ColumnParseNode.java @@ -0,0 +1,90 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; + +import org.apache.phoenix.query.QueryConstants; + +/** + * Node representing a reference to a column in a SQL expression + * + * @author jtaylor + * @since 0.1 + */ +public class ColumnParseNode extends NamedParseNode { + private final TableName tableName; + private final String fullName; + private final String alias; + + public ColumnParseNode(TableName tableName, String name, String alias) { + // Upper case here so our Maps can depend on this (and we don't have to upper case and create a string on every + // lookup + super(name); + this.alias = alias; + this.tableName = tableName; + fullName = tableName == null ? getName() : tableName.toString() + QueryConstants.NAME_SEPARATOR + getName(); + } + + public ColumnParseNode(TableName tableName, String name) { + this(tableName, name, null); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + return visitor.visit(this); + } + + public String getTableName() { + return tableName == null ? null : tableName.getTableName(); + } + + public String getSchemaName() { + return tableName == null ? null : tableName.getSchemaName(); + } + + public String getFullName() { + return fullName; + } + + @Override + public String getAlias() { + return alias; + } + + @Override + public String toString() { + return fullName; + } + + @Override + public int hashCode() { + return fullName.hashCode(); + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + ColumnParseNode other = (ColumnParseNode)obj; + return fullName.equals(other.fullName); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ComparisonParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ComparisonParseNode.java new file mode 100644 index 00000000..1a50fb73 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ComparisonParseNode.java @@ -0,0 +1,59 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +import org.apache.hadoop.hbase.filter.CompareFilter; + +/** + * + * Common base class for =, >, >=, <, <=, != + * + * @author jtaylor + * @since 0.1 + */ +public abstract class ComparisonParseNode extends BinaryParseNode { + + ComparisonParseNode(ParseNode lhs, ParseNode rhs) { + super(lhs, rhs); + } + + @Override + public final T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } + + /** + * Return the comparison operator associated with the given comparison expression node + */ + public abstract CompareFilter.CompareOp getFilterOp(); + + /** + * Return the inverted operator for the CompareOp + */ + public abstract CompareFilter.CompareOp getInvertFilterOp(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/CompoundParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/CompoundParseNode.java new file mode 100644 index 00000000..45c196e0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/CompoundParseNode.java @@ -0,0 +1,75 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Abstract node representing an expression node that has children + * + * @author jtaylor + * @since 0.1 + */ +public abstract class CompoundParseNode extends ParseNode { + private final List children; + private final boolean isStateless; + + CompoundParseNode(List children) { + this.children = Collections.unmodifiableList(children); + boolean isStateless = true; + for (ParseNode child : children) { + isStateless &= child.isStateless(); + if (!isStateless) { + break; + } + } + this.isStateless = isStateless; + } + + @Override + public boolean isStateless() { + return isStateless; + } + + @Override + public final List getChildren() { + return children; + } + + + final List acceptChildren(ParseNodeVisitor visitor) throws SQLException { + List l = visitor.newElementList(children.size()); + for (int i = 0; i < children.size(); i++) { + T e = children.get(i).accept(visitor); + visitor.addElement(l, e); + } + return l; + } + + @Override + public String toString() { + return this.getClass().getName() + children.toString(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ConcreteTableNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ConcreteTableNode.java new file mode 100644 index 00000000..cd00316f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ConcreteTableNode.java @@ -0,0 +1,43 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.phoenix.util.SchemaUtil; + +/** + * + * Abstract node representing a table reference in the FROM clause in SQL + * + * @author jtaylor + * @since 0.1 + */ +public abstract class ConcreteTableNode extends TableNode { + private final TableName name; + + ConcreteTableNode(String alias, TableName name) { + super(SchemaUtil.normalizeIdentifier(alias)); + this.name = name; + } + + public TableName getName() { + return name; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/CreateIndexStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/CreateIndexStatement.java new file mode 100644 index 00000000..95d3233b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/CreateIndexStatement.java @@ -0,0 +1,74 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.Collections; +import java.util.List; + +import org.apache.hadoop.hbase.util.Pair; + +import com.google.common.collect.ListMultimap; + + +public class CreateIndexStatement extends SingleTableSQLStatement { + private final TableName indexTableName; + private final PrimaryKeyConstraint indexConstraint; + private final List includeColumns; + private final List splitNodes; + private final ListMultimap> props; + private final boolean ifNotExists; + + public CreateIndexStatement(NamedNode indexTableName, NamedTableNode dataTable, + PrimaryKeyConstraint indexConstraint, List includeColumns, List splits, + ListMultimap> props, boolean ifNotExists, int bindCount) { + super(dataTable, bindCount); + this.indexTableName =TableName.createNormalized(dataTable.getName().getSchemaName(),indexTableName.getName()); + this.indexConstraint = indexConstraint == null ? PrimaryKeyConstraint.EMPTY : indexConstraint; + this.includeColumns = includeColumns == null ? Collections.emptyList() : includeColumns; + this.splitNodes = splits == null ? Collections.emptyList() : splits; + this.props = props; + this.ifNotExists = ifNotExists; + } + + public PrimaryKeyConstraint getIndexConstraint() { + return indexConstraint; + } + + public List getIncludeColumns() { + return includeColumns; + } + + public TableName getIndexTableName() { + return indexTableName; + } + + public List getSplitNodes() { + return splitNodes; + } + + public ListMultimap> getProps() { + return props; + } + + public boolean ifNotExists() { + return ifNotExists; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/CreateSequenceStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/CreateSequenceStatement.java new file mode 100644 index 00000000..b24c0767 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/CreateSequenceStatement.java @@ -0,0 +1,64 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +public class CreateSequenceStatement implements BindableStatement { + + private final TableName sequenceName; + private final ParseNode startWith; + private final ParseNode incrementBy; + private final ParseNode cacheSize; + private final boolean ifNotExists; + private final int bindCount; + + protected CreateSequenceStatement(TableName sequenceName, ParseNode startsWith, ParseNode incrementBy, ParseNode cacheSize, boolean ifNotExists, int bindCount) { + this.sequenceName = sequenceName; + this.startWith = startsWith == null ? LiteralParseNode.ONE : startsWith; + this.incrementBy = incrementBy == null ? LiteralParseNode.ONE : incrementBy; + this.cacheSize = cacheSize == null ? null : cacheSize; + this.ifNotExists = ifNotExists; + this.bindCount = bindCount; + } + + @Override + public int getBindCount() { + return this.bindCount; + } + + public ParseNode getIncrementBy() { + return incrementBy; + } + + public TableName getSequenceName() { + return sequenceName; + } + + public ParseNode getCacheSize() { + return cacheSize; + } + + public ParseNode getStartWith() { + return startWith; + } + + public boolean ifNotExists() { + return ifNotExists; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/CreateTableStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/CreateTableStatement.java new file mode 100644 index 00000000..9d6d551b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/CreateTableStatement.java @@ -0,0 +1,100 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.Collections; +import java.util.List; + +import org.apache.hadoop.hbase.util.Pair; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableListMultimap; +import com.google.common.collect.ListMultimap; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.schema.PTableType; + +public class CreateTableStatement implements BindableStatement { + private final TableName tableName; + private final PTableType tableType; + private final List columns; + private final PrimaryKeyConstraint pkConstraint; + private final List splitNodes; + private final int bindCount; + private final ListMultimap> props; + private final boolean ifNotExists; + private final TableName baseTableName; + private final ParseNode whereClause; + + protected CreateTableStatement(TableName tableName, ListMultimap> props, List columns, PrimaryKeyConstraint pkConstraint, + List splitNodes, PTableType tableType, boolean ifNotExists, + TableName baseTableName, ParseNode whereClause, int bindCount) { + this.tableName = tableName; + this.props = props == null ? ImmutableListMultimap.>of() : props; + this.tableType = PhoenixDatabaseMetaData.TYPE_SCHEMA.equals(tableName.getSchemaName()) ? PTableType.SYSTEM : tableType; + this.columns = columns == null ? ImmutableList.of() : ImmutableList.copyOf(columns); + this.pkConstraint = pkConstraint == null ? PrimaryKeyConstraint.EMPTY : pkConstraint; + this.splitNodes = splitNodes == null ? Collections.emptyList() : ImmutableList.copyOf(splitNodes); + this.bindCount = bindCount; + this.ifNotExists = ifNotExists; + this.baseTableName = baseTableName; + this.whereClause = whereClause; + } + + public ParseNode getWhereClause() { + return whereClause; + } + + @Override + public int getBindCount() { + return bindCount; + } + + public TableName getTableName() { + return tableName; + } + + public TableName getBaseTableName() { + return baseTableName; + } + + public List getColumnDefs() { + return columns; + } + + public List getSplitNodes() { + return splitNodes; + } + + public PTableType getTableType() { + return tableType; + } + + public ListMultimap> getProps() { + return props; + } + + public boolean ifNotExists() { + return ifNotExists; + } + + public PrimaryKeyConstraint getPrimaryKeyConstraint() { + return pkConstraint; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/CurrentDateParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/CurrentDateParseNode.java new file mode 100644 index 00000000..4ad29bc4 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/CurrentDateParseNode.java @@ -0,0 +1,41 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.CurrentDateFunction; +import org.apache.phoenix.expression.function.FunctionExpression; + + +public class CurrentDateParseNode extends FunctionParseNode { + + public CurrentDateParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public FunctionExpression create(List children, StatementContext context) throws SQLException { + return new CurrentDateFunction(context.getCurrentTime()); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/CurrentTimeParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/CurrentTimeParseNode.java new file mode 100644 index 00000000..ede88a2c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/CurrentTimeParseNode.java @@ -0,0 +1,41 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.CurrentTimeFunction; +import org.apache.phoenix.expression.function.FunctionExpression; + + +public class CurrentTimeParseNode extends FunctionParseNode { + + public CurrentTimeParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public FunctionExpression create(List children, StatementContext context) throws SQLException { + return new CurrentTimeFunction(context.getCurrentTime()); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/DelegateConstantToCountParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/DelegateConstantToCountParseNode.java new file mode 100644 index 00000000..8e436ce5 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/DelegateConstantToCountParseNode.java @@ -0,0 +1,48 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.CountAggregateFunction; +import org.apache.phoenix.expression.function.FunctionExpression; + + +public abstract class DelegateConstantToCountParseNode extends AggregateFunctionParseNode { + + public DelegateConstantToCountParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + protected CountAggregateFunction getDelegateFunction(List children, StatementContext context) { + CountAggregateFunction countFunc = null; + if (getChildren().get(0).isStateless()) { + countFunc = (CountAggregateFunction)context.getExpressionManager().addIfAbsent(new CountAggregateFunction(children)); + } + return countFunc; + } + + @Override + public abstract FunctionExpression create(List children, StatementContext context) throws SQLException; + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/DeleteStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/DeleteStatement.java new file mode 100644 index 00000000..8b23c9a0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/DeleteStatement.java @@ -0,0 +1,69 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.Collections; +import java.util.List; + +public class DeleteStatement extends SingleTableSQLStatement implements FilterableStatement { + private final ParseNode whereNode; + private final List orderBy; + private final LimitNode limit; + private final HintNode hint; + + public DeleteStatement(NamedTableNode table, HintNode hint, ParseNode whereNode, List orderBy, LimitNode limit, int bindCount) { + super(table, bindCount); + this.whereNode = whereNode; + this.orderBy = orderBy == null ? Collections.emptyList() : orderBy; + this.limit = limit; + this.hint = hint == null ? HintNode.EMPTY_HINT_NODE : hint; + } + + @Override + public ParseNode getWhere() { + return whereNode; + } + + @Override + public List getOrderBy() { + return orderBy; + } + + @Override + public LimitNode getLimit() { + return limit; + } + + @Override + public HintNode getHint() { + return hint; + } + + @Override + public boolean isDistinct() { + return false; + } + + @Override + public boolean isAggregate() { + return false; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/DerivedTableNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/DerivedTableNode.java new file mode 100644 index 00000000..3d76698a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/DerivedTableNode.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; + + + +/** + * + * Node representing a subselect in the FROM clause of SQL + * + * @author jtaylor + * @since 0.1 + */ +public class DerivedTableNode extends TableNode { + + private final SelectStatement select; + + DerivedTableNode(String alias, SelectStatement select) { + super(alias); + this.select = select; + } + + public SelectStatement getSelect() { + return select; + } + + @Override + public void accept(TableNodeVisitor visitor) throws SQLException { + visitor.visit(this); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/DistinctCountParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/DistinctCountParseNode.java new file mode 100644 index 00000000..d270f2ba --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/DistinctCountParseNode.java @@ -0,0 +1,46 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.DistinctCountAggregateFunction; +import org.apache.phoenix.expression.function.FunctionExpression; + +/** + * + * @author anoopsjohn + * @since 1.2.1 + */ +public class DistinctCountParseNode extends DelegateConstantToCountParseNode { + + public DistinctCountParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public FunctionExpression create(List children, StatementContext context) + throws SQLException { + return new DistinctCountAggregateFunction(children, getDelegateFunction(children, context)); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/DivideParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/DivideParseNode.java new file mode 100644 index 00000000..bef0d8c6 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/DivideParseNode.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing division in a SQL expression + * + * @author jtaylor + * @since 0.1 + */ +public class DivideParseNode extends ArithmeticParseNode { + + DivideParseNode(List children) { + super(children); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/DropColumnStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/DropColumnStatement.java new file mode 100644 index 00000000..71cf099f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/DropColumnStatement.java @@ -0,0 +1,43 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.List; + +import org.apache.phoenix.schema.PTableType; + +public class DropColumnStatement extends AlterTableStatement { + private final List columnRefs; + private final boolean ifExists; + + protected DropColumnStatement(NamedTableNode table, PTableType tableType, List columnRefs, boolean ifExists) { + super(table, tableType); + this.columnRefs = columnRefs; + this.ifExists = ifExists; + } + + public List getColumnRefs() { + return columnRefs; + } + + public boolean ifExists() { + return ifExists; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/DropIndexStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/DropIndexStatement.java new file mode 100644 index 00000000..8a01622d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/DropIndexStatement.java @@ -0,0 +1,50 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +public class DropIndexStatement implements BindableStatement { + private final TableName tableName; + private final NamedNode indexName; + private final boolean ifExists; + + public DropIndexStatement(NamedNode indexName, TableName tableName, boolean ifExists) { + this.indexName = indexName; + this.tableName = tableName; + this.ifExists = ifExists; + } + + public TableName getTableName() { + return tableName; + } + + public NamedNode getIndexName() { + return indexName; + } + + @Override + public int getBindCount() { + return 0; + } + + public boolean ifExists() { + return ifExists; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/DropSequenceStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/DropSequenceStatement.java new file mode 100644 index 00000000..6ae9652d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/DropSequenceStatement.java @@ -0,0 +1,46 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +public class DropSequenceStatement implements BindableStatement { + + private final TableName sequenceName; + private final boolean ifExists; + private final int bindCount; + + protected DropSequenceStatement(TableName sequenceName, boolean ifExists, int bindCount) { + this.sequenceName = sequenceName; + this.ifExists = ifExists; + this.bindCount = bindCount; + } + + @Override + public int getBindCount() { + return bindCount; + } + + public TableName getSequenceName() { + return sequenceName; + } + + public boolean ifExists() { + return ifExists; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/DropTableStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/DropTableStatement.java new file mode 100644 index 00000000..22c4530e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/DropTableStatement.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.phoenix.schema.PTableType; + +public class DropTableStatement implements BindableStatement { + private final TableName tableName; + private final boolean ifExists; + private final PTableType tableType; + + protected DropTableStatement(TableName tableName, PTableType tableType, boolean ifExists) { + this.tableName = tableName; + this.tableType = tableType; + this.ifExists = ifExists; + } + + @Override + public int getBindCount() { + return 0; // No binds for DROP + } + + public TableName getTableName() { + return tableName; + } + + public PTableType getTableType() { + return tableType; + } + + public boolean ifExists() { + return ifExists; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/EqualParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/EqualParseNode.java new file mode 100644 index 00000000..bdefff9c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/EqualParseNode.java @@ -0,0 +1,47 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; + + +/** + * + * Node representing the equal operator in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class EqualParseNode extends ComparisonParseNode { + + EqualParseNode(ParseNode lhs, ParseNode rhs) { + super(lhs, rhs); + } + + @Override + public CompareOp getFilterOp() { + return CompareOp.EQUAL; + } + + @Override + public CompareOp getInvertFilterOp() { + return CompareOp.EQUAL; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ExistsParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ExistsParseNode.java new file mode 100644 index 00000000..aaa1a738 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ExistsParseNode.java @@ -0,0 +1,55 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing EXISTS and NOT EXISTS expressions in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class ExistsParseNode extends BinaryParseNode { + private final boolean negate; + + ExistsParseNode(ParseNode l, ParseNode r, boolean negate) { + super(l, r); + this.negate = negate; + } + + public boolean isNegate() { + return negate; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ExplainStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ExplainStatement.java new file mode 100644 index 00000000..fc437c54 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ExplainStatement.java @@ -0,0 +1,37 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +public class ExplainStatement implements BindableStatement { + private final BindableStatement statement; + + public ExplainStatement(BindableStatement statement) { + this.statement = statement; + } + + public BindableStatement getStatement() { + return statement; + } + + @Override + public int getBindCount() { + return statement.getBindCount(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/FamilyWildcardParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/FamilyWildcardParseNode.java new file mode 100644 index 00000000..438146cb --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/FamilyWildcardParseNode.java @@ -0,0 +1,55 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.phoenix.parse; + +import java.sql.SQLException; + +/** + * + * Node representing the selection of all columns of a family (cf.*) in the SELECT clause of SQL + * + * @author nmaillard + * @since 1.2 + */ + +public class FamilyWildcardParseNode extends NamedParseNode { + private final boolean isRewrite; + + public FamilyWildcardParseNode(String familyName, boolean isRewrite){ + super(familyName); + this.isRewrite = isRewrite; + } + + public FamilyWildcardParseNode(FamilyWildcardParseNode familyName, boolean isRewrite){ + super(familyName); + this.isRewrite = isRewrite; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + return visitor.visit(this); + } + + public boolean isRewrite() { + return isRewrite; + } +} + diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/FilterableStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/FilterableStatement.java new file mode 100644 index 00000000..47969d91 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/FilterableStatement.java @@ -0,0 +1,31 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.List; + +public interface FilterableStatement extends BindableStatement { + public HintNode getHint(); + public ParseNode getWhere(); + public boolean isDistinct(); + public boolean isAggregate(); + public List getOrderBy(); + public LimitNode getLimit(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/FloorParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/FloorParseNode.java new file mode 100644 index 00000000..2f88bf2f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/FloorParseNode.java @@ -0,0 +1,81 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.FloorDateExpression; +import org.apache.phoenix.expression.function.FloorDecimalExpression; +import org.apache.phoenix.expression.function.FloorFunction; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.TypeMismatchException; + +/** + * Parse node corresponding to {@link FloorFunction}. + * It also acts as a factory for creating the right kind of + * floor expression according to the data type of the + * first child. + * + * @author samarth.jain + * @since 3.0.0 + */ +public class FloorParseNode extends FunctionParseNode { + + FloorParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public Expression create(List children, StatementContext context) throws SQLException { + return getFloorExpression(children); + } + + public static Expression getFloorExpression(List children) throws SQLException { + final Expression firstChild = children.get(0); + final PDataType firstChildDataType = firstChild.getDataType(); + + //FLOOR on timestamp doesn't really care about the nanos part i.e. it just sets it to zero. + //Which is exactly what FloorDateExpression does too. + if(firstChildDataType.isCoercibleTo(PDataType.TIMESTAMP)) { + return FloorDateExpression.create(children); + } else if(firstChildDataType.isCoercibleTo(PDataType.DECIMAL)) { + return new FloorDecimalExpression(children); + } else { + throw TypeMismatchException.newException(firstChildDataType, "1"); + } + } + + /** + * When rounding off decimals, user need not specify the scale. In such cases, + * we need to prevent the function from getting evaluated as null. This is really + * a hack. A better way would have been if {@link org.apache.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo} provided a + * way of associating default values for each permissible data type. + * Something like: @ Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValues = {"null", "1"} isConstant=true) + * Till then, this will have to do. + */ + @Override + public boolean evalToNullIfParamIsNull(StatementContext context, int index) throws SQLException { + return index == 0; + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/FunctionParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/FunctionParseNode.java similarity index 87% rename from phoenix-core/src/main/java/com/salesforce/phoenix/parse/FunctionParseNode.java rename to phoenix-core/src/main/java/org/apache/phoenix/parse/FunctionParseNode.java index 9fb052ff..e31a19e3 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/FunctionParseNode.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/FunctionParseNode.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; import java.lang.annotation.ElementType; import java.lang.annotation.Retention; @@ -43,15 +35,15 @@ import org.apache.http.annotation.Immutable; import com.google.common.collect.ImmutableSet; -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.function.AggregateFunction; -import com.salesforce.phoenix.expression.function.FunctionExpression; -import com.salesforce.phoenix.schema.ArgumentTypeMismatchException; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.ValueRangeExcpetion; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.function.AggregateFunction; +import org.apache.phoenix.expression.function.FunctionExpression; +import org.apache.phoenix.schema.ArgumentTypeMismatchException; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.ValueRangeExcpetion; +import org.apache.phoenix.util.SchemaUtil; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/GreaterThanOrEqualParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/GreaterThanOrEqualParseNode.java new file mode 100644 index 00000000..4780d560 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/GreaterThanOrEqualParseNode.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.hadoop.hbase.filter.CompareFilter; +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; + + +/** + * + * Node representing the greater than or equal to operator (>=) in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class GreaterThanOrEqualParseNode extends ComparisonParseNode { + + GreaterThanOrEqualParseNode(ParseNode lhs, ParseNode rhs) { + super(lhs, rhs); + } + + @Override + public CompareOp getFilterOp() { + return CompareFilter.CompareOp.GREATER_OR_EQUAL; + } + + + @Override + public CompareOp getInvertFilterOp() { + return CompareOp.LESS_OR_EQUAL; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/GreaterThanParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/GreaterThanParseNode.java new file mode 100644 index 00000000..ff0442bc --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/GreaterThanParseNode.java @@ -0,0 +1,48 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.hadoop.hbase.filter.CompareFilter; +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; + + +/** + * + * Node representing the greater than operator (>) in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class GreaterThanParseNode extends ComparisonParseNode { + + GreaterThanParseNode(ParseNode lhs, ParseNode rhs) { + super(lhs, rhs); + } + + @Override + public CompareOp getFilterOp() { + return CompareFilter.CompareOp.GREATER; + } + + @Override + public CompareOp getInvertFilterOp() { + return CompareOp.LESS; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/HintNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/HintNode.java new file mode 100644 index 00000000..fa823829 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/HintNode.java @@ -0,0 +1,160 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.HashMap; +import java.util.Map; + +import com.google.common.collect.ImmutableMap; +import org.apache.phoenix.util.SchemaUtil; + + +/** + * Node representing optimizer hints in SQL + */ +public class HintNode { + public static final HintNode EMPTY_HINT_NODE = new HintNode(); + + public static final char SEPARATOR = ' '; + public static final String PREFIX = "("; + public static final String SUFFIX = ")"; + // Split on whitespace and parenthesis, keeping the parenthesis in the token array + private static final String SPLIT_REGEXP = "\\s+|((?<=\\" + PREFIX + ")|(?=\\" + PREFIX + "))|((?<=\\" + SUFFIX + ")|(?=\\" + SUFFIX + "))"; + + public enum Hint { + /** + * Forces a range scan to be used to process the query. + */ + RANGE_SCAN, + /** + * Forces a skip scan to be used to process the query. + */ + SKIP_SCAN, + /** + * Prevents the spawning of multiple threads during + * query processing. + */ + NO_INTRA_REGION_PARALLELIZATION, + /** + * Prevents the usage of indexes, forcing usage + * of the data table for a query. + */ + NO_INDEX, + /** + * Hint of the form INDEX( ...) + * to suggest usage of the index if possible. The first + * usable index in the list of indexes will be choosen. + * Table and index names may be surrounded by double quotes + * if they are case sensitive. + */ + INDEX, + /** + * All things being equal, use the data table instead of + * the index table when optimizing. + */ + USE_DATA_OVER_INDEX_TABLE, + /** + * All things being equal, use the index table instead of + * the data table when optimizing. + */ + USE_INDEX_OVER_DATA_TABLE, + }; + + private final Map hints; + + public static HintNode create(HintNode hintNode, Hint hint) { + return create(hintNode, hint, ""); + } + + public static HintNode create(HintNode hintNode, Hint hint, String value) { + Map hints = new HashMap(hintNode.hints); + hints.put(hint, value); + return new HintNode(hints); + } + + private HintNode() { + hints = new HashMap(); + } + + private HintNode(Map hints) { + this.hints = ImmutableMap.copyOf(hints); + } + + public HintNode(String hint) { + Map hints = new HashMap(); + // Split on whitespace or parenthesis. We do not need to handle escaped or + // embedded whitespace/parenthesis, since we are parsing what will be HBase + // table names which are not allowed to contain whitespace or parenthesis. + String[] hintWords = hint.split(SPLIT_REGEXP); + for (int i = 0; i < hintWords.length; i++) { + String hintWord = hintWords[i]; + if (hintWord.isEmpty()) { + continue; + } + try { + Hint key = Hint.valueOf(hintWord.toUpperCase()); + String hintValue = ""; + if (i+1 < hintWords.length && PREFIX.equals(hintWords[i+1])) { + StringBuffer hintValueBuf = new StringBuffer(hint.length()); + hintValueBuf.append(PREFIX); + i+=2; + while (i < hintWords.length && !SUFFIX.equals(hintWords[i])) { + hintValueBuf.append(SchemaUtil.normalizeIdentifier(hintWords[i++])); + hintValueBuf.append(SEPARATOR); + } + // Replace trailing separator with suffix + hintValueBuf.replace(hintValueBuf.length()-1, hintValueBuf.length(), SUFFIX); + hintValue = hintValueBuf.toString(); + } + String oldValue = hints.put(key, hintValue); + // Concatenate together any old value with the new value + if (oldValue != null) { + hints.put(key, oldValue + hintValue); + } + } catch (IllegalArgumentException e) { // Ignore unknown/invalid hints + } + } + this.hints = ImmutableMap.copyOf(hints); + } + + public boolean isEmpty() { + return hints.isEmpty(); + } + + /** + * Gets the value of the hint or null if the hint is not present. + * @param hint the hint + * @return the value specified in parenthesis following the hint or null + * if the hint is not present. + * + */ + public String getHint(Hint hint) { + return hints.get(hint); + } + + /** + * Tests for the presence of a hint in a query + * @param hint the hint + * @return true if the hint is present and false otherwise + */ + public boolean hasHint(Hint hint) { + return hints.containsKey(hint); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/InListParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/InListParseNode.java new file mode 100644 index 00000000..7632de7e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/InListParseNode.java @@ -0,0 +1,66 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + + + +/** + * + * Node representing the IN literal list expression in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class InListParseNode extends CompoundParseNode { + private final boolean negate; + + InListParseNode(List children, boolean negate) { + super(children); + // All values in the IN must be constant. First child is the LHS + for (int i = 1; i < children.size(); i++) { + ParseNode child = children.get(i); + if (!child.isStateless()) { + throw new ParseException(new SQLExceptionInfo.Builder(SQLExceptionCode.VALUE_IN_LIST_NOT_CONSTANT) + .build().buildException()); + } + } + this.negate = negate; + } + + public boolean isNegate() { + return negate; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/InParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/InParseNode.java new file mode 100644 index 00000000..d7708fc0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/InParseNode.java @@ -0,0 +1,55 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing IN subquery expression in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class InParseNode extends BinaryParseNode { + private final boolean negate; + + InParseNode(ParseNode l, ParseNode r, boolean negate) { + super(l, r); + this.negate = negate; + } + + public boolean isNegate() { + return negate; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/IndexKeyConstraint.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/IndexKeyConstraint.java new file mode 100644 index 00000000..6dfc7448 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/IndexKeyConstraint.java @@ -0,0 +1,39 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.List; + +import org.apache.hadoop.hbase.util.Pair; + +import com.google.common.collect.ImmutableList; +import org.apache.phoenix.schema.ColumnModifier; + +public class IndexKeyConstraint { + private final List> columnNameToModifier; + + IndexKeyConstraint(List> columnNameAndModifier) { + this.columnNameToModifier = ImmutableList.copyOf(columnNameAndModifier); + } + + public List> getColumns() { + return columnNameToModifier; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/IsNullParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/IsNullParseNode.java new file mode 100644 index 00000000..a1afc2e4 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/IsNullParseNode.java @@ -0,0 +1,55 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing the IS NULL and IS NOT NULL expressions in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class IsNullParseNode extends UnaryParseNode { + private final boolean negate; + + IsNullParseNode(ParseNode expr, boolean negate) { + super(expr); + this.negate = negate; + } + + public boolean isNegate() { + return negate; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/JoinTableNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/JoinTableNode.java new file mode 100644 index 00000000..d6f7f820 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/JoinTableNode.java @@ -0,0 +1,64 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; + + + +/** + * + * Node representing the join specified in the FROM clause of SQL + * + * @author jtaylor + * @since 0.1 + */ +public class JoinTableNode extends TableNode { + public enum JoinType {Inner, Left, Right, Full}; + + private final JoinType type; + private final ParseNode on; + private final TableNode table; + + JoinTableNode(JoinType type, ParseNode on, TableNode table) { + super(table.getAlias()); + this.type = type; + this.on = on; + this.table = table; + } + + public JoinType getType() { + return type; + } + + public ParseNode getOnNode() { + return on; + } + + public TableNode getTable() { + return table; + } + + @Override + public void accept(TableNodeVisitor visitor) throws SQLException { + visitor.visit(this); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/LessThanOrEqualParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/LessThanOrEqualParseNode.java new file mode 100644 index 00000000..4fb91ddf --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/LessThanOrEqualParseNode.java @@ -0,0 +1,48 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.hadoop.hbase.filter.CompareFilter; +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; + + +/** + * + * Node representing the less than or equal to operator (<=) in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class LessThanOrEqualParseNode extends ComparisonParseNode { + + LessThanOrEqualParseNode(ParseNode lhs, ParseNode rhs) { + super(lhs, rhs); + } + + @Override + public CompareOp getFilterOp() { + return CompareFilter.CompareOp.LESS_OR_EQUAL; + } + + @Override + public CompareOp getInvertFilterOp() { + return CompareOp.GREATER_OR_EQUAL; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/LessThanParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/LessThanParseNode.java new file mode 100644 index 00000000..c9f69dc3 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/LessThanParseNode.java @@ -0,0 +1,48 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.hadoop.hbase.filter.CompareFilter; +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; + + +/** + * + * Node representing the less than operator (<) in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class LessThanParseNode extends ComparisonParseNode { + + LessThanParseNode(ParseNode lhs, ParseNode rhs) { + super(lhs, rhs); + } + + @Override + public CompareOp getFilterOp() { + return CompareFilter.CompareOp.LESS; + } + + @Override + public CompareOp getInvertFilterOp() { + return CompareOp.GREATER; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/LikeParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/LikeParseNode.java new file mode 100644 index 00000000..b941d569 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/LikeParseNode.java @@ -0,0 +1,55 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing LIKE and NOT LIKE in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class LikeParseNode extends BinaryParseNode { + private final boolean negate; + + LikeParseNode(ParseNode lhs, ParseNode rhs, boolean negate) { + super(lhs, rhs); + this.negate = negate; + } + + public boolean isNegate() { + return negate; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/LimitNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/LimitNode.java new file mode 100644 index 00000000..cd4d6a9f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/LimitNode.java @@ -0,0 +1,40 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + + +public class LimitNode { + private final BindParseNode bindNode; + private final LiteralParseNode limitNode; + + LimitNode(BindParseNode bindNode) { + this.bindNode = bindNode; + limitNode = null; + } + + LimitNode(LiteralParseNode limitNode) { + this.limitNode = limitNode; + this.bindNode = null; + } + + public ParseNode getLimitParseNode() { + return bindNode == null ? limitNode : bindNode; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/LiteralParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/LiteralParseNode.java new file mode 100644 index 00000000..25247ee8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/LiteralParseNode.java @@ -0,0 +1,77 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +import org.apache.phoenix.schema.PDataType; + + + +/** + * + * Node representing literal expressions such as 1,2.5,'foo', and NULL in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class LiteralParseNode extends TerminalParseNode { + public static final List STAR = Collections.singletonList(new LiteralParseNode(1)); + public static final ParseNode NULL = new LiteralParseNode(null); + public static final ParseNode ZERO = new LiteralParseNode(0); + public static final ParseNode ONE = new LiteralParseNode(1); + + private final Object value; + private final PDataType type; + + public LiteralParseNode(Object value) { + this.value = value; + this.type = PDataType.fromLiteral(value); + } + + public PDataType getType() { + return type; + } + + public Object getValue() { + return value; + } + + @Override + public boolean isStateless() { + return true; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + return visitor.visit(this); + } + + public byte[] getBytes() { + return type == null ? null : type.toBytes(value); + } + + @Override + public String toString() { + return type == PDataType.VARCHAR ? ("'" + value.toString() + "'") : value == null ? "null" : value.toString(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/MaxAggregateParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/MaxAggregateParseNode.java new file mode 100644 index 00000000..6f59a72b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/MaxAggregateParseNode.java @@ -0,0 +1,41 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.FunctionExpression; +import org.apache.phoenix.expression.function.MaxAggregateFunction; + + +public class MaxAggregateParseNode extends DelegateConstantToCountParseNode { + + public MaxAggregateParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public FunctionExpression create(List children, StatementContext context) throws SQLException { + return new MaxAggregateFunction(children, getDelegateFunction(children,context)); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/MinAggregateParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/MinAggregateParseNode.java new file mode 100644 index 00000000..f6450dd1 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/MinAggregateParseNode.java @@ -0,0 +1,41 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.FunctionExpression; +import org.apache.phoenix.expression.function.MinAggregateFunction; + + +public class MinAggregateParseNode extends DelegateConstantToCountParseNode { + + public MinAggregateParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public FunctionExpression create(List children, StatementContext context) throws SQLException { + return new MinAggregateFunction(children, getDelegateFunction(children,context)); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/MultiplyParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/MultiplyParseNode.java new file mode 100644 index 00000000..1b100e93 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/MultiplyParseNode.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing multiplication in a SQL expression + * + * @author jtaylor + * @since 0.1 + */ +public class MultiplyParseNode extends ArithmeticParseNode { + + MultiplyParseNode(List children) { + super(children); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NamedNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/NamedNode.java similarity index 92% rename from phoenix-core/src/main/java/com/salesforce/phoenix/parse/NamedNode.java rename to phoenix-core/src/main/java/org/apache/phoenix/parse/NamedNode.java index 54586db9..9c687aad 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/NamedNode.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/NamedNode.java @@ -1,6 +1,6 @@ -package com.salesforce.phoenix.parse; +package org.apache.phoenix.parse; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.phoenix.util.SchemaUtil; public class NamedNode { private final String name; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/NamedParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/NamedParseNode.java new file mode 100644 index 00000000..bd1c6cfb --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/NamedParseNode.java @@ -0,0 +1,53 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + + +/** + * + * Abstract node representing named nodes such as binds and column expressions in SQL + * + * @author jtaylor + * @since 0.1 + */ +public abstract class NamedParseNode extends TerminalParseNode{ + private final NamedNode namedNode; + + NamedParseNode(NamedParseNode node) { + this.namedNode = node.namedNode; + } + + NamedParseNode(String name) { + this.namedNode = new NamedNode(name); + } + + public String getName() { + return namedNode.getName(); + } + + public boolean isCaseSensitive() { + return namedNode.isCaseSensitive(); + } + + @Override + public String toString() { + return getName(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/NamedTableNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/NamedTableNode.java new file mode 100644 index 00000000..4be546db --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/NamedTableNode.java @@ -0,0 +1,64 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +import com.google.common.collect.ImmutableList; + +/** + * Node representing an explicit table reference in the FROM clause of SQL + * + * @author jtaylor + * @since 0.1 + */ +public class NamedTableNode extends ConcreteTableNode { + + private final List dynColumns; + + public static NamedTableNode create (String alias, TableName name, List dynColumns) { + return new NamedTableNode(alias, name, dynColumns); + } + + NamedTableNode(String alias, TableName name) { + super(alias, name); + dynColumns = Collections. emptyList(); + } + + NamedTableNode(String alias, TableName name, List dynColumns) { + super(alias, name); + if (dynColumns != null) { + this.dynColumns = ImmutableList.copyOf(dynColumns); + } else { + this.dynColumns = Collections. emptyList(); + } + } + + @Override + public void accept(TableNodeVisitor visitor) throws SQLException { + visitor.visit(this); + } + + public List getDynamicColumns() { + return dynColumns; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/NotEqualParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/NotEqualParseNode.java new file mode 100644 index 00000000..7872275c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/NotEqualParseNode.java @@ -0,0 +1,48 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; + + + +/** + * + * Node representing a not equal expression (!=,<>) in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class NotEqualParseNode extends ComparisonParseNode { + + NotEqualParseNode(ParseNode lhs, ParseNode rhs) { + super(lhs, rhs); + } + + @Override + public CompareOp getFilterOp() { + return CompareOp.NOT_EQUAL; + } + + @Override + public CompareOp getInvertFilterOp() { + return CompareOp.NOT_EQUAL; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/NotParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/NotParseNode.java new file mode 100644 index 00000000..580e7ca9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/NotParseNode.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing the NOT in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class NotParseNode extends UnaryParseNode { + + NotParseNode(ParseNode expr) { + super(expr); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/OrParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/OrParseNode.java new file mode 100644 index 00000000..0fe869ff --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/OrParseNode.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing an OR in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class OrParseNode extends CompoundParseNode { + + OrParseNode(List children) { + super(children); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/OrderByNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/OrderByNode.java new file mode 100644 index 00000000..6a48d10b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/OrderByNode.java @@ -0,0 +1,57 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + + +/** + * + * Node representing an ORDER BY clause (including asc/desc and nulls first/last) in SQL + * + * @author jtaylor + * @since 0.1 + */ +public final class OrderByNode { + private final ParseNode child; + private final boolean nullsLast; + private final boolean orderAscending; + + OrderByNode(ParseNode child, boolean nullsLast, boolean orderAscending) { + this.child = child; + this.nullsLast = nullsLast; + this.orderAscending = orderAscending; + } + + public boolean isNullsLast() { + return nullsLast; + } + + public boolean isAscending() { + return orderAscending; + } + + public ParseNode getNode() { + return child; + } + + @Override + public String toString() { + return child.toString() + (orderAscending ? " asc" : " desc") + " nulls " + (nullsLast ? "last" : "first"); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/OuterJoinParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/OuterJoinParseNode.java new file mode 100644 index 00000000..12ed92bb --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/OuterJoinParseNode.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing an outer join qualifier (+) in SQL + * TODO: remove Oracle specific syntax + * + * @author jtaylor + * @since 0.1 + */ +public class OuterJoinParseNode extends UnaryParseNode{ + OuterJoinParseNode(ParseNode node) { + super(node); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseContext.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseContext.java new file mode 100644 index 00000000..275ceb2a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseContext.java @@ -0,0 +1,60 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.List; + +import com.google.common.collect.Lists; + +public class ParseContext { + private boolean isAggregate; + + public ParseContext() { + } + + public boolean isAggregate() { + return isAggregate; + } + + public void setAggregate(boolean isAggregate) { + this.isAggregate |= isAggregate; + } + + public static class Stack { + private final List stack = Lists.newArrayListWithExpectedSize(5); + + public void push(ParseContext context) { + stack.add(context); + } + + public ParseContext pop() { + return stack.remove(stack.size()-1); + } + + public ParseContext peek() { + return stack.get(stack.size()-1); + } + + public boolean isEmpty() { + return stack.isEmpty(); + } + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseException.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseException.java new file mode 100644 index 00000000..010dd883 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseException.java @@ -0,0 +1,47 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +/** + * + * RuntimeException for exceptions occurring during parsing, + * since ANTLR doesn't handle typed exceptions well. + * + * @author jtaylor + * @since 2.0 + */ +public class ParseException extends RuntimeException { + + public ParseException() { + } + + public ParseException(String msg) { + super(msg); + } + + public ParseException(Throwable t) { + super(t); + } + + public ParseException(String msg, Throwable t) { + super(msg, t); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNode.java new file mode 100644 index 00000000..46a385cc --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNode.java @@ -0,0 +1,52 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + + + + +/** + * + * Abstract base class for a parse node in SQL + * + * @author jtaylor + * @since 0.1 + */ +public abstract class ParseNode { + public abstract List getChildren(); + public abstract T accept(ParseNodeVisitor visitor) throws SQLException; + + public boolean isStateless() { + return false; + } + + /** + * Allows node to override what the alias is for a given node. + * Useful for a column reference, as JDBC says that the alias + * name for "a.b" should be "b" + * @return the alias to use for this node or null for no alias + */ + public String getAlias() { + return null; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNodeFactory.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNodeFactory.java similarity index 87% rename from phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNodeFactory.java rename to phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNodeFactory.java index 0a1b8500..20082a2d 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNodeFactory.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNodeFactory.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; import java.lang.reflect.Constructor; import java.sql.SQLException; @@ -40,24 +32,24 @@ import com.google.common.collect.ListMultimap; import com.google.common.collect.Maps; -import com.salesforce.phoenix.exception.UnknownFunctionException; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.ExpressionType; -import com.salesforce.phoenix.expression.function.AvgAggregateFunction; -import com.salesforce.phoenix.expression.function.CountAggregateFunction; -import com.salesforce.phoenix.expression.function.CurrentDateFunction; -import com.salesforce.phoenix.expression.function.CurrentTimeFunction; -import com.salesforce.phoenix.expression.function.DistinctCountAggregateFunction; -import com.salesforce.phoenix.expression.function.FunctionExpression; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunction; -import com.salesforce.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo; -import com.salesforce.phoenix.parse.JoinTableNode.JoinType; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PIndexState; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.TypeMismatchException; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.phoenix.exception.UnknownFunctionException; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.ExpressionType; +import org.apache.phoenix.expression.function.AvgAggregateFunction; +import org.apache.phoenix.expression.function.CountAggregateFunction; +import org.apache.phoenix.expression.function.CurrentDateFunction; +import org.apache.phoenix.expression.function.CurrentTimeFunction; +import org.apache.phoenix.expression.function.DistinctCountAggregateFunction; +import org.apache.phoenix.expression.function.FunctionExpression; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo; +import org.apache.phoenix.parse.JoinTableNode.JoinType; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PIndexState; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.TypeMismatchException; +import org.apache.phoenix.util.SchemaUtil; /** diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNodeRewriter.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNodeRewriter.java similarity index 91% rename from phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNodeRewriter.java rename to phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNodeRewriter.java index 716abdc3..504b4968 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ParseNodeRewriter.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNodeRewriter.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; import java.sql.SQLException; import java.util.ArrayList; @@ -36,9 +28,9 @@ import com.google.common.collect.Lists; import com.google.common.collect.Maps; -import com.salesforce.phoenix.compile.ColumnResolver; -import com.salesforce.phoenix.schema.AmbiguousColumnException; -import com.salesforce.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.compile.ColumnResolver; +import org.apache.phoenix.schema.AmbiguousColumnException; +import org.apache.phoenix.schema.ColumnNotFoundException; /** * diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNodeVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNodeVisitor.java new file mode 100644 index 00000000..f300173b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ParseNodeVisitor.java @@ -0,0 +1,108 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + + + +/** + * + * Visitor for ParseNode in the node tree. Uses composite + * visitor pattern with enter/leave calls for any + * compound expression node. Only supported SQL constructs + * have visit methods. Unsupported constructs fall through + * to {@link #visitEnter(CompoundParseNode)} for + * compound parse nodes and {@link #visit(ParseNode)} + * for terminal parse nodes. + * + * @author jtaylor + * @since 0.1 + */ +public interface ParseNodeVisitor { + public List newElementList(int size); + public void addElement(List a, E element); + + public boolean visitEnter(LikeParseNode node) throws SQLException; + public E visitLeave(LikeParseNode node, List l) throws SQLException; + + public boolean visitEnter(AndParseNode node) throws SQLException; + public E visitLeave(AndParseNode node, List l) throws SQLException; + + public boolean visitEnter(OrParseNode node) throws SQLException; + public E visitLeave(OrParseNode node, List l) throws SQLException; + + public boolean visitEnter(FunctionParseNode node) throws SQLException; + public E visitLeave(FunctionParseNode node, List l) throws SQLException; + + public boolean visitEnter(ComparisonParseNode node) throws SQLException; + public E visitLeave(ComparisonParseNode node, List l) throws SQLException; + + public boolean visitEnter(CaseParseNode node) throws SQLException; + public E visitLeave(CaseParseNode node, List l) throws SQLException; + + public boolean visitEnter(CompoundParseNode node) throws SQLException; + public E visitLeave(CompoundParseNode node, List l) throws SQLException; + + public boolean visitEnter(AddParseNode node) throws SQLException; + public E visitLeave(AddParseNode node, List l) throws SQLException; + + public boolean visitEnter(MultiplyParseNode node) throws SQLException; + public E visitLeave(MultiplyParseNode node, List l) throws SQLException; + + public boolean visitEnter(DivideParseNode node) throws SQLException; + public E visitLeave(DivideParseNode node, List l) throws SQLException; + + public boolean visitEnter(SubtractParseNode node) throws SQLException; + public E visitLeave(SubtractParseNode node, List l) throws SQLException; + + public boolean visitEnter(NotParseNode node) throws SQLException; + public E visitLeave(NotParseNode node, List l) throws SQLException; + + public boolean visitEnter(InListParseNode node) throws SQLException; + public E visitLeave(InListParseNode node, List l) throws SQLException; + + public boolean visitEnter(IsNullParseNode node) throws SQLException; + public E visitLeave(IsNullParseNode node, List l) throws SQLException; + + public E visit(ColumnParseNode node) throws SQLException; + public E visit(LiteralParseNode node) throws SQLException; + public E visit(BindParseNode node) throws SQLException; + public E visit(WildcardParseNode node) throws SQLException; + public E visit(FamilyWildcardParseNode node) throws SQLException; + public E visit(ParseNode node) throws SQLException; + + public boolean visitEnter(StringConcatParseNode node) throws SQLException; + public E visitLeave(StringConcatParseNode node, List l) throws SQLException; + + public boolean visitEnter(BetweenParseNode node) throws SQLException; + public E visitLeave(BetweenParseNode node, List l) throws SQLException; + + public boolean visitEnter(CastParseNode node) throws SQLException; + public E visitLeave(CastParseNode node, List l) throws SQLException; + + public boolean visitEnter(RowValueConstructorParseNode node) throws SQLException; + public E visitLeave(RowValueConstructorParseNode node, List l) throws SQLException; + + public boolean visitEnter(ArrayConstructorNode node) throws SQLException; + public E visitLeave(ArrayConstructorNode node, List l) throws SQLException; + public E visit(SequenceValueParseNode node) throws SQLException; +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/PrimaryKeyConstraint.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/PrimaryKeyConstraint.java new file mode 100644 index 00000000..97c99cb1 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/PrimaryKeyConstraint.java @@ -0,0 +1,69 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.Collections; +import java.util.HashMap; +import java.util.List; + +import org.apache.hadoop.hbase.util.Pair; + +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Maps; +import org.apache.phoenix.schema.ColumnModifier; + +public class PrimaryKeyConstraint extends NamedNode { + public static final PrimaryKeyConstraint EMPTY = new PrimaryKeyConstraint(null, Collections.>emptyList()); + + private final List> columns; + private final HashMap> columnNameToModifier; + + PrimaryKeyConstraint(String name, List> columns) { + super(name); + this.columns = columns == null ? Collections.>emptyList() : ImmutableList.copyOf(columns); + this.columnNameToModifier = Maps.newHashMapWithExpectedSize(this.columns.size()); + for (Pair p : this.columns) { + this.columnNameToModifier.put(p.getFirst(), p); + } + } + + public List> getColumnNames() { + return columns; + } + + public Pair getColumn(ColumnName columnName) { + return columnNameToModifier.get(columnName); + } + + public boolean contains(ColumnName columnName) { + return columnNameToModifier.containsKey(columnName); + } + + @Override + public int hashCode() { + return super.hashCode(); + } + + @Override + public boolean equals(Object obj) { + return super.equals(obj); + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/PropertyName.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/PropertyName.java similarity index 87% rename from phoenix-core/src/main/java/com/salesforce/phoenix/parse/PropertyName.java rename to phoenix-core/src/main/java/org/apache/phoenix/parse/PropertyName.java index 4fc040df..167b3232 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/PropertyName.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/PropertyName.java @@ -1,6 +1,6 @@ -package com.salesforce.phoenix.parse; +package org.apache.phoenix.parse; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.phoenix.util.SchemaUtil; public class PropertyName { private final NamedNode familyName; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/RoundParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/RoundParseNode.java new file mode 100644 index 00000000..b7e2c0c8 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/RoundParseNode.java @@ -0,0 +1,83 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.RoundDateExpression; +import org.apache.phoenix.expression.function.RoundDecimalExpression; +import org.apache.phoenix.expression.function.RoundFunction; +import org.apache.phoenix.expression.function.RoundTimestampExpression; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.TypeMismatchException; + +/** + * + * Parse node corresponding to {@link RoundFunction}. + * It also acts as a factory for creating the right kind of + * round expression according to the data type of the + * first child. + * + * @author samarth.jain + * @since 3.0.0 + */ +public class RoundParseNode extends FunctionParseNode { + + RoundParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public Expression create(List children, StatementContext context) throws SQLException { + return getRoundExpression(children); + } + + public static Expression getRoundExpression(List children) throws SQLException { + final Expression firstChild = children.get(0); + final PDataType firstChildDataType = firstChild.getDataType(); + + if(firstChildDataType.isCoercibleTo(PDataType.DATE)) { + return RoundDateExpression.create(children); // FIXME: remove cast + } else if (firstChildDataType.isCoercibleTo(PDataType.TIMESTAMP)) { + return RoundTimestampExpression.create(children); // FIXME: remove cast + } else if(firstChildDataType.isCoercibleTo(PDataType.DECIMAL)) { + return new RoundDecimalExpression(children); + } else { + throw TypeMismatchException.newException(firstChildDataType, "1"); + } + } + + /** + * When rounding off decimals, user need not specify the scale. In such cases, + * we need to prevent the function from getting evaluated as null. This is really + * a hack. A better way would have been if {@link org.apache.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo} provided a + * way of associating default values for each permissible data type. + * Something like: @ Argument(allowedTypes={PDataType.VARCHAR, PDataType.INTEGER}, defaultValues = {"null", "1"} isConstant=true) + * Till then, this will have to do. + */ + @Override + public boolean evalToNullIfParamIsNull(StatementContext context, int index) throws SQLException { + return index == 0; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/RowValueConstructorParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/RowValueConstructorParseNode.java new file mode 100644 index 00000000..e35646ce --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/RowValueConstructorParseNode.java @@ -0,0 +1,48 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + +/** + * + * Node representing a row value constructor in SQL. + * + * @author samarth.jain + * @since 0.1 + */ +public class RowValueConstructorParseNode extends CompoundParseNode { + + public RowValueConstructorParseNode(List l) { + super(l); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SQLParser.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/SQLParser.java similarity index 77% rename from phoenix-core/src/main/java/com/salesforce/phoenix/parse/SQLParser.java rename to phoenix-core/src/main/java/org/apache/phoenix/parse/SQLParser.java index 7513b129..4a0139e0 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SQLParser.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/SQLParser.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; import java.io.IOException; import java.io.Reader; @@ -38,7 +30,7 @@ import org.antlr.runtime.CommonTokenStream; import org.antlr.runtime.RecognitionException; -import com.salesforce.phoenix.exception.PhoenixParserException; +import org.apache.phoenix.exception.PhoenixParserException; /** * diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/SelectStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/SelectStatement.java new file mode 100644 index 00000000..1f2be751 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/SelectStatement.java @@ -0,0 +1,179 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.Arrays; +import java.util.Collections; +import java.util.List; + +import org.apache.phoenix.expression.function.CountAggregateFunction; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunction; +import org.apache.phoenix.parse.FunctionParseNode.BuiltInFunctionInfo; + +/** + * + * Top level node representing a SQL statement + * + * @author jtaylor + * @since 0.1 + */ +public class SelectStatement implements FilterableStatement { + public static final SelectStatement SELECT_ONE = + new SelectStatement( + Collections.emptyList(), null, false, + Collections.singletonList(new AliasedNode(null,new LiteralParseNode(1))), + null, Collections.emptyList(), + null, Collections.emptyList(), + null, 0, false); + public static final SelectStatement COUNT_ONE = + new SelectStatement( + Collections.emptyList(), null, false, + Collections.singletonList( + new AliasedNode(null, + new AggregateFunctionParseNode( + CountAggregateFunction.NORMALIZED_NAME, + LiteralParseNode.STAR, + new BuiltInFunctionInfo(CountAggregateFunction.class, CountAggregateFunction.class.getAnnotation(BuiltInFunction.class))))), + null, Collections.emptyList(), + null, Collections.emptyList(), + null, 0, true); + public static SelectStatement create(SelectStatement select, HintNode hint) { + if (select.getHint() == hint || hint.isEmpty()) { + return select; + } + return new SelectStatement(select.getFrom(), hint, select.isDistinct(), + select.getSelect(), select.getWhere(), select.getGroupBy(), select.getHaving(), + select.getOrderBy(), select.getLimit(), select.getBindCount(), select.isAggregate()); + } + + public SelectStatement combine(ParseNode where) { + if (where == null) { + return this; + } + if (this.getWhere() != null) { + where = new AndParseNode(Arrays.asList(this.getWhere(), where)); + } + return new SelectStatement(this.getFrom(), this.getHint(), this.isDistinct(), + this.getSelect(), where, this.getGroupBy(), this.getHaving(), + this.getOrderBy(), this.getLimit(), this.getBindCount(), this.isAggregate()); + } + + public static SelectStatement create(SelectStatement select, List selects) { + return new SelectStatement(select.getFrom(), select.getHint(), select.isDistinct(), + selects, select.getWhere(), select.getGroupBy(), select.getHaving(), + select.getOrderBy(), select.getLimit(), select.getBindCount(), select.isAggregate()); + } + + private final List fromTable; + private final HintNode hint; + private final boolean isDistinct; + private final List select; + private final ParseNode where; + private final List groupBy; + private final ParseNode having; + private final List orderBy; + private final LimitNode limit; + private final int bindCount; + private final boolean isAggregate; + + // Count constant expressions + private static int countConstants(List nodes) { + int count = 0; + for (int i = 0; i < nodes.size(); i++) { + if (nodes.get(i).isStateless()) { + count++; + } + } + return count; + } + + protected SelectStatement(List from, HintNode hint, boolean isDistinct, List select, ParseNode where, List groupBy, ParseNode having, List orderBy, LimitNode limit, int bindCount, boolean isAggregate) { + this.fromTable = Collections.unmodifiableList(from); + this.hint = hint == null ? HintNode.EMPTY_HINT_NODE : hint; + this.isDistinct = isDistinct; + this.select = Collections.unmodifiableList(select); + this.where = where; + this.groupBy = Collections.unmodifiableList(groupBy); + this.having = having; + this.orderBy = Collections.unmodifiableList(orderBy); + this.limit = limit; + this.bindCount = bindCount; + this.isAggregate = isAggregate || groupBy.size() != countConstants(groupBy) || this.having != null; + } + + @Override + public boolean isDistinct() { + return isDistinct; + } + + @Override + public LimitNode getLimit() { + return limit; + } + + @Override + public int getBindCount() { + return bindCount; + } + + public List getFrom() { + return fromTable; + } + + @Override + public HintNode getHint() { + return hint; + } + + public List getSelect() { + return select; + } + /** + * Gets the where condition, or null if none. + */ + @Override + public ParseNode getWhere() { + return where; + } + + /** + * Gets the group-by, containing at least 1 element, or null, if none. + */ + public List getGroupBy() { + return groupBy; + } + + public ParseNode getHaving() { + return having; + } + + /** + * Gets the order-by, containing at least 1 element, or null, if none. + */ + @Override + public List getOrderBy() { + return orderBy; + } + + @Override + public boolean isAggregate() { + return isAggregate; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SelectStatementRewriter.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/SelectStatementRewriter.java similarity index 76% rename from phoenix-core/src/main/java/com/salesforce/phoenix/parse/SelectStatementRewriter.java rename to phoenix-core/src/main/java/org/apache/phoenix/parse/SelectStatementRewriter.java index d79f67f4..1d181540 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SelectStatementRewriter.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/SelectStatementRewriter.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; import java.sql.SQLException; import java.util.ArrayList; diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SequenceValueParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/SequenceValueParseNode.java similarity index 94% rename from phoenix-core/src/main/java/com/salesforce/phoenix/parse/SequenceValueParseNode.java rename to phoenix-core/src/main/java/org/apache/phoenix/parse/SequenceValueParseNode.java index 7fef793b..f72d60d0 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/SequenceValueParseNode.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/SequenceValueParseNode.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.parse; +package org.apache.phoenix.parse; import java.sql.SQLException; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/SingleTableSQLStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/SingleTableSQLStatement.java new file mode 100644 index 00000000..ec5976ea --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/SingleTableSQLStatement.java @@ -0,0 +1,39 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +public abstract class SingleTableSQLStatement implements BindableStatement { + private final NamedTableNode table; + private final int bindCount; + + public SingleTableSQLStatement(NamedTableNode table, int bindCount) { + this.table = table; + this.bindCount = bindCount; + } + + public NamedTableNode getTable() { + return table; + } + + @Override + public int getBindCount() { + return bindCount; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/StatelessTraverseAllParseNodeVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/StatelessTraverseAllParseNodeVisitor.java similarity index 98% rename from phoenix-core/src/main/java/com/salesforce/phoenix/parse/StatelessTraverseAllParseNodeVisitor.java rename to phoenix-core/src/main/java/org/apache/phoenix/parse/StatelessTraverseAllParseNodeVisitor.java index f886945a..beaa2df5 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/StatelessTraverseAllParseNodeVisitor.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/StatelessTraverseAllParseNodeVisitor.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.parse; +package org.apache.phoenix.parse; import java.sql.SQLException; import java.util.List; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/StringConcatParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/StringConcatParseNode.java new file mode 100644 index 00000000..b5134d25 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/StringConcatParseNode.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + + +/** + * + * Node representing || String concatenation in a SQL expression + * + * @author kmahadik + * @since 0.1 + */ +public class StringConcatParseNode extends CompoundParseNode { + + StringConcatParseNode(List children) { + super(children); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/SubqueryParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/SubqueryParseNode.java new file mode 100644 index 00000000..b2073df9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/SubqueryParseNode.java @@ -0,0 +1,49 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; + + + +/** + * + * Node representing a subquery in SQL + * + * @author jtaylor + * @since 0.1 + */ +public class SubqueryParseNode extends TerminalParseNode { + private final SelectStatement select; + + SubqueryParseNode(SelectStatement select) { + this.select = select; + } + + public SelectStatement getSelectNode() { + return select; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + return visitor.visit(this); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/SubtractParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/SubtractParseNode.java new file mode 100644 index 00000000..c12efef1 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/SubtractParseNode.java @@ -0,0 +1,48 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.Collections; +import java.util.List; + + + +/** + * + * Node representing subtraction in a SQL expression + * + * @author jtaylor + * @since 0.1 + */ +public class SubtractParseNode extends ArithmeticParseNode { + SubtractParseNode(List children) { + super(children); + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + List l = Collections.emptyList(); + if (visitor.visitEnter(this)) { + l = acceptChildren(visitor); + } + return visitor.visitLeave(this, l); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/SumAggregateParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/SumAggregateParseNode.java new file mode 100644 index 00000000..8669b8d0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/SumAggregateParseNode.java @@ -0,0 +1,41 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.function.FunctionExpression; +import org.apache.phoenix.expression.function.SumAggregateFunction; + + +public class SumAggregateParseNode extends DelegateConstantToCountParseNode { + + public SumAggregateParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public FunctionExpression create(List children, StatementContext context) throws SQLException { + return new SumAggregateFunction(children, getDelegateFunction(children,context)); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/TableName.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/TableName.java new file mode 100644 index 00000000..e5525928 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/TableName.java @@ -0,0 +1,89 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.util.SchemaUtil; + +public class TableName { + private final String tableName; + private final String schemaName; + + public static TableName createNormalized(String schemaName, String tableName) { + schemaName = schemaName == null ? null : SchemaUtil.normalizeIdentifier(schemaName); + tableName = SchemaUtil.normalizeIdentifier(tableName); + return new TableName(schemaName, tableName); + } + + public static TableName create(String schemaName, String tableName) { + return new TableName(schemaName,tableName); + } + + private TableName(String schemaName, String tableName) { + this.schemaName = schemaName; + this.tableName = tableName; + } + + public String getTableName() { + return tableName; + } + + public String getSchemaName() { + return schemaName; + } + + @Override + public String toString() { + return (schemaName == null ? "" : schemaName + QueryConstants.NAME_SEPARATOR) + tableName; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + + ((schemaName == null) ? 0 : schemaName.hashCode()); + result = prime * result + + ((tableName == null) ? 0 : tableName.hashCode()); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) + return true; + if (obj == null) + return false; + if (getClass() != obj.getClass()) + return false; + TableName other = (TableName) obj; + if (schemaName == null) { + if (other.schemaName != null) + return false; + } else if (!schemaName.equals(other.schemaName)) + return false; + if (tableName == null) { + if (other.tableName != null) + return false; + } else if (!tableName.equals(other.tableName)) + return false; + return true; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/TableNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/TableNode.java new file mode 100644 index 00000000..0eaf140f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/TableNode.java @@ -0,0 +1,45 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; + + + +/** + * + * Abstract base class for FROM clause data sources + * + * @author jtaylor + * @since 0.1 + */ +public abstract class TableNode { + private final String alias; + + TableNode(String alias) { + this.alias = alias; + } + + public String getAlias() { + return alias; + } + + public abstract void accept(TableNodeVisitor visitor) throws SQLException; +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/TableNodeVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/TableNodeVisitor.java new file mode 100644 index 00000000..6a14da83 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/TableNodeVisitor.java @@ -0,0 +1,37 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; + + +/** + * + * Visitor for nodes in the FROM clause + * + * @author jtaylor + * @since 0.1 + */ +public interface TableNodeVisitor { + void visit(BindTableNode boundTableNode) throws SQLException; + void visit(JoinTableNode joinNode) throws SQLException; + void visit(NamedTableNode namedTableNode) throws SQLException; + void visit(DerivedTableNode subselectNode) throws SQLException; +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/TerminalParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/TerminalParseNode.java new file mode 100644 index 00000000..2c31341c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/TerminalParseNode.java @@ -0,0 +1,37 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.Collections; +import java.util.List; + +/** + * + * Abstract node for expressions that have no children + * + * @author jtaylor + * @since 0.1 + */ +public abstract class TerminalParseNode extends ParseNode { + @Override + public final List getChildren() { + return Collections.emptyList(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ToCharParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ToCharParseNode.java new file mode 100644 index 00000000..ef4d390b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ToCharParseNode.java @@ -0,0 +1,67 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.text.Format; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.function.FunctionArgumentType; +import org.apache.phoenix.expression.function.FunctionExpression; +import org.apache.phoenix.expression.function.ToCharFunction; +import org.apache.phoenix.schema.PDataType; + + +public class ToCharParseNode extends FunctionParseNode { + + public ToCharParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public FunctionExpression create(List children, StatementContext context) throws SQLException { + PDataType dataType = children.get(0).getDataType(); + String formatString = (String)((LiteralExpression)children.get(1)).getValue(); // either date or number format string + Format formatter; + FunctionArgumentType type; + if (dataType.isCoercibleTo(PDataType.TIMESTAMP)) { + if (formatString == null) { + formatString = context.getDateFormat(); + formatter = context.getDateFormatter(); + } else { + formatter = FunctionArgumentType.TEMPORAL.getFormatter(formatString); + } + type = FunctionArgumentType.TEMPORAL; + } + else if (dataType.isCoercibleTo(PDataType.DECIMAL)) { + if (formatString == null) + formatString = context.getNumberFormat(); + formatter = FunctionArgumentType.NUMERIC.getFormatter(formatString); + type = FunctionArgumentType.NUMERIC; + } + else { + throw new SQLException(dataType + " type is unsupported for TO_CHAR(). Numeric and temporal types are supported."); + } + return new ToCharFunction(children, type, formatString, formatter); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/ToDateParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ToDateParseNode.java new file mode 100644 index 00000000..9863902d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ToDateParseNode.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.text.Format; +import java.util.List; + +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.function.FunctionExpression; +import org.apache.phoenix.expression.function.ToDateFunction; +import org.apache.phoenix.util.DateUtil; + + +public class ToDateParseNode extends FunctionParseNode { + public ToDateParseNode(String name, List children, BuiltInFunctionInfo info) { + super(name, children, info); + } + + @Override + public FunctionExpression create(List children, StatementContext context) throws SQLException { + Format dateParser; + String dateFormat = (String)((LiteralExpression)children.get(1)).getValue(); + if (dateFormat == null) { + dateFormat = context.getDateFormat(); + dateParser = context.getDateParser(); + } else { + dateParser = DateUtil.getDateParser(dateFormat); + } + return new ToDateFunction(children, dateFormat, dateParser); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ToNumberParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/ToNumberParseNode.java similarity index 77% rename from phoenix-core/src/main/java/com/salesforce/phoenix/parse/ToNumberParseNode.java rename to phoenix-core/src/main/java/org/apache/phoenix/parse/ToNumberParseNode.java index 00d9d405..cb2d1491 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/ToNumberParseNode.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/ToNumberParseNode.java @@ -1,16 +1,16 @@ -package com.salesforce.phoenix.parse; +package org.apache.phoenix.parse; import java.sql.SQLException; import java.text.Format; import java.util.List; -import com.salesforce.phoenix.compile.StatementContext; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.expression.LiteralExpression; -import com.salesforce.phoenix.expression.function.FunctionArgumentType; -import com.salesforce.phoenix.expression.function.FunctionExpression; -import com.salesforce.phoenix.expression.function.ToNumberFunction; -import com.salesforce.phoenix.schema.PDataType; +import org.apache.phoenix.compile.StatementContext; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.expression.LiteralExpression; +import org.apache.phoenix.expression.function.FunctionArgumentType; +import org.apache.phoenix.expression.function.FunctionExpression; +import org.apache.phoenix.expression.function.ToNumberFunction; +import org.apache.phoenix.schema.PDataType; public class ToNumberParseNode extends FunctionParseNode { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/TraverseAllParseNodeVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/TraverseAllParseNodeVisitor.java new file mode 100644 index 00000000..1a4a4e64 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/TraverseAllParseNodeVisitor.java @@ -0,0 +1,158 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + + +/** + * + * Visitor that traverses into all parse nodes + * + * @author jtaylor + * @since 0.1 + */ +public abstract class TraverseAllParseNodeVisitor extends BaseParseNodeVisitor { + @Override + public boolean visitEnter(AndParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(OrParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(FunctionParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(CaseParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(ComparisonParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(LikeParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(NotParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(CastParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(InListParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(IsNullParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(MultiplyParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(SubtractParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(AddParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(DivideParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(BetweenParseNode node) throws SQLException { + return true; + } + + @Override + public T visit(ColumnParseNode node) throws SQLException { + return null; + } + + @Override + public T visitLeave(IsNullParseNode node, List l) throws SQLException { + return null; + } + + @Override + public T visit(LiteralParseNode node) throws SQLException { + return null; + } + + @Override + public T visit(BindParseNode node) throws SQLException { + return null; + } + + @Override + public T visit(WildcardParseNode node) throws SQLException { + return null; + } + + @Override + public T visit(FamilyWildcardParseNode node) throws SQLException { + return null; + } + + @Override + public boolean visitEnter(StringConcatParseNode node) throws SQLException { + return true; + } + + @Override + public boolean visitEnter(RowValueConstructorParseNode node) throws SQLException { + return true; + } + + @Override + public T visit(SequenceValueParseNode node) throws SQLException { + return null; + } + + @Override + public boolean visitEnter(ArrayConstructorNode node) throws SQLException { + return true; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/TraverseNoParseNodeVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/TraverseNoParseNodeVisitor.java new file mode 100644 index 00000000..c3d62fd0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/TraverseNoParseNodeVisitor.java @@ -0,0 +1,242 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; +import java.util.List; + + +/** + * + * Visitor that traverses into no parse nodes + * + * @author jtaylor + * @since 0.1 + */ +public abstract class TraverseNoParseNodeVisitor extends BaseParseNodeVisitor { + @Override + public boolean visitEnter(AndParseNode node) throws SQLException { + return false; + } + + @Override + public boolean visitEnter(OrParseNode node) throws SQLException { + return false; + } + + @Override + public boolean visitEnter(FunctionParseNode node) throws SQLException { + return false; + } + + @Override + public boolean visitEnter(ComparisonParseNode node) throws SQLException { + return false; + } + + @Override + public boolean visitEnter(CaseParseNode node) throws SQLException { + return false; + } + + @Override + public boolean visitEnter(LikeParseNode node) throws SQLException { + return false; + } + + @Override + public boolean visitEnter(BetweenParseNode node) throws SQLException { + return false; + } + + @Override + public T visitLeave(LikeParseNode node, List l) throws SQLException { + return null; + } + + @Override + public boolean visitEnter(NotParseNode node) throws SQLException { + return false; + } + + @Override + public boolean visitEnter(CastParseNode node) throws SQLException { + return false; + } + + @Override + public T visitLeave(NotParseNode node, List l) throws SQLException { + return null; + } + + @Override + public T visitLeave(CastParseNode node, List l) throws SQLException { + return null; + } + + @Override + public boolean visitEnter(InListParseNode node) throws SQLException { + return false; + } + + @Override + public T visitLeave(InListParseNode node, List l) throws SQLException { + return null; + } + + @Override + public boolean visitEnter(IsNullParseNode node) throws SQLException { + return false; + } + + @Override + public T visitLeave(IsNullParseNode node, List l) throws SQLException { + return null; + } + + @Override + public T visit(ColumnParseNode node) throws SQLException { + return null; + } + + @Override + public T visit(LiteralParseNode node) throws SQLException { + return null; + } + + @Override + public T visit(BindParseNode node) throws SQLException { + return null; + } + + @Override + public T visit(WildcardParseNode node) throws SQLException { + return null; + } + + @Override + public T visit(FamilyWildcardParseNode node) throws SQLException { + return null; + } + + @Override + public T visitLeave(AndParseNode node, List l) throws SQLException { + return null; + } + + @Override + public T visitLeave(OrParseNode node, List l) throws SQLException { + return null; + } + + @Override + public T visitLeave(FunctionParseNode node, List l) throws SQLException { + return null; + } + + @Override + public T visitLeave(ComparisonParseNode node, List l) throws SQLException { + return null; + } + + @Override + public T visitLeave(CaseParseNode node, List l) throws SQLException { + return null; + } + + @Override + public boolean visitEnter(MultiplyParseNode node) throws SQLException { + return false; + } + + @Override + public T visitLeave(MultiplyParseNode node, List l) throws SQLException { + return null; + } + + @Override + public boolean visitEnter(SubtractParseNode node) throws SQLException { + return false; + } + + @Override + public T visitLeave(SubtractParseNode node, List l) throws SQLException { + return null; + } + + @Override + public boolean visitEnter(AddParseNode node) throws SQLException { + return false; + } + + @Override + public T visitLeave(AddParseNode node, List l) throws SQLException { + return null; + } + + @Override + public boolean visitEnter(DivideParseNode node) throws SQLException { + return false; + } + + @Override + public T visitLeave(DivideParseNode node, List l) throws SQLException { + return null; + } + @Override + public boolean visitEnter(StringConcatParseNode node) throws SQLException { + return false; + } + + @Override + public T visitLeave(StringConcatParseNode node, List l) throws SQLException { + return null; + } + + @Override + public T visitLeave(BetweenParseNode node, List l) throws SQLException { + return null; + } + + @Override + public boolean visitEnter(RowValueConstructorParseNode node) throws SQLException { + return false; + } + + @Override + public T visitLeave(RowValueConstructorParseNode node, List l) throws SQLException { + return null; + } + + @Override + public T visit(SequenceValueParseNode node) throws SQLException { + return null; + } + + @Override + public boolean visitEnter(ArrayConstructorNode node) throws SQLException { + return false; + } + + @Override + public T visitLeave(ArrayConstructorNode node, List l) throws SQLException { + return null; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/UnaryParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/UnaryParseNode.java new file mode 100644 index 00000000..356d0476 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/UnaryParseNode.java @@ -0,0 +1,35 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.Collections; + +/** + * + * Abstract node representing an expression that has a single child in SQL + * + * @author jtaylor + * @since 0.1 + */ +public abstract class UnaryParseNode extends CompoundParseNode { + UnaryParseNode(ParseNode expr) { + super(Collections.singletonList(expr)); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/UnsupportedAllParseNodeVisitor.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/UnsupportedAllParseNodeVisitor.java similarity index 78% rename from phoenix-core/src/main/java/com/salesforce/phoenix/parse/UnsupportedAllParseNodeVisitor.java rename to phoenix-core/src/main/java/org/apache/phoenix/parse/UnsupportedAllParseNodeVisitor.java index 6f71937d..24bfcf56 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/parse/UnsupportedAllParseNodeVisitor.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/UnsupportedAllParseNodeVisitor.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.parse; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; import java.sql.SQLException; import java.sql.SQLFeatureNotSupportedException; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/UpsertStatement.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/UpsertStatement.java new file mode 100644 index 00000000..777a64c9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/UpsertStatement.java @@ -0,0 +1,54 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.util.Collections; +import java.util.List; + +public class UpsertStatement extends SingleTableSQLStatement { + private final List columns; + private final List values; + private final SelectStatement select; + private final HintNode hint; + + public UpsertStatement(NamedTableNode table, HintNode hint, List columns, List values, SelectStatement select, int bindCount) { + super(table, bindCount); + this.columns = columns == null ? Collections.emptyList() : columns; + this.values = values; + this.select = select; + this.hint = hint == null ? HintNode.EMPTY_HINT_NODE : hint; + } + + public List getColumns() { + return columns; + } + + public List getValues() { + return values; + } + + public SelectStatement getSelect() { + return select; + } + + public HintNode getHint() { + return hint; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/parse/WildcardParseNode.java b/phoenix-core/src/main/java/org/apache/phoenix/parse/WildcardParseNode.java new file mode 100644 index 00000000..1ae27fc9 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/parse/WildcardParseNode.java @@ -0,0 +1,58 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.parse; + +import java.sql.SQLException; + + + +/** + * + * Node representing the selection of all columns (*) in the SELECT clause of SQL + * + * @author jtaylor + * @since 0.1 + */ +public class WildcardParseNode extends TerminalParseNode { + public static final String NAME = "*"; + public static final WildcardParseNode INSTANCE = new WildcardParseNode(false); + public static final WildcardParseNode REWRITE_INSTANCE = new WildcardParseNode(true); + + private final boolean isRewrite; + + private WildcardParseNode(boolean isRewrite) { + this.isRewrite = isRewrite; + } + + @Override + public T accept(ParseNodeVisitor visitor) throws SQLException { + return visitor.visit(this); + } + + @Override + public String toString() { + return NAME; + } + + public boolean isRewrite() { + return isRewrite; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/BaseQueryServicesImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/query/BaseQueryServicesImpl.java new file mode 100644 index 00000000..27fa10e6 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/BaseQueryServicesImpl.java @@ -0,0 +1,80 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import java.util.concurrent.ExecutorService; + +import org.apache.phoenix.job.JobManager; +import org.apache.phoenix.memory.GlobalMemoryManager; +import org.apache.phoenix.memory.MemoryManager; +import org.apache.phoenix.optimize.QueryOptimizer; +import org.apache.phoenix.util.ReadOnlyProps; + + + +/** + * + * Base class for QueryService implementors. + * + * @author jtaylor + * @since 0.1 + */ +public abstract class BaseQueryServicesImpl implements QueryServices { + private final ExecutorService executor; + private final MemoryManager memoryManager; + private final ReadOnlyProps props; + private final QueryOptimizer queryOptimizer; + + public BaseQueryServicesImpl(QueryServicesOptions options) { + this.executor = JobManager.createThreadPoolExec( + options.getKeepAliveMs(), + options.getThreadPoolSize(), + options.getQueueSize()); + this.memoryManager = new GlobalMemoryManager( + Runtime.getRuntime().totalMemory() * options.getMaxMemoryPerc() / 100, + options.getMaxMemoryWaitMs()); + this.props = options.getProps(); + this.queryOptimizer = new QueryOptimizer(this); + } + + @Override + public ExecutorService getExecutor() { + return executor; + } + + @Override + public MemoryManager getMemoryManager() { + return memoryManager; + } + + @Override + public final ReadOnlyProps getProps() { + return props; + } + + @Override + public void close() { + } + + @Override + public QueryOptimizer getOptimizer() { + return queryOptimizer; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/ChildQueryServices.java b/phoenix-core/src/main/java/org/apache/phoenix/query/ChildQueryServices.java new file mode 100644 index 00000000..93613d6c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/ChildQueryServices.java @@ -0,0 +1,47 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import org.apache.phoenix.memory.ChildMemoryManager; +import org.apache.phoenix.memory.MemoryManager; + +/** + * + * Child QueryServices that delegates through to global QueryService. + * Used to track memory used by each org to allow a max percentage threshold. + * + * @author jtaylor + * @since 0.1 + */ +public class ChildQueryServices extends DelegateConnectionQueryServices { + private final MemoryManager memoryManager; + private static final int DEFAULT_MAX_ORG_MEMORY_PERC = 30; + + public ChildQueryServices(ConnectionQueryServices services) { + super(services); + int maxOrgMemPerc = getProps().getInt(MAX_TENANT_MEMORY_PERC_ATTRIB, DEFAULT_MAX_ORG_MEMORY_PERC); + this.memoryManager = new ChildMemoryManager(services.getMemoryManager(), maxOrgMemPerc); + } + + @Override + public MemoryManager getMemoryManager() { + return memoryManager; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/ConfigurationFactory.java b/phoenix-core/src/main/java/org/apache/phoenix/query/ConfigurationFactory.java new file mode 100644 index 00000000..77d737ed --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/ConfigurationFactory.java @@ -0,0 +1,46 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.HBaseConfiguration; + +/** + * Creates {@link Configuration} instances that contain HBase/Hadoop settings. + * + * @author aaraujo + * @since 2.0 + */ +public interface ConfigurationFactory { + /** + * @return Configuration containing HBase/Hadoop settings + */ + Configuration getConfiguration(); + + /** + * Default implementation uses {@link org.apache.hadoop.hbase.HBaseConfiguration#create()}. + */ + static class ConfigurationFactoryImpl implements ConfigurationFactory { + @Override + public Configuration getConfiguration() { + return HBaseConfiguration.create(); + } + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServices.java b/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServices.java new file mode 100644 index 00000000..47eae18c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServices.java @@ -0,0 +1,102 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import java.sql.SQLException; +import java.util.List; +import java.util.Map; +import java.util.Properties; + +import org.apache.hadoop.hbase.HRegionLocation; +import org.apache.hadoop.hbase.HTableDescriptor; +import org.apache.hadoop.hbase.client.HBaseAdmin; +import org.apache.hadoop.hbase.client.HTableInterface; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Pair; +import org.apache.phoenix.client.KeyValueBuilder; +import org.apache.phoenix.compile.MutationPlan; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.SequenceKey; + + +public interface ConnectionQueryServices extends QueryServices, MetaDataMutated { + /** + * Get (and create if necessary) a child QueryService for a given tenantId. + * The QueryService will be cached for the lifetime of the parent QueryService + * @param tenantId the organization ID + * @return the child QueryService + */ + public ConnectionQueryServices getChildQueryServices(ImmutableBytesWritable tenantId); + + /** + * Get an HTableInterface by the given name. It is the callers + * responsibility to close the returned HTableInterface. + * @param tableName the name of the HTable + * @return the HTableInterface + * @throws SQLException + */ + public HTableInterface getTable(byte[] tableName) throws SQLException; + + public HTableDescriptor getTableDescriptor(byte[] tableName) throws SQLException; + + public StatsManager getStatsManager(); + + public List getAllTableRegions(byte[] tableName) throws SQLException; + + public PhoenixConnection connect(String url, Properties info) throws SQLException; + + public MetaDataMutationResult getTable(byte[] tenantId, byte[] schemaName, byte[] tableName, long tableTimestamp, long clientTimetamp) throws SQLException; + public MetaDataMutationResult createTable(List tableMetaData, byte[] tableName, PTableType tableType, Map tableProps, List>> families, byte[][] splits) throws SQLException; + public MetaDataMutationResult dropTable(List tableMetadata, PTableType tableType) throws SQLException; + public MetaDataMutationResult addColumn(List tableMetaData, PTableType tableType, List>> families) throws SQLException; + public MetaDataMutationResult dropColumn(List tableMetadata, PTableType tableType) throws SQLException; + public MetaDataMutationResult updateIndexState(List tableMetadata, String parentTableName) throws SQLException; + public MutationState updateData(MutationPlan plan) throws SQLException; + + public void init(String url, Properties props) throws SQLException; + + public int getLowestClusterHBaseVersion(); + public HBaseAdmin getAdmin() throws SQLException; + + void clearTableRegionCache(byte[] tableName) throws SQLException; + + boolean hasInvalidIndexConfiguration(); + + long createSequence(String tenantId, String schemaName, String sequenceName, long startWith, long incrementBy, int batchSize, long timestamp) throws SQLException; + long dropSequence(String tenantId, String schemaName, String sequenceName, long timestamp) throws SQLException; + void reserveSequenceValues(List sequenceKeys, long timestamp, long[] values, SQLException[] exceptions) throws SQLException; + void incrementSequenceValues(List sequenceKeys, long timestamp, long[] values, SQLException[] exceptions) throws SQLException; + long getSequenceValue(SequenceKey sequenceKey, long timestamp) throws SQLException; + void returnSequenceValues(List sequenceKeys, long timestamp, SQLException[] exceptions) throws SQLException; + void addConnection(PhoenixConnection connection) throws SQLException; + void removeConnection(PhoenixConnection connection) throws SQLException; + + /** + * @return the {@link KeyValueBuilder} that is valid for the locally installed version of HBase. + */ + public KeyValueBuilder getKeyValueBuilder(); + + public enum Feature {REVERSE_SCAN}; + public boolean supportsFeature(Feature feature); +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/ConnectionQueryServicesImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java similarity index 91% rename from phoenix-core/src/main/java/com/salesforce/phoenix/query/ConnectionQueryServicesImpl.java rename to phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java index c58c4ac4..4faac665 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/ConnectionQueryServicesImpl.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java @@ -1,34 +1,26 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_TABLE_NAME_BYTES; -import static com.salesforce.phoenix.query.QueryServicesOptions.DEFAULT_DROP_METADATA; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_TABLE_NAME_BYTES; +import static org.apache.phoenix.query.QueryServicesOptions.DEFAULT_DROP_METADATA; import java.io.IOException; import java.sql.SQLException; @@ -58,59 +50,59 @@ import org.apache.hadoop.hbase.client.Mutation; import org.apache.hadoop.hbase.client.Result; import org.apache.hadoop.hbase.client.coprocessor.Batch; +import org.apache.hadoop.hbase.index.Indexer; +import org.apache.hadoop.hbase.index.covered.CoveredColumnsIndexBuilder; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.hbase.util.Bytes; import org.apache.hadoop.hbase.util.Pair; import org.apache.hadoop.hbase.util.VersionInfo; import org.apache.hadoop.hbase.zookeeper.ZKConfig; +import org.apache.phoenix.client.KeyValueBuilder; +import org.apache.phoenix.compile.MutationPlan; +import org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver; +import org.apache.phoenix.coprocessor.MetaDataEndpointImpl; +import org.apache.phoenix.coprocessor.MetaDataProtocol; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MutationCode; +import org.apache.phoenix.coprocessor.MetaDataRegionObserver; +import org.apache.phoenix.coprocessor.ScanRegionObserver; +import org.apache.phoenix.coprocessor.SequenceRegionObserver; +import org.apache.phoenix.coprocessor.ServerCachingEndpointImpl; +import org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver; +import org.apache.phoenix.exception.PhoenixIOException; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.index.PhoenixIndexBuilder; +import org.apache.phoenix.index.PhoenixIndexCodec; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.ConnectionInfo; +import org.apache.phoenix.schema.EmptySequenceCacheException; +import org.apache.phoenix.schema.MetaDataSplitPolicy; +import org.apache.phoenix.schema.NewerTableAlreadyExistsException; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PMetaData; +import org.apache.phoenix.schema.PMetaDataImpl; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.ReadOnlyTableException; +import org.apache.phoenix.schema.Sequence; +import org.apache.phoenix.schema.SequenceKey; +import org.apache.phoenix.schema.TableNotFoundException; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.JDBCUtil; +import org.apache.phoenix.util.MetaDataUtil; +import org.apache.phoenix.util.PhoenixRuntime; +import org.apache.phoenix.util.ReadOnlyProps; +import org.apache.phoenix.util.SchemaUtil; +import org.apache.phoenix.util.ServerUtil; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import com.google.common.collect.Lists; import com.google.common.collect.Maps; import com.google.common.collect.Sets; -import com.salesforce.hbase.index.Indexer; -import com.salesforce.hbase.index.covered.CoveredColumnsIndexBuilder; -import com.salesforce.phoenix.client.KeyValueBuilder; -import com.salesforce.phoenix.compile.MutationPlan; -import com.salesforce.phoenix.coprocessor.GroupedAggregateRegionObserver; -import com.salesforce.phoenix.coprocessor.MetaDataEndpointImpl; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MutationCode; -import com.salesforce.phoenix.coprocessor.MetaDataRegionObserver; -import com.salesforce.phoenix.coprocessor.ScanRegionObserver; -import com.salesforce.phoenix.coprocessor.SequenceRegionObserver; -import com.salesforce.phoenix.coprocessor.ServerCachingEndpointImpl; -import com.salesforce.phoenix.coprocessor.UngroupedAggregateRegionObserver; -import com.salesforce.phoenix.exception.PhoenixIOException; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.index.PhoenixIndexBuilder; -import com.salesforce.phoenix.index.PhoenixIndexCodec; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.jdbc.PhoenixEmbeddedDriver.ConnectionInfo; -import com.salesforce.phoenix.schema.EmptySequenceCacheException; -import com.salesforce.phoenix.schema.MetaDataSplitPolicy; -import com.salesforce.phoenix.schema.NewerTableAlreadyExistsException; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PMetaData; -import com.salesforce.phoenix.schema.PMetaDataImpl; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.ReadOnlyTableException; -import com.salesforce.phoenix.schema.Sequence; -import com.salesforce.phoenix.schema.SequenceKey; -import com.salesforce.phoenix.schema.TableNotFoundException; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.JDBCUtil; -import com.salesforce.phoenix.util.MetaDataUtil; -import com.salesforce.phoenix.util.PhoenixRuntime; -import com.salesforce.phoenix.util.ReadOnlyProps; -import com.salesforce.phoenix.util.SchemaUtil; -import com.salesforce.phoenix.util.ServerUtil; public class ConnectionQueryServicesImpl extends DelegateQueryServices implements ConnectionQueryServices { private static final Logger logger = LoggerFactory.getLogger(ConnectionQueryServicesImpl.class); @@ -500,7 +492,6 @@ private HTableDescriptor generateTableDescriptor(byte[] tableName, HTableDescrip if (SchemaUtil.isMetaTable(tableName)) { descriptor.setValue(SchemaUtil.UPGRADE_TO_2_0, Boolean.TRUE.toString()); descriptor.setValue(SchemaUtil.UPGRADE_TO_2_1, Boolean.TRUE.toString()); - descriptor.setValue(SchemaUtil.UPGRADE_TO_3_0, Boolean.TRUE.toString()); if (!descriptor.hasCoprocessor(MetaDataEndpointImpl.class.getName())) { descriptor.addCoprocessor(MetaDataEndpointImpl.class.getName(), null, 1, null); } @@ -653,19 +644,8 @@ private boolean ensureTableCreated(byte[] tableName, PTableType tableType , Map< return false; } - boolean updateTo3_0 = false; if (isMetaTable) { checkClientServerCompatibility(); - - updateTo3_0 = existingDesc.getValue(SchemaUtil.UPGRADE_TO_3_0) == null; - if (updateTo3_0) { - // Check if SYSTEM.TABLE has multiple regions which is problematic, as - // we can't update it atomically. It's unlikely that this is the case - // but we'll find out if it is. - if (this.getAllTableRegions(tableName).size() > 1) { - throw new SQLException("Unable to convert SYSTEM.TABLE automatically as it is too big. Please contact customer support."); - } - } } // We'll do this alter at the end of the upgrade @@ -809,7 +789,7 @@ public void update(byte[] region, byte[] row, MetaDataMutationResult value) { @Override - public MetaDataMutationResult createTable(final List tableMetaData, byte[] tableName, PTableType tableType, + public MetaDataMutationResult createTable(final List tableMetaData, byte[] physicalTableName, PTableType tableType, Map tableProps, final List>> families, byte[][] splits) throws SQLException { byte[][] rowKeyMetadata = new byte[3][]; Mutation m = tableMetaData.get(0); @@ -818,9 +798,11 @@ public MetaDataMutationResult createTable(final List tableMetaData, by byte[] tenantIdBytes = rowKeyMetadata[PhoenixDatabaseMetaData.TENANT_ID_INDEX]; byte[] schemaBytes = rowKeyMetadata[PhoenixDatabaseMetaData.SCHEMA_NAME_INDEX]; byte[] tableBytes = rowKeyMetadata[PhoenixDatabaseMetaData.TABLE_NAME_INDEX]; - if (tableType != PTableType.VIEW || tableName != null) { - tableName = tableName == null ? SchemaUtil.getTableNameAsBytes(schemaBytes, tableBytes) : tableName; - ensureTableCreated(tableName, tableType, tableProps, families, splits); + if (tableType != PTableType.VIEW || physicalTableName != null) { + if (physicalTableName == null) { + physicalTableName = SchemaUtil.getTableNameAsBytes(schemaBytes, tableBytes); + } + ensureTableCreated(physicalTableName, tableType, tableProps, families, splits); } byte[] tableKey = SchemaUtil.getTableKey(tenantIdBytes, schemaBytes, tableBytes); @@ -1436,4 +1418,11 @@ public void removeConnection(PhoenixConnection connection) throws SQLException { public KeyValueBuilder getKeyValueBuilder() { return this.kvBuilder; } + + @Override + public boolean supportsFeature(Feature feature) { + // TODO: Keep map of Feature -> min HBase version + // For now, only Feature is REVERSE_SCAN and it's not supported in any version yet + return false; + } } diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/ConnectionlessQueryServicesImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java similarity index 77% rename from phoenix-core/src/main/java/com/salesforce/phoenix/query/ConnectionlessQueryServicesImpl.java rename to phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java index f4cfc0aa..dd7cc5f0 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/ConnectionlessQueryServicesImpl.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java @@ -1,33 +1,25 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_FAMILY_BYTES; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_FAMILY_BYTES; import java.sql.SQLException; import java.util.Collections; @@ -47,34 +39,34 @@ import org.apache.hadoop.hbase.util.Bytes; import org.apache.hadoop.hbase.util.Pair; import org.apache.hadoop.hbase.util.VersionInfo; +import org.apache.phoenix.client.KeyValueBuilder; +import org.apache.phoenix.compile.MutationPlan; +import org.apache.phoenix.coprocessor.MetaDataProtocol; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MutationCode; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.schema.NewerTableAlreadyExistsException; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PIndexState; +import org.apache.phoenix.schema.PMetaData; +import org.apache.phoenix.schema.PMetaDataImpl; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableImpl; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.SequenceAlreadyExistsException; +import org.apache.phoenix.schema.SequenceKey; +import org.apache.phoenix.schema.SequenceNotFoundException; +import org.apache.phoenix.schema.TableAlreadyExistsException; +import org.apache.phoenix.schema.TableNotFoundException; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.PhoenixRuntime; +import org.apache.phoenix.util.SchemaUtil; import com.google.common.collect.Maps; -import com.salesforce.phoenix.client.KeyValueBuilder; -import com.salesforce.phoenix.compile.MutationPlan; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MutationCode; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.schema.NewerTableAlreadyExistsException; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PIndexState; -import com.salesforce.phoenix.schema.PMetaData; -import com.salesforce.phoenix.schema.PMetaDataImpl; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.PTableImpl; -import com.salesforce.phoenix.schema.PTableType; -import com.salesforce.phoenix.schema.SequenceAlreadyExistsException; -import com.salesforce.phoenix.schema.SequenceKey; -import com.salesforce.phoenix.schema.SequenceNotFoundException; -import com.salesforce.phoenix.schema.TableAlreadyExistsException; -import com.salesforce.phoenix.schema.TableNotFoundException; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.PhoenixRuntime; -import com.salesforce.phoenix.util.SchemaUtil; /** @@ -176,7 +168,7 @@ public MetaDataMutationResult getTable(byte[] tenantId, byte[] schemaBytes, byte } @Override - public MetaDataMutationResult createTable(List tableMetaData, byte[] tableName, PTableType tableType, Map tableProps, List>> families, byte[][] splits) throws SQLException { + public MetaDataMutationResult createTable(List tableMetaData, byte[] physicalName, PTableType tableType, Map tableProps, List>> families, byte[][] splits) throws SQLException { return new MetaDataMutationResult(MutationCode.TABLE_NOT_FOUND, 0, null); } @@ -360,4 +352,9 @@ public void removeConnection(PhoenixConnection connection) throws SQLException { public KeyValueBuilder getKeyValueBuilder() { return this.kvBuilder; } -} + + @Override + public boolean supportsFeature(Feature feature) { + return false; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/DelegateConnectionQueryServices.java b/phoenix-core/src/main/java/org/apache/phoenix/query/DelegateConnectionQueryServices.java new file mode 100644 index 00000000..0649dafb --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/DelegateConnectionQueryServices.java @@ -0,0 +1,227 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import java.sql.SQLException; +import java.util.List; +import java.util.Map; +import java.util.Properties; + +import org.apache.hadoop.hbase.HRegionLocation; +import org.apache.hadoop.hbase.HTableDescriptor; +import org.apache.hadoop.hbase.client.HBaseAdmin; +import org.apache.hadoop.hbase.client.HTableInterface; +import org.apache.hadoop.hbase.client.Mutation; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Pair; +import org.apache.phoenix.client.KeyValueBuilder; +import org.apache.phoenix.compile.MutationPlan; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PMetaData; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.PTableType; +import org.apache.phoenix.schema.SequenceKey; + + +public class DelegateConnectionQueryServices extends DelegateQueryServices implements ConnectionQueryServices { + + public DelegateConnectionQueryServices(ConnectionQueryServices delegate) { + super(delegate); + } + + @Override + protected ConnectionQueryServices getDelegate() { + return (ConnectionQueryServices)super.getDelegate(); + } + + @Override + public ConnectionQueryServices getChildQueryServices(ImmutableBytesWritable tenantId) { + return getDelegate().getChildQueryServices(tenantId); + } + + @Override + public HTableInterface getTable(byte[] tableName) throws SQLException { + return getDelegate().getTable(tableName); + } + + @Override + public StatsManager getStatsManager() { + return getDelegate().getStatsManager(); + } + + @Override + public List getAllTableRegions(byte[] tableName) throws SQLException { + return getDelegate().getAllTableRegions(tableName); + } + + @Override + public PMetaData addTable(PTable table) throws SQLException { + return getDelegate().addTable(table); + } + + @Override + public PMetaData addColumn(String tableName, List columns, long tableTimeStamp, long tableSeqNum, + boolean isImmutableRows) throws SQLException { + return getDelegate().addColumn(tableName, columns, tableTimeStamp, tableSeqNum, isImmutableRows); + } + + @Override + public PMetaData removeTable(String tableName) + throws SQLException { + return getDelegate().removeTable(tableName); + } + + @Override + public PMetaData removeColumn(String tableName, String familyName, String columnName, long tableTimeStamp, + long tableSeqNum) throws SQLException { + return getDelegate().removeColumn(tableName, familyName, columnName, tableTimeStamp, tableSeqNum); + } + + @Override + public PhoenixConnection connect(String url, Properties info) throws SQLException { + return getDelegate().connect(url, info); + } + + @Override + public MetaDataMutationResult getTable(byte[] tenantId, byte[] schemaBytes, byte[] tableBytes, long tableTimestamp, long clientTimestamp) throws SQLException { + return getDelegate().getTable(tenantId, schemaBytes, tableBytes, tableTimestamp, clientTimestamp); + } + + @Override + public MetaDataMutationResult createTable(List tableMetaData, byte[] physicalName, + PTableType tableType, Map tableProps, List>> families, byte[][] splits) + throws SQLException { + return getDelegate().createTable(tableMetaData, physicalName, tableType, tableProps, families, splits); + } + + @Override + public MetaDataMutationResult dropTable(List tabeMetaData, PTableType tableType) throws SQLException { + return getDelegate().dropTable(tabeMetaData, tableType); + } + + @Override + public MetaDataMutationResult addColumn(List tabeMetaData, PTableType tableType, List>> families ) throws SQLException { + return getDelegate().addColumn(tabeMetaData, tableType, families); + } + + + @Override + public MetaDataMutationResult dropColumn(List tabeMetaData, PTableType tableType) throws SQLException { + return getDelegate().dropColumn(tabeMetaData, tableType); + } + + @Override + public MetaDataMutationResult updateIndexState(List tableMetadata, String parentTableName) throws SQLException { + return getDelegate().updateIndexState(tableMetadata, parentTableName); + } + + @Override + public void init(String url, Properties props) throws SQLException { + getDelegate().init(url, props); + } + + @Override + public MutationState updateData(MutationPlan plan) throws SQLException { + return getDelegate().updateData(plan); + } + + @Override + public int getLowestClusterHBaseVersion() { + return getDelegate().getLowestClusterHBaseVersion(); + } + + @Override + public HBaseAdmin getAdmin() throws SQLException { + return getDelegate().getAdmin(); + } + + @Override + public HTableDescriptor getTableDescriptor(byte[] tableName) throws SQLException { + return getDelegate().getTableDescriptor(tableName); + } + + @Override + public void clearTableRegionCache(byte[] tableName) throws SQLException { + getDelegate().clearTableRegionCache(tableName); + } + + @Override + public boolean hasInvalidIndexConfiguration() { + return getDelegate().hasInvalidIndexConfiguration(); + } + + @Override + public long createSequence(String tenantId, String schemaName, String sequenceName, long startWith, + long incrementBy, int batchSize, long timestamp) throws SQLException { + return getDelegate().createSequence(tenantId, schemaName, sequenceName, startWith, incrementBy, batchSize, timestamp); + } + + @Override + public long dropSequence(String tenantId, String schemaName, String sequenceName, long timestamp) + throws SQLException { + return getDelegate().dropSequence(tenantId, schemaName, sequenceName, timestamp); + } + + @Override + public void reserveSequenceValues(List sequenceKeys, long timestamp, long[] values, + SQLException[] exceptions) throws SQLException { + getDelegate().reserveSequenceValues(sequenceKeys, timestamp, values, exceptions); + } + + @Override + public void incrementSequenceValues(List sequenceKeys, long timestamp, long[] values, + SQLException[] exceptions) throws SQLException { + getDelegate().incrementSequenceValues(sequenceKeys, timestamp, values, exceptions); + } + + @Override + public long getSequenceValue(SequenceKey sequenceKey, long timestamp) throws SQLException { + return getDelegate().getSequenceValue(sequenceKey, timestamp); + } + + @Override + public void returnSequenceValues(List sequenceKeys, long timestamp, SQLException[] exceptions) + throws SQLException { + getDelegate().returnSequenceValues(sequenceKeys, timestamp, exceptions); + } + + @Override + public void addConnection(PhoenixConnection connection) throws SQLException { + getDelegate().addConnection(connection); + } + + @Override + public void removeConnection(PhoenixConnection connection) throws SQLException { + getDelegate().removeConnection(connection); + } + + @Override + public KeyValueBuilder getKeyValueBuilder() { + return getDelegate().getKeyValueBuilder(); + } + + @Override + public boolean supportsFeature(Feature feature) { + return getDelegate().supportsFeature(feature); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/DelegateQueryServices.java b/phoenix-core/src/main/java/org/apache/phoenix/query/DelegateQueryServices.java new file mode 100644 index 00000000..6ddf5c58 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/DelegateQueryServices.java @@ -0,0 +1,74 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import java.sql.SQLException; +import java.util.concurrent.ExecutorService; + +import org.apache.phoenix.memory.MemoryManager; +import org.apache.phoenix.optimize.QueryOptimizer; +import org.apache.phoenix.util.ReadOnlyProps; + + + +/** + * + * Class that delegates QueryService calls through to + * a parent QueryService. + * + * @author jtaylor + * @since 0.1 + */ +public class DelegateQueryServices implements QueryServices { + private final QueryServices parent; + + public DelegateQueryServices(QueryServices queryServices) { + parent = queryServices; + } + + protected QueryServices getDelegate() { + return parent; + } + + @Override + public ExecutorService getExecutor() { + return parent.getExecutor(); + } + + @Override + public MemoryManager getMemoryManager() { + return parent.getMemoryManager(); + } + + @Override + public void close() throws SQLException { + parent.close(); + } + + @Override + public ReadOnlyProps getProps() { + return parent.getProps(); + } + + @Override + public QueryOptimizer getOptimizer() { + return parent.getOptimizer(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/HBaseFactoryProvider.java b/phoenix-core/src/main/java/org/apache/phoenix/query/HBaseFactoryProvider.java new file mode 100644 index 00000000..08e85759 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/HBaseFactoryProvider.java @@ -0,0 +1,57 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import org.apache.phoenix.util.InstanceResolver; + +/** + * Manages factories that provide extension points for HBase. + *

+ * Dependent modules may register their own implementations of the following using {@link java.util.ServiceLoader}: + *

    + *
  • {@link ConfigurationFactory}
    • + *
  • {@link HTableFactory}
    • + *
  • {@link HConnectionFactory}
    • + *
+ * + * If a custom implementation is not registered, the default implementations will be used. + * + * @author aaraujo + * @since 0.2 + */ +public class HBaseFactoryProvider { + + private static final HTableFactory DEFAULT_HTABLE_FACTORY = new HTableFactory.HTableFactoryImpl(); + private static final HConnectionFactory DEFAULT_HCONNECTION_FACTORY = + new HConnectionFactory.HConnectionFactoryImpl(); + private static final ConfigurationFactory DEFAULT_CONFIGURATION_FACTORY = new ConfigurationFactory.ConfigurationFactoryImpl(); + + public static HTableFactory getHTableFactory() { + return InstanceResolver.getSingleton(HTableFactory.class, DEFAULT_HTABLE_FACTORY); + } + + public static HConnectionFactory getHConnectionFactory() { + return InstanceResolver.getSingleton(HConnectionFactory.class, DEFAULT_HCONNECTION_FACTORY); + } + + public static ConfigurationFactory getConfigurationFactory() { + return InstanceResolver.getSingleton(ConfigurationFactory.class, DEFAULT_CONFIGURATION_FACTORY); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/HConnectionFactory.java b/phoenix-core/src/main/java/org/apache/phoenix/query/HConnectionFactory.java new file mode 100644 index 00000000..d40c5400 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/HConnectionFactory.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.hbase.ZooKeeperConnectionException; +import org.apache.hadoop.hbase.client.HConnection; +import org.apache.hadoop.hbase.client.HConnectionManager; + +/** + * Factory for creating {@link HConnection} + * + * @author ukuchibhotla + */ +public interface HConnectionFactory { + + /** + * Creates HConnection to access HBase clusters. + * + * @param configuration object + * @return A HConnection instance + */ + HConnection createConnection(Configuration conf) throws ZooKeeperConnectionException; + + /** + * Default implementation. Uses standard HBase HConnections. + */ + static class HConnectionFactoryImpl implements HConnectionFactory { + @Override + public HConnection createConnection(Configuration conf) throws ZooKeeperConnectionException { + return HConnectionManager.createConnection(conf); + } + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/HTableFactory.java b/phoenix-core/src/main/java/org/apache/phoenix/query/HTableFactory.java new file mode 100644 index 00000000..cf5b4dad --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/HTableFactory.java @@ -0,0 +1,56 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import java.io.IOException; +import java.util.concurrent.ExecutorService; + +import org.apache.hadoop.hbase.client.HConnection; +import org.apache.hadoop.hbase.client.HTable; +import org.apache.hadoop.hbase.client.HTableInterface; + +/** + * Creates clients to access HBase tables. + * + * @author aaraujo + * @since 0.2 + */ +public interface HTableFactory { + /** + * Creates an HBase client using an externally managed HConnection and Thread pool. + * + * @param tableName Name of the table. + * @param connection HConnection to use. + * @param pool ExecutorService to use. + * @return An client to access an HBase table. + * @throws IOException if a server or network exception occurs + */ + HTableInterface getTable(byte[] tableName, HConnection connection, ExecutorService pool) throws IOException; + + /** + * Default implementation. Uses standard HBase HTables. + */ + static class HTableFactoryImpl implements HTableFactory { + @Override + public HTableInterface getTable(byte[] tableName, HConnection connection, ExecutorService pool) throws IOException { + return new HTable(tableName, connection, pool); + } + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/KeyRange.java b/phoenix-core/src/main/java/org/apache/phoenix/query/KeyRange.java similarity index 91% rename from phoenix-core/src/main/java/com/salesforce/phoenix/query/KeyRange.java rename to phoenix-core/src/main/java/org/apache/phoenix/query/KeyRange.java index ba6b52f7..bc37c57c 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/KeyRange.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/KeyRange.java @@ -1,33 +1,25 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. +/* + * Copyright 2010 The Apache Software Foundation * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. + * http://www.apache.org/licenses/LICENSE-2.0 * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; - -import static com.salesforce.phoenix.query.QueryConstants.SEPARATOR_BYTE_ARRAY; + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import static org.apache.phoenix.query.QueryConstants.SEPARATOR_BYTE_ARRAY; import java.io.DataInput; import java.io.DataOutput; @@ -46,8 +38,8 @@ import com.google.common.base.Function; import com.google.common.collect.ComparisonChain; import com.google.common.collect.Lists; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.util.ByteUtil; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.util.ByteUtil; import edu.umd.cs.findbugs.annotations.NonNull; @@ -127,7 +119,7 @@ public static KeyRange getKeyRange(byte[] lowerRange, byte[] upperRange) { return getKeyRange(lowerRange, true, upperRange, false); } - // TODO: make non public and move to com.salesforce.phoenix.type soon + // TODO: make non public and move to org.apache.phoenix.type soon public static KeyRange getKeyRange(byte[] lowerRange, boolean lowerInclusive, byte[] upperRange, boolean upperInclusive) { if (lowerRange == null || upperRange == null) { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/MetaDataMutated.java b/phoenix-core/src/main/java/org/apache/phoenix/query/MetaDataMutated.java new file mode 100644 index 00000000..0ed1d564 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/MetaDataMutated.java @@ -0,0 +1,42 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import java.sql.SQLException; +import java.util.List; + +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PMetaData; +import org.apache.phoenix.schema.PTable; + + +/** + * + * Interface for applying schema mutations to our client-side schema cache + * + * @author jtaylor + * @since 0.1 + */ +public interface MetaDataMutated { + PMetaData addTable(PTable table) throws SQLException; + PMetaData removeTable(String tableName) throws SQLException; + PMetaData addColumn(String tableName, List columns, long tableTimeStamp, long tableSeqNum, boolean isImmutableRows) throws SQLException; + PMetaData removeColumn(String tableName, String familyName, String columnName, long tableTimeStamp, long tableSeqNum) throws SQLException; +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryConstants.java b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryConstants.java new file mode 100644 index 00000000..48bd7f7f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryConstants.java @@ -0,0 +1,215 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + + +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.ARRAY_SIZE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.BUFFER_LENGTH; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.CACHE_SIZE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.CHAR_OCTET_LENGTH; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_COUNT; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_DEF; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_MODIFIER; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_SIZE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.CURRENT_VALUE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TABLE_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TYPE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DECIMAL_DIGITS; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DEFAULT_COLUMN_FAMILY_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DISABLE_WAL; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.IMMUTABLE_ROWS; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.INCREMENT_BY; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.INDEX_STATE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.IS_AUTOINCREMENT; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.IS_NULLABLE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.LINK_TYPE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.MULTI_TENANT; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.NULLABLE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.NUM_PREC_RADIX; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.ORDINAL_POSITION; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.PK_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.REF_GENERATION_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.REMARKS_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SALT_BUCKETS; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SCOPE_CATALOG; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SCOPE_SCHEMA; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SCOPE_TABLE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SELF_REFERENCING_COL_NAME_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SEQUENCE_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SEQUENCE_SCHEMA; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SOURCE_DATA_TYPE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SQL_DATA_TYPE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SQL_DATETIME_SUB; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.START_WITH; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_CAT_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_NAME_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_SCHEM_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_SEQ_NUM; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_TYPE_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TENANT_ID; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_SCHEMA; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_SEQUENCE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_TABLE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_STATEMENT; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_TYPE; + +import java.math.BigDecimal; + +import org.apache.hadoop.hbase.HConstants; +import org.apache.hadoop.hbase.HTableDescriptor; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.phoenix.coprocessor.MetaDataProtocol; +import org.apache.phoenix.schema.MetaDataSplitPolicy; +import org.apache.phoenix.schema.PName; +import org.apache.phoenix.schema.PNormalizedName; + + +/** + * + * Constants used during querying + * + * @author jtaylor + * @since 0.1 + */ +public interface QueryConstants { + public static final String NAME_SEPARATOR = "."; + public final static byte[] NAME_SEPARATOR_BYTES = Bytes.toBytes(NAME_SEPARATOR); + public static final byte NAME_SEPARATOR_BYTE = NAME_SEPARATOR_BYTES[0]; + public static final String NULL_SCHEMA_NAME = ""; + public static final String NULL_DISPLAY_TEXT = ""; + public static final long UNSET_TIMESTAMP = -1; + + public enum JoinType {INNER, LEFT_OUTER} + public final static String PHOENIX_SCHEMA = "system"; + public final static String PHOENIX_METADATA = "table"; + + public final static PName SINGLE_COLUMN_NAME = new PNormalizedName("s"); + public final static PName SINGLE_COLUMN_FAMILY_NAME = new PNormalizedName("s"); + public final static byte[] SINGLE_COLUMN = SINGLE_COLUMN_NAME.getBytes(); + public final static byte[] SINGLE_COLUMN_FAMILY = SINGLE_COLUMN_FAMILY_NAME.getBytes(); + + public static final long AGG_TIMESTAMP = HConstants.LATEST_TIMESTAMP; + /** + * Key used for a single row aggregation where there is no group by + */ + public final static byte[] UNGROUPED_AGG_ROW_KEY = Bytes.toBytes("a"); + public final static PName AGG_COLUMN_NAME = SINGLE_COLUMN_NAME; + public final static PName AGG_COLUMN_FAMILY_NAME = SINGLE_COLUMN_FAMILY_NAME; + + public static final byte[] TRUE = new byte[] {1}; + + /** + * Separator used between variable length keys for a composite key. + * Variable length data types may not use this byte value. + */ + public static final byte SEPARATOR_BYTE = (byte) 0; + public static final byte[] SEPARATOR_BYTE_ARRAY = new byte[] {SEPARATOR_BYTE}; + + public static final String DEFAULT_COPROCESS_PATH = "phoenix.jar"; + public final static int MILLIS_IN_DAY = 1000 * 60 * 60 * 24; + + public static final String EMPTY_COLUMN_NAME = "_0"; + public static final byte[] EMPTY_COLUMN_BYTES = Bytes.toBytes(EMPTY_COLUMN_NAME); + public static final ImmutableBytesPtr EMPTY_COLUMN_BYTES_PTR = new ImmutableBytesPtr( + EMPTY_COLUMN_BYTES); + public static final String DEFAULT_COLUMN_FAMILY = EMPTY_COLUMN_NAME; + public static final byte[] DEFAULT_COLUMN_FAMILY_BYTES = EMPTY_COLUMN_BYTES; + public static final String ALL_FAMILY_PROPERTIES_KEY = ""; + public static final String SYSTEM_TABLE_PK_NAME = "pk"; + + public static final double MILLIS_TO_NANOS_CONVERTOR = Math.pow(10, 6); + public static final BigDecimal BD_MILLIS_NANOS_CONVERSION = BigDecimal.valueOf(MILLIS_TO_NANOS_CONVERTOR); + public static final BigDecimal BD_MILLIS_IN_DAY = BigDecimal.valueOf(QueryConstants.MILLIS_IN_DAY); + + + public static final String CREATE_TABLE_METADATA = + // Do not use IF NOT EXISTS as we sometimes catch the TableAlreadyExists exception + // and add columns to the SYSTEM.TABLE dynamically. + "CREATE TABLE " + TYPE_SCHEMA + ".\"" + TYPE_TABLE + "\"(\n" + + // PK columns + TENANT_ID + " VARCHAR NULL," + + TABLE_SCHEM_NAME + " VARCHAR NULL," + + TABLE_NAME_NAME + " VARCHAR NOT NULL," + + COLUMN_NAME + " VARCHAR NULL," + // null only for table row + TABLE_CAT_NAME + " VARCHAR NULL," + // using for CF - ensures uniqueness for columns + // Table metadata (will be null for column rows) + TABLE_TYPE_NAME + " CHAR(1)," + + REMARKS_NAME + " VARCHAR," + + DATA_TYPE + " INTEGER," + + PK_NAME + " VARCHAR," + + TYPE_NAME + " VARCHAR," + + SELF_REFERENCING_COL_NAME_NAME + " VARCHAR," + + REF_GENERATION_NAME + " VARCHAR," + + TABLE_SEQ_NUM + " BIGINT," + + COLUMN_COUNT + " INTEGER," + + // Column metadata (will be null for table row) + COLUMN_SIZE + " INTEGER," + + BUFFER_LENGTH + " INTEGER," + + DECIMAL_DIGITS + " INTEGER," + + NUM_PREC_RADIX + " INTEGER," + + NULLABLE + " INTEGER," + + COLUMN_DEF + " VARCHAR," + + SQL_DATA_TYPE + " INTEGER," + + SQL_DATETIME_SUB + " INTEGER," + + CHAR_OCTET_LENGTH + " INTEGER," + + ORDINAL_POSITION + " INTEGER," + + IS_NULLABLE + " VARCHAR," + + SCOPE_CATALOG + " VARCHAR," + + SCOPE_SCHEMA + " VARCHAR," + + SCOPE_TABLE + " VARCHAR," + + SOURCE_DATA_TYPE + " INTEGER," + // supposed to be SHORT + IS_AUTOINCREMENT + " VARCHAR," + + // Columns added in 1.2.1 + COLUMN_MODIFIER + " INTEGER," + + SALT_BUCKETS + " INTEGER," + + // Columns added in 2.0.0 + DATA_TABLE_NAME + " VARCHAR," + + INDEX_STATE + " CHAR(1),\n" + + IMMUTABLE_ROWS + " BOOLEAN,\n" + + // Columns added in 3.0.0 + VIEW_STATEMENT + " VARCHAR,\n" + + DEFAULT_COLUMN_FAMILY_NAME + " VARCHAR,\n" + + DISABLE_WAL + " BOOLEAN,\n" + + MULTI_TENANT + " BOOLEAN,\n" + + VIEW_TYPE + " UNSIGNED_TINYINT,\n" + + LINK_TYPE + " UNSIGNED_TINYINT,\n" + + ARRAY_SIZE + " INTEGER,\n" + + "CONSTRAINT " + SYSTEM_TABLE_PK_NAME + " PRIMARY KEY (" + TENANT_ID + "," + + TABLE_SCHEM_NAME + "," + TABLE_NAME_NAME + "," + COLUMN_NAME + "," + TABLE_CAT_NAME + "))\n" + + HConstants.VERSIONS + "=" + MetaDataProtocol.DEFAULT_MAX_META_DATA_VERSIONS + ",\n" + + DEFAULT_COLUMN_FAMILY_NAME + "=" + "'_0'" + ",\n" + // Use original default for b/w compat + HTableDescriptor.SPLIT_POLICY + "='" + MetaDataSplitPolicy.class.getName() + "'\n"; + + public static final String CREATE_SEQUENCE_METADATA = + "CREATE TABLE IF NOT EXISTS " + TYPE_SCHEMA + ".\"" + TYPE_SEQUENCE + "\"(\n" + + TENANT_ID + " VARCHAR NULL," + + SEQUENCE_SCHEMA + " VARCHAR NULL, \n" + + SEQUENCE_NAME + " VARCHAR NOT NULL, \n" + + START_WITH + " BIGINT NOT NULL, \n" + + CURRENT_VALUE + " BIGINT NOT NULL, \n" + + INCREMENT_BY + " BIGINT NOT NULL, \n" + + CACHE_SIZE + " INTEGER NOT NULL \n" + + " CONSTRAINT " + SYSTEM_TABLE_PK_NAME + " PRIMARY KEY (" + TENANT_ID + "," + SEQUENCE_SCHEMA + "," + SEQUENCE_NAME + "))\n" + + HConstants.VERSIONS + "=" + MetaDataProtocol.DEFAULT_MAX_META_DATA_VERSIONS + "\n"; + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java new file mode 100644 index 00000000..320bac08 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java @@ -0,0 +1,119 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import java.util.concurrent.ExecutorService; + +import org.apache.http.annotation.Immutable; + +import org.apache.phoenix.iterate.SpoolTooBigToDiskException; +import org.apache.phoenix.memory.MemoryManager; +import org.apache.phoenix.optimize.QueryOptimizer; +import org.apache.phoenix.util.ReadOnlyProps; +import org.apache.phoenix.util.SQLCloseable; + + + +/** + * + * Interface to group together services needed during querying. The + * parameters that may be set in {@link org.apache.hadoop.conf.Configuration} + * are documented here: https://github.com/forcedotcom/phoenix/wiki/Tuning + * + * @author jtaylor + * @since 0.1 + */ +@Immutable +public interface QueryServices extends SQLCloseable { + public static final String KEEP_ALIVE_MS_ATTRIB = "phoenix.query.keepAliveMs"; + public static final String THREAD_POOL_SIZE_ATTRIB = "phoenix.query.threadPoolSize"; + public static final String QUEUE_SIZE_ATTRIB = "phoenix.query.queueSize"; + public static final String THREAD_TIMEOUT_MS_ATTRIB = "phoenix.query.timeoutMs"; + public static final String SPOOL_THRESHOLD_BYTES_ATTRIB = "phoenix.query.spoolThresholdBytes"; + + /** + * max size to spool the the result into + * ${java.io.tmpdir}/ResultSpoolerXXX.bin if + * {@link QueryServices#SPOOL_THRESHOLD_BYTES_ATTRIB } is reached. + *

+ * default is unlimited(-1) + *

+ * if the threshold is reached, a {@link SpoolTooBigToDiskException } will be thrown + */ + public static final String MAX_SPOOL_TO_DISK_BYTES_ATTRIB = "phoenix.query.maxSpoolToDiskBytes"; + + public static final String MAX_MEMORY_PERC_ATTRIB = "phoenix.query.maxGlobalMemoryPercentage"; + public static final String MAX_MEMORY_WAIT_MS_ATTRIB = "phoenix.query.maxGlobalMemoryWaitMs"; + public static final String MAX_TENANT_MEMORY_PERC_ATTRIB = "phoenix.query.maxTenantMemoryPercentage"; + public static final String MAX_SERVER_CACHE_SIZE_ATTRIB = "phoenix.query.maxServerCacheBytes"; + public static final String TARGET_QUERY_CONCURRENCY_ATTRIB = "phoenix.query.targetConcurrency"; + public static final String MAX_QUERY_CONCURRENCY_ATTRIB = "phoenix.query.maxConcurrency"; + public static final String DATE_FORMAT_ATTRIB = "phoenix.query.dateFormat"; + public static final String NUMBER_FORMAT_ATTRIB = "phoenix.query.numberFormat"; + public static final String STATS_UPDATE_FREQ_MS_ATTRIB = "phoenix.query.statsUpdateFrequency"; + public static final String MAX_STATS_AGE_MS_ATTRIB = "phoenix.query.maxStatsAge"; + public static final String CALL_QUEUE_ROUND_ROBIN_ATTRIB = "ipc.server.callqueue.roundrobin"; + public static final String SCAN_CACHE_SIZE_ATTRIB = "hbase.client.scanner.caching"; + public static final String MAX_MUTATION_SIZE_ATTRIB = "phoenix.mutate.maxSize"; + public static final String MUTATE_BATCH_SIZE_ATTRIB = "phoenix.mutate.batchSize"; + public static final String MAX_SERVER_CACHE_TIME_TO_LIVE_MS = "phoenix.coprocessor.maxServerCacheTimeToLiveMs"; + public static final String MAX_INTRA_REGION_PARALLELIZATION_ATTRIB = "phoenix.query.maxIntraRegionParallelization"; + public static final String ROW_KEY_ORDER_SALTED_TABLE_ATTRIB = "phoenix.query.rowKeyOrderSaltedTable"; + public static final String USE_INDEXES_ATTRIB = "phoenix.query.useIndexes"; + public static final String IMMUTABLE_ROWS_ATTRIB = "phoenix.mutate.immutableRows"; + public static final String INDEX_MUTATE_BATCH_SIZE_THRESHOLD_ATTRIB = "phoenix.index.mutableBatchSizeThreshold"; + public static final String DROP_METADATA_ATTRIB = "phoenix.schema.dropMetaData"; + public static final String GROUPBY_SPILLABLE_ATTRIB = "phoenix.groupby.spillable"; + public static final String GROUPBY_SPILL_FILES_ATTRIB = "phoenix.groupby.spillFiles"; + public static final String GROUPBY_MAX_CACHE_SIZE_ATTRIB = "phoenix.groupby.maxCacheSize"; + + public static final String CALL_QUEUE_PRODUCER_ATTRIB_NAME = "CALL_QUEUE_PRODUCER"; + + public static final String MASTER_INFO_PORT_ATTRIB = "hbase.master.info.port"; + public static final String REGIONSERVER_INFO_PORT_ATTRIB = "hbase.regionserver.info.port"; + public static final String REGIONSERVER_LEASE_PERIOD_ATTRIB = "hbase.regionserver.lease.period"; + public static final String RPC_TIMEOUT_ATTRIB = "hbase.rpc.timeout"; + public static final String ZOOKEEPER_QUARUM_ATTRIB = "hbase.zookeeper.quorum"; + public static final String ZOOKEEPER_PORT_ATTRIB = "hbase.zookeeper.property.clientPort"; + public static final String ZOOKEEPER_ROOT_NODE_ATTRIB = "zookeeper.znode.parent"; + public static final String DISTINCT_VALUE_COMPRESS_THRESHOLD_ATTRIB = "phoenix.distinct.value.compress.threshold"; + public static final String SEQUENCE_CACHE_SIZE_ATTRIB = "phoenix.sequence.cacheSize"; + + + /** + * Get executor service used for parallel scans + */ + public ExecutorService getExecutor(); + /** + * Get the memory manager used to track memory usage + */ + public MemoryManager getMemoryManager(); + + /** + * Get the properties from the HBase configuration in a + * read-only structure that avoids any synchronization + */ + public ReadOnlyProps getProps(); + + /** + * Get query optimizer used to choose the best query plan + */ + public QueryOptimizer getOptimizer(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesImpl.java new file mode 100644 index 00000000..5ebde598 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesImpl.java @@ -0,0 +1,38 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + + + + + +/** + * + * Real implementation of QueryServices for use in runtime and perf testing + * + * @author jtaylor + * @since 0.1 + */ +public final class QueryServicesImpl extends BaseQueryServicesImpl { + + public QueryServicesImpl() { + super(QueryServicesOptions.withDefaults()); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryServicesOptions.java b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java similarity index 75% rename from phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryServicesOptions.java rename to phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java index 94041db9..7f3025c5 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/QueryServicesOptions.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java @@ -1,66 +1,58 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; -import static com.salesforce.phoenix.query.QueryServices.CALL_QUEUE_PRODUCER_ATTRIB_NAME; -import static com.salesforce.phoenix.query.QueryServices.CALL_QUEUE_ROUND_ROBIN_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.DATE_FORMAT_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.DROP_METADATA_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.GROUPBY_MAX_CACHE_SIZE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.GROUPBY_SPILL_FILES_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.GROUPBY_SPILLABLE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.IMMUTABLE_ROWS_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.INDEX_MUTATE_BATCH_SIZE_THRESHOLD_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.KEEP_ALIVE_MS_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MASTER_INFO_PORT_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MAX_INTRA_REGION_PARALLELIZATION_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MAX_MEMORY_PERC_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MAX_MEMORY_WAIT_MS_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MAX_MUTATION_SIZE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MAX_QUERY_CONCURRENCY_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MAX_SERVER_CACHE_SIZE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MAX_SERVER_CACHE_TIME_TO_LIVE_MS; -import static com.salesforce.phoenix.query.QueryServices.MAX_SPOOL_TO_DISK_BYTES_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MAX_TENANT_MEMORY_PERC_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.MUTATE_BATCH_SIZE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.QUEUE_SIZE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.REGIONSERVER_INFO_PORT_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.REGIONSERVER_LEASE_PERIOD_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.ROW_KEY_ORDER_SALTED_TABLE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.RPC_TIMEOUT_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.SCAN_CACHE_SIZE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.SEQUENCE_CACHE_SIZE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.STATS_UPDATE_FREQ_MS_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.TARGET_QUERY_CONCURRENCY_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.THREAD_POOL_SIZE_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.THREAD_TIMEOUT_MS_ATTRIB; -import static com.salesforce.phoenix.query.QueryServices.USE_INDEXES_ATTRIB; +import static org.apache.phoenix.query.QueryServices.CALL_QUEUE_PRODUCER_ATTRIB_NAME; +import static org.apache.phoenix.query.QueryServices.CALL_QUEUE_ROUND_ROBIN_ATTRIB; +import static org.apache.phoenix.query.QueryServices.DATE_FORMAT_ATTRIB; +import static org.apache.phoenix.query.QueryServices.DROP_METADATA_ATTRIB; +import static org.apache.phoenix.query.QueryServices.GROUPBY_MAX_CACHE_SIZE_ATTRIB; +import static org.apache.phoenix.query.QueryServices.GROUPBY_SPILL_FILES_ATTRIB; +import static org.apache.phoenix.query.QueryServices.GROUPBY_SPILLABLE_ATTRIB; +import static org.apache.phoenix.query.QueryServices.IMMUTABLE_ROWS_ATTRIB; +import static org.apache.phoenix.query.QueryServices.INDEX_MUTATE_BATCH_SIZE_THRESHOLD_ATTRIB; +import static org.apache.phoenix.query.QueryServices.KEEP_ALIVE_MS_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MASTER_INFO_PORT_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MAX_INTRA_REGION_PARALLELIZATION_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MAX_MEMORY_PERC_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MAX_MEMORY_WAIT_MS_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MAX_MUTATION_SIZE_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MAX_QUERY_CONCURRENCY_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MAX_SERVER_CACHE_SIZE_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MAX_SERVER_CACHE_TIME_TO_LIVE_MS; +import static org.apache.phoenix.query.QueryServices.MAX_SPOOL_TO_DISK_BYTES_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MAX_TENANT_MEMORY_PERC_ATTRIB; +import static org.apache.phoenix.query.QueryServices.MUTATE_BATCH_SIZE_ATTRIB; +import static org.apache.phoenix.query.QueryServices.QUEUE_SIZE_ATTRIB; +import static org.apache.phoenix.query.QueryServices.REGIONSERVER_INFO_PORT_ATTRIB; +import static org.apache.phoenix.query.QueryServices.REGIONSERVER_LEASE_PERIOD_ATTRIB; +import static org.apache.phoenix.query.QueryServices.ROW_KEY_ORDER_SALTED_TABLE_ATTRIB; +import static org.apache.phoenix.query.QueryServices.RPC_TIMEOUT_ATTRIB; +import static org.apache.phoenix.query.QueryServices.SCAN_CACHE_SIZE_ATTRIB; +import static org.apache.phoenix.query.QueryServices.SEQUENCE_CACHE_SIZE_ATTRIB; +import static org.apache.phoenix.query.QueryServices.SPOOL_THRESHOLD_BYTES_ATTRIB; +import static org.apache.phoenix.query.QueryServices.STATS_UPDATE_FREQ_MS_ATTRIB; +import static org.apache.phoenix.query.QueryServices.TARGET_QUERY_CONCURRENCY_ATTRIB; +import static org.apache.phoenix.query.QueryServices.THREAD_POOL_SIZE_ATTRIB; +import static org.apache.phoenix.query.QueryServices.THREAD_TIMEOUT_MS_ATTRIB; +import static org.apache.phoenix.query.QueryServices.USE_INDEXES_ATTRIB; import java.util.Map.Entry; @@ -68,8 +60,8 @@ import org.apache.hadoop.hbase.HConstants; import org.apache.hadoop.hbase.regionserver.wal.WALEditCodec; -import com.salesforce.phoenix.util.DateUtil; -import com.salesforce.phoenix.util.ReadOnlyProps; +import org.apache.phoenix.util.DateUtil; +import org.apache.phoenix.util.ReadOnlyProps; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/query/StatsManager.java b/phoenix-core/src/main/java/org/apache/phoenix/query/StatsManager.java new file mode 100644 index 00000000..7d02e021 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/StatsManager.java @@ -0,0 +1,59 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; + +import java.sql.SQLException; + +import org.apache.phoenix.schema.TableRef; + + +/** + * + * Interface for managing and caching table statistics. + * The frequency of updating the table statistics are controlled + * by {@link org.apache.phoenix.query.QueryServices#STATS_UPDATE_FREQ_MS_ATTRIB}. + * Table stats may also be manually updated through {@link #updateStats(TableRef)}. + * + * + * @author jtaylor + * @since 0.1 + */ +public interface StatsManager { + /** + * Get the minimum key for the given table + * @param table the table + * @return the minimum key or null if unknown + */ + byte[] getMinKey(TableRef table); + + /** + * Get the maximum key for the given table + * @param table the table + * @return the maximum key or null if unknown + */ + byte[] getMaxKey(TableRef table); + + /** + * Manually update the cached table statistics + * @param table the table + * @throws SQLException + */ + void updateStats(TableRef table) throws SQLException; +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/query/StatsManagerImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/query/StatsManagerImpl.java similarity index 79% rename from phoenix-core/src/main/java/com/salesforce/phoenix/query/StatsManagerImpl.java rename to phoenix-core/src/main/java/org/apache/phoenix/query/StatsManagerImpl.java index 374ec5eb..a36f275a 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/query/StatsManagerImpl.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/query/StatsManagerImpl.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.query; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.query; import java.io.IOException; import java.sql.SQLException; @@ -41,9 +33,9 @@ import org.apache.hadoop.hbase.client.Scan; import org.apache.hadoop.hbase.filter.KeyOnlyFilter; -import com.salesforce.phoenix.schema.TableRef; -import com.salesforce.phoenix.util.SchemaUtil; -import com.salesforce.phoenix.util.ServerUtil; +import org.apache.phoenix.schema.TableRef; +import org.apache.phoenix.util.SchemaUtil; +import org.apache.phoenix.util.ServerUtil; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/AmbiguousColumnException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/AmbiguousColumnException.java new file mode 100644 index 00000000..f460d639 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/AmbiguousColumnException.java @@ -0,0 +1,52 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + +/** + * + * Exception thrown when a column name is used without being qualified with an alias + * and more than one table contains that column. + * + * @author jtaylor + * @since 0.1 + */ +public class AmbiguousColumnException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.AMBIGUOUS_COLUMN; + + public AmbiguousColumnException() { + super(new SQLExceptionInfo.Builder(code).build().toString(), code.getSQLState(), code.getErrorCode()); + } + + public AmbiguousColumnException(String columnName) { + super(new SQLExceptionInfo.Builder(code).setColumnName(columnName).build().toString(), + code.getSQLState(), code.getErrorCode()); + } + + public AmbiguousColumnException(String columnName, Throwable cause) { + super(new SQLExceptionInfo.Builder(code).setColumnName(columnName).build().toString(), + code.getSQLState(), code.getErrorCode(), cause); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/AmbiguousTableException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/AmbiguousTableException.java new file mode 100644 index 00000000..56815f96 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/AmbiguousTableException.java @@ -0,0 +1,52 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + +/** + * + * Exception thrown when a table name is used without being qualified with an alias + * and more than one schema contains that table. + * + * @author jtaylor + * @since 0.1 + */ +public class AmbiguousTableException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.AMBIGUOUS_TABLE; + + public AmbiguousTableException() { + super(new SQLExceptionInfo.Builder(code).build().toString(), code.getSQLState(), code.getErrorCode(), null); + } + + public AmbiguousTableException(String tableName) { + super(new SQLExceptionInfo.Builder(code).setTableName(tableName).toString(), + code.getSQLState(), code.getErrorCode(), null); + } + + public AmbiguousTableException(String tableName, Throwable cause) { + super(new SQLExceptionInfo.Builder(code).setTableName(tableName).toString(), + code.getSQLState(), code.getErrorCode(), cause); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ArgumentTypeMismatchException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ArgumentTypeMismatchException.java new file mode 100644 index 00000000..51cea073 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ArgumentTypeMismatchException.java @@ -0,0 +1,44 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + +/** + * Exception thrown when we try to use use an argument that has the wrong type. + * + * @author zhuang + * @since 1.0 + */ +public class ArgumentTypeMismatchException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.TYPE_MISMATCH; + + public ArgumentTypeMismatchException(PDataType expected, PDataType actual, String location) { + super(new SQLExceptionInfo.Builder(code).setMessage("expected: " + expected + " but was: " + actual + " at " + location).build().toString(), code.getSQLState(), code.getErrorCode()); + } + + public ArgumentTypeMismatchException(String expected, String actual, String location) { + super(new SQLExceptionInfo.Builder(code).setMessage("expected: " + expected + " but was: " + actual + " at " + location).build().toString(), code.getSQLState(), code.getErrorCode()); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnAlreadyExistsException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnAlreadyExistsException.java new file mode 100644 index 00000000..6c270c17 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnAlreadyExistsException.java @@ -0,0 +1,62 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + + +/** + * + * Exception thrown when a column already exists. + * + * @author jtaylor + * @since 0.1 + */ +public class ColumnAlreadyExistsException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.COLUMN_EXIST_IN_DEF; + private final String schemaName; + private final String tableName; + private final String columnName; + + public ColumnAlreadyExistsException(String schemaName, String tableName, String columnName) { + super(new SQLExceptionInfo.Builder(code).setColumnName(columnName) + .setSchemaName(schemaName).setTableName(tableName).build().toString(), + code.getSQLState(), code.getErrorCode(), null); + this.schemaName = schemaName; + this.tableName = tableName; + this.columnName = columnName; + } + + public String getTableName() { + return tableName; + } + + public String getSchemaName() { + return schemaName; + } + + public String getColumnName() { + return columnName; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnFamilyNotFoundException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnFamilyNotFoundException.java new file mode 100644 index 00000000..a450b71b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnFamilyNotFoundException.java @@ -0,0 +1,46 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + +/** + * + * Exception thrown when a family name could not be found in the schema + * + * @author jtaylor + * @since 0.1 + */ +public class ColumnFamilyNotFoundException extends MetaDataEntityNotFoundException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.COLUMN_FAMILY_NOT_FOUND; + private final String familyName; + + public ColumnFamilyNotFoundException(String familyName) { + super(new SQLExceptionInfo.Builder(code).setFamilyName(familyName).build().toString(), + code.getSQLState(), code.getErrorCode(), null); + this.familyName = familyName; + } + + public String getFamilyName() { + return familyName; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnModifier.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnModifier.java new file mode 100644 index 00000000..7b4bc3da --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnModifier.java @@ -0,0 +1,149 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import org.apache.hadoop.hbase.filter.CompareFilter.CompareOp; + +import com.google.common.base.Preconditions; + +/** + * A ColumnModifier implementation modifies how bytes are stored in a primary key column.

+ * The {@link ColumnModifier#apply apply} method is called when the bytes for a specific column are first written to HBase and again + * when they are read back. Phoenix attemps to minimize calls to apply when bytes are read out of HBase. + * + * @author simontoens + * @since 1.2 + */ +public enum ColumnModifier { + /** + * Invert the bytes in the src byte array to support descending ordering of row keys. + */ + SORT_DESC(1) { + @Override + public byte[] apply(byte[] src, int srcOffset, byte[] dest, int dstOffset, int length) { + Preconditions.checkNotNull(src); + Preconditions.checkNotNull(dest); + for (int i = 0; i < length; i++) { + dest[dstOffset+i] = (byte)(src[srcOffset+i] ^ 0xFF); + } + return dest; + } + + @Override + public byte apply(byte b) { + return (byte)(b ^ 0xFF); + } + + @Override + public CompareOp transform(CompareOp op) { + switch (op) { + case EQUAL: + return op; + case GREATER: + return CompareOp.LESS; + case GREATER_OR_EQUAL: + return CompareOp.LESS_OR_EQUAL; + case LESS: + return CompareOp.GREATER; + case LESS_OR_EQUAL: + return CompareOp.GREATER_OR_EQUAL; + default: + throw new IllegalArgumentException("Unknown operator " + op); + } + } + + @Override + public byte[] apply(byte[] src, int srcOffset, int length) { + return apply(src, srcOffset, new byte[length], 0, length); + } + }; + + private final int serializationId; + + ColumnModifier(int serializationId) { + this.serializationId = serializationId; + } + + public int getSerializationId() { + return serializationId; + } + /** + * Returns the ColumnModifier for the specified DDL stmt keyword. + */ + public static ColumnModifier fromDDLValue(String modifier) { + if (modifier == null) { + return null; + } else if (modifier.equalsIgnoreCase("ASC")) { + return null; + } else if (modifier.equalsIgnoreCase("DESC")) { + return SORT_DESC; + } else { + return null; + } + } + + /** + * Returns the ColumnModifier for the specified internal value. + */ + public static ColumnModifier fromSystemValue(int value) { + for (ColumnModifier mod : ColumnModifier.values()) { + if (mod.getSerializationId() == value) { + return mod; + } + } + return null; + } + + /** + * Returns an internal value representing the specified ColumnModifier. + */ + public static int toSystemValue(ColumnModifier columnModifier) { + if (columnModifier == null) { + return 0; + } + return columnModifier.getSerializationId(); + } + + /** + * Copies the bytes from source array to destination array and applies the column modifier operation on the bytes + * starting at the specified offsets. The column modifier is applied to the number of bytes matching the + * specified length. + * @param src the source byte array to copy from, cannot be null + * @param srcOffset the offset into the source byte array at which to begin. + * @param dest the destination byte array into which to transfer the modified bytes. + * @param dstOffset the offset into the destination byte array at which to begin + * @param length the number of bytes for which to apply the modification + * @return the destination byte array + */ + public abstract byte[] apply(byte[] src, int srcOffset, byte[] dest, int dstOffset, int length); + /** + * Copies the bytes from source array to a newly allocated destination array and applies the column + * modifier operation on the bytes starting at the specified offsets. The column modifier is applied + * to the number of bytes matching the specified length. + * @param src the source byte array to copy from, cannot be null + * @param srcOffset the offset into the source byte array at which to begin. + * @param length the number of bytes for which to apply the modification + * @return the newly allocated destination byte array + */ + public abstract byte[] apply(byte[] src, int srcOffset, int length); + public abstract byte apply(byte b); + + public abstract CompareOp transform(CompareOp op); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnNotFoundException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnNotFoundException.java new file mode 100644 index 00000000..c3b331b6 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnNotFoundException.java @@ -0,0 +1,65 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + + +/** + * + * Exception thrown when a column name referenced in a select + * statement cannot be found in any table. + * + * @author jtaylor + * @since 0.1 + */ +public class ColumnNotFoundException extends MetaDataEntityNotFoundException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.COLUMN_NOT_FOUND; + private final String schemaName; + private final String tableName; + private final String columnName; + + public ColumnNotFoundException(String columnName) { + this(null, null, null, columnName); + } + + public ColumnNotFoundException(String schemaName, String tableName, String familyName, String columnName) { + super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName) + .setFamilyName(familyName).setColumnName(columnName).build().toString(), + code.getSQLState(), code.getErrorCode(), null); + this.schemaName = schemaName; + this.tableName = tableName; + this.columnName = columnName; + } + + public String getTableName() { + return tableName; + } + + public String getSchemaName() { + return schemaName; + } + + public String getColumnName() { + return columnName; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnRef.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnRef.java new file mode 100644 index 00000000..ed033b77 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ColumnRef.java @@ -0,0 +1,135 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import org.apache.http.annotation.Immutable; + +import org.apache.phoenix.expression.ColumnExpression; +import org.apache.phoenix.expression.IndexKeyValueColumnExpression; +import org.apache.phoenix.expression.KeyValueColumnExpression; +import org.apache.phoenix.expression.ProjectedColumnExpression; +import org.apache.phoenix.expression.RowKeyColumnExpression; +import org.apache.phoenix.util.IndexUtil; +import org.apache.phoenix.util.SchemaUtil; + + +/** + * + * Class that represents a reference to a PColumn in a PTable + * + * @author jtaylor + * @since 0.1 + */ +@Immutable +public final class ColumnRef { + private final TableRef tableRef; + private final int columnPosition; + private final int pkSlotPosition; + + public ColumnRef(ColumnRef columnRef, long timeStamp) { + this.tableRef = new TableRef(columnRef.tableRef, timeStamp); + this.columnPosition = columnRef.columnPosition; + this.pkSlotPosition = columnRef.pkSlotPosition; + } + + public ColumnRef(TableRef tableRef, int columnPosition) { + if (tableRef == null) { + throw new NullPointerException(); + } + if (columnPosition < 0 || columnPosition >= tableRef.getTable().getColumns().size()) { + throw new IllegalArgumentException("Column position of " + columnPosition + " must be between 0 and " + tableRef.getTable().getColumns().size() + " for table " + tableRef.getTable().getName().getString()); + } + this.tableRef = tableRef; + this.columnPosition = columnPosition; + PColumn column = getColumn(); + int i = -1; + if (SchemaUtil.isPKColumn(column)) { + for (PColumn pkColumn : tableRef.getTable().getPKColumns()) { + i++; + if (pkColumn == column) { + break; + } + } + } + pkSlotPosition = i; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + columnPosition; + result = prime * result + tableRef.hashCode(); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + ColumnRef other = (ColumnRef)obj; + if (columnPosition != other.columnPosition) return false; + if (!tableRef.equals(other.tableRef)) return false; + return true; + } + + public ColumnExpression newColumnExpression() { + boolean isIndex = tableRef.getTable().getType() == PTableType.INDEX; + if (SchemaUtil.isPKColumn(this.getColumn())) { + String name = this.getColumn().getName().getString(); + if (isIndex) { + name = IndexUtil.getDataColumnName(name); + } + return new RowKeyColumnExpression( + getColumn(), + new RowKeyValueAccessor(this.getTable().getPKColumns(), pkSlotPosition), + name); + } + + if (isIndex) + return new IndexKeyValueColumnExpression(getColumn()); + + if (tableRef.getTable().getType() == PTableType.JOIN) + return new ProjectedColumnExpression(getColumn(), tableRef.getTable()); + + return new KeyValueColumnExpression(getColumn()); + } + + public int getColumnPosition() { + return columnPosition; + } + + public int getPKSlotPosition() { + return pkSlotPosition; + } + + public PColumn getColumn() { + return tableRef.getTable().getColumns().get(columnPosition); + } + + public PTable getTable() { + return tableRef.getTable(); + } + + public TableRef getTableRef() { + return tableRef; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ConcurrentTableMutationException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ConcurrentTableMutationException.java new file mode 100644 index 00000000..c3b540b3 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ConcurrentTableMutationException.java @@ -0,0 +1,48 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + + +public class ConcurrentTableMutationException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.CONCURRENT_TABLE_MUTATION; + private final String schemaName; + private final String tableName; + + public ConcurrentTableMutationException(String schemaName, String tableName) { + super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).build().toString(), + code.getSQLState(), code.getErrorCode()); + this.schemaName = schemaName; + this.tableName = tableName; + } + + public String getTableName() { + return tableName; + } + + public String getSchemaName() { + return schemaName; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ConstraintViolationException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ConstraintViolationException.java new file mode 100644 index 00000000..5a8b2607 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ConstraintViolationException.java @@ -0,0 +1,48 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +/** + * + * Exception thrown when a schema constraint is violated at the + * time of data insertion. + * + * @author jtaylor + * @since 180 + */ +public class ConstraintViolationException extends RuntimeException { + private static final long serialVersionUID = 1L; + + public ConstraintViolationException() { + } + + public ConstraintViolationException(String message) { + super(message); + } + + public ConstraintViolationException(Throwable cause) { + super(cause); + } + + public ConstraintViolationException(String message, Throwable cause) { + super(message, cause); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/DelegateColumn.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/DelegateColumn.java new file mode 100644 index 00000000..af6b77ed --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/DelegateColumn.java @@ -0,0 +1,71 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; + +public class DelegateColumn extends DelegateDatum implements PColumn { + + public DelegateColumn(PColumn delegate) { + super(delegate); + } + + @Override + protected PColumn getDelegate() { + return (PColumn)super.getDelegate(); + } + + @Override + public PName getName() { + return getDelegate().getName(); + } + + @Override + public ColumnModifier getColumnModifier() { + return getDelegate().getColumnModifier(); + } + + @Override + public PName getFamilyName() { + return getDelegate().getFamilyName(); + } + + @Override + public void readFields(DataInput input) throws IOException { + getDelegate().readFields(input); + } + + @Override + public void write(DataOutput output) throws IOException { + getDelegate().write(output); + } + + @Override + public int getPosition() { + return getDelegate().getPosition(); + } + + @Override + public Integer getArraySize() { + return getDelegate().getArraySize(); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/DelegateDatum.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/DelegateDatum.java new file mode 100644 index 00000000..86e4cc9d --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/DelegateDatum.java @@ -0,0 +1,62 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +public class DelegateDatum implements PDatum { + private final PDatum delegate; + + public DelegateDatum(PDatum delegate) { + this.delegate = delegate; + } + + @Override + public boolean isNullable() { + return delegate.isNullable(); + } + + @Override + public PDataType getDataType() { + return delegate.getDataType(); + } + + @Override + public Integer getByteSize() { + return delegate.getByteSize(); + } + + @Override + public Integer getMaxLength() { + return delegate.getByteSize(); + } + + @Override + public Integer getScale() { + return delegate.getScale(); + } + + @Override + public ColumnModifier getColumnModifier() { + return delegate.getColumnModifier(); + } + + protected PDatum getDelegate() { + return delegate; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/EmptySequenceCacheException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/EmptySequenceCacheException.java similarity index 79% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/EmptySequenceCacheException.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/EmptySequenceCacheException.java index 6fb95a2b..1964983a 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/EmptySequenceCacheException.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/EmptySequenceCacheException.java @@ -1,9 +1,9 @@ -package com.salesforce.phoenix.schema; +package org.apache.phoenix.schema; import java.sql.SQLException; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; public class EmptySequenceCacheException extends SQLException { private static final long serialVersionUID = 1L; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ExecuteQueryNotApplicableException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ExecuteQueryNotApplicableException.java new file mode 100644 index 00000000..8812bf85 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ExecuteQueryNotApplicableException.java @@ -0,0 +1,39 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + +public class ExecuteQueryNotApplicableException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.EXECUTE_QUERY_NOT_APPLICABLE; + + public ExecuteQueryNotApplicableException(String query) { + super(new SQLExceptionInfo.Builder(code).setMessage("Query: " + query).build().toString(), code.getSQLState(), code.getErrorCode()); + } + + public ExecuteQueryNotApplicableException(String command, String statement) { + super(new SQLExceptionInfo.Builder(code).setMessage("Command: " + command + ". Statement: " + statement).build().toString(), code.getSQLState(), code.getErrorCode()); + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ExecuteUpdateNotApplicableException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ExecuteUpdateNotApplicableException.java new file mode 100644 index 00000000..b99d87bf --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ExecuteUpdateNotApplicableException.java @@ -0,0 +1,38 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + +public class ExecuteUpdateNotApplicableException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.EXECUTE_QUERY_NOT_APPLICABLE; + + public ExecuteUpdateNotApplicableException(String query) { + super(new SQLExceptionInfo.Builder(code).setMessage("Query: " + query).build().toString(), code.getSQLState(), code.getErrorCode()); + } + + public ExecuteUpdateNotApplicableException(String command, String statement) { + super(new SQLExceptionInfo.Builder(code).setMessage("Command: " + command + ". Statement: " + statement).build().toString(), code.getSQLState(), code.getErrorCode()); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/IllegalDataException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/IllegalDataException.java new file mode 100644 index 00000000..83ae565a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/IllegalDataException.java @@ -0,0 +1,47 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +/** + * + * Exception thrown when an invalid or illegal data value is found + * + * @author jtaylor + * @since 0.1 + */ +public class IllegalDataException extends ConstraintViolationException { + private static final long serialVersionUID = 1L; + + public IllegalDataException() { + } + + public IllegalDataException(String message) { + super(message); + } + + public IllegalDataException(Throwable cause) { + super(cause); + } + + public IllegalDataException(String message, Throwable cause) { + super(message, cause); + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/KeyValueSchema.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/KeyValueSchema.java similarity index 79% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/KeyValueSchema.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/KeyValueSchema.java index 323f3007..bca09a95 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/KeyValueSchema.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/KeyValueSchema.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; import java.util.List; @@ -33,9 +25,9 @@ import org.apache.hadoop.io.WritableUtils; import org.apache.http.annotation.Immutable; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.schema.tuple.Tuple; -import com.salesforce.phoenix.util.ByteUtil; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.schema.tuple.Tuple; +import org.apache.phoenix.util.ByteUtil; /** diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/MetaDataClient.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java similarity index 90% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/MetaDataClient.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java index 39e940b0..f107350b 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/MetaDataClient.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java @@ -1,67 +1,59 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; import static com.google.common.collect.Lists.newArrayListWithExpectedSize; import static com.google.common.collect.Sets.newLinkedHashSet; import static com.google.common.collect.Sets.newLinkedHashSetWithExpectedSize; -import static com.salesforce.phoenix.exception.SQLExceptionCode.INSUFFICIENT_MULTI_TENANT_COLUMNS; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.BASE_SCHEMA_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.BASE_TABLE_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_COUNT; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_MODIFIER; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_SIZE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TABLE_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TYPE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DECIMAL_DIGITS; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DEFAULT_COLUMN_FAMILY_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.DISABLE_WAL; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.IMMUTABLE_ROWS; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.INDEX_STATE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.MULTI_TENANT; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.NULLABLE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.ORDINAL_POSITION; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.PK_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SALT_BUCKETS; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_CAT_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_NAME_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_SCHEM_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_SEQ_NUM; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_TYPE_NAME; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TENANT_ID; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_SCHEMA; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_TABLE; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_EXPRESSION; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_TYPE; -import static com.salesforce.phoenix.query.QueryServices.DROP_METADATA_ATTRIB; -import static com.salesforce.phoenix.query.QueryServicesOptions.DEFAULT_DROP_METADATA; -import static com.salesforce.phoenix.schema.PDataType.VARCHAR; +import static org.apache.phoenix.exception.SQLExceptionCode.INSUFFICIENT_MULTI_TENANT_COLUMNS; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.ARRAY_SIZE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_COUNT; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_MODIFIER; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.COLUMN_SIZE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TABLE_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DATA_TYPE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DECIMAL_DIGITS; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DEFAULT_COLUMN_FAMILY_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.DISABLE_WAL; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.IMMUTABLE_ROWS; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.INDEX_STATE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.LINK_TYPE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.MULTI_TENANT; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.NULLABLE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.ORDINAL_POSITION; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.PK_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SALT_BUCKETS; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_CAT_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_NAME_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_SCHEM_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_SEQ_NUM; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TABLE_TYPE_NAME; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TENANT_ID; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_SCHEMA; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_TABLE; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_STATEMENT; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.VIEW_TYPE; +import static org.apache.phoenix.query.QueryServices.DROP_METADATA_ATTRIB; +import static org.apache.phoenix.query.QueryServicesOptions.DEFAULT_DROP_METADATA; +import static org.apache.phoenix.schema.PDataType.VARCHAR; import java.sql.DriverManager; import java.sql.PreparedStatement; @@ -86,6 +78,44 @@ import org.apache.hadoop.hbase.client.Put; import org.apache.hadoop.hbase.util.Bytes; import org.apache.hadoop.hbase.util.Pair; +import org.apache.phoenix.compile.ColumnResolver; +import org.apache.phoenix.compile.FromCompiler; +import org.apache.phoenix.compile.MutationPlan; +import org.apache.phoenix.compile.PostDDLCompiler; +import org.apache.phoenix.compile.PostIndexDDLCompiler; +import org.apache.phoenix.coprocessor.MetaDataProtocol; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; +import org.apache.phoenix.coprocessor.MetaDataProtocol.MutationCode; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.execute.MutationState; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.parse.AddColumnStatement; +import org.apache.phoenix.parse.AlterIndexStatement; +import org.apache.phoenix.parse.ColumnDef; +import org.apache.phoenix.parse.ColumnName; +import org.apache.phoenix.parse.CreateIndexStatement; +import org.apache.phoenix.parse.CreateSequenceStatement; +import org.apache.phoenix.parse.CreateTableStatement; +import org.apache.phoenix.parse.DropColumnStatement; +import org.apache.phoenix.parse.DropIndexStatement; +import org.apache.phoenix.parse.DropSequenceStatement; +import org.apache.phoenix.parse.DropTableStatement; +import org.apache.phoenix.parse.NamedTableNode; +import org.apache.phoenix.parse.ParseNodeFactory; +import org.apache.phoenix.parse.PrimaryKeyConstraint; +import org.apache.phoenix.parse.TableName; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.PTable.LinkType; +import org.apache.phoenix.schema.PTable.ViewType; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.IndexUtil; +import org.apache.phoenix.util.MetaDataUtil; +import org.apache.phoenix.util.PhoenixRuntime; +import org.apache.phoenix.util.SchemaUtil; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -94,45 +124,6 @@ import com.google.common.collect.Lists; import com.google.common.collect.Maps; import com.google.common.primitives.Ints; -import com.salesforce.phoenix.compile.ColumnResolver; -import com.salesforce.phoenix.compile.FromCompiler; -import com.salesforce.phoenix.compile.MutationPlan; -import com.salesforce.phoenix.compile.PostDDLCompiler; -import com.salesforce.phoenix.compile.PostIndexDDLCompiler; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MetaDataMutationResult; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol.MutationCode; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.execute.MutationState; -import com.salesforce.phoenix.expression.AndExpression; -import com.salesforce.phoenix.expression.Expression; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.parse.AddColumnStatement; -import com.salesforce.phoenix.parse.AlterIndexStatement; -import com.salesforce.phoenix.parse.ColumnDef; -import com.salesforce.phoenix.parse.ColumnName; -import com.salesforce.phoenix.parse.CreateIndexStatement; -import com.salesforce.phoenix.parse.CreateSequenceStatement; -import com.salesforce.phoenix.parse.CreateTableStatement; -import com.salesforce.phoenix.parse.DropColumnStatement; -import com.salesforce.phoenix.parse.DropIndexStatement; -import com.salesforce.phoenix.parse.DropSequenceStatement; -import com.salesforce.phoenix.parse.DropTableStatement; -import com.salesforce.phoenix.parse.NamedTableNode; -import com.salesforce.phoenix.parse.ParseNodeFactory; -import com.salesforce.phoenix.parse.PrimaryKeyConstraint; -import com.salesforce.phoenix.parse.TableName; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.query.QueryServices; -import com.salesforce.phoenix.query.QueryServicesOptions; -import com.salesforce.phoenix.schema.PTable.ViewType; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.IndexUtil; -import com.salesforce.phoenix.util.MetaDataUtil; -import com.salesforce.phoenix.util.PhoenixRuntime; -import com.salesforce.phoenix.util.SchemaUtil; public class MetaDataClient { private static final Logger logger = LoggerFactory.getLogger(MetaDataClient.class); @@ -152,20 +143,19 @@ public class MetaDataClient { INDEX_STATE + "," + IMMUTABLE_ROWS + "," + DEFAULT_COLUMN_FAMILY_NAME + "," + - VIEW_EXPRESSION + "," + + VIEW_STATEMENT + "," + DISABLE_WAL + "," + MULTI_TENANT + "," + - VIEW_TYPE + "," + - BASE_SCHEMA_NAME + "," + - BASE_TABLE_NAME + - ") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"; - private static final String CREATE_INDEX_LINK = + VIEW_TYPE + + ") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"; + private static final String CREATE_LINK = "UPSERT INTO " + TYPE_SCHEMA + ".\"" + TYPE_TABLE + "\"( " + TENANT_ID + "," + TABLE_SCHEM_NAME + "," + TABLE_NAME_NAME + "," + - TABLE_CAT_NAME + - ") VALUES (?, ?, ?, ?)"; + TABLE_CAT_NAME + "," + + LINK_TYPE + + ") VALUES (?, ?, ?, ?, ?)"; private static final String INCREMENT_SEQ_NUM = "UPSERT INTO " + TYPE_SCHEMA + ".\"" + TYPE_TABLE + "\"( " + TENANT_ID + "," + @@ -205,8 +195,9 @@ public class MetaDataClient { DECIMAL_DIGITS + "," + ORDINAL_POSITION + "," + COLUMN_MODIFIER + "," + - DATA_TABLE_NAME + // write this both in the column and table rows for access by metadata APIs - ") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"; + DATA_TABLE_NAME + "," + // write this both in the column and table rows for access by metadata APIs + ARRAY_SIZE + + ") VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"; private static final String UPDATE_COLUMN_POSITION = "UPSERT INTO " + TYPE_SCHEMA + ".\"" + TYPE_TABLE + "\" ( " + TENANT_ID + "," + @@ -340,6 +331,11 @@ private void addColumnMutation(String schemaName, String tableName, PColumn colu colUpsert.setInt(10, column.getPosition() + 1); colUpsert.setInt(11, ColumnModifier.toSystemValue(column.getColumnModifier())); colUpsert.setString(12, parentTableName); + if (column.getArraySize() == null) { + colUpsert.setNull(13, Types.INTEGER); + } else { + colUpsert.setInt(13, column.getArraySize()); + } colUpsert.execute(); } @@ -377,15 +373,15 @@ private PColumn newColumn(int position, ColumnDef def, PrimaryKeyConstraint pkCo } PColumn column = new PColumnImpl(PNameFactory.newName(columnName), familyName, def.getDataType(), - def.getMaxLength(), def.getScale(), def.isNull(), position, columnModifier); + def.getMaxLength(), def.getScale(), def.isNull(), position, columnModifier, def.getArraySize()); return column; } catch (IllegalArgumentException e) { // Based on precondition check in constructor throw new SQLException(e); } } - public MutationState createTable(CreateTableStatement statement, byte[][] splits, PTable parent, Expression viewExpression, ViewType viewType) throws SQLException { - PTable table = createTableInternal(statement, splits, parent, viewExpression, viewType); + public MutationState createTable(CreateTableStatement statement, byte[][] splits, PTable parent, String viewStatement, ViewType viewType) throws SQLException { + PTable table = createTableInternal(statement, splits, parent, viewStatement, viewType); if (table == null || table.getType() == PTableType.VIEW) { return new MutationState(0,connection); } @@ -618,7 +614,7 @@ private static ColumnDef findColumnDefOrNull(List colDefs, ColumnName return null; } - private PTable createTableInternal(CreateTableStatement statement, byte[][] splits, final PTable parent, Expression viewExpression, ViewType viewType) throws SQLException { + private PTable createTableInternal(CreateTableStatement statement, byte[][] splits, final PTable parent, String viewStatement, ViewType viewType) throws SQLException { final PTableType tableType = statement.getTableType(); boolean wasAutoCommit = connection.getAutoCommit(); connection.rollback(); @@ -630,7 +626,6 @@ private PTable createTableInternal(CreateTableStatement statement, byte[][] spli String schemaName = tableNameNode.getSchemaName(); String tableName = tableNameNode.getTableName(); String parentTableName = null; - String viewExpressionStr = viewExpression == null ? null : viewExpression.toString(); String tenantId = connection.getTenantId() == null ? null : connection.getTenantId().getString(); boolean isParentImmutableRows = false; if (parent != null && tableType == PTableType.INDEX) { @@ -649,11 +644,12 @@ private PTable createTableInternal(CreateTableStatement statement, byte[][] spli connection.rollback(); // Add row linking from data table row to index table row - PreparedStatement linkStatement = connection.prepareStatement(CREATE_INDEX_LINK); + PreparedStatement linkStatement = connection.prepareStatement(CREATE_LINK); linkStatement.setString(1, connection.getTenantId() == null ? null : connection.getTenantId().getString()); linkStatement.setString(2, schemaName); linkStatement.setString(3, parentTableName); linkStatement.setString(4, tableName); + linkStatement.setByte(5, LinkType.INDEX_TABLE.getSerializedValue()); linkStatement.execute(); } @@ -690,7 +686,7 @@ private PTable createTableInternal(CreateTableStatement statement, byte[][] spli boolean isImmutableRows = false; boolean multiTenant = false; // Although unusual, it's possible to set a mapped VIEW as having immutable rows. - // This tells Phoenix that you've managing the index maintenance yourself. + // This tells Phoenix that you're managing the index maintenance yourself. if (tableType != PTableType.VIEW || viewType == ViewType.MAPPED) { Boolean isImmutableRowsProp = (Boolean) tableProps.remove(PTable.IS_IMMUTABLE_ROWS_PROP_NAME); if (isImmutableRowsProp == null) { @@ -741,36 +737,40 @@ private PTable createTableInternal(CreateTableStatement statement, byte[][] spli List columns; LinkedHashSet pkColumns; - TableName baseName = statement.getBaseTableName(); - String baseTableName = null; - String baseSchemaName = null; - if (baseName != null) { - baseTableName = baseName.getTableName(); - baseSchemaName = baseName.getSchemaName(); - } - - if (tenantId != null && baseTableName == null) { - throw new SQLExceptionInfo.Builder(SQLExceptionCode.CANNOT_CREATE_TENANT_SPECIFIC_BASE_TABLE) + if (tenantId != null && tableType != PTableType.VIEW) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.CANNOT_CREATE_TENANT_SPECIFIC_TABLE) .setSchemaName(schemaName).setTableName(tableName).build().buildException(); } - if (tableType == PTableType.VIEW && viewType != ViewType.MAPPED) { - if (parent.getType() == PTableType.VIEW) { - viewExpressionStr = AndExpression.combine(viewExpressionStr, parent.getViewExpression()); - viewType = ViewType.combine(viewType, parent.getViewType()); - baseTableName = parent.getBaseTableName().getString(); - baseSchemaName = parent.getBaseSchemaName() == null ? null : parent.getBaseSchemaName().getString(); + List physicalNames = Collections.emptyList(); + if (tableType == PTableType.VIEW) { + physicalNames = Collections.singletonList(PNameFactory.newName(parent.getPhysicalName().getString())); + if (viewType == ViewType.MAPPED) { + columns = newArrayListWithExpectedSize(colDefs.size()); + pkColumns = newLinkedHashSetWithExpectedSize(colDefs.size()); + } else { + // Propagate property values to VIEW. + // TODO: formalize the known set of these properties + multiTenant = parent.isMultiTenant(); + saltBucketNum = parent.getBucketNum(); + isImmutableRows = parent.isImmutableRows(); + disableWAL = (disableWALProp == null ? parent.isWALDisabled() : disableWALProp); + defaultFamilyName = parent.getDefaultFamilyName() == null ? null : parent.getDefaultFamilyName().getString(); + columns = newArrayListWithExpectedSize(parent.getColumns().size() + colDefs.size()); + columns.addAll(parent.getColumns()); + pkColumns = newLinkedHashSet(parent.getPKColumns()); + + // Add row linking from data table row to physical table row + PreparedStatement linkStatement = connection.prepareStatement(CREATE_LINK); + for (PName physicalName : physicalNames) { + linkStatement.setString(1, connection.getTenantId() == null ? null : connection.getTenantId().getString()); + linkStatement.setString(2, schemaName); + linkStatement.setString(3, tableName); + linkStatement.setString(4, physicalName.getString()); + linkStatement.setByte(5, LinkType.PHYSICAL_TABLE.getSerializedValue()); + linkStatement.execute(); + } } - // Propagate property values to VIEW. - // TODO: formalize the known set of these properties - multiTenant = parent.isMultiTenant(); - saltBucketNum = parent.getBucketNum(); - isImmutableRows = parent.isImmutableRows(); - disableWAL = (disableWALProp == null ? parent.isWALDisabled() : disableWALProp); - defaultFamilyName = parent.getDefaultFamilyName() == null ? null : parent.getDefaultFamilyName().getString(); - columns = newArrayListWithExpectedSize(parent.getColumns().size() + colDefs.size()); - columns.addAll(parent.getColumns()); - pkColumns = newLinkedHashSet(parent.getPKColumns()); } else { columns = newArrayListWithExpectedSize(colDefs.size()); pkColumns = newLinkedHashSetWithExpectedSize(colDefs.size() + 1); // in case salted @@ -909,7 +909,7 @@ private PTable createTableInternal(CreateTableStatement statement, byte[][] spli PTable table = PTableImpl.makePTable(PNameFactory.newName(schemaName),PNameFactory.newName(tableName), tableType, null, MetaDataProtocol.MIN_TABLE_TIMESTAMP, PTable.INITIAL_SEQ_NUM, PNameFactory.newName(QueryConstants.SYSTEM_TABLE_PK_NAME), null, columns, null, Collections.emptyList(), isImmutableRows, - null, null, defaultFamilyName == null ? null : PNameFactory.newName(defaultFamilyName), + Collections.emptyList(), defaultFamilyName == null ? null : PNameFactory.newName(defaultFamilyName), null, Boolean.TRUE.equals(disableWAL), false, null); connection.addTable(table); } else if (tableType == PTableType.INDEX) { @@ -962,7 +962,7 @@ private PTable createTableInternal(CreateTableStatement statement, byte[][] spli tableUpsert.setString(10, indexState == null ? null : indexState.getSerializedValue()); tableUpsert.setBoolean(11, isImmutableRows); tableUpsert.setString(12, defaultFamilyName); - tableUpsert.setString(13, viewExpressionStr); + tableUpsert.setString(13, viewStatement); tableUpsert.setBoolean(14, disableWAL); tableUpsert.setBoolean(15, multiTenant); if (viewType == null) { @@ -970,8 +970,6 @@ private PTable createTableInternal(CreateTableStatement statement, byte[][] spli } else { tableUpsert.setByte(16, viewType.getSerializedValue()); } - tableUpsert.setString(17, baseSchemaName); - tableUpsert.setString(18, baseTableName); tableUpsert.execute(); tableMetaData.addAll(connection.getMutationState().toMutations().next().getSecond()); @@ -989,7 +987,7 @@ private PTable createTableInternal(CreateTableStatement statement, byte[][] spli QueryServices.ROW_KEY_ORDER_SALTED_TABLE_ATTRIB, QueryServicesOptions.DEFAULT_ROW_KEY_ORDER_SALTED_TABLE)); MetaDataMutationResult result = connection.getQueryServices().createTable( tableMetaData, - viewType == ViewType.MAPPED ? SchemaUtil.getTableNameAsBytes(baseSchemaName, baseTableName) : null, + viewType == ViewType.MAPPED ? physicalNames.get(0).getBytes() : null, tableType, tableProps, familyPropList, splits); MutationCode code = result.getMutationCode(); switch(code) { @@ -1013,9 +1011,9 @@ private PTable createTableInternal(CreateTableStatement statement, byte[][] spli PTable table = PTableImpl.makePTable( PNameFactory.newName(schemaName), PNameFactory.newName(tableName), tableType, indexState, result.getMutationTime(), PTable.INITIAL_SEQ_NUM, pkName == null ? null : PNameFactory.newName(pkName), saltBucketNum, columns, dataTableName == null ? null : PNameFactory.newName(dataTableName), - Collections.emptyList(), isImmutableRows, baseSchemaName == null ? null : PNameFactory.newName(baseSchemaName), - baseTableName == null ? null : PNameFactory.newName(baseTableName), defaultFamilyName == null ? null : PNameFactory.newName(defaultFamilyName), - viewExpressionStr, Boolean.TRUE.equals(disableWAL), multiTenant, viewType); + Collections.emptyList(), isImmutableRows, physicalNames, + defaultFamilyName == null ? null : PNameFactory.newName(defaultFamilyName), + viewStatement, Boolean.TRUE.equals(disableWAL), multiTenant, viewType); connection.addTable(table); return table; } @@ -1122,7 +1120,7 @@ private MutationState dropTable(String schemaName, String tableName, String pare } catch (TableNotFoundException ignore) { } // Ignore - just means wasn't cached boolean dropMetaData = connection.getQueryServices().getProps().getBoolean(DROP_METADATA_ATTRIB, DEFAULT_DROP_METADATA); - if (result.getTable() != null && !(tableType == PTableType.VIEW && result.getTable().getViewType() != ViewType.UPDATABLE) && !dropMetaData) { + if (result.getTable() != null && tableType != PTableType.VIEW && !dropMetaData) { connection.setAutoCommit(true); // Delete everything in the column. You'll still be able to do queries at earlier timestamps long ts = (scn == null ? result.getMutationTime() : scn); diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataEntityNotFoundException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataEntityNotFoundException.java new file mode 100644 index 00000000..e304e4f6 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataEntityNotFoundException.java @@ -0,0 +1,30 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +public abstract class MetaDataEntityNotFoundException extends SQLException { + private static final long serialVersionUID = 1L; + + public MetaDataEntityNotFoundException(String reason, String sqlState, int code, Throwable cause) { + super(reason, sqlState, code, cause); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataSplitPolicy.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataSplitPolicy.java new file mode 100644 index 00000000..a5a9bd28 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataSplitPolicy.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy; + +import org.apache.phoenix.util.SchemaUtil; + + +public class MetaDataSplitPolicy extends ConstantSizeRegionSplitPolicy { + + @Override + protected byte[] getSplitPoint() { + byte[] splitPoint = super.getSplitPoint(); + int offset = SchemaUtil.getVarCharLength(splitPoint, 0, splitPoint.length); + // Split only on Phoenix schema name, so this is ok b/c we won't be splitting + // in the middle of a Phoenix table. + if (offset == splitPoint.length) { + return splitPoint; + } +// offset = SchemaUtil.getVarCharLength(splitPoint, offset+1, splitPoint.length-offset-1); +// // Split only on Phoenix schema and table name, so this is ok b/c we won't be splitting +// // in the middle of a Phoenix table. +// if (offset == splitPoint.length) { +// return splitPoint; +// } + // Otherwise, an attempt is being made to split in the middle of a table. + // Just return a split point at the schema boundary instead + byte[] newSplitPoint = new byte[offset + 1]; + System.arraycopy(splitPoint, 0, newSplitPoint, 0, offset+1); + return newSplitPoint; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/NewerTableAlreadyExistsException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/NewerTableAlreadyExistsException.java new file mode 100644 index 00000000..b5979e81 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/NewerTableAlreadyExistsException.java @@ -0,0 +1,29 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +public class NewerTableAlreadyExistsException extends TableAlreadyExistsException { + private static final long serialVersionUID = 1L; + + public NewerTableAlreadyExistsException(String schemaName, String tableName) { + super(schemaName, tableName); + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PArrayDataType.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PArrayDataType.java similarity index 87% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/PArrayDataType.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/PArrayDataType.java index 8889e2f8..dbff35b9 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PArrayDataType.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PArrayDataType.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. +/* + * Copyright 2010 The Apache Software Foundation * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. + * http://www.apache.org/licenses/LICENSE-2.0 * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; import java.nio.ByteBuffer; import java.sql.Types; @@ -33,7 +25,7 @@ import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.phoenix.util.ByteUtil; +import org.apache.phoenix.util.ByteUtil; /** * The datatype for PColummns that are Arrays @@ -391,4 +383,4 @@ public static int getArrayLength(ImmutableBytesWritable ptr, return Bytes.toInt(bytes, ptr.getOffset() + Bytes.SIZEOF_BYTE); } -} +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PBaseColumn.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PBaseColumn.java new file mode 100644 index 00000000..5169e295 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PBaseColumn.java @@ -0,0 +1,52 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + + +/** + * + * Base class for PColumn implementors that provides + * some reasonable default implementations. + * + * @author jtaylor + * @since 0.1 + */ +public abstract class PBaseColumn implements PColumn { + + @Override + public final Integer getByteSize() { + return getDataType().getByteSize(); + } + + @Override + public final Integer getMaxLength() { + return null; + } + + @Override + public final Integer getScale() { + return null; + } + + @Override + public boolean isNullable() { + return false; + } + } diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumn.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumn.java new file mode 100644 index 00000000..4436a118 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumn.java @@ -0,0 +1,51 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import org.apache.hadoop.io.Writable; + +/** + * Definition of a Phoenix column + * + * @author wmacklem,jtaylor + * @since 0.1 + */ +public interface PColumn extends PDatum, Writable { + + /** + * @return the name of the column qualifier + */ + PName getName(); + + /** + * @return the name of the column family + */ + PName getFamilyName(); + + /** + * @return the zero-based ordinal position of the column + */ + int getPosition(); + + /** + * @return the declared array size or zero if this is not an array + */ + Integer getArraySize(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumnFamily.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumnFamily.java new file mode 100644 index 00000000..6927c63c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumnFamily.java @@ -0,0 +1,54 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.util.Collection; + +/** + * + * Definition of a Phoenix Column Family + * + * @author jtaylor + * @since 0.1 + */ +public interface PColumnFamily { + + /** + * @return The column family name. + */ + PName getName(); + + /** + * @return All the PColumns in this column family. + */ + Collection getColumns(); + + /** + * @return The PColumn for the specified column qualifier. + * @throws ColumnNotFoundException if the column cannot be found + */ + PColumn getColumn(byte[] qualifier) throws ColumnNotFoundException; + + /** + * @return The PColumn for the specified column qualifier. + * @throws ColumnNotFoundException if the column cannot be found + */ + PColumn getColumn(String name) throws ColumnNotFoundException; +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumnFamilyImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumnFamilyImpl.java new file mode 100644 index 00000000..4b097620 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumnFamilyImpl.java @@ -0,0 +1,75 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.util.List; +import java.util.Map; + +import org.apache.hadoop.hbase.util.Bytes; + +import com.google.common.collect.*; + +public class PColumnFamilyImpl implements PColumnFamily { + private final PName name; + private final List columns; + private final Map columnByString; + private final Map columnByBytes; + + public PColumnFamilyImpl(PName name, List columns) { + this.name = name; + this.columns = ImmutableList.copyOf(columns); + ImmutableMap.Builder columnByStringBuilder = ImmutableMap.builder(); + ImmutableSortedMap.Builder columnByBytesBuilder = ImmutableSortedMap.orderedBy(Bytes.BYTES_COMPARATOR); + for (PColumn column : columns) { + columnByBytesBuilder.put(column.getName().getBytes(), column); + columnByStringBuilder.put(column.getName().getString(), column); + } + this.columnByBytes = columnByBytesBuilder.build(); + this.columnByString = columnByStringBuilder.build(); + } + + @Override + public PName getName() { + return name; + } + + @Override + public List getColumns() { + return columns; + } + + @Override + public PColumn getColumn(byte[] qualifier) throws ColumnNotFoundException { + PColumn column = columnByBytes.get(qualifier); + if (column == null) { + throw new ColumnNotFoundException(Bytes.toString(qualifier)); + } + return column; + } + + @Override + public PColumn getColumn(String name) throws ColumnNotFoundException { + PColumn column = columnByString.get(name); + if (column == null) { + throw new ColumnNotFoundException(name); + } + return column; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumnImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumnImpl.java new file mode 100644 index 00000000..c20cc8c6 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PColumnImpl.java @@ -0,0 +1,207 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.io.DataInput; +import java.io.DataOutput; +import java.io.IOException; + +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.io.WritableUtils; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.util.ByteUtil; + +import com.google.common.base.Preconditions; + + +public class PColumnImpl implements PColumn { + private static final Integer NO_MAXLENGTH = Integer.MIN_VALUE; + private static final Integer NO_SCALE = Integer.MIN_VALUE; + + private PName name; + private PName familyName; + private PDataType dataType; + private Integer maxLength; + private Integer scale; + private boolean nullable; + private int position; + private ColumnModifier columnModifier; + private Integer arraySize; + + public PColumnImpl() { + } + + public PColumnImpl(PName name, + PName familyName, + PDataType dataType, + Integer maxLength, + Integer scale, + boolean nullable, + int position, + ColumnModifier sortOrder, Integer arrSize) { + init(name, familyName, dataType, maxLength, scale, nullable, position, sortOrder, arrSize); + } + + public PColumnImpl(PColumn column, int position) { + this(column.getName(), column.getFamilyName(), column.getDataType(), column.getMaxLength(), + column.getScale(), column.isNullable(), position, column.getColumnModifier(), column.getArraySize()); + } + + private void init(PName name, + PName familyName, + PDataType dataType, + Integer maxLength, + Integer scale, + boolean nullable, + int position, + ColumnModifier columnModifier, + Integer arrSize) { + this.dataType = dataType; + if (familyName == null) { + // Allow nullable columns in PK, but only if they're variable length. + // Variable length types may be null, since we use a null-byte terminator + // (which is a disallowed character in variable length types). However, + // fixed width types do not have a way of representing null. + // TODO: we may be able to allow this for columns at the end of the PK + Preconditions.checkArgument(!nullable || !dataType.isFixedWidth(), + "PK columns may not be both fixed width and nullable: " + name.getString()); + } + this.name = name; + this.familyName = familyName == null ? null : familyName; + this.maxLength = maxLength; + this.scale = scale; + this.nullable = nullable; + this.position = position; + this.columnModifier = columnModifier; + this.arraySize = arrSize; + } + + @Override + public PName getName() { + return name; + } + + @Override + public PName getFamilyName() { + return familyName; + } + + @Override + public PDataType getDataType() { + return dataType; + } + + @Override + public Integer getMaxLength() { + return maxLength; + } + + @Override + public Integer getScale() { + return scale; + } + + @Override + public Integer getByteSize() { + Integer dataTypeMaxLength = dataType.getByteSize(); + return dataTypeMaxLength == null ? dataType.estimateByteSizeFromLength(maxLength) + : dataTypeMaxLength; + } + + @Override + public boolean isNullable() { + return nullable; + } + + @Override + public int getPosition() { + return position; + } + + @Override + public ColumnModifier getColumnModifier() { + return columnModifier; + } + + @Override + public String toString() { + return (familyName == null ? "" : familyName.toString() + QueryConstants.NAME_SEPARATOR) + name.toString(); + } + + @Override + public void readFields(DataInput input) throws IOException { + byte[] columnNameBytes = Bytes.readByteArray(input); + PName columnName = PNameFactory.newName(columnNameBytes); + byte[] familyNameBytes = Bytes.readByteArray(input); + PName familyName = familyNameBytes.length == 0 ? null : PNameFactory.newName(familyNameBytes); + // TODO: optimize the reading/writing of this b/c it could likely all fit in a single byte or two + PDataType dataType = PDataType.values()[WritableUtils.readVInt(input)]; + int maxLength = WritableUtils.readVInt(input); + int scale = WritableUtils.readVInt(input); + boolean nullable = input.readBoolean(); + int position = WritableUtils.readVInt(input); + ColumnModifier columnModifier = ColumnModifier.fromSystemValue(WritableUtils.readVInt(input)); + int arrSize = WritableUtils.readVInt(input); + init(columnName, familyName, dataType, maxLength == NO_MAXLENGTH ? null : maxLength, + scale == NO_SCALE ? null : scale, nullable, position, columnModifier, arrSize == -1 ? null : arrSize); + } + + @Override + public void write(DataOutput output) throws IOException { + Bytes.writeByteArray(output, name.getBytes()); + Bytes.writeByteArray(output, familyName == null ? ByteUtil.EMPTY_BYTE_ARRAY : familyName.getBytes()); + WritableUtils.writeVInt(output, dataType.ordinal()); + WritableUtils.writeVInt(output, maxLength == null ? NO_MAXLENGTH : maxLength); + WritableUtils.writeVInt(output, scale == null ? NO_SCALE : scale); + output.writeBoolean(nullable); + WritableUtils.writeVInt(output, position); + WritableUtils.writeVInt(output, ColumnModifier.toSystemValue(columnModifier)); + WritableUtils.writeVInt(output, arraySize == null ? -1 : arraySize); + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + ((familyName == null) ? 0 : familyName.hashCode()); + result = prime * result + ((name == null) ? 0 : name.hashCode()); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + PColumnImpl other = (PColumnImpl)obj; + if (familyName == null) { + if (other.familyName != null) return false; + } else if (!familyName.equals(other.familyName)) return false; + if (name == null) { + if (other.name != null) return false; + } else if (!name.equals(other.name)) return false; + return true; + } + + @Override + public Integer getArraySize() { + return arraySize; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PDataType.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PDataType.java similarity index 98% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/PDataType.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/PDataType.java index 4de4bfd5..850ae136 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PDataType.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PDataType.java @@ -1,58 +1,39 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. +/* + * Copyright 2010 The Apache Software Foundation * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. + * http://www.apache.org/licenses/LICENSE-2.0 * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import java.math.BigDecimal; -import java.math.BigInteger; -import java.math.MathContext; -import java.math.RoundingMode; -import java.sql.Date; -import java.sql.Time; -import java.sql.Timestamp; -import java.sql.Types; + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.math.*; +import java.sql.*; import java.text.Format; import java.util.Map; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.hbase.util.Base64; import org.apache.hadoop.hbase.util.Bytes; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.util.*; import com.google.common.collect.ImmutableMap; import com.google.common.math.LongMath; -import com.google.common.primitives.Booleans; -import com.google.common.primitives.Doubles; -import com.google.common.primitives.Longs; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.DateUtil; -import com.salesforce.phoenix.util.NumberUtil; -import com.salesforce.phoenix.util.StringUtil; +import com.google.common.primitives.*; /** @@ -5201,9 +5182,6 @@ public Integer estimateByteSizeFromLength(Integer length) { if (isFixedWidth()) { return getByteSize(); } - if(isArrayType()) { - return null; - } // If not fixed width, default to say the byte size is the same as length. return length; } diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PDateColumn.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PDateColumn.java new file mode 100644 index 00000000..8ed6c4a5 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PDateColumn.java @@ -0,0 +1,27 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +abstract public class PDateColumn extends PBaseColumn { + @Override + public PDataType getDataType() { + return PDataType.DATE; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PDatum.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PDatum.java new file mode 100644 index 00000000..e1946f5c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PDatum.java @@ -0,0 +1,54 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + + +public interface PDatum { + /** + * @return is this column nullable? + */ + boolean isNullable(); + + /** + * @return data type of the column + */ + PDataType getDataType(); + + /** + * @return maximum byte length of the column + */ + Integer getByteSize(); + + /** + * @return the actual length of the column. For decimal, it would be its precision. For char or + * varchar, it would be the maximum length as specified during schema definition. + */ + Integer getMaxLength(); + + /** + * @return scale of a decimal number. + */ + Integer getScale(); + + /** + * @return The modifier for this column or null if it doesn't have a modifier + */ + ColumnModifier getColumnModifier(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PIndexState.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PIndexState.java new file mode 100644 index 00000000..330c5c91 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PIndexState.java @@ -0,0 +1,92 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + + +public enum PIndexState { + BUILDING("b"), + USABLE("e"), + UNUSABLE("d"), + ACTIVE("a"), + INACTIVE("i"), + DISABLE("x"), + REBUILD("r"); + + private final String serializedValue; + private final byte[] serializedBytes; + private final byte[] nameBytesValue; + + private PIndexState(String value) { + this.serializedValue = value; + this.serializedBytes = PDataType.VARCHAR.toBytes(value); + this.nameBytesValue = PDataType.VARCHAR.toBytes(this.toString()); + } + + public String getSerializedValue() { + return serializedValue; + } + + public byte[] getSerializedBytes() { + return serializedBytes; + } + + public byte[] toBytes() { + return nameBytesValue; + } + + private static final PIndexState[] FROM_VALUE; + private static final int FROM_VALUE_OFFSET; + static { + int minChar = Integer.MAX_VALUE; + int maxChar = Integer.MIN_VALUE; + for (PIndexState state: PIndexState.values()) { + char c = state.getSerializedValue().charAt(0); + if (c < minChar) { + minChar = c; + } + if (c > maxChar) { + maxChar = c; + } + } + FROM_VALUE_OFFSET = minChar; + FROM_VALUE = new PIndexState[maxChar - minChar + 1]; + for (PIndexState state: PIndexState.values()) { + FROM_VALUE[state.getSerializedValue().charAt(0) - minChar] = state; + } + } + + public static PIndexState fromSerializedValue(String serializedValue) { + if (serializedValue.length() == 1) { + int i = serializedValue.charAt(0) - FROM_VALUE_OFFSET; + if (i >= 0 && i < FROM_VALUE.length && FROM_VALUE[i] != null) { + return FROM_VALUE[i]; + } + } + throw new IllegalArgumentException("Unable to PIndexState enum for serialized value of '" + serializedValue + "'"); + } + + public static PIndexState fromSerializedValue(byte serializedByte) { + int i = serializedByte - FROM_VALUE_OFFSET; + if (i >= 0 && i < FROM_VALUE.length && FROM_VALUE[i] != null) { + return FROM_VALUE[i]; + } + throw new IllegalArgumentException("Unable to PIndexState enum for serialized value of '" + (char)serializedByte + "'"); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PIntegerColumn.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PIntegerColumn.java new file mode 100644 index 00000000..5c00117b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PIntegerColumn.java @@ -0,0 +1,35 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + + +/** + * + * Base class for PColumn implementors of type Integer. + * + * @author jtaylor + * @since 0.1 + */ +public abstract class PIntegerColumn extends PBaseColumn { + @Override + public final PDataType getDataType() { + return PDataType.INTEGER; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PLongColumn.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PLongColumn.java new file mode 100644 index 00000000..69fd7e95 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PLongColumn.java @@ -0,0 +1,35 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + + +/** + * + * Base class for PColumn implementors of type Long. + * + * @author jtaylor + * @since 0.1 + */ +public abstract class PLongColumn extends PBaseColumn { + @Override + public final PDataType getDataType() { + return PDataType.LONG; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PMetaData.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PMetaData.java new file mode 100644 index 00000000..1af5c4ab --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PMetaData.java @@ -0,0 +1,30 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.util.Map; + +import org.apache.phoenix.query.MetaDataMutated; + + +public interface PMetaData extends MetaDataMutated { + public PTable getTable(String name) throws TableNotFoundException; + public Map getTables(); +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PMetaDataImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PMetaDataImpl.java similarity index 79% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/PMetaDataImpl.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/PMetaDataImpl.java index 5b9f9972..cbe5d508 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PMetaDataImpl.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PMetaDataImpl.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; import java.sql.SQLException; import java.util.Collections; @@ -142,7 +134,7 @@ public PMetaData removeColumn(String tableName, String familyName, String column // Update position of columns that follow removed column for (int i = position+1; i < oldColumns.size(); i++) { PColumn oldColumn = oldColumns.get(i); - PColumn newColumn = new PColumnImpl(oldColumn.getName(), oldColumn.getFamilyName(), oldColumn.getDataType(), oldColumn.getMaxLength(), oldColumn.getScale(), oldColumn.isNullable(), i-1+positionOffset, oldColumn.getColumnModifier()); + PColumn newColumn = new PColumnImpl(oldColumn.getName(), oldColumn.getFamilyName(), oldColumn.getDataType(), oldColumn.getMaxLength(), oldColumn.getScale(), oldColumn.isNullable(), i-1+positionOffset, oldColumn.getColumnModifier(), oldColumn.getArraySize()); columns.add(newColumn); } diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PName.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PName.java new file mode 100644 index 00000000..97d8989f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PName.java @@ -0,0 +1,96 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.util.ByteUtil; + + +/** + * + * Interface to encapsulate both the client-side name + * together with the server-side name for a named object + * + * @author jtaylor + * @since 0.1 + */ +public interface PName { + public static PName EMPTY_NAME = new PName() { + @Override + public String getString() { + return ""; + } + + @Override + public byte[] getBytes() { + return ByteUtil.EMPTY_BYTE_ARRAY; + } + + @Override + public String toString() { + return getString(); + } + + @Override + public ImmutableBytesPtr getBytesPtr() { + return ByteUtil.EMPTY_BYTE_ARRAY_PTR; + } + }; + public static PName EMPTY_COLUMN_NAME = new PName() { + @Override + public String getString() { + return QueryConstants.EMPTY_COLUMN_NAME; + } + + @Override + public byte[] getBytes() { + return QueryConstants.EMPTY_COLUMN_BYTES; + } + + @Override + public String toString() { + return getString(); + } + + @Override + public ImmutableBytesPtr getBytesPtr() { + return QueryConstants.EMPTY_COLUMN_BYTES_PTR; + } + }; + /** + * Get the client-side, normalized name as referenced + * in a SQL statement. + * @return the normalized string name + */ + String getString(); + + /** + * Get the server-side name as referenced in HBase-related + * APIs such as Scan, Filter, etc. + * @return the name as a byte array + */ + byte[] getBytes(); + + /** + * @return a pointer to the underlying bytes + */ + ImmutableBytesPtr getBytesPtr(); +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PNameFactory.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PNameFactory.java similarity index 87% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/PNameFactory.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/PNameFactory.java index 5e8ac0c8..cbea54c9 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PNameFactory.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PNameFactory.java @@ -1,8 +1,8 @@ -package com.salesforce.phoenix.schema; +package org.apache.phoenix.schema; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.phoenix.query.QueryConstants; +import org.apache.phoenix.query.QueryConstants; public class PNameFactory { diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PNameImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PNameImpl.java similarity index 96% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/PNameImpl.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/PNameImpl.java index caeefb66..8feca89e 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PNameImpl.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PNameImpl.java @@ -15,12 +15,12 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package com.salesforce.phoenix.schema; +package org.apache.phoenix.schema; import org.apache.hadoop.hbase.util.Bytes; import org.apache.http.annotation.Immutable; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; @Immutable public class PNameImpl implements PName { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PNormalizedName.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PNormalizedName.java new file mode 100644 index 00000000..4876de98 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PNormalizedName.java @@ -0,0 +1,53 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import org.apache.http.annotation.Immutable; + +import org.apache.phoenix.util.SchemaUtil; + + +@Immutable +public class PNormalizedName extends PNameImpl { + + public PNormalizedName(String nonNormalizedName) { + super(SchemaUtil.normalizeIdentifier(nonNormalizedName)); + } + + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + getString().hashCode(); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + PNormalizedName other = (PNormalizedName)obj; + // Compare normalized stringName for equality, since bytesName + // may differ since it remains case sensitive. + if (!getString().equals(other.getString())) return false; + return true; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PRow.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PRow.java new file mode 100644 index 00000000..22decfc7 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PRow.java @@ -0,0 +1,75 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.util.List; +import java.util.Map; + +import org.apache.hadoop.hbase.client.Mutation; + +import com.google.common.collect.ImmutableMap; + +/** + * + * Provide a client API for updating rows. The updates are processed in + * the calling order. Calling setValue after calling delete will cause the + * delete to be canceled. Conversely, calling delete after calling + * setValue will cause all prior setValue calls to be canceled. + * + * @author jtaylor + * @since 0.1 + */ +public interface PRow { + Map DELETE_MARKER = ImmutableMap.of(); + + /** + * Get the list of {@link org.apache.hadoop.hbase.client.Mutation} used to + * update an HTable after all mutations through calls to + * {@link #setValue(PColumn, Object)} or {@link #delete()}. + * @return the list of mutations representing all changes made to a row + * @throws ConstraintViolationException if row data violates schema + * constraint + */ + public List toRowMutations(); + + /** + * Set a column value in the row + * @param col the column for which the value is being set + * @param value the value + * @throws ConstraintViolationException if row data violates schema + * constraint + */ + public void setValue(PColumn col, Object value); + + /** + * Set a column value in the row + * @param col the column for which the value is being set + * @param value the value + * @throws ConstraintViolationException if row data violates schema + * constraint + */ + public void setValue(PColumn col, byte[] value); + + /** + * Delete the row. Note that a delete take precedence over any + * values that may have been set before or after the delete call. + */ + public void delete(); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PStringColumn.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PStringColumn.java new file mode 100644 index 00000000..606b5415 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PStringColumn.java @@ -0,0 +1,36 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +/** + * + * Abstract class for columns of type {@link org.apache.phoenix.schema.PDataType#VARCHAR} + * + * @author jtaylor + * @since 0.1 + */ +public abstract class PStringColumn extends PBaseColumn { + + @Override + public PDataType getDataType() { + return PDataType.VARCHAR; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PTable.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PTable.java new file mode 100644 index 00000000..10a568e1 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PTable.java @@ -0,0 +1,280 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.util.List; + +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.io.Writable; +import org.apache.phoenix.client.KeyValueBuilder; +import org.apache.phoenix.index.IndexMaintainer; +import org.apache.phoenix.schema.stat.PTableStats; + + +/** + * Definition of a Phoenix table + * + * @author wmacklem,jtaylor + * @since 0.1 + */ +public interface PTable extends Writable { + public static final long INITIAL_SEQ_NUM = 0; + public static final String IS_IMMUTABLE_ROWS_PROP_NAME = "IMMUTABLE_ROWS"; + public static final boolean DEFAULT_DISABLE_WAL = false; + + public enum ViewType { + MAPPED((byte)1), + READ_ONLY((byte)2), + UPDATABLE((byte)3); + + private final byte[] byteValue; + private final byte serializedValue; + + ViewType(byte serializedValue) { + this.serializedValue = serializedValue; + this.byteValue = Bytes.toBytes(this.name()); + } + + public byte[] getBytes() { + return byteValue; + } + + public boolean isReadOnly() { + return this != UPDATABLE; + } + + public byte getSerializedValue() { + return this.serializedValue; + } + + public static ViewType fromSerializedValue(byte serializedValue) { + if (serializedValue < 1 || serializedValue > ViewType.values().length) { + throw new IllegalArgumentException("Invalid ViewType " + serializedValue); + } + return ViewType.values()[serializedValue-1]; + } + + public ViewType combine(ViewType otherType) { + if (otherType == null) { + return this; + } + if (this == UPDATABLE && otherType == UPDATABLE) { + return UPDATABLE; + } + return READ_ONLY; + } + } + + public enum LinkType { + /** + * Link from a table to its index table + */ + INDEX_TABLE((byte)1), + /** + * Link from a view to its physical table + */ + PHYSICAL_TABLE((byte)2); + + private final byte[] byteValue; + private final byte serializedValue; + + LinkType(byte serializedValue) { + this.serializedValue = serializedValue; + this.byteValue = Bytes.toBytes(this.name()); + } + + public byte[] getBytes() { + return byteValue; + } + + public byte getSerializedValue() { + return this.serializedValue; + } + + public static LinkType fromSerializedValue(byte serializedValue) { + if (serializedValue < 1 || serializedValue > LinkType.values().length) { + return null; + } + return LinkType.values()[serializedValue-1]; + } + } + + long getTimeStamp(); + long getSequenceNumber(); + /** + * @return table name + */ + PName getName(); + PName getSchemaName(); + PName getTableName(); + + /** + * @return the table type + */ + PTableType getType(); + + PName getPKName(); + + /** + * Get the PK columns ordered by position. + * @return a list of the PK columns + */ + List getPKColumns(); + + /** + * Get all columns ordered by position. + * @return a list of all columns + */ + List getColumns(); + + /** + * @return A list of the column families of this table + * ordered by position. + */ + List getColumnFamilies(); + + /** + * Get the column family with the given name + * @param family the column family name + * @return the PColumnFamily with the given name + * @throws ColumnFamilyNotFoundException if the column family cannot be found + */ + PColumnFamily getColumnFamily(byte[] family) throws ColumnFamilyNotFoundException; + + PColumnFamily getColumnFamily(String family) throws ColumnFamilyNotFoundException; + + /** + * Get the column with the given string name. + * @param name the column name + * @return the PColumn with the given name + * @throws ColumnNotFoundException if no column with the given name + * can be found + * @throws AmbiguousColumnException if multiple columns are found with the given name + */ + PColumn getColumn(String name) throws ColumnNotFoundException, AmbiguousColumnException; + + /** + * Get the PK column with the given name. + * @param name the column name + * @return the PColumn with the given name + * @throws ColumnNotFoundException if no PK column with the given name + * can be found + * @throws ColumnNotFoundException + */ + PColumn getPKColumn(String name) throws ColumnNotFoundException; + + /** + * Creates a new row at the specified timestamp using the key + * for the PK values (from {@link #newKey(ImmutableBytesWritable, byte[][])} + * and the optional key values specified using values. + * @param ts the timestamp that the key value will have when committed + * @param key the row key of the key value + * @param values the optional key values + * @return the new row. Use {@link org.apache.phoenix.schema.PRow#toRowMutations()} to + * generate the Row to send to the HBase server. + * @throws ConstraintViolationException if row data violates schema + * constraint + */ + PRow newRow(KeyValueBuilder builder, long ts, ImmutableBytesWritable key, byte[]... values); + + /** + * Creates a new row for the PK values (from {@link #newKey(ImmutableBytesWritable, byte[][])} + * and the optional key values specified using values. The timestamp of the key value + * will be set by the HBase server. + * @param key the row key of the key value + * @param values the optional key values + * @return the new row. Use {@link org.apache.phoenix.schema.PRow#toRowMutations()} to + * generate the row to send to the HBase server. + * @throws ConstraintViolationException if row data violates schema + * constraint + */ + PRow newRow(KeyValueBuilder builder, ImmutableBytesWritable key, byte[]... values); + + /** + * Formulates a row key using the values provided. The values must be in + * the same order as {@link #getPKColumns()}. + * @param key bytes pointer that will be filled in with the row key + * @param values the PK column values + * @return the number of values that were used from values to set + * the row key + */ + int newKey(ImmutableBytesWritable key, byte[][] values); + + /** + * Return the statistics table associated with this PTable. + * @return the statistics table. + */ + PTableStats getTableStats(); + + RowKeySchema getRowKeySchema(); + + /** + * Return the number of buckets used by this table for salting. If the table does + * not use salting, returns null. + * @return number of buckets used by this table for salting, or null if salting is not used. + */ + Integer getBucketNum(); + + /** + * Return the list of indexes defined on this table. + * @return the list of indexes. + */ + List getIndexes(); + + /** + * For a table of index type, return the state of the table. + * @return the state of the index. + */ + PIndexState getIndexState(); + + /** + * Gets the full name of the data table for an index table. + * @return the name of the data table that this index is on + * or null if not an index. + */ + PName getParentName(); + /** + * Gets the table name of the data table for an index table. + * @return the table name of the data table that this index is + * on or null if not an index. + */ + PName getParentTableName(); + + /** + * For a view, return the name of table in Phoenix that physically stores data. + * Currently a single name, but when views are allowed over multiple tables, will become multi-valued. + * @return the name of the physical table storing the data. + */ + public List getPhysicalNames(); + + PName getPhysicalName(); + boolean isImmutableRows(); + + void getIndexMaintainers(ImmutableBytesWritable ptr); + IndexMaintainer getIndexMaintainer(PTable dataTable); + PName getDefaultFamilyName(); + + boolean isWALDisabled(); + boolean isMultiTenant(); + + ViewType getViewType(); + String getViewStatement(); +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PTableImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PTableImpl.java similarity index 83% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/PTableImpl.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/PTableImpl.java index 1be6e390..11d6c94d 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PTableImpl.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PTableImpl.java @@ -1,36 +1,28 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; - -import static com.salesforce.phoenix.client.KeyValueBuilder.addQuietly; -import static com.salesforce.phoenix.client.KeyValueBuilder.deleteQuietly; -import static com.salesforce.phoenix.query.QueryConstants.SEPARATOR_BYTE; -import static com.salesforce.phoenix.schema.SaltingUtil.SALTING_COLUMN; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import static org.apache.phoenix.client.KeyValueBuilder.addQuietly; +import static org.apache.phoenix.client.KeyValueBuilder.deleteQuietly; +import static org.apache.phoenix.query.QueryConstants.SEPARATOR_BYTE; +import static org.apache.phoenix.schema.SaltingUtil.SALTING_COLUMN; import java.io.DataInput; import java.io.DataOutput; @@ -48,9 +40,20 @@ import org.apache.hadoop.hbase.client.Delete; import org.apache.hadoop.hbase.client.Mutation; import org.apache.hadoop.hbase.client.Put; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.hbase.util.Bytes; import org.apache.hadoop.io.WritableUtils; +import org.apache.phoenix.client.KeyValueBuilder; +import org.apache.phoenix.index.IndexMaintainer; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.RowKeySchema.RowKeySchemaBuilder; +import org.apache.phoenix.schema.stat.PTableStats; +import org.apache.phoenix.schema.stat.PTableStatsImpl; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.SchemaUtil; +import org.apache.phoenix.util.StringUtil; +import org.apache.phoenix.util.TrustedByteArrayOutputStream; import com.google.common.base.Objects; import com.google.common.collect.ArrayListMultimap; @@ -60,17 +63,6 @@ import com.google.common.collect.ListMultimap; import com.google.common.collect.Lists; import com.google.common.collect.Maps; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.client.KeyValueBuilder; -import com.salesforce.phoenix.index.IndexMaintainer; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.RowKeySchema.RowKeySchemaBuilder; -import com.salesforce.phoenix.schema.stat.PTableStats; -import com.salesforce.phoenix.schema.stat.PTableStatsImpl; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.SchemaUtil; -import com.salesforce.phoenix.util.StringUtil; -import com.salesforce.phoenix.util.TrustedByteArrayOutputStream; /** @@ -109,14 +101,12 @@ public class PTableImpl implements PTable { // Data table name that the index is created on. private PName parentName; private PName parentTableName; - private PName baseSchemaName; - private PName baseTableName; - private PName baseName; + private List physicalNames; private boolean isImmutableRows; private IndexMaintainer indexMaintainer; private ImmutableBytesWritable indexMaintainersPtr; private PName defaultFamilyName; - private String viewExpression; + private String viewStatement; private boolean disableWAL; private boolean multiTenant; private ViewType viewType; @@ -176,48 +166,50 @@ public static PTableImpl makePTable(PTable table, long timeStamp, List i return new PTableImpl( table.getSchemaName(), table.getTableName(), table.getType(), table.getIndexState(), timeStamp, table.getSequenceNumber() + 1, table.getPKName(), table.getBucketNum(), getColumnsToClone(table), table.getParentTableName(), indexes, table.isImmutableRows(), - table.getBaseSchemaName(), table.getBaseTableName(), table.getDefaultFamilyName(), table.getViewExpression(), table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); + table.getPhysicalNames(), table.getDefaultFamilyName(), table.getViewStatement(), table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); } public static PTableImpl makePTable(PTable table, List columns) throws SQLException { return new PTableImpl( table.getSchemaName(), table.getTableName(), table.getType(), table.getIndexState(), table.getTimeStamp(), table.getSequenceNumber(), - table.getPKName(), table.getBucketNum(), columns, table.getParentTableName(), table.getIndexes(), table.isImmutableRows(), table.getBaseSchemaName(), table.getBaseTableName(), table.getDefaultFamilyName(), table.getViewExpression(), table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); + table.getPKName(), table.getBucketNum(), columns, table.getParentTableName(), table.getIndexes(), table.isImmutableRows(), table.getPhysicalNames(), + table.getDefaultFamilyName(), table.getViewStatement(), table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); } public static PTableImpl makePTable(PTable table, long timeStamp, long sequenceNumber, List columns) throws SQLException { return new PTableImpl( table.getSchemaName(), table.getTableName(), table.getType(), table.getIndexState(), timeStamp, sequenceNumber, - table.getPKName(), table.getBucketNum(), columns, table.getParentTableName(), table.getIndexes(), table.isImmutableRows(), table.getBaseSchemaName(), table.getBaseTableName(), table.getDefaultFamilyName(), table.getViewExpression(), table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); + table.getPKName(), table.getBucketNum(), columns, table.getParentTableName(), table.getIndexes(), table.isImmutableRows(), table.getPhysicalNames(), + table.getDefaultFamilyName(), table.getViewStatement(), table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); } public static PTableImpl makePTable(PTable table, long timeStamp, long sequenceNumber, List columns, boolean isImmutableRows) throws SQLException { return new PTableImpl( table.getSchemaName(), table.getTableName(), table.getType(), table.getIndexState(), timeStamp, sequenceNumber, table.getPKName(), table.getBucketNum(), columns, table.getParentTableName(), table.getIndexes(), isImmutableRows, - table.getBaseSchemaName(), table.getBaseTableName(), table.getDefaultFamilyName(), table.getViewExpression(), table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); + table.getPhysicalNames(), table.getDefaultFamilyName(), table.getViewStatement(), table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); } public static PTableImpl makePTable(PTable table, PIndexState state) throws SQLException { return new PTableImpl( table.getSchemaName(), table.getTableName(), table.getType(), state, table.getTimeStamp(), table.getSequenceNumber(), table.getPKName(), table.getBucketNum(), getColumnsToClone(table), table.getParentTableName(), - table.getIndexes(), table.isImmutableRows(), table.getBaseSchemaName(), table.getBaseTableName(), - table.getDefaultFamilyName(), table.getViewExpression(), table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); + table.getIndexes(), table.isImmutableRows(), table.getPhysicalNames(), + table.getDefaultFamilyName(), table.getViewStatement(), table.isWALDisabled(), table.isMultiTenant(), table.getViewType()); } public static PTableImpl makePTable(PName schemaName, PName tableName, PTableType type, PIndexState state, long timeStamp, long sequenceNumber, PName pkName, - Integer bucketNum, List columns, PName dataTableName, List indexes, boolean isImmutableRows, PName baseSchemaName, - PName baseTableName, PName defaultFamilyName, String viewExpression, boolean disableWAL, boolean multiTenant, ViewType viewType) throws SQLException { + Integer bucketNum, List columns, PName dataTableName, List indexes, boolean isImmutableRows, List physicalNames, + PName defaultFamilyName, String viewExpression, boolean disableWAL, boolean multiTenant, ViewType viewType) throws SQLException { return new PTableImpl(schemaName, tableName, type, state, timeStamp, sequenceNumber, pkName, bucketNum, columns, dataTableName, indexes, - isImmutableRows, baseSchemaName, baseTableName, defaultFamilyName, viewExpression, disableWAL, multiTenant, viewType); + isImmutableRows, physicalNames, defaultFamilyName, viewExpression, disableWAL, multiTenant, viewType); } private PTableImpl(PName schemaName, PName tableName, PTableType type, PIndexState state, long timeStamp, long sequenceNumber, PName pkName, - Integer bucketNum, List columns, PName dataTableName, List indexes, boolean isImmutableRows, PName baseSchemaName, - PName baseTableName, PName defaultFamilyName, String viewExpression, boolean disableWAL, boolean multiTenant, ViewType viewType) throws SQLException { + Integer bucketNum, List columns, PName dataTableName, List indexes, boolean isImmutableRows, List physicalNames, + PName defaultFamilyName, String viewExpression, boolean disableWAL, boolean multiTenant, ViewType viewType) throws SQLException { init(schemaName, tableName, type, state, timeStamp, sequenceNumber, pkName, bucketNum, columns, new PTableStatsImpl(), - dataTableName, indexes, isImmutableRows, baseSchemaName, baseTableName, defaultFamilyName, viewExpression, disableWAL, multiTenant, viewType); + dataTableName, indexes, isImmutableRows, physicalNames, defaultFamilyName, viewExpression, disableWAL, multiTenant, viewType); } @Override @@ -232,7 +224,7 @@ public ViewType getViewType() { private void init(PName schemaName, PName tableName, PTableType type, PIndexState state, long timeStamp, long sequenceNumber, PName pkName, Integer bucketNum, List columns, PTableStats stats, PName parentTableName, List indexes, boolean isImmutableRows, - PName baseSchemaName, PName baseTableName, PName defaultFamilyName, String viewExpression, boolean disableWAL, boolean multiTenant, ViewType viewType) throws SQLException { + List physicalNames, PName defaultFamilyName, String viewExpression, boolean disableWAL, boolean multiTenant, ViewType viewType) throws SQLException { if (schemaName == null) { throw new NullPointerException(); } @@ -246,7 +238,7 @@ private void init(PName schemaName, PName tableName, PTableType type, PIndexStat this.pkName = pkName; this.isImmutableRows = isImmutableRows; this.defaultFamilyName = defaultFamilyName; - this.viewExpression = viewExpression; + this.viewStatement = viewExpression; this.disableWAL = disableWAL; this.multiTenant = multiTenant; this.viewType = viewType; @@ -328,9 +320,7 @@ private void init(PName schemaName, PName tableName, PTableType type, PIndexStat this.indexes = indexes; this.parentTableName = parentTableName; this.parentName = parentTableName == null ? null : PNameFactory.newName(SchemaUtil.getTableName(schemaName.getString(), parentTableName.getString())); - this.baseSchemaName = baseSchemaName; - this.baseTableName = baseTableName; - this.baseName = baseTableName == null ? null : PNameFactory.newName(SchemaUtil.getTableName(baseSchemaName == null ? null : baseSchemaName.getString(), baseTableName.getString())); + this.physicalNames = physicalNames == null ? ImmutableList.of() : ImmutableList.copyOf(physicalNames); } @Override @@ -703,24 +693,25 @@ public void readFields(DataInput input) throws IOException { boolean disableWAL = input.readBoolean(); boolean multiTenant = input.readBoolean(); ViewType viewType = null; - String viewExpression = null; - PName baseSchemaName = null; - PName baseTableName = null; + String viewStatement = null; + List physicalNames = Collections.emptyList(); if (tableType == PTableType.VIEW) { viewType = ViewType.fromSerializedValue(input.readByte()); - byte[] viewExpressionBytes = Bytes.readByteArray(input); - viewExpression = viewExpressionBytes.length == 0 ? null : (String)PDataType.VARCHAR.toObject(viewExpressionBytes); - byte[] baseSchemaNameBytes = Bytes.readByteArray(input); - baseSchemaName = baseSchemaNameBytes.length == 0 ? null : PNameFactory.newName(baseSchemaNameBytes); - byte[] baseTableNameBytes = Bytes.readByteArray(input); - baseTableName = baseTableNameBytes.length == 0 ? null : PNameFactory.newName(baseTableNameBytes); + byte[] viewStatementBytes = Bytes.readByteArray(input); + viewStatement = viewStatementBytes.length == 0 ? null : (String)PDataType.VARCHAR.toObject(viewStatementBytes); + int nPhysicalNames = WritableUtils.readVInt(input); + physicalNames = Lists.newArrayListWithExpectedSize(nPhysicalNames); + for (int i = 0; i < nPhysicalNames; i++) { + byte[] physicalNameBytes = Bytes.readByteArray(input); + physicalNames.add(PNameFactory.newName(physicalNameBytes)); + } } PTableStats stats = new PTableStatsImpl(guidePosts); try { init(schemaName, tableName, tableType, indexState, timeStamp, sequenceNumber, pkName, bucketNum.equals(NO_SALTING) ? null : bucketNum, columns, stats, dataTableName, - indexes, isImmutableRows, baseSchemaName, baseTableName, defaultFamilyName, - viewExpression, disableWAL, multiTenant, viewType); + indexes, isImmutableRows, physicalNames, defaultFamilyName, + viewStatement, disableWAL, multiTenant, viewType); } catch (SQLException e) { throw new RuntimeException(e); // Impossible } @@ -762,9 +753,11 @@ public void write(DataOutput output) throws IOException { output.writeBoolean(multiTenant); if (type == PTableType.VIEW) { output.writeByte(viewType.getSerializedValue()); - Bytes.writeByteArray(output, viewExpression == null ? ByteUtil.EMPTY_BYTE_ARRAY : PDataType.VARCHAR.toBytes(viewExpression)); - Bytes.writeByteArray(output, baseSchemaName == null ? ByteUtil.EMPTY_BYTE_ARRAY : baseSchemaName.getBytes()); - Bytes.writeByteArray(output, baseTableName.getBytes()); + Bytes.writeByteArray(output, viewStatement == null ? ByteUtil.EMPTY_BYTE_ARRAY : PDataType.VARCHAR.toBytes(viewStatement)); + WritableUtils.writeVInt(output, physicalNames.size()); + for (int i = 0; i < physicalNames.size(); i++) { + Bytes.writeByteArray(output, physicalNames.get(i).getBytes()); + } } } @@ -845,22 +838,12 @@ public synchronized void getIndexMaintainers(ImmutableBytesWritable ptr) { @Override public PName getPhysicalName() { - return type == PTableType.VIEW ? getBaseName() : getName(); + return physicalNames.isEmpty() ? getName() : physicalNames.get(0); } @Override - public PName getBaseName() { - return baseName; - } - - @Override - public PName getBaseSchemaName() { - return baseSchemaName; - } - - @Override - public PName getBaseTableName() { - return baseTableName; + public List getPhysicalNames() { + return physicalNames; } @Override @@ -869,8 +852,8 @@ public PName getDefaultFamilyName() { } @Override - public String getViewExpression() { - return viewExpression; + public String getViewStatement() { + return viewStatement; } @Override diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/PTableType.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PTableType.java new file mode 100644 index 00000000..f079a92c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PTableType.java @@ -0,0 +1,111 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.util.Map; + +import com.google.common.collect.Maps; + + +public enum PTableType { + SYSTEM("s", "SYSTEM TABLE"), + TABLE("u", "TABLE"), + VIEW("v", "VIEW"), + INDEX("i", "INDEX"), + JOIN("j", "JOIN"); + + private final PName value; + private final String serializedValue; + + private PTableType(String serializedValue, String value) { + this.serializedValue = serializedValue; + this.value = PNameFactory.newName(value); + } + + public String getSerializedValue() { + return serializedValue; + } + + public PName getValue() { + return value; + } + + @Override + public String toString() { + return value.getString(); + } + + private static final PTableType[] FROM_SERIALIZED_VALUE; + private static final int FROM_SERIALIZED_VALUE_OFFSET; + private static final Map FROM_VALUE = Maps.newHashMapWithExpectedSize(PTableType.values().length); + + static { + int minChar = Integer.MAX_VALUE; + int maxChar = Integer.MIN_VALUE; + for (PTableType type : PTableType.values()) { + char c = type.getSerializedValue().charAt(0); + if (c < minChar) { + minChar = c; + } + if (c > maxChar) { + maxChar = c; + } + } + FROM_SERIALIZED_VALUE_OFFSET = minChar; + FROM_SERIALIZED_VALUE = new PTableType[maxChar - minChar + 1]; + for (PTableType type : PTableType.values()) { + FROM_SERIALIZED_VALUE[type.getSerializedValue().charAt(0) - minChar] = type; + } + } + + static { + for (PTableType type : PTableType.values()) { + if (FROM_VALUE.put(type.getValue().getString(),type) != null) { + throw new IllegalStateException("Duplicate PTableType value of " + type.getValue().getString() + " is not allowed"); + } + } + } + + public static PTableType fromValue(String value) { + PTableType type = FROM_VALUE.get(value); + if (type == null) { + throw new IllegalArgumentException("Unable to PTableType enum for value of '" + value + "'"); + } + return type; + } + + public static PTableType fromSerializedValue(String serializedValue) { + if (serializedValue.length() == 1) { + int i = serializedValue.charAt(0) - FROM_SERIALIZED_VALUE_OFFSET; + if (i >= 0 && i < FROM_SERIALIZED_VALUE.length && FROM_SERIALIZED_VALUE[i] != null) { + return FROM_SERIALIZED_VALUE[i]; + } + } + throw new IllegalArgumentException("Unable to PTableType enum for serialized value of '" + serializedValue + "'"); + } + + public static PTableType fromSerializedValue(byte serializedByte) { + int i = serializedByte - FROM_SERIALIZED_VALUE_OFFSET; + if (i >= 0 && i < FROM_SERIALIZED_VALUE.length && FROM_SERIALIZED_VALUE[i] != null) { + return FROM_SERIALIZED_VALUE[i]; + } + throw new IllegalArgumentException("Unable to PTableType enum for serialized value of '" + (char)serializedByte + "'"); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PhoenixArray.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/PhoenixArray.java similarity index 86% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/PhoenixArray.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/PhoenixArray.java index 5834f2d7..e58cb507 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/PhoenixArray.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/PhoenixArray.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. +/* + * Copyright 2010 The Apache Software Foundation * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. + * http://www.apache.org/licenses/LICENSE-2.0 * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; import java.sql.Array; import java.sql.ResultSet; @@ -33,7 +25,7 @@ import java.util.Arrays; import java.util.Map; -import com.salesforce.phoenix.util.SQLCloseable; +import org.apache.phoenix.util.SQLCloseable; /** * java.sql.Array implementation for Phoenix diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ReadOnlyTableException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ReadOnlyTableException.java new file mode 100644 index 00000000..8672f8fa --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ReadOnlyTableException.java @@ -0,0 +1,57 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + +/** + * + * Exception thrown when an attempt is made to modify or write to a read-only table. + * + * @author jtaylor + * @since 0.1 + */ +public class ReadOnlyTableException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.READ_ONLY_TABLE; + + public ReadOnlyTableException(String schemaName, String tableName) { + super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).toString(), + code.getSQLState(), code.getErrorCode()); + } + + public ReadOnlyTableException(String message, String schemaName, String tableName) { + super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).setMessage(message).toString(), + code.getSQLState(), code.getErrorCode()); + } + + public ReadOnlyTableException(String message, String schemaName, String tableName, Throwable cause) { + super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).setRootCause(cause).setMessage(message).toString(), + code.getSQLState(), code.getErrorCode(), cause); + } + + public ReadOnlyTableException(String message, String schemaName, String tableName, String familyName) { + super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).setFamilyName(familyName).setMessage(message).toString(), + code.getSQLState(), code.getErrorCode()); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/RowKeySchema.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/RowKeySchema.java similarity index 82% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/RowKeySchema.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/RowKeySchema.java index b3cee889..ec5fd1b2 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/RowKeySchema.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/RowKeySchema.java @@ -1,40 +1,32 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; -import static com.salesforce.phoenix.query.QueryConstants.SEPARATOR_BYTE; +import static org.apache.phoenix.query.QueryConstants.SEPARATOR_BYTE; import java.util.Collections; import java.util.List; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; -import com.salesforce.phoenix.query.QueryConstants; +import org.apache.phoenix.query.QueryConstants; /** diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/RowKeyValueAccessor.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/RowKeyValueAccessor.java similarity index 77% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/RowKeyValueAccessor.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/RowKeyValueAccessor.java index 78442c23..01f51c7f 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/RowKeyValueAccessor.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/RowKeyValueAccessor.java @@ -1,33 +1,25 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; -import static com.salesforce.phoenix.query.QueryConstants.SEPARATOR_BYTE; +import static org.apache.phoenix.query.QueryConstants.SEPARATOR_BYTE; import java.io.DataInput; import java.io.DataOutput; @@ -39,7 +31,7 @@ import org.apache.hadoop.io.Writable; import org.apache.hadoop.io.WritableUtils; -import com.salesforce.phoenix.util.ByteUtil; +import org.apache.phoenix.util.ByteUtil; /** diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/SaltingUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/SaltingUtil.java similarity index 75% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/SaltingUtil.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/SaltingUtil.java index 44b69b9c..cb02b792 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/SaltingUtil.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/SaltingUtil.java @@ -1,43 +1,35 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; import java.util.Arrays; import java.util.Collections; import java.util.List; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.phoenix.compile.ScanRanges; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.query.KeyRange.Bound; +import org.apache.phoenix.util.ScanUtil; import com.google.common.collect.Lists; -import com.salesforce.phoenix.compile.ScanRanges; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.query.KeyRange.Bound; -import com.salesforce.phoenix.util.ScanUtil; /** @@ -49,7 +41,7 @@ public class SaltingUtil { public static final String SALTING_COLUMN_NAME = "_SALT"; public static final String SALTED_ROW_KEY_NAME = "_SALTED_KEY"; public static final PColumnImpl SALTING_COLUMN = new PColumnImpl( - PNameFactory.newName(SALTING_COLUMN_NAME), null, PDataType.BINARY, 1, 0, false, 0, null); + PNameFactory.newName(SALTING_COLUMN_NAME), null, PDataType.BINARY, 1, 0, false, 0, null, 0); public static List generateAllSaltingRanges(int bucketNum) { List allRanges = Lists.newArrayListWithExpectedSize(bucketNum); diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/Sequence.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/Sequence.java similarity index 95% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/Sequence.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/Sequence.java index a362aed3..f88475d9 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/Sequence.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/Sequence.java @@ -1,9 +1,9 @@ -package com.salesforce.phoenix.schema; +package org.apache.phoenix.schema; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.CACHE_SIZE_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.CURRENT_VALUE_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.INCREMENT_BY_BYTES; -import static com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData.SEQUENCE_FAMILY_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.CACHE_SIZE_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.CURRENT_VALUE_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.INCREMENT_BY_BYTES; +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.SEQUENCE_FAMILY_BYTES; import java.io.IOException; import java.sql.SQLException; @@ -21,15 +21,15 @@ import org.apache.hadoop.hbase.util.Bytes; import com.google.common.collect.Lists; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.coprocessor.SequenceRegionObserver; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.util.ByteUtil; -import com.salesforce.phoenix.util.KeyValueUtil; -import com.salesforce.phoenix.util.SchemaUtil; +import org.apache.phoenix.coprocessor.MetaDataProtocol; +import org.apache.phoenix.coprocessor.SequenceRegionObserver; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.util.ByteUtil; +import org.apache.phoenix.util.KeyValueUtil; +import org.apache.phoenix.util.SchemaUtil; public class Sequence { public static final int SUCCESS = 0; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/SequenceAlreadyExistsException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/SequenceAlreadyExistsException.java new file mode 100644 index 00000000..cdb7bdb3 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/SequenceAlreadyExistsException.java @@ -0,0 +1,48 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + + +public class SequenceAlreadyExistsException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.SEQUENCE_ALREADY_EXIST; + private final String schemaName; + private final String sequenceName; + + + public SequenceAlreadyExistsException(String schemaName, String sequenceName) { + super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(sequenceName).build().toString(), + code.getSQLState(), code.getErrorCode()); + this.schemaName = schemaName; + this.sequenceName = sequenceName; + } + + public String getSequenceName() { + return sequenceName; + } + + public String getSchemaName() { + return schemaName; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/SequenceKey.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/SequenceKey.java new file mode 100644 index 00000000..2be5acef --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/SequenceKey.java @@ -0,0 +1,76 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + + +public class SequenceKey implements Comparable { + @Override + public int hashCode() { + final int prime = 31; + int result = 1; + result = prime * result + ((tenantId == null) ? 0 : tenantId.hashCode()); + result = prime * result + ((schemaName == null) ? 0 : schemaName.hashCode()); + result = prime * result + sequenceName.hashCode(); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + SequenceKey other = (SequenceKey)obj; + return this.compareTo(other) == 0; + } + + private final String tenantId; + private final String schemaName; + private final String sequenceName; + + public SequenceKey(String tenantId, String schemaName, String sequenceName) { + this.tenantId = tenantId; + this.schemaName = schemaName; + this.sequenceName = sequenceName; + } + + public String getTenantId() { + return tenantId; + } + + public String getSchemaName() { + return schemaName; + } + + public String getSequenceName() { + return sequenceName; + } + + @Override + public int compareTo(SequenceKey that) { + int c = this.tenantId == that.getTenantId() ? 0 : this.tenantId == null ? -1 : that.getTenantId() == null ? 1 : this.tenantId.compareTo(that.getTenantId()); + if (c == 0) { + c = this.schemaName == that.getSchemaName() ? 0 : this.schemaName == null ? -1 : that.getSchemaName() == null ? 1 : this.schemaName.compareTo(that.getSchemaName()); + if (c == 0) { + return sequenceName.compareTo(that.getSequenceName()); + } + } + return c; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/SequenceNotFoundException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/SequenceNotFoundException.java new file mode 100644 index 00000000..466a3b97 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/SequenceNotFoundException.java @@ -0,0 +1,50 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + + +public class SequenceNotFoundException extends MetaDataEntityNotFoundException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.SEQUENCE_UNDEFINED; + private final String schemaName; + private final String tableName; + + public SequenceNotFoundException(String tableName) { + this(null, tableName); + } + + public SequenceNotFoundException(String schemaName, String tableName) { + super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).build().toString(), + code.getSQLState(), code.getErrorCode(), null); + this.tableName = tableName; + this.schemaName = schemaName; + } + + public String getTableName() { + return tableName; + } + + public String getSchemaName() { + return schemaName; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/TableAlreadyExistsException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/TableAlreadyExistsException.java new file mode 100644 index 00000000..7781ba22 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/TableAlreadyExistsException.java @@ -0,0 +1,55 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + + +/** + * + * Exception thrown when a table name could not be found in the schema + * + * @author jtaylor + * @since 0.1 + */ +public class TableAlreadyExistsException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.TABLE_ALREADY_EXIST; + private final String schemaName; + private final String tableName; + + public TableAlreadyExistsException(String schemaName, String tableName) { + super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).build().toString(), + code.getSQLState(), code.getErrorCode()); + this.schemaName = schemaName; + this.tableName = tableName; + } + + public String getTableName() { + return tableName; + } + + public String getSchemaName() { + return schemaName; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/TableNotFoundException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/TableNotFoundException.java new file mode 100644 index 00000000..7b66603c --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/TableNotFoundException.java @@ -0,0 +1,73 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import org.apache.hadoop.hbase.HConstants; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + + +/** + * + * Exception thrown when a table name could not be found in the schema + * + * @author jtaylor + * @since 0.1 + */ +public class TableNotFoundException extends MetaDataEntityNotFoundException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.TABLE_UNDEFINED; + private final String schemaName; + private final String tableName; + private final long timestamp; + + public TableNotFoundException(TableNotFoundException e, long timestamp) { + this(e.schemaName,e.tableName, timestamp); + } + + public TableNotFoundException(String tableName) { + this(null, tableName); + } + + public TableNotFoundException(String schemaName, String tableName) { + this(schemaName, tableName, HConstants.LATEST_TIMESTAMP); + } + + public TableNotFoundException(String schemaName, String tableName, long timestamp) { + super(new SQLExceptionInfo.Builder(code).setSchemaName(schemaName).setTableName(tableName).build().toString(), + code.getSQLState(), code.getErrorCode(), null); + this.tableName = tableName; + this.schemaName = schemaName; + this.timestamp = timestamp; + } + + public String getTableName() { + return tableName; + } + + public String getSchemaName() { + return schemaName; + } + + public long getTimeStamp() { + return timestamp; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/TableRef.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/TableRef.java new file mode 100644 index 00000000..14705cac --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/TableRef.java @@ -0,0 +1,82 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import org.apache.hadoop.hbase.HConstants; + + + +public final class TableRef { + private final PTable table; + private final String alias; + private final long timeStamp; + private final boolean hasDynamicCols; + + public TableRef(TableRef tableRef, long timeStamp) { + this(tableRef.alias, tableRef.table, timeStamp, tableRef.hasDynamicCols); + } + + public TableRef(PTable table) { + this(null, table, HConstants.LATEST_TIMESTAMP, false); + } + + public TableRef(String alias, PTable table, long timeStamp, boolean hasDynamicCols) { + this.alias = alias; + this.table = table; + this.timeStamp = timeStamp; + this.hasDynamicCols = hasDynamicCols; + } + + public PTable getTable() { + return table; + } + + public String getTableAlias() { + return alias; + } + + @Override + public int hashCode() { + final int prime = 31; + int result = alias == null ? 0 : alias.hashCode(); + result = prime * result + this.table.getName().getString().hashCode(); + return result; + } + + @Override + public boolean equals(Object obj) { + if (this == obj) return true; + if (obj == null) return false; + if (getClass() != obj.getClass()) return false; + TableRef other = (TableRef)obj; + if ((alias == null && other.alias != null) || (alias != null && !alias.equals(other.alias))) return false; + if (!table.getName().getString().equals(other.table.getName().getString())) return false; + return true; + } + + public long getTimeStamp() { + return timeStamp; + } + + public boolean hasDynamicCols() { + return hasDynamicCols; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/TypeMismatchException.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/TypeMismatchException.java new file mode 100644 index 00000000..ae248738 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/TypeMismatchException.java @@ -0,0 +1,60 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + +/** + * Exception thrown when we try to convert one type into a different incompatible type. + * + * @author zhuang + * @since 1.0 + */ +public class TypeMismatchException extends SQLException { + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.TYPE_MISMATCH; + + public TypeMismatchException(String msg) { + super(new SQLExceptionInfo.Builder(code).setMessage(msg).build().toString(), code.getSQLState(), code.getErrorCode()); + } + + public static TypeMismatchException newException(PDataType lhs) { + return new TypeMismatchException(getMessage(lhs,null,null)); + } + + public static TypeMismatchException newException(PDataType lhs, String location) { + return new TypeMismatchException(getMessage(lhs,null,location)); + } + + public static TypeMismatchException newException(PDataType lhs, PDataType rhs) { + return new TypeMismatchException(getMessage(lhs,rhs,null)); + } + + public static TypeMismatchException newException(PDataType lhs, PDataType rhs, String location) { + return new TypeMismatchException(getMessage(lhs,rhs,location)); + } + + public static String getMessage(PDataType lhs, PDataType rhs, String location) { + return lhs + (rhs == null ? "" : " and " + rhs) + (location == null ? "" : " for " + location); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ValueBitSet.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ValueBitSet.java similarity index 77% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/ValueBitSet.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/ValueBitSet.java index 61315bd0..2b1937ba 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ValueBitSet.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ValueBitSet.java @@ -1,38 +1,30 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; import java.util.Arrays; import org.apache.hadoop.hbase.io.ImmutableBytesWritable; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.phoenix.util.SizedUtil; +import org.apache.phoenix.util.SizedUtil; /** diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/ValueRangeExcpetion.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ValueRangeExcpetion.java new file mode 100644 index 00000000..aff2ef78 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ValueRangeExcpetion.java @@ -0,0 +1,40 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; + +import java.sql.SQLException; + +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; + +/** + * Exception thrown when we try to use use an argument that has the wrong type. + * + * @author anoopsjohn + * @since 1.1.2 + */ +public class ValueRangeExcpetion extends SQLException{ + private static final long serialVersionUID = 1L; + private static SQLExceptionCode code = SQLExceptionCode.VALUE_OUTSIDE_RANGE; + + public ValueRangeExcpetion(Object minValue, Object maxValue, Object actualValue, String location){ + super(new SQLExceptionInfo.Builder(code).setMessage("expected: [" + minValue + " , " + maxValue + "] but was: " + actualValue + " at " + location).build().toString(), code.getSQLState(), code.getErrorCode()); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ValueSchema.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/ValueSchema.java similarity index 85% rename from phoenix-core/src/main/java/com/salesforce/phoenix/schema/ValueSchema.java rename to phoenix-core/src/main/java/org/apache/phoenix/schema/ValueSchema.java index 88172195..41daf2a7 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/schema/ValueSchema.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/ValueSchema.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.schema; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema; import java.io.DataInput; import java.io.DataOutput; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/stat/PTableStats.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/stat/PTableStats.java new file mode 100644 index 00000000..5e38a275 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/stat/PTableStats.java @@ -0,0 +1,47 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema.stat; + +import java.io.DataOutput; +import java.io.IOException; + +import org.apache.hadoop.hbase.HRegionInfo; + + +/** + * Interface for Phoenix table statistics. Statistics is collected on the server + * side and can be used for various purpose like splitting region for scanning, etc. + * + * The table is defined on the client side, but it is populated on the server side. The client + * should not populate any data to the statistics object. + */ +public interface PTableStats { + + /** + * Given the region info, returns an array of bytes that is the current estimate of key + * distribution inside that region. The keys should split that region into equal chunks. + * + * @param region + * @return array of keys + */ + byte[][] getRegionGuidePosts(HRegionInfo region); + + void write(DataOutput output) throws IOException; +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/stat/PTableStatsImpl.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/stat/PTableStatsImpl.java new file mode 100644 index 00000000..27bb6650 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/stat/PTableStatsImpl.java @@ -0,0 +1,70 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema.stat; + +import java.io.DataOutput; +import java.io.IOException; +import java.util.Map; +import java.util.Map.Entry; + +import org.apache.hadoop.hbase.HRegionInfo; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.io.WritableUtils; + +import com.google.common.collect.ImmutableMap; + + +/** + * Implementation for PTableStats. + */ +public class PTableStatsImpl implements PTableStats { + + // The map for guide posts should be immutable. We only take the current snapshot from outside + // method call and store it. + private Map regionGuidePosts; + + public PTableStatsImpl() { } + + public PTableStatsImpl(Map stats) { + regionGuidePosts = ImmutableMap.copyOf(stats); + } + + @Override + public byte[][] getRegionGuidePosts(HRegionInfo region) { + return regionGuidePosts.get(region.getRegionNameAsString()); + } + + @Override + public void write(DataOutput output) throws IOException { + if (regionGuidePosts == null) { + WritableUtils.writeVInt(output, 0); + return; + } + WritableUtils.writeVInt(output, regionGuidePosts.size()); + for (Entry entry : regionGuidePosts.entrySet()) { + WritableUtils.writeString(output, entry.getKey()); + byte[][] value = entry.getValue(); + WritableUtils.writeVInt(output, value.length); + for (int i=0; i values; + + public MultiKeyValueTuple(List values) { + setKeyValues(values); + } + + public MultiKeyValueTuple() { + } + + public void setKeyValues(List values) { + this.values = ImmutableList.copyOf(values); + } + + @Override + public void getKey(ImmutableBytesWritable ptr) { + KeyValue value = values.get(0); + ptr.set(value.getBuffer(), value.getRowOffset(), value.getRowLength()); + } + + @Override + public boolean isImmutable() { + return true; + } + + @Override + public KeyValue getValue(byte[] family, byte[] qualifier) { + return KeyValueUtil.getColumnLatest(values, family, qualifier); + } + + @Override + public String toString() { + return values.toString(); + } + + @Override + public int size() { + return values.size(); + } + + @Override + public KeyValue getValue(int index) { + return values.get(index); + } + + @Override + public boolean getValue(byte[] family, byte[] qualifier, + ImmutableBytesWritable ptr) { + KeyValue kv = getValue(family, qualifier); + if (kv == null) + return false; + ptr.set(kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); + return true; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/tuple/ResultTuple.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/tuple/ResultTuple.java new file mode 100644 index 00000000..c99a9906 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/tuple/ResultTuple.java @@ -0,0 +1,104 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema.tuple; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.client.Result; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; + +import org.apache.phoenix.util.ResultUtil; + + +public class ResultTuple implements Tuple { + private Result result; + + public ResultTuple(Result result) { + this.result = result; + } + + public ResultTuple() { + } + + public Result getResult() { + return this.result; + } + + public void setResult(Result result) { + this.result = result; + } + + @Override + public void getKey(ImmutableBytesWritable ptr) { + ResultUtil.getKey(result, ptr); + } + + @Override + public boolean isImmutable() { + return true; + } + + @Override + public KeyValue getValue(byte[] family, byte[] qualifier) { + return result.getColumnLatest(family, qualifier); + } + + @Override + public String toString() { + StringBuilder sb = new StringBuilder(); + sb.append("keyvalues="); + if(this.result.isEmpty()) { + sb.append("NONE"); + return sb.toString(); + } + sb.append("{"); + boolean moreThanOne = false; + for(KeyValue kv : this.result.list()) { + if(moreThanOne) { + sb.append(", \n"); + } else { + moreThanOne = true; + } + sb.append(kv.toString()+"/value="+Bytes.toString(kv.getValue())); + } + sb.append("}\n"); + return sb.toString(); + } + + @Override + public int size() { + return result.size(); + } + + @Override + public KeyValue getValue(int index) { + return result.raw()[index]; + } + + @Override + public boolean getValue(byte[] family, byte[] qualifier, + ImmutableBytesWritable ptr) { + KeyValue kv = getValue(family, qualifier); + if (kv == null) + return false; + ptr.set(kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); + return true; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/tuple/SingleKeyValueTuple.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/tuple/SingleKeyValueTuple.java new file mode 100644 index 00000000..5d76604b --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/tuple/SingleKeyValueTuple.java @@ -0,0 +1,111 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema.tuple; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; + + +public class SingleKeyValueTuple implements Tuple { + private static final byte[] UNITIALIZED_KEY_BUFFER = new byte[0]; + private KeyValue keyValue; + private ImmutableBytesWritable keyPtr = new ImmutableBytesWritable(UNITIALIZED_KEY_BUFFER); + + public SingleKeyValueTuple() { + } + + public SingleKeyValueTuple(KeyValue keyValue) { + if (keyValue == null) { + throw new NullPointerException(); + } + setKeyValue(keyValue); + } + + public boolean hasKey() { + return keyPtr.get() != UNITIALIZED_KEY_BUFFER; + } + + public void reset() { + this.keyValue = null; + keyPtr.set(UNITIALIZED_KEY_BUFFER); + } + + public void setKeyValue(KeyValue keyValue) { + if (keyValue == null) { + throw new IllegalArgumentException(); + } + this.keyValue = keyValue; + setKey(keyValue); + } + + public void setKey(ImmutableBytesWritable ptr) { + keyPtr.set(ptr.get(), ptr.getOffset(), ptr.getLength()); + } + + public void setKey(KeyValue keyValue) { + if (keyValue == null) { + throw new IllegalArgumentException(); + } + keyPtr.set(keyValue.getBuffer(), keyValue.getRowOffset(), keyValue.getRowLength()); + } + + @Override + public void getKey(ImmutableBytesWritable ptr) { + ptr.set(keyPtr.get(), keyPtr.getOffset(), keyPtr.getLength()); + } + + @Override + public KeyValue getValue(byte[] cf, byte[] cq) { + return keyValue; + } + + @Override + public boolean isImmutable() { + return true; + } + + @Override + public String toString() { + return "SingleKeyValueTuple[" + keyValue == null ? keyPtr.get() == UNITIALIZED_KEY_BUFFER ? "null" : Bytes.toStringBinary(keyPtr.get(),keyPtr.getOffset(),keyPtr.getLength()) : keyValue.toString() + "]"; + } + + @Override + public int size() { + return keyValue == null ? 0 : 1; + } + + @Override + public KeyValue getValue(int index) { + if (index != 0 || keyValue == null) { + throw new IndexOutOfBoundsException(Integer.toString(index)); + } + return keyValue; + } + + @Override + public boolean getValue(byte[] family, byte[] qualifier, + ImmutableBytesWritable ptr) { + if (keyValue == null) + return false; + ptr.set(keyValue.getBuffer(), keyValue.getValueOffset(), keyValue.getValueLength()); + return true; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/schema/tuple/Tuple.java b/phoenix-core/src/main/java/org/apache/phoenix/schema/tuple/Tuple.java new file mode 100644 index 00000000..ffa3f72e --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/schema/tuple/Tuple.java @@ -0,0 +1,84 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.schema.tuple; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; + +/** + * + * Interface representing an ordered list of KeyValues returned as the + * result of a query. Each tuple represents a row (i.e. all its KeyValues + * will have the same key), and each KeyValue represents a column value. + * + * @author jtaylor + * @since 0.1 + */ +public interface Tuple { + /** + * @return Number of KeyValues contained by the Tuple. + */ + public int size(); + + /** + * Determines whether or not the Tuple is immutable (the typical case) + * or will potentially have additional KeyValues added to it (the case + * during filter evaluation when we see one KeyValue at a time). + * @return true if Tuple is immutable and false otherwise. + */ + public boolean isImmutable(); + + /** + * Get the row key for the Tuple + * @param ptr the bytes pointer that will be updated to point to + * the key buffer. + */ + public void getKey(ImmutableBytesWritable ptr); + + /** + * Get the KeyValue at the given index. + * @param index the zero-based KeyValue index between 0 and {@link #size()} exclusive + * @return the KeyValue at the given index + * @throws IndexOutOfBoundsException if an invalid index is used + */ + public KeyValue getValue(int index); + + /*** + * Get the KeyValue contained by the Tuple with the given family and + * qualifier name. + * @param family the column family of the KeyValue being retrieved + * @param qualifier the column qualify of the KeyValue being retrieved + * @return the KeyValue with the given family and qualifier name or + * null if not found. + */ + public KeyValue getValue(byte [] family, byte [] qualifier); + + /*** + * Get the value byte array of the KeyValue contained by the Tuple with + * the given family and qualifier name. + * @param family the column family of the KeyValue being retrieved + * @param qualifier the column qualify of the KeyValue being retrieved + * @param ptr the bytes pointer that will be updated to point to the + * value buffer. + * @return true if the KeyValue with the given family and qualifier name + * exists; otherwise false. + */ + public boolean getValue(byte [] family, byte [] qualifier, ImmutableBytesWritable ptr); +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/BigDecimalUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/BigDecimalUtil.java new file mode 100644 index 00000000..4b49f0fa --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/BigDecimalUtil.java @@ -0,0 +1,70 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import org.apache.hadoop.hbase.util.Pair; + +/** + * + * @author anoopsjohn + * @since 1.2.1 + */ +public class BigDecimalUtil { + + /** + * Calculates the precision and scale for BigDecimal arithmetic operation results. It uses the algorithm mentioned + * here + * @param lp precision of the left operand + * @param ls scale of the left operand + * @param rp precision of the right operand + * @param rs scale of the right operand + * @param op The operation type + * @return {@link Pair} comprising of the precision and scale. + */ + public static Pair getResultPrecisionScale(int lp, int ls, int rp, int rs, Operation op) { + int resultPrec = 0, resultScale = 0; + switch (op) { + case MULTIPLY: + resultPrec = lp + rp; + resultScale = ls + rs; + break; + case DIVIDE: + resultPrec = lp - ls + rp + Math.max(ls + rp - rs + 1, 4); + resultScale = 31 - lp + ls - rs; + break; + case ADD: + resultPrec = 2 * (lp - ls) + ls; // Is this correct? The page says addition -> 2 * (p - s) + s. + resultScale = Math.max(ls, rs); + break; + case AVG: + resultPrec = Math.max(lp - ls, rp - rs) + 1 + Math.max(ls, rs); + resultScale = Math.max(Math.max(ls, rs), 4); + break; + case OTHERS: + resultPrec = Math.max(lp - ls, rp - rs) + 1 + Math.max(ls, rs); + resultScale = Math.max(ls, rs); + } + return new Pair(resultPrec, resultScale); + } + + public static enum Operation { + MULTIPLY, DIVIDE, ADD, AVG, OTHERS; + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/BitSet.java b/phoenix-core/src/main/java/org/apache/phoenix/util/BitSet.java similarity index 98% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/BitSet.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/BitSet.java index 58221471..7cdbe6a1 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/BitSet.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/BitSet.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.util; +package org.apache.phoenix.util; import java.io.*; diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ByteUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/ByteUtil.java similarity index 90% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/ByteUtil.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/ByteUtil.java index 82e52b32..93c4df2c 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ByteUtil.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/ByteUtil.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; import java.io.ByteArrayInputStream; import java.io.DataInput; @@ -41,10 +33,10 @@ import org.apache.hadoop.hbase.util.Bytes; import org.apache.hadoop.io.WritableUtils; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.schema.ColumnModifier; -import com.salesforce.phoenix.schema.PDataType; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.PDataType; /** diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/CSVLoader.java b/phoenix-core/src/main/java/org/apache/phoenix/util/CSVLoader.java similarity index 80% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/CSVLoader.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/CSVLoader.java index 400cb9c7..508818d4 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/CSVLoader.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/CSVLoader.java @@ -1,38 +1,30 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; import au.com.bytecode.opencsv.CSVReader; import com.google.common.collect.Maps; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.jdbc.PhoenixConnection; -import com.salesforce.phoenix.schema.PDataType; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.schema.PDataType; import java.io.FileReader; import java.sql.DatabaseMetaData; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/Closeables.java b/phoenix-core/src/main/java/org/apache/phoenix/util/Closeables.java new file mode 100644 index 00000000..b09ebe0f --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/Closeables.java @@ -0,0 +1,124 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import java.io.Closeable; +import java.io.IOException; +import java.util.*; + +import com.google.common.collect.Iterables; + + +/** + * Utilities for operating on {@link Closeable}s. + * + */ +public class Closeables { + /** Not constructed */ + private Closeables() { } + + /** + * Allows you to close as many of the {@link Closeable}s as possible. + * + * If any of the close's fail with an IOException, those exception(s) will + * be thrown after attempting to close all of the inputs. + */ + public static void closeAll(Iterable iterable) throws IOException { + IOException ex = closeAllQuietly(iterable); + if (ex != null) throw ex; + } + + public static IOException closeAllQuietly(Iterable iterable) { + if (iterable == null) return null; + + LinkedList exceptions = null; + for (Closeable closeable : iterable) { + try { + closeable.close(); + } catch (IOException x) { + if (exceptions == null) exceptions = new LinkedList(); + exceptions.add(x); + } + } + + IOException ex = MultipleCausesIOException.fromIOExceptions(exceptions); + return ex; + } + + static private class MultipleCausesIOException extends IOException { + private static final long serialVersionUID = 1L; + + static IOException fromIOExceptions(Collection exceptions) { + if (exceptions == null || exceptions.isEmpty()) return null; + if (exceptions.size() == 1) return Iterables.getOnlyElement(exceptions); + + return new MultipleCausesIOException(exceptions); + } + + private final Collection exceptions; + private boolean hasSetStackTrace; + + /** + * Use the {@link #fromIOExceptions(Collection) factory}. + */ + private MultipleCausesIOException(Collection exceptions) { + this.exceptions = exceptions; + } + + @Override + public String getMessage() { + StringBuilder sb = new StringBuilder(this.exceptions.size() * 50); + int exceptionNum = 0; + for (IOException ex : this.exceptions) { + sb.append("Cause Number " + exceptionNum + ": " + ex.getMessage() + "\n"); + exceptionNum++; + } + return sb.toString(); + } + + @Override + public StackTraceElement[] getStackTrace() { + if (!this.hasSetStackTrace) { + ArrayList frames = new ArrayList(this.exceptions.size() * 20); + + int exceptionNum = 0; + for (IOException exception : this.exceptions) { + StackTraceElement header = new StackTraceElement(MultipleCausesIOException.class.getName(), + "Exception Number " + exceptionNum, + "", + 0); + + frames.add(header); + for (StackTraceElement ste : exception.getStackTrace()) { + frames.add(ste); + } + exceptionNum++; + } + + setStackTrace(frames.toArray(new StackTraceElement[frames.size()])); + this.hasSetStackTrace = true; + } + + return super.getStackTrace(); + } + + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ColumnInfo.java b/phoenix-core/src/main/java/org/apache/phoenix/util/ColumnInfo.java similarity index 91% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/ColumnInfo.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/ColumnInfo.java index edaae90c..9ca05e76 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ColumnInfo.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/ColumnInfo.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.util; +package org.apache.phoenix.util; /** * ColumnInfo used to store Column Name and its associated PDataType diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/DateUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/DateUtil.java new file mode 100644 index 00000000..b7460f8a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/DateUtil.java @@ -0,0 +1,154 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import java.math.BigDecimal; +import java.sql.Date; +import java.sql.Time; +import java.sql.Timestamp; +import java.text.Format; +import java.text.ParseException; +import java.text.SimpleDateFormat; +import java.util.TimeZone; + +import org.apache.commons.lang.time.FastDateFormat; + +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.IllegalDataException; + + + +@SuppressWarnings("serial") +public class DateUtil { + public static final TimeZone DATE_TIME_ZONE = TimeZone.getTimeZone("GMT"); + public static final String DEFAULT_DATE_FORMAT = "yyyy-MM-dd HH:mm:ss"; // This is the format the app sets in NLS settings for every connection. + public static final Format DEFAULT_DATE_FORMATTER = FastDateFormat.getInstance(DEFAULT_DATE_FORMAT, DATE_TIME_ZONE); + + public static final String DEFAULT_MS_DATE_FORMAT = "yyyy-MM-dd HH:mm:ss.SSS"; + public static final Format DEFAULT_MS_DATE_FORMATTER = FastDateFormat.getInstance(DEFAULT_MS_DATE_FORMAT, DATE_TIME_ZONE); + + private DateUtil() { + } + + public static Format getDateParser(String pattern) { + SimpleDateFormat format = new SimpleDateFormat(pattern) { + @Override + public java.util.Date parseObject(String source) throws ParseException { + java.util.Date date = super.parse(source); + return new java.sql.Date(date.getTime()); + } + }; + format.setTimeZone(DateUtil.DATE_TIME_ZONE); + return format; + } + + public static Format getTimeParser(String pattern) { + SimpleDateFormat format = new SimpleDateFormat(pattern) { + @Override + public java.util.Date parseObject(String source) throws ParseException { + java.util.Date date = super.parse(source); + return new java.sql.Time(date.getTime()); + } + }; + format.setTimeZone(DateUtil.DATE_TIME_ZONE); + return format; + } + + public static Format getTimestampParser(String pattern) { + SimpleDateFormat format = new SimpleDateFormat(pattern) { + @Override + public java.util.Date parseObject(String source) throws ParseException { + java.util.Date date = super.parse(source); + return new java.sql.Timestamp(date.getTime()); + } + }; + format.setTimeZone(DateUtil.DATE_TIME_ZONE); + return format; + } + + public static Format getDateFormatter(String pattern) { + return DateUtil.DEFAULT_DATE_FORMAT.equals(pattern) ? DateUtil.DEFAULT_DATE_FORMATTER : FastDateFormat.getInstance(pattern, DateUtil.DATE_TIME_ZONE); + } + + private static ThreadLocal dateFormat = + new ThreadLocal < Format > () { + @Override protected Format initialValue() { + return getDateParser(DEFAULT_DATE_FORMAT); + } + }; + + public static Date parseDate(String dateValue) { + try { + return (Date)dateFormat.get().parseObject(dateValue); + } catch (ParseException e) { + throw new IllegalDataException(e); + } + } + + private static ThreadLocal timeFormat = + new ThreadLocal < Format > () { + @Override protected Format initialValue() { + return getTimeParser(DEFAULT_DATE_FORMAT); + } + }; + + public static Time parseTime(String timeValue) { + try { + return (Time)timeFormat.get().parseObject(timeValue); + } catch (ParseException e) { + throw new IllegalDataException(e); + } + } + + private static ThreadLocal timestampFormat = + new ThreadLocal < Format > () { + @Override protected Format initialValue() { + return getTimestampParser(DEFAULT_DATE_FORMAT); + } + }; + + public static Timestamp parseTimestamp(String timeValue) { + try { + return (Timestamp)timestampFormat.get().parseObject(timeValue); + } catch (ParseException e) { + throw new IllegalDataException(e); + } + } + + /** + * Utility function to work around the weirdness of the {@link Timestamp} constructor. + * This method takes the milli-seconds that spills over to the nanos part as part of + * constructing the {@link Timestamp} object. + * If we just set the nanos part of timestamp to the nanos passed in param, we + * end up losing the sub-second part of timestamp. + */ + public static Timestamp getTimestamp(long millis, int nanos) { + Timestamp ts = new Timestamp(millis); + ts.setNanos(ts.getNanos() + nanos); + return ts; + } + + /** + * Utility function to convert a {@link BigDecimal} value to {@link Timestamp}. + */ + public static Timestamp getTimestamp(BigDecimal bd) { + return DateUtil.getTimestamp(bd.longValue(), ((bd.remainder(BigDecimal.ONE).multiply(BigDecimal.valueOf(QueryConstants.MILLIS_TO_NANOS_CONVERTOR))).intValue())); + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/IndexUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/IndexUtil.java similarity index 76% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/IndexUtil.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/IndexUtil.java index 0c7291c1..a7c975ce 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/IndexUtil.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/IndexUtil.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; import java.io.IOException; import java.sql.SQLException; @@ -39,20 +31,20 @@ import org.apache.hadoop.hbase.util.Bytes; import com.google.common.collect.Lists; -import com.salesforce.hbase.index.ValueGetter; -import com.salesforce.hbase.index.covered.update.ColumnReference; -import com.salesforce.hbase.index.util.ImmutableBytesPtr; -import com.salesforce.phoenix.client.KeyValueBuilder; -import com.salesforce.phoenix.exception.SQLExceptionCode; -import com.salesforce.phoenix.exception.SQLExceptionInfo; -import com.salesforce.phoenix.index.IndexMaintainer; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.ColumnFamilyNotFoundException; -import com.salesforce.phoenix.schema.ColumnNotFoundException; -import com.salesforce.phoenix.schema.PColumn; -import com.salesforce.phoenix.schema.PColumnFamily; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PTable; +import org.apache.hadoop.hbase.index.ValueGetter; +import org.apache.hadoop.hbase.index.covered.update.ColumnReference; +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.client.KeyValueBuilder; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.index.IndexMaintainer; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.ColumnFamilyNotFoundException; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PColumnFamily; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PTable; public class IndexUtil { public static final String INDEX_COLUMN_NAME_SEP = ":"; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/InstanceResolver.java b/phoenix-core/src/main/java/org/apache/phoenix/util/InstanceResolver.java new file mode 100644 index 00000000..85cf54d0 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/InstanceResolver.java @@ -0,0 +1,66 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import java.util.ServiceLoader; +import java.util.concurrent.ConcurrentHashMap; + +/** + * Resolves object instances registered using the JDK 6+ {@link java.util.ServiceLoader}. + * + * @author aaraujo + * @since 2.0 + */ +public class InstanceResolver { + private static final ConcurrentHashMap RESOLVED_SINGLETONS = new ConcurrentHashMap(); + + private InstanceResolver() {/* not allowed */} + + /** + * Resolves an instance of the specified class if it has not already been resolved. + * @param clazz The type of instance to resolve + * @param defaultInstance The instance to use if a custom instance has not been registered + * @return The resolved instance or the default instance provided. + * {@code null} if an instance is not registered and a default is not provided. + */ + @SuppressWarnings("unchecked") + public static T getSingleton(Class clazz, T defaultInstance) { + Object obj = RESOLVED_SINGLETONS.get(clazz); + if(obj != null) { + return (T)obj; + } + if (defaultInstance != null && !clazz.isInstance(defaultInstance)) throw new IllegalArgumentException("defaultInstance is not of type " + clazz.getName()); + final Object o = resolveSingleton(clazz, defaultInstance); + obj = RESOLVED_SINGLETONS.putIfAbsent(clazz, o); + if(obj == null) { + obj = o; + } + return (T)obj; + } + + private synchronized static T resolveSingleton(Class clazz, T defaultInstance) { + ServiceLoader loader = ServiceLoader.load(clazz); + // returns the first registered instance found + for (T singleton : loader) { + return singleton; + } + return defaultInstance; + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/JDBCUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/JDBCUtil.java new file mode 100644 index 00000000..eb973cc7 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/JDBCUtil.java @@ -0,0 +1,83 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import java.sql.SQLException; +import java.util.Properties; + +import javax.annotation.Nullable; + +import org.apache.phoenix.query.QueryServices; +import org.apache.phoenix.query.QueryServicesOptions; +import org.apache.phoenix.schema.PName; +import org.apache.phoenix.schema.PNameFactory; + + + +/** + * Utilities for JDBC + * + * @author jtaylor + * @since 178 + */ +public class JDBCUtil { + + private JDBCUtil() { + } + + /** + * Find the propName by first looking in the url string and if not found, + * next in the info properties. If not found, null is returned. + * @param url JDBC connection URL + * @param info JDBC connection properties + * @param propName the name of the property to find + * @return the property value or null if not found + */ + public static String findProperty(String url, Properties info, String propName) { + String urlPropName = ";" + propName + "="; + String propValue = info.getProperty(propName); + if (propValue == null) { + int begIndex = url.indexOf(urlPropName); + if (begIndex >= 0) { + int endIndex = url.indexOf(';',begIndex + urlPropName.length()); + if (endIndex < 0) { + endIndex = url.length(); + } + propValue = url.substring(begIndex + urlPropName.length(), endIndex); + } + } + return propValue; + } + + public static Long getCurrentSCN(String url, Properties info) throws SQLException { + String scnStr = findProperty(url, info, PhoenixRuntime.CURRENT_SCN_ATTRIB); + return (scnStr == null ? null : Long.parseLong(scnStr)); + } + + public static int getMutateBatchSize(String url, Properties info, ReadOnlyProps props) throws SQLException { + String batchSizeStr = findProperty(url, info, PhoenixRuntime.UPSERT_BATCH_SIZE_ATTRIB); + return (batchSizeStr == null ? props.getInt(QueryServices.MUTATE_BATCH_SIZE_ATTRIB, QueryServicesOptions.DEFAULT_MUTATE_BATCH_SIZE) : Integer.parseInt(batchSizeStr)); + } + + public static @Nullable PName getTenantId(String url, Properties info) throws SQLException { + String tenantId = findProperty(url, info, PhoenixRuntime.TENANT_ID_ATTRIB); + return (tenantId == null ? null : PNameFactory.newName(tenantId)); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/KeyValueUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/KeyValueUtil.java new file mode 100644 index 00000000..657845da --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/KeyValueUtil.java @@ -0,0 +1,170 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import java.util.Collections; +import java.util.Comparator; +import java.util.List; + +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.KeyValue.Type; +import org.apache.hadoop.hbase.client.Result; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; + +/** + * + * Utilities for KeyValue. Where there's duplication with KeyValue methods, + * these avoid creating new objects when not necessary (primary preventing + * byte array copying). + * + * @author jtaylor + * @since 0.1 + */ +public class KeyValueUtil { + private KeyValueUtil() { + } + + public static KeyValue newKeyValue(Result r, byte[] cf, byte[] cq, long ts, byte[] value, int valueOffset, int valueLength) { + byte[] bytes = ResultUtil.getRawBytes(r); + return new KeyValue(bytes, ResultUtil.getKeyOffset(r), ResultUtil.getKeyLength(r), + cf, 0, cf.length, + cq, 0, cq.length, + ts, Type.Put, + value, valueOffset, valueLength); + } + + public static KeyValue newKeyValue(byte[] key, byte[] cf, byte[] cq, long ts, byte[] value, int valueOffset, int valueLength) { + return new KeyValue(key, 0, key.length, + cf, 0, cf.length, + cq, 0, cq.length, + ts, Type.Put, + value, valueOffset, valueLength); + } + + public static KeyValue newKeyValue(ImmutableBytesWritable key, byte[] cf, byte[] cq, long ts, byte[] value, int valueOffset, int valueLength) { + return new KeyValue(key.get(), key.getOffset(), key.getLength(), + cf, 0, cf.length, + cq, 0, cq.length, + ts, Type.Put, + value, valueOffset, valueLength); + } + + public static KeyValue newKeyValue(byte[] key, int keyOffset, int keyLength, byte[] cf, byte[] cq, long ts, byte[] value, int valueOffset, int valueLength) { + return new KeyValue(key, keyOffset, keyLength, + cf, 0, cf.length, + cq, 0, cq.length, + ts, Type.Put, + value, valueOffset, valueLength); + } + + public static KeyValue newKeyValue(byte[] key, byte[] cf, byte[] cq, long ts, byte[] value) { + return newKeyValue(key,cf,cq,ts,value,0,value.length); + } + + public static KeyValue newKeyValue(Result r, byte[] cf, byte[] cq, long ts, byte[] value) { + return newKeyValue(r,cf,cq,ts,value,0,value.length); + } + + /** + * Binary search for latest column value without allocating memory in the process + * @param kvs + * @param family + * @param qualifier + */ + public static KeyValue getColumnLatest(Listkvs, byte[] family, byte[] qualifier) { + if (kvs.size() == 0) { + return null; + } + KeyValue row = kvs.get(0); + Comparator comp = new SearchComparator(row.getBuffer(), row.getRowOffset(), row.getRowLength(), family, qualifier); + // pos === ( -(insertion point) - 1) + int pos = Collections.binarySearch(kvs, null, comp); + // never will exact match + if (pos < 0) { + pos = (pos+1) * -1; + // pos is now insertion point + } + if (pos == kvs.size()) { + return null; // doesn't exist + } + + KeyValue kv = kvs.get(pos); + if (Bytes.compareTo(kv.getBuffer(), kv.getFamilyOffset(), kv.getFamilyLength(), + family, 0, family.length) != 0) { + return null; + } + if (Bytes.compareTo(kv.getBuffer(), kv.getQualifierOffset(), kv.getQualifierLength(), + qualifier, 0, qualifier.length) != 0) { + return null; + } + return kv; + } + + /* + * Special comparator, *only* works for binary search. + * Current JDKs only uses the search term on the right side, + * Making use of that saves instanceof checks, and allows us + * to inline the search term in the comparator + */ + private static class SearchComparator implements Comparator { + private final byte[] row; + private final byte[] family; + private final byte[] qualifier; + private final int rowOff; + private final int rowLen; + + public SearchComparator(byte[] r, int rOff, int rLen, byte[] f, byte[] q) { + row = r; + family = f; + qualifier = q; + rowOff = rOff; + rowLen = rLen; + } + + @Override + public int compare(final KeyValue l, final KeyValue ignored) { + assert ignored == null; + final byte[] buf = l.getBuffer(); + final int rOff = l.getRowOffset(); + final short rLen = l.getRowLength(); + // row + int val = Bytes.compareTo(buf, rOff, rLen, row, rowOff, rowLen); + if (val != 0) { + return val; + } + // family + final int fOff = l.getFamilyOffset(rLen); + final byte fLen = l.getFamilyLength(fOff); + val = Bytes.compareTo(buf, fOff, fLen, family, 0, family.length); + if (val != 0) { + return val; + } + // qualifier + val = Bytes.compareTo(buf, l.getQualifierOffset(fOff), + l.getQualifierLength(rLen, fLen), qualifier, 0, qualifier.length); + if (val != 0) { + return val; + } + // want latest TS and type, so we get the first + return 1; + } + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/MetaDataUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/MetaDataUtil.java similarity index 78% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/MetaDataUtil.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/MetaDataUtil.java index 4f69a902..0d8a60a3 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/MetaDataUtil.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/MetaDataUtil.java @@ -1,33 +1,25 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; - -import static com.salesforce.phoenix.util.SchemaUtil.getVarChars; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import static org.apache.phoenix.util.SchemaUtil.getVarChars; import java.util.Collection; import java.util.List; @@ -36,12 +28,11 @@ import org.apache.hadoop.hbase.client.Mutation; import org.apache.hadoop.hbase.client.Put; import org.apache.hadoop.hbase.util.Bytes; - -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.jdbc.PhoenixDatabaseMetaData; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PTableType; +import org.apache.phoenix.coprocessor.MetaDataProtocol; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PTableType; public class MetaDataUtil { @@ -171,14 +162,6 @@ public static long getSequenceNumber(Mutation tableMutation) { throw new IllegalStateException(); } - public static byte[] getBaseTableName(List tableMutations) { - return getMutationKVByteValue(getPutOnlyTableHeaderRow(tableMutations), PhoenixDatabaseMetaData.BASE_TABLE_NAME_BYTES); - } - - public static byte[] getBaseSchemaName(List tableMutations) { - return getMutationKVByteValue(getPutOnlyTableHeaderRow(tableMutations), PhoenixDatabaseMetaData.BASE_SCHEMA_NAME_BYTES); - } - public static long getSequenceNumber(List tableMetaData) { return getSequenceNumber(getTableHeaderRow(tableMetaData)); } @@ -196,7 +179,7 @@ public static Mutation getTableHeaderRow(List tableMetaData) { return tableMetaData.get(0); } - private static byte[] getMutationKVByteValue(Mutation headerRow, byte[] key) { + public static byte[] getMutationKVByteValue(Mutation headerRow, byte[] key) { KeyValue kv = getMutationKeyValue(headerRow, key); // FIXME: byte copy return kv == null ? ByteUtil.EMPTY_BYTE_ARRAY : kv.getValue(); diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/NumberUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/NumberUtil.java new file mode 100644 index 00000000..ece1104a --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/NumberUtil.java @@ -0,0 +1,54 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import java.math.BigDecimal; + +import org.apache.phoenix.schema.PDataType; + +/** + * Utility methods for numbers like decimal, long, etc. + * + * @author elevine + * @since 0.1 + */ +public class NumberUtil { + + public static final String DEFAULT_NUMBER_FORMAT = "#,##0.###"; + + /** + * Strip all trailing zeros to ensure that no digit will be zero and + * round using our default context to ensure precision doesn't exceed max allowed. + * @return new {@link BigDecimal} instance + */ + public static BigDecimal normalize(BigDecimal bigDecimal) { + return bigDecimal.stripTrailingZeros().round(PDataType.DEFAULT_MATH_CONTEXT); + } + + public static BigDecimal setDecimalWidthAndScale(BigDecimal decimal, int precision, int scale) { + // If we could not fit all the digits before decimal point into the new desired precision and + // scale, return null and the caller method should handle the error. + if (((precision - scale) < (decimal.precision() - decimal.scale()))){ + return null; + } + decimal = decimal.setScale(scale, BigDecimal.ROUND_DOWN); + return decimal; + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/PhoenixRuntime.java b/phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixRuntime.java similarity index 86% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/PhoenixRuntime.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixRuntime.java index 07283bd3..102a0f3e 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/PhoenixRuntime.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixRuntime.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; import java.io.File; import java.io.FileReader; @@ -46,7 +38,7 @@ import org.apache.hadoop.hbase.util.Pair; import com.google.common.collect.Lists; -import com.salesforce.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixConnection; /** * @@ -87,7 +79,7 @@ public class PhoenixRuntime { * Use this connection property to help with fairness of resource allocation * for the client and server. The value of the attribute determines the * bucket used to rollup resource usage for a particular tenant/organization. Each tenant - * may only use a percentage of total resources, governed by the {@link com.salesforce.phoenix.query.QueryServices} + * may only use a percentage of total resources, governed by the {@link org.apache.phoenix.query.QueryServices} * configuration properties */ public static final String TENANT_ID_ATTRIB = "TenantId"; @@ -232,7 +224,7 @@ private PhoenixRuntime() { /** * Runs a series of semicolon-terminated SQL statements using the connection provided, returning * the number of SQL statements executed. Note that if the connection has specified an SCN through - * the {@link com.salesforce.phoenix.util.PhoenixRuntime#CURRENT_SCN_ATTRIB} connection property, then the timestamp + * the {@link org.apache.phoenix.util.PhoenixRuntime#CURRENT_SCN_ATTRIB} connection property, then the timestamp * is bumped up by one after each statement execution. * @param conn an open JDBC connection * @param reader a reader for semicolumn separated SQL statements diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/QueryUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/QueryUtil.java similarity index 98% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/QueryUtil.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/QueryUtil.java index 0c18fec8..bb0c9cc4 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/QueryUtil.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/QueryUtil.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.util; +package org.apache.phoenix.util; import java.sql.DatabaseMetaData; import java.sql.ResultSet; diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ReadOnlyProps.java b/phoenix-core/src/main/java/org/apache/phoenix/util/ReadOnlyProps.java similarity index 99% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/ReadOnlyProps.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/ReadOnlyProps.java index cd047f8a..6fcbc3d6 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ReadOnlyProps.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/ReadOnlyProps.java @@ -1,4 +1,4 @@ -package com.salesforce.phoenix.util; +package org.apache.phoenix.util; import java.util.*; import java.util.Map.Entry; diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ResultUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/ResultUtil.java similarity index 77% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/ResultUtil.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/ResultUtil.java index 1427daa8..8339cc0a 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ResultUtil.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/ResultUtil.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; import java.util.Arrays; import java.util.Comparator; diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/SQLCloseable.java b/phoenix-core/src/main/java/org/apache/phoenix/util/SQLCloseable.java new file mode 100644 index 00000000..11cf02f6 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/SQLCloseable.java @@ -0,0 +1,34 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import java.sql.SQLException; + +/** + * + * Interface for a SQL resource that should be closed + * after it is no longer in use. + * + * @author jtaylor + * @since 0.1 + */ +public interface SQLCloseable { + void close() throws SQLException; +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/SQLCloseables.java b/phoenix-core/src/main/java/org/apache/phoenix/util/SQLCloseables.java new file mode 100644 index 00000000..c255c1da --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/SQLCloseables.java @@ -0,0 +1,133 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import java.sql.SQLException; +import java.util.*; + +import com.google.common.collect.Iterables; + + +/** + * Utilities for operating on {@link SQLCloseable}s. + * + * @author jtaylor + * @since 0.1 + */ +public class SQLCloseables { + /** Not constructed */ + private SQLCloseables() { } + + /** + * Allows you to close as many of the {@link SQLCloseable}s as possible. + * + * If any of the close's fail with an IOException, those exception(s) will + * be thrown after attempting to close all of the inputs. + */ + public static void closeAll(Iterable iterable) throws SQLException { + SQLException ex = closeAllQuietly(iterable); + if (ex != null) throw ex; + } + + public static SQLException closeAllQuietly(Iterable iterable) { + if (iterable == null) return null; + + LinkedList exceptions = null; + for (SQLCloseable closeable : iterable) { + try { + closeable.close(); + } catch (SQLException x) { + if (exceptions == null) exceptions = new LinkedList(); + exceptions.add(x); + } + } + + SQLException ex = MultipleCausesSQLException.fromSQLExceptions(exceptions); + return ex; + } + + /** + * A subclass of {@link SQLException} that allows you to chain multiple + * causes together. + * + * @author jtaylor + * @since 0.1 + * @see SQLCloseables + */ + static private class MultipleCausesSQLException extends SQLException { + private static final long serialVersionUID = 1L; + + static SQLException fromSQLExceptions(Collection exceptions) { + if (exceptions == null || exceptions.isEmpty()) return null; + if (exceptions.size() == 1) return Iterables.getOnlyElement(exceptions); + + return new MultipleCausesSQLException(exceptions); + } + + private final Collection exceptions; + private boolean hasSetStackTrace; + + /** + * Use the {@link #fromIOExceptions(Collection) factory}. + */ + private MultipleCausesSQLException(Collection exceptions) { + this.exceptions = exceptions; + } + + @Override + public String getMessage() { + StringBuilder sb = new StringBuilder(this.exceptions.size() * 50); + int exceptionNum = 0; + for (SQLException ex : this.exceptions) { + sb.append("Cause Number " + exceptionNum + ": " + ex.getMessage() + "\n"); + exceptionNum++; + } + return sb.toString(); + } + + @Override + public StackTraceElement[] getStackTrace() { + if (!this.hasSetStackTrace) { + ArrayList frames = new ArrayList(this.exceptions.size() * 20); + + int exceptionNum = 0; + for (SQLException exception : this.exceptions) { + StackTraceElement header = new StackTraceElement(MultipleCausesSQLException.class.getName(), + "Exception Number " + exceptionNum, + "", + 0); + + frames.add(header); + for (StackTraceElement ste : exception.getStackTrace()) { + frames.add(ste); + } + exceptionNum++; + } + + setStackTrace(frames.toArray(new StackTraceElement[frames.size()])); + this.hasSetStackTrace = true; + } + + return super.getStackTrace(); + } + + } + +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ScanUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/ScanUtil.java similarity index 88% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/ScanUtil.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/ScanUtil.java index 5d59cb70..4ea8a114 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/ScanUtil.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/ScanUtil.java @@ -1,31 +1,23 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; import java.io.IOException; import java.sql.SQLException; @@ -45,15 +37,15 @@ import org.apache.hadoop.hbase.util.Bytes; import com.google.common.collect.Lists; -import com.salesforce.phoenix.compile.ScanRanges; -import com.salesforce.phoenix.coprocessor.MetaDataProtocol; -import com.salesforce.phoenix.filter.SkipScanFilter; -import com.salesforce.phoenix.query.KeyRange; -import com.salesforce.phoenix.query.KeyRange.Bound; -import com.salesforce.phoenix.query.QueryConstants; -import com.salesforce.phoenix.schema.PDataType; -import com.salesforce.phoenix.schema.PTable; -import com.salesforce.phoenix.schema.RowKeySchema; +import org.apache.phoenix.compile.ScanRanges; +import org.apache.phoenix.coprocessor.MetaDataProtocol; +import org.apache.phoenix.filter.SkipScanFilter; +import org.apache.phoenix.query.KeyRange; +import org.apache.phoenix.query.KeyRange.Bound; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.RowKeySchema; /** @@ -430,4 +422,26 @@ public static byte[] nextKey(byte[] key, PTable table, ImmutableBytesWritable pt ByteUtil.nextKey(key, key.length); return key; } -} + + private static final String REVERSED_ATTR = "_reversed_"; + + public static void setReversed(Scan scan) { + // TODO: set attribute dynamically here to prevent dependency on newer HBase release + scan.setAttribute(REVERSED_ATTR, PDataType.TRUE_BYTES); + } + + // Start/stop row must be swapped if scan is being done in reverse + public static void swapStartStopRowIfReversed(Scan scan) { + if (isReversed(scan)) { + byte[] startRow = scan.getStartRow(); + byte[] stopRow = scan.getStopRow(); + scan.setStartRow(stopRow); + scan.setStopRow(startRow); + } + } + + public static boolean isReversed(Scan scan) { + byte[] reversed = scan.getAttribute(REVERSED_ATTR); + return (PDataType.TRUE_BYTES.equals(reversed)); + } +} \ No newline at end of file diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/SchemaUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/SchemaUtil.java new file mode 100644 index 00000000..7b070f74 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/SchemaUtil.java @@ -0,0 +1,507 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import static org.apache.phoenix.jdbc.PhoenixDatabaseMetaData.TYPE_TABLE_NAME_BYTES; + +import java.sql.SQLException; +import java.sql.Statement; +import java.util.ArrayList; +import java.util.Iterator; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Properties; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.phoenix.exception.SQLExceptionCode; +import org.apache.phoenix.exception.SQLExceptionInfo; +import org.apache.phoenix.jdbc.PhoenixConnection; +import org.apache.phoenix.jdbc.PhoenixDatabaseMetaData; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.AmbiguousColumnException; +import org.apache.phoenix.schema.ColumnFamilyNotFoundException; +import org.apache.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.ColumnNotFoundException; +import org.apache.phoenix.schema.PColumn; +import org.apache.phoenix.schema.PColumnFamily; +import org.apache.phoenix.schema.PDataType; +import org.apache.phoenix.schema.PDatum; +import org.apache.phoenix.schema.PMetaData; +import org.apache.phoenix.schema.PName; +import org.apache.phoenix.schema.PTable; +import org.apache.phoenix.schema.RowKeySchema; +import org.apache.phoenix.schema.RowKeySchema.RowKeySchemaBuilder; +import org.apache.phoenix.schema.SaltingUtil; + + + +/** + * + * Static class for various schema-related utilities + * + * @author jtaylor + * @since 0.1 + */ +public class SchemaUtil { + private static final int VAR_LENGTH_ESTIMATE = 10; + + public static final DataBlockEncoding DEFAULT_DATA_BLOCK_ENCODING = DataBlockEncoding.FAST_DIFF; + public static final RowKeySchema VAR_BINARY_SCHEMA = new RowKeySchemaBuilder(1).addField(new PDatum() { + + @Override + public boolean isNullable() { + return false; + } + + @Override + public PDataType getDataType() { + return PDataType.VARBINARY; + } + + @Override + public Integer getByteSize() { + return null; + } + + @Override + public Integer getMaxLength() { + return null; + } + + @Override + public Integer getScale() { + return null; + } + + @Override + public ColumnModifier getColumnModifier() { + return null; + } + + }, false, null).build(); + + /** + * May not be instantiated + */ + private SchemaUtil() { + } + + public static boolean isPKColumn(PColumn column) { + return column.getFamilyName() == null; + } + + /** + * Estimate the max key length in bytes of the PK for a given table + * @param table the table + * @return the max PK length + */ + public static int estimateKeyLength(PTable table) { + int maxKeyLength = 0; + // Calculate the max length of a key (each part must currently be of a fixed width) + int i = 0; + List columns = table.getPKColumns(); + while (i < columns.size()) { + PColumn keyColumn = columns.get(i++); + Integer byteSize = keyColumn.getByteSize(); + maxKeyLength += (byteSize == null) ? VAR_LENGTH_ESTIMATE : byteSize; + } + return maxKeyLength; + } + + /** + * Normalize an identifier. If name is surrounded by double quotes, + * it is used as-is, otherwise the name is upper caased. + * @param name the parsed identifier + * @return the normalized identifier + */ + public static String normalizeIdentifier(String name) { + if (name == null) { + return name; + } + if (isCaseSensitive(name)) { + // Don't upper case if in quotes + return name.substring(1, name.length()-1); + } + return name.toUpperCase(); + } + + public static boolean isCaseSensitive(String name) { + return name.length() > 0 && name.charAt(0)=='"'; + } + + public static List concat(List l1, List l2) { + int size1 = l1.size(); + if (size1 == 0) { + return l2; + } + int size2 = l2.size(); + if (size2 == 0) { + return l1; + } + List l3 = new ArrayList(size1 + size2); + l3.addAll(l1); + l3.addAll(l2); + return l3; + } + + public static byte[] getSequenceKey(byte[] tenantId, byte[] schemaName, byte[] sequenceName) { + return getTableKey(tenantId, schemaName, sequenceName); + } + + public static byte[] getSequenceKey(String tenantId, String schemaName, String sequenceName) { + return getTableKey(tenantId, schemaName, sequenceName); + } + + /** + * Get the key used in the Phoenix metadata row for a table definition + * @param schemaName + * @param tableName + */ + public static byte[] getTableKey(byte[] tenantId, byte[] schemaName, byte[] tableName) { + return ByteUtil.concat(tenantId, QueryConstants.SEPARATOR_BYTE_ARRAY, schemaName, QueryConstants.SEPARATOR_BYTE_ARRAY, tableName); + } + + public static byte[] getTableKey(String tenantId, String schemaName, String tableName) { + return ByteUtil.concat(tenantId == null ? ByteUtil.EMPTY_BYTE_ARRAY : Bytes.toBytes(tenantId), QueryConstants.SEPARATOR_BYTE_ARRAY, schemaName == null ? ByteUtil.EMPTY_BYTE_ARRAY : Bytes.toBytes(schemaName), QueryConstants.SEPARATOR_BYTE_ARRAY, Bytes.toBytes(tableName)); + } + + public static String getTableName(String schemaName, String tableName) { + return getName(schemaName,tableName); + } + + private static String getName(String optionalQualifier, String name) { + if (optionalQualifier == null || optionalQualifier.isEmpty()) { + return name; + } + return optionalQualifier + QueryConstants.NAME_SEPARATOR + name; + } + + public static String getTableName(byte[] schemaName, byte[] tableName) { + return getName(schemaName, tableName); + } + + public static String getColumnDisplayName(byte[] cf, byte[] cq) { + return getName(cf == null || cf.length == 0 || Bytes.compareTo(cf, QueryConstants.DEFAULT_COLUMN_FAMILY_BYTES) == 0 ? ByteUtil.EMPTY_BYTE_ARRAY : cf, cq); + } + + public static String getColumnDisplayName(String cf, String cq) { + return getName(cf == null || cf.isEmpty() || QueryConstants.DEFAULT_COLUMN_FAMILY.equals(cf) ? null : cf, cq); + } + + public static String getMetaDataEntityName(String schemaName, String tableName, String familyName, String columnName) { + if ((schemaName == null || schemaName.isEmpty()) && (tableName == null || tableName.isEmpty())) { + return getName(familyName, columnName); + } + if ((familyName == null || familyName.isEmpty()) && (columnName == null || columnName.isEmpty())) { + return getName(schemaName, tableName); + } + return getName(getName(schemaName, tableName), getName(familyName, columnName)); + } + + public static String getColumnName(String familyName, String columnName) { + return getName(familyName, columnName); + } + + public static byte[] getTableNameAsBytes(String schemaName, String tableName) { + if (schemaName == null || schemaName.length() == 0) { + return StringUtil.toBytes(tableName); + } + return getTableNameAsBytes(StringUtil.toBytes(schemaName),StringUtil.toBytes(tableName)); + } + + public static byte[] getTableNameAsBytes(byte[] schemaName, byte[] tableName) { + return getNameAsBytes(schemaName, tableName); + } + + private static byte[] getNameAsBytes(byte[] nameOne, byte[] nameTwo) { + if (nameOne == null || nameOne.length == 0) { + return nameTwo; + } else if ((nameTwo == null || nameTwo.length == 0)) { + return nameOne; + } else { + return ByteUtil.concat(nameOne, QueryConstants.NAME_SEPARATOR_BYTES, nameTwo); + } + } + + public static String getName(byte[] nameOne, byte[] nameTwo) { + return Bytes.toString(getNameAsBytes(nameOne,nameTwo)); + } + + public static int getVarCharLength(byte[] buf, int keyOffset, int maxLength) { + return getVarCharLength(buf, keyOffset, maxLength, 1); + } + + public static int getVarCharLength(byte[] buf, int keyOffset, int maxLength, int skipCount) { + int length = 0; + for (int i=0; i 0; i++) { + int length = getVarCharLength(rowKey, offset, keyLength); + byte[] b = new byte[length]; + System.arraycopy(rowKey, offset, b, 0, length); + offset += length + 1; + keyLength -= length + 1; + colMetaData[i] = b; + } + return i; + } + + public static String findExistingColumn(PTable table, List columns) { + for (PColumn column : columns) { + PName familyName = column.getFamilyName(); + if (familyName == null) { + try { + return table.getPKColumn(column.getName().getString()).getName().getString(); + } catch (ColumnNotFoundException e) { + continue; + } + } else { + try { + return table.getColumnFamily(familyName.getString()).getColumn(column.getName().getString()).getName().getString(); + } catch (ColumnFamilyNotFoundException e) { + continue; // Shouldn't happen + } catch (ColumnNotFoundException e) { + continue; + } + } + } + return null; + } + + public static String toString(byte[][] values) { + if (values == null) { + return "null"; + } + StringBuilder buf = new StringBuilder("["); + for (byte[] value : values) { + buf.append(Bytes.toStringBinary(value)); + buf.append(','); + } + buf.setCharAt(buf.length()-1, ']'); + return buf.toString(); + } + + public static String toString(PDataType type, byte[] value) { + boolean isString = type.isCoercibleTo(PDataType.VARCHAR); + return isString ? ("'" + type.toObject(value).toString() + "'") : type.toObject(value).toString(); + } + + public static byte[] getEmptyColumnFamily(List families) { + return families.isEmpty() ? QueryConstants.EMPTY_COLUMN_BYTES : families.get(0).getName().getBytes(); + } + + public static ImmutableBytesPtr getEmptyColumnFamilyPtr(List families) { + return families.isEmpty() ? QueryConstants.EMPTY_COLUMN_BYTES_PTR : families.get(0) + .getName().getBytesPtr(); + } + + public static boolean isMetaTable(byte[] tableName) { + return Bytes.compareTo(tableName, TYPE_TABLE_NAME_BYTES) == 0; + } + + public static boolean isSequenceTable(byte[] tableName) { + return Bytes.compareTo(tableName, PhoenixDatabaseMetaData.SEQUENCE_TABLE_NAME_BYTES) == 0; + } + + public static boolean isMetaTable(PTable table) { + return PhoenixDatabaseMetaData.TYPE_SCHEMA.equals(table.getSchemaName().getString()) && PhoenixDatabaseMetaData.TYPE_TABLE.equals(table.getTableName().getString()); + } + + public static boolean isMetaTable(byte[] schemaName, byte[] tableName) { + return Bytes.compareTo(schemaName, PhoenixDatabaseMetaData.TYPE_SCHEMA_BYTES) == 0 && Bytes.compareTo(tableName, PhoenixDatabaseMetaData.TYPE_TABLE_BYTES) == 0; + } + + public static boolean isMetaTable(String schemaName, String tableName) { + return PhoenixDatabaseMetaData.TYPE_SCHEMA.equals(schemaName) && PhoenixDatabaseMetaData.TYPE_TABLE.equals(tableName); + } + + // Given the splits and the rowKeySchema, find out the keys that + public static byte[][] processSplits(byte[][] splits, LinkedHashSet pkColumns, Integer saltBucketNum, boolean defaultRowKeyOrder) throws SQLException { + // FIXME: shouldn't this return if splits.length == 0? + if (splits == null) return null; + // We do not accept user specified splits if the table is salted and we specify defaultRowKeyOrder. In this case, + // throw an exception. + if (splits.length > 0 && saltBucketNum != null && defaultRowKeyOrder) { + throw new SQLExceptionInfo.Builder(SQLExceptionCode.NO_SPLITS_ON_SALTED_TABLE).build().buildException(); + } + // If the splits are not specified and table is salted, pre-split the table. + if (splits.length == 0 && saltBucketNum != null) { + splits = SaltingUtil.getSalteByteSplitPoints(saltBucketNum); + } + byte[][] newSplits = new byte[splits.length][]; + for (int i=0; i pkColumns) { + int pos = 0, offset = 0, maxOffset = split.length; + Iterator iterator = pkColumns.iterator(); + while (pos < pkColumns.size()) { + PColumn column = iterator.next(); + if (column.getDataType().isFixedWidth()) { // Fixed width + int length = column.getByteSize(); + if (maxOffset - offset < length) { + // The split truncates the field. Fill in the rest of the part and any fields that + // are missing after this field. + int fillInLength = length - (maxOffset - offset); + fillInLength += estimatePartLength(pos + 1, iterator); + return ByteUtil.fillKey(split, split.length + fillInLength); + } + // Account for this field, move to next position; + offset += length; + pos++; + } else { // Variable length + // If we are the last slot, then we are done. Nothing needs to be filled in. + if (pos == pkColumns.size() - 1) { + break; + } + while (offset < maxOffset && split[offset] != QueryConstants.SEPARATOR_BYTE) { + offset++; + } + if (offset == maxOffset) { + // The var-length field does not end with a separator and it's not the last field. + int fillInLength = 1; // SEPARATOR byte for the current var-length slot. + fillInLength += estimatePartLength(pos + 1, iterator); + return ByteUtil.fillKey(split, split.length + fillInLength); + } + // Move to the next position; + offset += 1; // skip separator; + pos++; + } + } + return split; + } + + // Estimate the key length after pos slot for schema. + private static int estimatePartLength(int pos, Iterator iterator) { + int length = 0; + while (iterator.hasNext()) { + PColumn column = iterator.next(); + if (column.getDataType().isFixedWidth()) { + length += column.getByteSize(); + } else { + length += 1; // SEPARATOR byte. + } + } + return length; + } + + public static final String UPGRADE_TO_2_0 = "UpgradeTo20"; + public static final String UPGRADE_TO_2_1 = "UpgradeTo21"; + + public static String getEscapedTableName(String schemaName, String tableName) { + if (schemaName == null || schemaName.length() == 0) { + return "\"" + tableName + "\""; + } + return "\"" + schemaName + "\"." + "\"" + tableName + "\""; + } + + protected static PhoenixConnection addMetaDataColumn(PhoenixConnection conn, long scn, String columnDef) throws SQLException { + String url = conn.getURL(); + Properties props = conn.getClientInfo(); + PMetaData metaData = conn.getPMetaData(); + props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(scn)); + PhoenixConnection metaConnection = null; + + Statement stmt = null; + try { + metaConnection = new PhoenixConnection(conn.getQueryServices(), url, props, metaData); + try { + stmt = metaConnection.createStatement(); + stmt.executeUpdate("ALTER TABLE SYSTEM.\"TABLE\" ADD IF NOT EXISTS " + columnDef); + return metaConnection; + } finally { + if(stmt != null) { + stmt.close(); + } + } + } finally { + if(metaConnection != null) { + metaConnection.close(); + } + } + } + + public static boolean columnExists(PTable table, String columnName) { + try { + table.getColumn(columnName); + return true; + } catch (ColumnNotFoundException e) { + return false; + } catch (AmbiguousColumnException e) { + return true; + } + } + + public static String getSchemaNameFromFullName(String tableName) { + int index = tableName.indexOf(QueryConstants.NAME_SEPARATOR); + if (index < 0) { + return ""; + } + return tableName.substring(0, index); + } + + public static String getTableNameFromFullName(String tableName) { + int index = tableName.indexOf(QueryConstants.NAME_SEPARATOR); + if (index < 0) { + return tableName; + } + return tableName.substring(index+1, tableName.length()); + } + + public static byte[] getTableKeyFromFullName(String fullTableName) { + int index = fullTableName.indexOf(QueryConstants.NAME_SEPARATOR); + if (index < 0) { + return getTableKey(null, null, fullTableName); + } + String schemaName = fullTableName.substring(0, index); + String tableName = fullTableName.substring(index+1); + return getTableKey(null, schemaName, tableName); + } +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/ServerUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/ServerUtil.java new file mode 100644 index 00000000..819bcc48 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/ServerUtil.java @@ -0,0 +1,123 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import java.io.IOException; +import java.sql.SQLException; +import java.util.HashMap; +import java.util.Map; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +import org.apache.hadoop.hbase.DoNotRetryIOException; + +import org.apache.phoenix.exception.PhoenixIOException; +import org.apache.phoenix.exception.SQLExceptionCode; + + +public class ServerUtil { + + private static final String FORMAT = "ERROR %d (%s): %s"; + private static final Pattern PATTERN = Pattern.compile("ERROR (\\d+) \\((\\w+)\\): (.*)"); + private static final Map, SQLExceptionCode> errorcodeMap + = new HashMap, SQLExceptionCode>(); + static { + // Map a normal exception into a corresponding SQLException. + errorcodeMap.put(ArithmeticException.class, SQLExceptionCode.SERVER_ARITHMETIC_ERROR); + } + + public static void throwIOException(String msg, Throwable t) throws IOException { + // First unwrap SQLExceptions if it's root cause is an IOException. + if (t instanceof SQLException) { + Throwable cause = t.getCause(); + if (cause instanceof IOException) { + t = cause; + } + } + // Throw immediately if DoNotRetryIOException + if (t instanceof DoNotRetryIOException) { + throw (DoNotRetryIOException)t; + } else if (t instanceof IOException) { + // If the IOException does not wrap any exception, then bubble it up. + Throwable cause = t.getCause(); + if (cause == null || cause instanceof IOException) { + throw (IOException)t; + } + // Else assume it's been wrapped, so throw as DoNotRetryIOException to prevent client hanging while retrying + throw new DoNotRetryIOException(t.getMessage(), cause); + } else if (t instanceof SQLException) { + // If it's already an SQLException, construct an error message so we can parse and reconstruct on the client side. + throw new DoNotRetryIOException(constructSQLErrorMessage((SQLException) t, msg), t); + } else { + // Not a DoNotRetryIOException, IOException or SQLException. Map the exception type to a general SQLException + // and construct the error message so it can be reconstruct on the client side. + // + // If no mapping exists, rethrow it as a generic exception. + SQLExceptionCode code = errorcodeMap.get(t.getClass()); + if (code == null) { + throw new DoNotRetryIOException(msg + ": " + t.getMessage(), t); + } else { + throw new DoNotRetryIOException(constructSQLErrorMessage(code, t, msg), t); + } + } + } + + private static String constructSQLErrorMessage(SQLExceptionCode code, Throwable e, String message) { + return constructSQLErrorMessage(code.getErrorCode(), code.getSQLState(), code.getMessage() + " " + e.getMessage() + " " + message); + } + + private static String constructSQLErrorMessage(SQLException e, String message) { + return constructSQLErrorMessage(e.getErrorCode(), e.getSQLState(), e.getMessage() + " " + message); + } + + private static String constructSQLErrorMessage(int errorCode, String SQLState, String message) { + return String.format(FORMAT, errorCode, SQLState, message); + } + + public static SQLException parseServerException(Throwable t) { + SQLException e = parseServerExceptionOrNull(t); + if (e != null) { + return e; + } + return new PhoenixIOException(t); + } + + public static SQLException parseServerExceptionOrNull(Throwable t) { + while (t.getCause() != null) { + t = t.getCause(); + } + return parseRemoteException(t); + } + + private static SQLException parseRemoteException(Throwable t) { + String message = t.getLocalizedMessage(); + if (message != null) { + // If the message matches the standard pattern, recover the SQLException and throw it. + Matcher matcher = PATTERN.matcher(t.getLocalizedMessage()); + if (matcher.find()) { + int errorCode = Integer.parseInt(matcher.group(1)); + String sqlState = matcher.group(2); + return new SQLException(matcher.group(), sqlState, errorCode, t); + } + } + return null; + } + +} diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/SizedUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/SizedUtil.java new file mode 100644 index 00000000..62f08165 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/SizedUtil.java @@ -0,0 +1,63 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + + +/** + * Utilities for computing an object's size. All size measurements are in bytes. + * Note, all of the sizes here, but especially OBJECT_SIZE and ARRAY_SIZE are estimates and will + * depend on the JVM itself (and which JVM, 64bit vs. 32bit, etc). + * The current values are based on: + * Java HotSpot(TM) 64-Bit Server VM/14.2-b01 + * + * (Uncomment out and run the main w/ appropriate object to test) + * Also, see this link: + * https://sites.google.com/a/salesforce.com/development/Home/old-wiki-home-page/i-wish-i-knew#TOC-How-to-measure-the-size-of-a-Java-O + * For another way to measure. + */ +public class SizedUtil { + public static final int POINTER_SIZE = 8; // 64 bit jvm. + public static final int OBJECT_SIZE = 16; // measured, see class comment. + public static final int ARRAY_SIZE = 24; // measured, see class comment. + public static final int CHAR_SIZE = 2; + public static final int INT_SIZE = 4; + public static final int LONG_SIZE = 8; + + public static final int MAP_ENTRY_SIZE = OBJECT_SIZE + 3 * POINTER_SIZE + INT_SIZE; + public static final int IMMUTABLE_BYTES_WRITABLE_SIZE = OBJECT_SIZE + INT_SIZE * 2 + ARRAY_SIZE; + public static final int IMMUTABLE_BYTES_PTR_SIZE = IMMUTABLE_BYTES_WRITABLE_SIZE + INT_SIZE;// Extra is an int field which caches hashcode. + public static final int KEY_VALUE_SIZE = 2 * INT_SIZE + LONG_SIZE + 2 * ARRAY_SIZE; + public static final int RESULT_SIZE = OBJECT_SIZE + 3 * POINTER_SIZE + IMMUTABLE_BYTES_WRITABLE_SIZE; + public static final int INT_OBJECT_SIZE = INT_SIZE + OBJECT_SIZE; + public static final int LONG_OBJECT_SIZE = LONG_SIZE + OBJECT_SIZE; + public static final int BIG_DECIMAL_SIZE = + OBJECT_SIZE + 2 * INT_SIZE + LONG_SIZE + 2 * POINTER_SIZE + + OBJECT_SIZE /* BigInteger */ + 5 * INT_SIZE + ARRAY_SIZE /*mag[]*/ + 2 * INT_SIZE /* est mag[2] */; + + private SizedUtil() { + } + + public static int sizeOfMap(int nRows, int keySize, int valueSize) { + return nRows * ( + SizedUtil.MAP_ENTRY_SIZE + // entry + keySize + // key size + valueSize); // value size + } +} diff --git a/phoenix-core/src/main/java/com/salesforce/phoenix/util/StringUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/StringUtil.java similarity index 82% rename from phoenix-core/src/main/java/com/salesforce/phoenix/util/StringUtil.java rename to phoenix-core/src/main/java/org/apache/phoenix/util/StringUtil.java index 6a953ccf..1e9d961d 100644 --- a/phoenix-core/src/main/java/com/salesforce/phoenix/util/StringUtil.java +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/StringUtil.java @@ -1,38 +1,30 @@ -/******************************************************************************* - * Copyright (c) 2013, Salesforce.com, Inc. - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * Neither the name of Salesforce.com nor the names of its contributors may - * be used to endorse or promote products derived from this software without - * specific prior written permission. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - ******************************************************************************/ -package com.salesforce.phoenix.util; +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; import java.io.UnsupportedEncodingException; import java.util.Arrays; import org.apache.hadoop.hbase.util.Bytes; -import com.salesforce.phoenix.schema.ColumnModifier; +import org.apache.phoenix.schema.ColumnModifier; public class StringUtil { diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/TrustedByteArrayOutputStream.java b/phoenix-core/src/main/java/org/apache/phoenix/util/TrustedByteArrayOutputStream.java new file mode 100644 index 00000000..f503cb35 --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/TrustedByteArrayOutputStream.java @@ -0,0 +1,56 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import java.io.ByteArrayOutputStream; +import java.io.IOException; + +/** + * + * Derived version of {@link java.io.ByteArrayOutputStream} that provides access + * to underlying byte array buffer so that it doesn't have to be copied + * + * @author jtaylor + * @since 0.1 + */ +public class TrustedByteArrayOutputStream extends ByteArrayOutputStream { + public TrustedByteArrayOutputStream(int initialSize) { + super(initialSize); + } + public byte[] getBuffer() { + return buf; + } + @Override + public byte[] toByteArray() { + if (buf.length == size()) { + return buf; + } + return super.toByteArray(); + } + @Override + public void write(byte[] b) { + try { + super.write(b); + } catch (IOException e) { + throw new RuntimeException(e); // Impossible + } + } +} + diff --git a/phoenix-core/src/main/java/org/apache/phoenix/util/TupleUtil.java b/phoenix-core/src/main/java/org/apache/phoenix/util/TupleUtil.java new file mode 100644 index 00000000..9a8608ad --- /dev/null +++ b/phoenix-core/src/main/java/org/apache/phoenix/util/TupleUtil.java @@ -0,0 +1,155 @@ +/* + * Copyright 2010 The Apache Software Foundation + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.phoenix.util; + +import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN; +import static org.apache.phoenix.query.QueryConstants.SINGLE_COLUMN_FAMILY; + +import java.io.DataOutput; +import java.io.IOException; +import java.util.List; + +import org.apache.hadoop.hbase.DoNotRetryIOException; +import org.apache.hadoop.hbase.KeyValue; +import org.apache.hadoop.hbase.io.ImmutableBytesWritable; +import org.apache.hadoop.hbase.util.Bytes; +import org.apache.hadoop.io.WritableUtils; + +import org.apache.hadoop.hbase.index.util.ImmutableBytesPtr; +import org.apache.phoenix.expression.Expression; +import org.apache.phoenix.query.QueryConstants; +import org.apache.phoenix.schema.tuple.Tuple; + + +/** + * + * Utilities for Tuple + * + * @author jtaylor + * @since 0.1 + */ +public class TupleUtil { + private TupleUtil() { + } + + public static boolean equals(Tuple t1, Tuple t2, ImmutableBytesWritable ptr) { + t1.getKey(ptr); + byte[] buf = ptr.get(); + int offset = ptr.getOffset(); + int length = ptr.getLength(); + t2.getKey(ptr); + return Bytes.compareTo(buf, offset, length, ptr.get(), ptr.getOffset(), ptr.getLength()) == 0; + } + + public static int compare(Tuple t1, Tuple t2, ImmutableBytesWritable ptr) { + return compare(t1, t2, ptr, 0); + } + + public static int compare(Tuple t1, Tuple t2, ImmutableBytesWritable ptr, int keyOffset) { + t1.getKey(ptr); + byte[] buf = ptr.get(); + int offset = ptr.getOffset() + keyOffset; + int length = ptr.getLength() - keyOffset; + t2.getKey(ptr); + return Bytes.compareTo(buf, offset, length, ptr.get(), ptr.getOffset() + keyOffset, ptr.getLength() - keyOffset); + } + + /** + * Set ptr to point to the value contained in the first KeyValue without + * exploding Result into KeyValue array. + * @param r + * @param ptr + */ + public static void getAggregateValue(Tuple r, ImmutableBytesWritable ptr) { + if (r.size() == 1) { + KeyValue kv = r.getValue(0); // Just one KV for aggregation + if (Bytes.compareTo(SINGLE_COLUMN_FAMILY, 0, SINGLE_COLUMN_FAMILY.length, kv.getBuffer(), kv.getFamilyOffset(), kv.getFamilyLength()) == 0) { + if (Bytes.compareTo(SINGLE_COLUMN, 0, SINGLE_COLUMN.length, kv.getBuffer(), kv.getQualifierOffset(), kv.getQualifierLength()) == 0) { + ptr.set(kv.getBuffer(), kv.getValueOffset(), kv.getValueLength()); + return; + } + } + } + throw new IllegalStateException("Expected single, aggregated KeyValue from coprocessor, but instead received " + r + ". Ensure aggregating coprocessors are loaded correctly on server"); + } + + /** Concatenate results evaluated against a list of expressions + * + * @param result the tuple for expression evaluation + * @param expressions + * @return the concatenated byte array as ImmutableBytesWritable + * @throws IOException + */ + public static ImmutableBytesPtr getConcatenatedValue(Tuple result, List expressions) throws IOException { + ImmutableBytesPtr value = new ImmutableBytesPtr(ByteUtil.EMPTY_BYTE_ARRAY); + Expression expression = expressions.get(0); + boolean evaluated = expression.evaluate(result, value); + + if (expressions.size() == 1) { + if (!evaluated) { + value.set(ByteUtil.EMPTY_BYTE_ARRAY); + } + return value; + } else { + TrustedByteArrayOutputStream output = new TrustedByteArrayOutputStream(value.getLength() * expressions.size()); + try { + if (evaluated) { + output.write(value.get(), value.getOffset(), value.getLength()); + } + for (int i = 1; i < expressions.size(); i++) { + if (!expression.getDataType().isFixedWidth()) { + output.write(QueryConstants.SEPARATOR_BYTE); + } + expression = expressions.get(i); + // TODO: should we track trailing null values and omit the separator bytes? + if (expression.evaluate(result, value)) { + output.write(value.get(), value.getOffset(), value.getLength()); + } else if (i < expressions.size()-1 && expression.getDataType().isFixedWidth()) { + // This should never happen, because any non terminating nullable fixed width type (i.e. INT or LONG) is + // converted to a variable length type (i.e. DECIMAL) to allow an empty byte array to represent null. + throw new DoNotRetryIOException("Non terminating null value found for fixed width expression (" + expression + ") in row: " + result); + } + } + byte[] outputBytes = output.getBuffer(); + value.set(outputBytes, 0, output.size()); + return value; + } finally { + output.close(); + } + } + } + + public static int write(Tuple result, DataOutput out) throws IOException { + int size = 0; + for(int i = 0; i < result.size(); i++) { + KeyValue kv = result.getValue(i); + size += kv.getLength(); + size += Bytes.SIZEOF_INT; // kv.getLength + } + + WritableUtils.writeVInt(out, size); + for(int i = 0; i < result.size(); i++) { + KeyValue kv = result.getValue(i); + out.writeInt(kv.getLength()); + out.write(kv.getBuffer(), kv.getOffset(), kv.getLength()); + } + return size; + } +} diff --git a/phoenix-core/src/main/resources/java.sql.Driver b/phoenix-core/src/main/resources/java.sql.Driver index 8067ad71..fc51a045 100644 --- a/phoenix-core/src/main/resources/java.sql.Driver +++ b/phoenix-core/src/main/resources/java.sql.Driver @@ -1 +1 @@ -com.salesforce.phoenix.jdbc.PhoenixDriver +org.apache.phoenix.jdbc.PhoenixDriver diff --git a/phoenix-core/src/site/bin/merge.jar b/phoenix-core/src/site/bin/merge.jar new file mode 100644 index 00000000..2b329d69 Binary files /dev/null and b/phoenix-core/src/site/bin/merge.jar differ diff --git a/phoenix-core/src/site/bin/merge.sh b/phoenix-core/src/site/bin/merge.sh new file mode 100755 index 00000000..5ac0641a --- /dev/null +++ b/phoenix-core/src/site/bin/merge.sh @@ -0,0 +1,10 @@ +current_dir=$(cd $(dirname $0);pwd) +cd $current_dir +SITE_TARGET="../../../target/site" +java -jar merge.jar ../language_reference_source/index.html $SITE_TARGET/language/index.html +java -jar merge.jar ../language_reference_source/functions.html $SITE_TARGET/language/functions.html +java -jar merge.jar ../language_reference_source/datatypes.html $SITE_TARGET/language/datatypes.html +cd $SITE_TARGET + +grep -rl class=\"nav-collapse\" . | xargs sed -i 's/class=\"nav-collapse\"/class=\"nav-collapse collapse\"/g';grep -rl class=\"active\" . | xargs sed -i 's/class=\"active\"/class=\"divider\"/g' +grep -rl "dropdown active" . | xargs sed -i 's/dropdown active/dropdown/g' diff --git a/phoenix-core/src/site/language_reference_source/datatypes.html b/phoenix-core/src/site/language_reference_source/datatypes.html new file mode 100644 index 00000000..9efca102 --- /dev/null +++ b/phoenix-core/src/site/language_reference_source/datatypes.html @@ -0,0 +1,493 @@ + + + +Data Types + + + + + + +

Index

+ + + + + + +
+ + INTEGER Type
+ + UNSIGNED_INT Type
+ + BIGINT Type
+ + UNSIGNED_LONG Type
+ + TINYINT Type
+ + UNSIGNED_TINYINT Type
+ + SMALLINT Type
+ 		+ + UNSIGNED_SMALLINT Type
+ + FLOAT Type
+ + UNSIGNED_FLOAT Type
+ + DOUBLE Type
+ + UNSIGNED_DOUBLE Type
+ + DECIMAL Type
+ + BOOLEAN Type
+ 		+ + TIME Type
+ + DATE Type
+ + TIMESTAMP Type
+ + VARCHAR Type
+ + CHAR Type
+ + BINARY Type
+ + VARBINARY Type
+
+ + + +

INTEGER Type

+ +
+INTEGER
+
+
+INTEGER +
+ + +

Possible values: -2147483648 to 2147483647.

Mapped to java.lang.Integer. The binary representation is a 4 byte integer with the sign bit flipped (so that negative values sorts before positive values).

+

Example:

+

INTEGER

+ +

UNSIGNED_INT Type

+ +
+UNSIGNED_INT
+
+
+UNSIGNED_INT +
+ + +

Possible values: 0 to 2147483647. Mapped to java.lang.Integer. The binary representation is a 4 byte integer, matching the HBase Bytes.toBytes(int) method. The purpose of this type is to map to existing HBase data that was serialized using this HBase utility method. If that is not the case, use the regular signed type instead.

+

Example:

+

UNSIGNED_INT

+ +

BIGINT Type

+ +
+BIGINT
+
+
+BIGINT +
+ + +

Possible values: -9223372036854775807 to 9223372036854775807. Mapped to java.lang.Long. The binary representation is an 8 byte long with the sign bit flipped (so that negative values sorts before positive values).

+

Example:

+

BIGINT

+ +

UNSIGNED_LONG Type

+ +
+UNSIGNED_LONG
+
+
+UNSIGNED_LONG +
+ + +

Possible values: 0 to 9223372036854775807. Mapped to java.lang.Long. The binary representation is an 8 byte integer, matching the HBase Bytes.toBytes(long) method. The purpose of this type is to map to existing HBase data that was serialized using this HBase utility method. If that is not the case, use the regular signed type instead.

+

Example:

+

UNSIGNED_LONG

+ +

TINYINT Type

+ +
+TINYINT
+
+
+TINYINT +
+ + +

Possible values: -128 to 127. Mapped to java.lang.Byte. The binary representation is a single byte, with the sign bit flipped (so that negative values sorts before positive values).

+

Example:

+

TINYINT

+ +

UNSIGNED_TINYINT Type

+ +
+UNSIGNED_TINYINT
+
+
+UNSIGNED_TINYINT +
+ + +

Possible values: 0 to 127. Mapped to java.lang.Byte. The binary representation is a single byte, matching the HBase Bytes.toBytes(byte) method. The purpose of this type is to map to existing HBase data that was serialized using this HBase utility method. If that is not the case, use the regular signed type instead.

+

Example:

+

UNSIGNED_TINYINT

+ +

SMALLINT Type

+ +
+SMALLINT
+
+
+SMALLINT +
+ + +

Possible values: -32768 to 32767. Mapped to java.lang.Short. The binary representation is a 2 byte short with the sign bit flipped (so that negative values sort before positive values).

+

Example:

+

SMALLINT

+ +

UNSIGNED_SMALLINT Type

+ +
+UNSIGNED_SMALLINT
+
+
+UNSIGNED_SMALLINT +
+ + +

Possible values: 0 to 32767. Mapped to java.lang.Short. The binary representation is an 2 byte integer, matching the HBase Bytes.toBytes(short) method. The purpose of this type is to map to existing HBase data that was serialized using this HBase utility method. If that is not the case, use the regular signed type instead.

+

Example:

+

UNSIGNED_SMALLINT

+ +

FLOAT Type

+ +
+FLOAT
+
+
+FLOAT +
+ + +

Possible values: -3.402823466 E + 38 to 3.402823466 E + 38. Mapped to java.lang.Float. The binary representation is an 4 byte float with the sign bit flipped (so that negative values sort before positive values).

+

Example:

+

FLOAT

+ +

UNSIGNED_FLOAT Type

+ +
+UNSIGNED_FLOAT
+
+
+UNSIGNED_FLOAT +
+ + +

Possible values: 0 to 3.402823466 E + 38. Mapped to java.lang.Float. The binary representation is an 4 byte float matching the HBase Bytes.toBytes(float) method. The purpose of this type is to map to existing HBase data that was serialized using this HBase utility method. If that is not the case, use the regular signed type instead.

+

Example:

+

UNSIGNED_FLOAT

+ +

DOUBLE Type

+ +
+DOUBLE
+
+
+DOUBLE +
+ + +

Possible values: -1.7976931348623158 E + 308 to 1.7976931348623158 E + 308. Mapped to java.lang.Double. The binary representation is an 8 byte double with the sign bit flipped (so that negative values sort before positive value).

+

Example:

+

DOUBLE

+ +

UNSIGNED_DOUBLE Type

+ +
+UNSIGNED_DOUBLE
+
+
+UNSIGNED_DOUBLE +
+ + +

Possible values: 0 to  1.7976931348623158 E + 308. Mapped to java.lang.Double. The binary representation is an 8 byte double matching the HBase Bytes.toBytes(double) method. The purpose of this type is to map to existing HBase data that was serialized using this HBase utility method. If that is not the case, use the regular signed type instead.

+

Example:

+

UNSIGNED_DOUBLE

+ +

DECIMAL Type

+ +
+DECIMAL
+
+
+DECIMAL +
+ + +

Data type with fixed precision and scale. The maximum precision is 18 digits. Mapped to java.math.BigDecimal. The binary representation is binary comparable, variable length format. When used in a row key, it is terminated with a null byte unless it is the last column.

+

Example:

+

DECIMAL

+ +

BOOLEAN Type

+ +
+BOOLEAN
+
+
+BOOLEAN +
+ + +

Possible values: TRUE and FALSE.

Mapped to java.lang.Boolean. The binary representation is a single byte with 0 for false and 1 for true

+

Example:

+

BOOLEAN

+ +

TIME Type

+ +
+TIME
+
+
+TIME +
+ + +

The time data type. The format is yyyy-MM-dd hh:mm:ss, with both the date and time parts maintained. Mapped to java.sql.Time. The binary representation is an 8 byte long (the number of milliseconds from the epoch).

+

Example:

+

TIME

+ +

DATE Type

+ +
+DATE
+
+
+DATE +
+ + +

The date data type. The format is yyyy-MM-dd hh:mm:ss, with both the date and time parts maintained to a millisecond accuracy. Mapped to java.sql.Date. The binary representation is an 8 byte long (the number of milliseconds from the epoch).

+

Example:

+

DATE

+ +

TIMESTAMP Type

+ +
+TIMESTAMP
+
+
+TIMESTAMP +
+ + +

The timestamp data type. The format is yyyy-MM-dd hh:mm:ss[.nnnnnnnnn]. Mapped to java.sql.Timestamp with an internal representation of the number of nanos from the epoch. The binary representation is 12 bytes: an 8 byte long for the epoch time plus a 4 byte integer for the nanos.

+

Example:

+

TIMESTAMP

+ +

VARCHAR Type

+ +
+VARCHAR  [ ( precisionInt ) ]
+
+
+
VARCHAR
 
( precisionInt )
+
+ + +

A variable length String with an optional max byte length. The binary representation is UTF8 matching the HBase Bytes.toBytes(String) method. When used in a row key, it is terminated with a null byte unless it is the last column.

Mapped to java.lang.String.

+

Example:

+

VARCHAR
VARCHAR(255)

+ +

CHAR Type

+ +
+CHAR ( precisionInt )
+
+
+
CHAR ( precisionInt )
+
+ + +

A fixed length String with single-byte characters. The binary representation is UTF8 matching the HBase Bytes.toBytes(String) method.

Mapped to java.lang.String.

+

Example:

+

CHAR(10)

+ +

BINARY Type

+ +
+BINARY ( precisionInt )
+
+
+
BINARY ( precisionInt )
+
+ + +

Raw fixed length byte array.

Mapped to byte[].

+

Example:

+

BINARY

+ +

VARBINARY Type

+ +
+VARBINARY
+
+
+VARBINARY +
+ + +

Raw variable length byte array.

Mapped to byte[].

+

Example:

+

VARBINARY

+ + + + + diff --git a/phoenix-core/src/site/language_reference_source/functions.html b/phoenix-core/src/site/language_reference_source/functions.html new file mode 100644 index 00000000..53056290 --- /dev/null +++ b/phoenix-core/src/site/language_reference_source/functions.html @@ -0,0 +1,740 @@ + + + +Functions + + + + + + +

Aggregate Functions

+ + + + + + +
+ + AVG
+ + COUNT
+ + MAX
+ + MIN
+ 		+ + SUM
+ + PERCENTILE_CONT
+ + PERCENTILE_DISC
+ + PERCENT_RANK
+ 		+ + STDDEV_POP
+ + STDDEV_SAMP
+
+ + +

String Functions

+ + + + + + +
+ + SUBSTR
+ + TRIM
+ + LTRIM
+ + RTRIM
+ 		+ + LENGTH
+ + REGEXP_SUBSTR
+ + REGEXP_REPLACE
+ + UPPER
+ 		+ + LOWER
+ + REVERSE
+ + TO_CHAR
+
+ + +

Time and Date Functions

+ + + + + + +
+ + ROUND
+ + TRUNCATE
+ 		+ + TO_DATE
+ + CURRENT_DATE
+ 		+ + CURRENT_TIME
+
+ + +

Other Functions

+ + + + + + +
+ + MD5
+ + INVERT
+ 		+ + TO_NUMBER
+ + COALESCE
+ 		+
+ + + + +

AVG

+ +
+AVG ( { numericTerm } )
+
+
+
AVG ( numericTerm )
+
+ + +

The average (mean) value. If no rows are selected, the result is NULL. Aggregates are only allowed in select statements. The returned value is of the same data type as the parameter.

+

Example:

+

AVG(X)

+ +

COUNT

+ +
+COUNT( [ DISTINCT ] { * | { term } } )
+
+
+
COUNT (
 
DISTINCT
*
term
)
+
+ + +

The count of all row, or of the non-null values. This method returns a long. When DISTINCT is used, it counts only distinct values. If no rows are selected, the result is 0. Aggregates are only allowed in select statements.

+

Example:

+

COUNT(*)

+ +

MAX

+ +
+MAX(term)
+
+
+
MAX ( term )
+
+ + +

The highest value. If no rows are selected, the result is NULL. Aggregates are only allowed in select statements. The returned value is of the same data type as the parameter.

+

Example:

+

MAX(NAME)

+ +

MIN

+ +
+MIN(term)
+
+
+
MIN ( term )
+
+ + +

The lowest value. If no rows are selected, the result is NULL. Aggregates are only allowed in select statements. The returned value is of the same data type as the parameter.

+

Example:

+

MIN(NAME)

+ +

SUM

+ +
+SUM( { numericTerm } )
+
+
+
SUM ( numericTerm )
+
+ + +

The sum of all values. If no rows are selected, the result is NULL. Aggregates are only allowed in select statements. The returned value is of the same data type as the parameter.

+

Example:

+

SUM(X)

+ +

PERCENTILE_CONT

+ +
+PERCENTILE_CONT( { numeric } ) WITHIN GROUP (ORDER BY { numericTerm } { ASC | DESC } )
+
+
+
PERCENTILE_CONT ( numeric ) WITHIN GROUP ( ORDER BY numericTerm
ASC
DESC
)
+
+ + +

The nth percentile of values in the column. The percentile value can be between 0 and 1 inclusive. Aggregates are only allowed in select statements. The returned value is of decimal data type.

+

Example:

+

PERCENTILE_CONT( 0.9 ) WITHIN GROUP (ORDER BY X ASC)

+ +

PERCENTILE_DISC

+ +
+PERCENTILE_DIST( { numeric } ) WITHIN GROUP (ORDER BY { numericTerm } { ASC | DESC } )
+
+
+
PERCENTILE_DIST ( numeric ) WITHIN GROUP ( ORDER BY numericTerm
ASC
DESC
)
+
+ + +

PERCENTILE_DISC is an inverse distribution function that assumes a discrete distribution model. It takes a percentile value and a sort specification and returns an element from the set. Nulls are ignored in the calculation.

+

Example:

+

PERCENTILE_DISC( 0.9 ) WITHIN GROUP (ORDER BY X DESC)

+ +

PERCENT_RANK

+ +
+PERCENT_RANK( { numeric } ) WITHIN GROUP (ORDER BY { numericTerm } { ASC | DESC } )
+
+
+
PERCENT_RANK ( numeric ) WITHIN GROUP ( ORDER BY numericTerm
ASC
DESC
)
+
+ + +

The percentile rank for a hypothetical value, if inserted into the column. Aggregates are only allowed in select statements. The returned value is of decimal data type.

+

Example:

+

PERCENT_RANK( 100 ) WITHIN GROUP (ORDER BY X ASC)

+ +

STDDEV_POP

+ +
+STDDEV_POP( { numericTerm } )
+
+
+
STDDEV_POP ( numericTerm )
+
+ + +

The population standard deviation of all values. Aggregates are only allowed in select statements. The returned value is of decimal data type.

+

Example:

+

STDDEV_POP( X )

+ +

STDDEV_SAMP

+ +
+STDDEV_SAMP( { numericTerm } )
+
+
+
STDDEV_SAMP ( numericTerm )
+
+ + +

The sample standard deviation of all values. Aggregates are only allowed in select statements. The returned value is of decimal data type.

+

Example:

+

STDDEV_SAMP( X )

+ +

MD5

+ +
+MD5( term )
+
+
+
MD5 ( term )
+
+ + +

Computes the MD5 hash of the argument, returning the result as a BINARY(16).

+

Example:

+

MD5(my_column)

+ +

INVERT

+ +
+INVERT( term )
+
+
+
INVERT ( term )
+
+ + +

Inverts the bits of the argument. The return type will be the same as the argument.

+

Example:

+

INVERT(my_column)

+ +

TO_NUMBER

+ +
+TO_NUMBER( stringTerm | timeStampTerm [, formatString] )
+
+ + + +

Formats a string or date/time/timeStamp as a number, optionally accepting a format string. For details on the format, see java.text.DecimalFormat. For date, time, and timeStamp terms, the result is the time in milliseconds since the epoch. This method returns a decimal number.

+

Example:

+

TO_NUMBER('$123.33', '\u00A4###.##')

+ +

COALESCE

+ +
+COALESCE( firstTerm, secondTerm )
+
+
+
COALESCE ( firstTerm , secondTerm )
+
+ + +

Returns the value of the first argument if not null and the second argument otherwise. Useful to guarantee that a column in an UPSERT SELECT command will evaluate to a non null value.

+

Example:

+

COALESCE(last_update_date, CURRENT_DATE())

+ +

SUBSTR

+ +
+SUBSTR( stringTerm, startInt [, lengthInt ] )
+
+
+
SUBSTR ( stringTerm , startInt
 
, lengthInt
)
+
+ + +

Returns a substring of a string starting at the one-based position. If zero is used, the position is zero-based. If the start index is negative, then the start index is relative to the end of the string. The length is optional and if not supplied, the rest of the string will be returned.

+

Example:

+

SUBSTR('[Hello]', 2, 5)
SUBSTR('Hello World', -5)

+ +

TRIM

+ +
+TRIM( stringTerm )
+
+
+
TRIM ( stringTerm )
+
+ + +

Removes leading and trailing spaces from the input string.

+

Example:

+

TRIM('  Hello  ')

+ +

LTRIM

+ +
+LTRIM( stringTerm )
+
+
+
LTRIM ( stringTerm )
+
+ + +

Removes leading spaces from the input string.

+

Example:

+

LTRIM('  Hello')

+ +

RTRIM

+ +
+RTRIM( stringTerm )
+
+
+
RTRIM ( stringTerm )
+
+ + +

Removes trailing spaces from the input string.

+

Example:

+

RTRIM('Hello   ')

+ +

LENGTH

+ +
+LENGTH( stringTerm )
+
+
+
LENGTH ( stringTerm )
+
+ + +

Returns the length of the string in characters.

+

Example:

+

LENGTH('Hello')

+ +

REGEXP_SUBSTR

+ +
+REGEXP_SUBSTR( stringTerm, patternString [, startInt ] )
+
+
+
REGEXP_SUBSTR ( stringTerm , patternString
 
, startInt
)
+
+ + +

Returns a substring of a string by applying a regular expression start from the offset of a one-based position. Just like with SUBSTR, if the start index is negative, then it is relative to the end of the string. If not specified, the start index defaults to 1.

+

Example:

+

REGEXP_SUBSTR('na1-appsrv35-sj35', '[^-]+') evaluates to 'na1'

+ +

REGEXP_REPLACE

+ +
+REGEXP_REPLACE( stringTerm, patternString [, replacementString ] )
+
+
+
REGEXP_REPLACE ( stringTerm , patternString
 
, replacementString
)
+
+ + +

Returns a string by applying a regular expression and replacing the matches with the replacement string. If the replacement string is not specified, it defaults to an empty string.

+

Example:

+

REGEXP_REPLACE('abc123ABC', '[0-9]+', '#') evaluates to 'abc#ABC'

+ +

UPPER

+ +
+UPPER( stringTerm )
+
+
+
UPPER ( stringTerm )
+
+ + +

Returns upper case string of the string argument.

+

Example:

+

UPPER('Hello')

+ +

LOWER

+ +
+LOWER( stringTerm )
+
+
+
LOWER ( stringTerm )
+
+ + +

Returns lower case string of the string argument.

+

Example:

+

LOWER('HELLO')

+ +

REVERSE

+ +
+REVERSE( stringTerm )
+
+
+
REVERSE ( stringTerm )
+
+ + +

Returns reversed string of the string argument.

+

Example:

+

REVERSE('Hello')

+ +

TO_CHAR

+ +
+TO_CHAR( timestampTerm | numberTerm [, formatString] )
+
+ + + +

Formats a date, time, timestamp, or number as a string. The default date format is yyyy-MM-dd HH:mm:ss and the default number format is #,##0.###. For details, see java.text.SimpleDateFormat for date/time values and java.text.DecimalFormat for numbers. This method returns a string.

+

Example:

+

TO_CHAR(myDate, '2001-02-03 04:05:06')
TO_CHAR(myDecimal, '#,##0.###')

+ +

ROUND

+ +
+ROUND(timestampTerm, {'DAY' | 'HOUR' | 'MINUTE' | 'SECOND' | 'MILLISECOND'} [, multiplierNumber])
+
+
+
ROUND ( timestampTerm ,
' DAY '
' HOUR '
' MINUTE '
' SECOND '
' MILLISECOND '
 
, multiplierNumber
)
+
+ + +

Rounds the timestamp to the nearest time unit specified. The multiplier is used to round to a multiple of a time unit (i.e. 10 minute) and defaults to 1 if not specified. This method returns a date.

+

Example:

+

ROUND(date, 'MINUTE', 30)
ROUND(time, 'HOUR')

+ +

TRUNCATE

+ +
+TRUNC(timestampTerm, {'DAY' | 'HOUR' | 'MINUTE' | 'SECOND' | 'MILLISECOND'} [, multiplierInt])
+
+
+
TRUNC ( timestampTerm ,
' DAY '
' HOUR '
' MINUTE '
' SECOND '
' MILLISECOND '
 
, multiplierInt
)
+
+ + +

Truncates the timestamp to the next time unit closer to 0. The multiplier is used to truncate to a multiple of a time unit (i.e. 10 minute) and defaults to 1 if not specified. This method returns a date.

+

Example:

+

TRUNCATE(timestamp, 'SECOND', 30)
TRUNCATE(date, 'DAY', 7)

+ +

TO_DATE

+ +
+TO_DATE( stringTerm [, formatString] )
+
+
+
TO_DATE ( stringTerm
 
, formatString
)
+
+ + +

Parses a string and returns a date. The most important format characters are: y year, M month, d day, H hour, m minute, s second. The default format string is yyyy-MM-dd HH:mm:ss. For details of the format, see java.text.SimpleDateFormat.

+

Example:

+

TO_DATE('Sat, 3 Feb 2001 03:05:06 GMT', 'EEE, d MMM yyyy HH:mm:ss z')

+ +

CURRENT_DATE

+ +
+CURRENT_DATE()
+
+
+
CURRENT_DATE ( )
+
+ + +

Returns the current server-side date, bound at the start of the execution of a query based on the current time on the region server owning the metadata of the table being queried.

+

Example:

+

CURRENT_DATE()

+ +

CURRENT_TIME

+ +
+CURRENT_TIME()
+
+
+
CURRENT_TIME ( )
+
+ + +

Same as CURRENT_DATE(), except returns a value of type TIME. In either case, the underlying representation is the epoch time as a long value.

+

Example:

+

CURRENT_TIME()

+ + + + + diff --git a/phoenix-core/src/site/language_reference_source/index.html b/phoenix-core/src/site/language_reference_source/index.html new file mode 100644 index 00000000..04bcec67 --- /dev/null +++ b/phoenix-core/src/site/language_reference_source/index.html @@ -0,0 +1,947 @@ + + + + +SQL Grammar + + + + + + +

Commands

+ + + + + + +
+ + SELECT
+ + UPSERT VALUES
+ + UPSERT SELECT
+ + DELETE
+ 		+ + CREATE
+ + DROP
+ + ALTER TABLE
+ + CREATE INDEX
+ 		+ + DROP INDEX
+ + ALTER INDEX
+ + EXPLAIN
+
+ + +

Other Grammar

+ + + + + + +
+ + Constraint
+ + Options
+ + Hint
+ + Column Def
+ + Table Ref
+ + Column Ref
+ + Select Expression
+ + Split Point
+ + Table Expression
+ + Order
+ + Expression
+ + And Condition
+ 		+ + Condition
+ + Compare
+ + Operand
+ + Summand
+ + Factor
+ + Term
+ + Row Value Constructor
+ + Bind Parameter
+ + Value
+ + Case
+ + Case When
+ + Name
+ 		+ + Quoted Name
+ + Alias
+ + Null
+ + Data Type
+ + String
+ + Boolean
+ + Numeric
+ + Int
+ + Long
+ + Decimal
+ + Number
+ + Comments
+
+ + + +

SELECT

+ +
+SELECT [/*+ hint */] [DISTINCT | ALL] selectExpression [,...]
+FROM tableExpression [( columnDef [,...] )] [ WHERE expression ]
+[ GROUP BY expression [,...] ] [ HAVING expression ]
+[ ORDER BY order [,...] ] [ LIMIT {bindParameter | number} ]
+
+
+
SELECT
 
/ * + hint * /
 
DISTINCT
ALL
selectExpression
 
, ...

FROM tableExpression
 
( columnDef
 
, ...
)
 
WHERE expression

 
GROUP BY expression
 
, ...
 
HAVING expression

 
ORDER BY order
 
, ...
 
LIMIT
bindParameter
number
+
+ + +

Selects data from a table. DISTINCT filters out duplicate results while ALL, the default, includes all results. FROM identifies the table being queried (single table only currently - no joins or derived tables yet). Dynamic columns not declared at create time may be defined in parenthesis after the table name and then used in the query. GROUP BY groups the the result by the given expression(s). HAVING filter rows after grouping. ORDER BY sorts the result by the given column(s) or expression(s) and is only allowed for aggregate queries or queries with a LIMIT clause. LIMIT limits the number of rows returned by the query with no limit applied if specified as null or less than zero. The LIMIT clause is executed after the ORDER BY clause to support TopN type queries. An optional hint overrides the default query plan.

+

Example:

+

+SELECT * FROM TEST;
SELECT a.* FROM TEST;
SELECT DISTINCT NAME FROM TEST;
SELECT ID, COUNT(1) FROM TEST GROUP BY ID;
SELECT NAME, SUM(VAL) FROM TEST GROUP BY NAME HAVING COUNT(1) > 2;
SELECT 'ID' COL, MAX(ID) AS MAX FROM TEST;
SELECT * FROM TEST LIMIT 1000;

+ +

UPSERT VALUES

+ +
+UPSERT INTO tableName [( { columnRef | columnDef } [,...] )] VALUES ( constantTerm [,...] )
+
+
+
UPSERT INTO tableName
 
(
columnRef
columnDef
 
, ...
)
VALUES ( constantTerm
 
, ...
)
+
+ + +

Inserts if not present and updates otherwise the value in the table. The list of columns is optional and if not present, the values will map to the column in the order they are declared in the schema. The values must evaluate to constants.

+

Example:

+

+UPSERT INTO TEST VALUES('foo','bar',3);
UPSERT INTO TEST(NAME,ID) VALUES('foo',123);

+ +

UPSERT SELECT

+ +
+UPSERT [/*+ hint */] INTO tableName [( { columnRef | columnDef } [,...] )] select
+
+
+
UPSERT
 
/ * + hint * /
INTO tableName
 
(
columnRef
columnDef
 
, ...
)
select
+
+ + +

Inserts if not present and updates otherwise rows in the table based on the results of running another query. The values are set based on their matching position between the source and target tables. The list of columns is optional and if not present will map to the column in the order they are declared in the schema. If auto commit is on, and both a) the target table matches the source table, and b) the select performs no aggregation, then the population of the target table will be done completely on the server-side (with constraint violations logged, but otherwise ignored). Otherwise, data is buffered on the client and, if auto commit is on, committed in row batches as specified by the UpsertBatchSize connection property (or the phoenix.mutate.upsertBatchSize HBase config property which defaults to 10000 rows)

+

Example:

+

+UPSERT INTO test.targetTable(col1, col2) SELECT col3, col4 FROM test.sourceTable WHERE col5 < 100
UPSERT INTO foo SELECT * FROM bar;

+ +

DELETE

+ +
+DELETE [/*+ hint */] FROM tableName [ WHERE expression ]
+[ ORDER BY order [,...] ] [ LIMIT {bindParameter | number} ]
+
+
+
DELETE
 
/ * + hint * /
FROM tableName
 
WHERE expression

 
ORDER BY order
 
, ...
 
LIMIT
bindParameter
number
+
+ + +

Deletes the rows selected by the where clause. If auto commit is on, the deletion is performed completely server-side.

+

Example:

+

+DELETE FROM TEST;
DELETE FROM TEST WHERE ID=123;
DELETE FROM TEST WHERE NAME LIKE 'foo%';

+ +

CREATE

+ +
+CREATE { TABLE | VIEW } [IF NOT EXISTS] tableRef
+( columnDef [,...] [constraint] )
+[tableOptions] [ SPLIT ON ( splitPoint [,...] ) ]
+
+
+
CREATE
TABLE
VIEW
 
IF NOT EXISTS
tableRef

( columnDef
 
, ...
 
constraint
)

 
tableOptions
 
SPLIT ON ( splitPoint
 
, ...
)
+
+ + +

Creates a new table or view. For the creation of a table, the HBase table and any column families referenced are created if they don't already exist (using uppercase names unless they are double quoted in which case they are case sensitive). Column families outside of the ones listed are not affected. At create time, an empty key value is added to the first column family of any existing rows. Upserts will also add this empty key value. This is done to improve query performance by having a key value column we can guarantee always being there (minimizing the amount of data that must be projected). Alternately, if a view is created, the HBase table and column families must already exist. No empty key value is added to existing rows and no data mutations are allowed - the view is read-only. Query performance for a view will not be as good as performance for a table. For a table only, HBase table and column configuration options may be passed through as key/value pairs to setup the HBase table as needed.

+

Example:

+

+CREATE TABLE my_schema.my_table ( id BIGINT not null primary key, date DATE not null)
CREATE TABLE my_table ( id INTEGER not null primary key desc, date DATE not null,
    m.db_utilization DECIMAL, i.db_utilization)
    m.DATA_BLOCK_ENCODING='DIFF'
CREATE TABLE stats.prod_metrics ( host char(50) not null, created_date date not null,
    txn_count bigint CONSTRAINT pk PRIMARY KEY (host, created_date) )
CREATE TABLE IF NOT EXISTS my_table ( id char(10) not null primary key, value integer)
    DATA_BLOCK_ENCODING='NONE',VERSIONS=?,MAX_FILESIZE=2000000 split on (?, ?, ?)

+ +

DROP

+ +
+DROP {TABLE | VIEW} [IF EXISTS] tableRef
+
+
+
DROP
TABLE
VIEW
 
IF EXISTS
tableRef
+
+ + +

Drops a table or view. When dropping a table, the data in the table is deleted. For a view, on the other hand, the data is not affected. Note that the schema is versioned, such that snapshot queries connecting at an earlier time stamp may still query against the dropped table, as the HBase table itself is not deleted.

+

Example:

+

+DROP TABLE my_schema.my_table
DROP VIEW my_view

+ +

ALTER TABLE

+ +
+ALTER TABLE tableRef { { ADD [IF NOT EXISTS] columnDef [options] } | { DROP COLUMN [IF EXISTS] columnRef } | { SET options } }
+
+
+
ALTER TABLE tableRef
ADD
 
IF NOT EXISTS
columnDef
 
options
DROP COLUMN
 
IF EXISTS
columnRef
SET options
+
+ + +

Alters an existing table by adding or removing a column or updating table options. When a column is dropped from a table, the data in that column is deleted as well. PK columns may not be dropped, and only nullable PK columns may be added. For a view, the data is not affected when a column is dropped. Note that creating or dropping columns only affects subsequent queries and data modifications. Snapshot queries that are connected at an earlier timestamp will still use the prior schema that was in place when the data was written.

+

Example:

+

+ALTER TABLE my_schema.my_table ADD d.dept_id char(10) VERSIONS=10
ALTER TABLE my_table ADD dept_name char(50)
ALTER TABLE my_table ADD parent_id char(15) null primary key
ALTER TABLE my_table DROP COLUMN d.dept_id
ALTER TABLE my_table DROP COLUMN dept_name
ALTER TABLE my_table DROP COLUMN parent_id
ALTER TABLE my_table SET IMMUTABLE_ROWS=true

+ +

CREATE INDEX

+ +
+CREATE INDEX [IF NOT EXISTS] indexName
+ON tableRef ( columnRef [ASC | DESC] [,...] )
+[ INCLUDE ( columnRef [,...] ) ]
+[indexOptions] [ SPLIT ON ( splitPoint [,...] ) ]
+
+
+
CREATE INDEX
 
IF NOT EXISTS
indexName

ON tableRef ( columnRef
 
ASC
DESC
 
, ...
)

 
INCLUDE ( columnRef
 
, ...
)

 
indexOptions
 
SPLIT ON ( splitPoint
 
, ...
)
+
+ + +

Creates a new secondary index on a table or view. The index will be automatically kept in sync with the table as the data changes. At query time, the optimizer will use the index if it contains all columns referenced in the query and produces the most efficient execution plan. If a table has rows that are write-once and append-only, then the table may set the IMMUTABLE_ROWS property to true (either up-front in the CREATE TABLE statement or afterwards in an ALTER TABLE statement). This reduces the overhead at write time to maintain the index. Otherwise, if this property is not set on the table, then incremental index maintenance will be performed on the server side when the data changes.

+

Example:

+

+CREATE INDEX my_idx ON sales.opportunity(last_updated_date DESC)
CREATE INDEX my_idx ON log.event(created_date DESC) INCLUDE (name, payload) SALT_BUCKETS=10
CREATE INDEX IF NOT EXISTS my_comp_idx ON server_metrics ( gc_time DESC, created_date DESC )
    DATA_BLOCK_ENCODING='NONE',VERSIONS=?,MAX_FILESIZE=2000000 split on (?, ?, ?)

+ +

DROP INDEX

+ +
+DROP INDEX [IF EXISTS] indexName ON tableRef
+
+
+
DROP INDEX
 
IF EXISTS
indexName ON tableRef
+
+ + +

Drops an index from a table. When dropping an index, the data in the index is deleted. Note that since metadata is versioned, snapshot queries connecting at an earlier time stamp may still use the index, as the HBase table backing the index is not deleted.

+

Example:

+

+DROP INDEX my_idx ON sales.opportunity
DROP INDEX IF EXISTS my_idx ON server_metrics

+ +

ALTER INDEX

+ +
+ALTER INDEX [IF EXISTS] indexName ON tableRef { DISABLE | REBUILD | UNUSABLE | USABLE }
+
+
+
ALTER INDEX
 
IF EXISTS
indexName ON tableRef
DISABLE
REBUILD
UNUSABLE
USABLE
+
+ + +

Alters the state of an existing index.  DISABLE will cause the no further index maintenance to be performed on the index and it will no longer be considered for use in queries. REBUILD will completely rebuild the index and upon completion will enable the index to be used in queries again. UNUSABLE will cause the index to no longer be considered for use in queries, however index maintenance will continue to be performed. USABLE will cause the index to again be considered for use in queries. Note that a disabled index must be rebuild and cannot be set as USABLE.

+

Example:

+

+ALTER INDEX my_idx ON sales.opportunity DISABLE
ALTER INDEX IF EXISTS my_idx ON server_metrics REBUILD

+ +

EXPLAIN

+ +
+EXPLAIN {select|upsertSelect|delete}
+
+ + + +

Computes the logical steps necessary to execute the given command. Each step is represented as a string in a single column result set row.

+

Example:

+

+EXPLAIN SELECT NAME, COUNT(*) FROM TEST GROUP BY NAME HAVING COUNT(*) > 2;
EXPLAIN SELECT entity_id FROM CORE.CUSTOM_ENTITY_DATA WHERE organization_id='00D300000000XHP' AND SUBSTR(entity_id,1,3) = '002' AND created_date < CURRENT_DATE()-1;

+ + +

Constraint

+ +
CONSTRAINT constraintName PRIMARY KEY ( columnName
 
ASC
DESC
 
, ...
)
+ + +

Defines a multi-part primary key constraint. Each column may be declared to be sorted in ascending or descending ordering. The default is ascending.

+

Example:

+

CONSTRAINT my_pk PRIMARY KEY (host,created_date)
CONSTRAINT my_pk PRIMARY KEY (host ASC,created_date DESC)

+ +

Options

+ +
 
familyName .
name =
value
bindParameter
 
, ...
+ + +

Sets an option on an HBase table or column by modifying the respective HBase metadata. The option applies to the named family or if omitted to all families if the name references an HColumnDescriptor property. Otherwise, the option applies to the HTableDescriptor.

One built-in option is SALT_BUCKETS. This option causes an extra byte to be transparently prepended to every row key to ensure an even distribution of write load across all your region servers. This is useful when your row key is always monotonically increasing causing hot spotting on a single region server. The byte is determined by hashing the row key and modding it with the SALT_BUCKETS value. The value may be from 1 to 256. If not split points are defined for the table, it will automatically be pre-split at each possible salt bucket value. For an excellent write-up of this technique, see http://blog.sematext.com/2012/04/09/hbasewd-avoid-regionserver-hotspotting-despite-writing-records-with-sequential-keys/

Another built-in options is IMMUTABLE_ROWS. Only tables with immutable rows are allowed to have indexes. Immutable rows are expected to be inserted once in their entirety and then never updated. This limitation will be removed once incremental index maintenance has been implemented. The current implementation inserts the index rows when the data row is inserted.

+

Example:

+

IMMUTABLE_ROWS=true
SALT_BUCKETS=10
DATA_BLOCK_ENCODING='NONE',a.VERSIONS=10
MAX_FILESIZE=2000000000,MEMSTORE_FLUSHSIZE=80000000

+ +

Hint

+ +
name
 
, ...
+ + +

Advanced features that overrides default query processing behavior. The supported hints include 1) SKIP_SCAN to force a skip scan to be performed on the query when it otherwise would not be. This option may improve performance if a query does not include the leading primary key column, but does include other, very selective primary key columns. 2) RANGE_SCAN to force a range scan to be performed on the query. This option may improve performance if a query filters on a range for non selective leading primary key column along with other primary key columns 3) NO_INTRA_REGION_PARALLELIZATION to prevent the spawning of multiple threads to process data within a single region. This option is useful when the overall data set being queries is known to be small. 4) NO_INDEX to force the data table to be used for a query, and 5) INDEX(<table_name> <index_name>...) to suggest which index to use for a given query. Double quotes may be used to surround a table_name and/or index_name that is case sensitive.

+

Example:

+

/*+ SKIP_SCAN */
/*+ RANGE_SCAN */
/*+ NO_INTRA_REGION_PARALLELIZATION */
/*+ NO_INDEX */
/*+ INDEX(employee emp_name_idx emp_start_date_idx) */

+ +

Column Def

+ +
columnRef dataType
 
 
NOT
NULL
 
PRIMARY KEY
 
ASC
DESC
+ + +

Define a new primary key column. The column name is case insensitive by default and case sensitive if double quoted. The sort order of a primary key may be ascending (ASC) or descending. The default is ascending.

+

Example:

+

id char(15) not null primary key
key integer null
m.response_time bigint

+ +

Table Ref

+ +
 
schemaName .
tableName
+ + +

References a table with an optional schema name qualifier

+

Example:

+

Sales.Contact
HR.Employee
Department

+ +

Column Ref

+ +
 
familyName .
columnName
+ + +

References a column with an optional family name qualifier

+

Example:

+

e.salary
dept_name

+ +

Select Expression

+ +
*
( familyName . * )
term
 
 
AS
columnAlias
+ + +

An expression in a SELECT statement. All columns in a table may be selected using *, and all columns in a column family may be selected using <familyName>.*.

+

Example:

+

*
cf.*
ID AS VALUE
VALUE + 1 VALUE_PLUS_ONE

+ +

Split Point

+ +
value
bindParameter
+ + +

Defines a split point for a table. Use a bind parameter with preparedStatement.setBinary(int,byte[]) to supply arbitrary bytes.

+

Example:

+

'A'

+ +

Table Expression

+ +
 
schemaName .
tableName
 
 
AS
tableAlias
+ + +

A reference to a table. Joins and sub queries are not currently supported.

+

Example:

+

PRODUCT_METRICS AS PM

+ +

Order

+ +
expression
 
ASC
DESC
 
NULLS
FIRST
LAST
+ + +

Sorts the result by an expression.

+

Example:

+

NAME DESC NULLS LAST

+ +

Expression

+ +
andCondition
 
OR andCondition
 
...
+ + +

Value or condition.

+

Example:

+

ID=1 OR NAME='Hi'

+ +

And Condition

+ +
condition
 
AND condition
 
...
+ + +

Value or condition.

+

Example:

+

ID=1 AND NAME='Hi'

+ +

Condition

+ +
operand
 
compare operand
 
NOT
IN ( constantOperand
 
, ...
)
 
NOT
LIKE operand
 
NOT
BETWEEN operand AND operand
IS
 
NOT
NULL
NOT expression
( expression )
+ + +

Boolean value or condition. When comparing with LIKE, the wildcards characters are _ (any one character) and % (any characters). To search for the characters % and _, the characters need to be escaped. The escape character is \ (backslash). Patterns that end with an escape character are invalid and the expression returns NULL. BETWEEN does an inclusive comparison for both operands.

+

Example:

+

NAME LIKE 'Jo%'

+ +

Compare

+ +
< >
< =
> =
=
<
>
! =
+ + +

Comparison operator. The operator != is the same as <>.

+

Example:

+

<>

+ +

Operand

+ +
summand
 
|| summand
 
...
+ + +

A string concatenation.

+

Example:

+

'foo'|| s

+ +

Summand

+ +
factor
 
+
-
factor
 
...
+ + +

An addition or subtraction of numeric or date type values

+

Example:

+

a + b
a - b

+ +

Factor

+ +
term
 
*
/
term
 
...
+ + +

A multiplication or division.

+

Example:

+

c * d
e / 5

+ +

Term

+ +
value
bindParameter
Function
case
caseWhen
( operand )
 
tableAlias .
columnRef
rowValueConstructor
+ + +

A value.

+

Example:

+

'Hello'

+ +

Row Value Constructor

+ +
( term , term
 
...
)
+ + +

A row value constructor is a list of other terms which are treated together as a kind of composite structure. They may be compared to each other or to other other terms. The main use case is 1) to enable efficiently stepping through a set of rows in support of query-more type functionality, or 2) to allow IN clause to perform point gets on composite row keys.

+

Example:

+

(col1, col2, 5)

+ +

Bind Parameter

+ +
?
: number
+ + +

A parameters can be indexed, for example :1 meaning the first parameter.

+

Example:

+

:1
?

+ +

Value

+ +
string
numeric
boolean
null
+ + +

A literal value of any data type, or null.

+

Example:

+

10

+ +

Case

+ +
CASE term WHEN expression THEN term
 
...

 
ELSE expression
END
+ + +

Returns the first expression where the value is equal to the test expression. If no else part is specified, return NULL.

+

Example:

+

CASE CNT WHEN 0 THEN 'No' WHEN 1 THEN 'One' ELSE 'Some' END

+ +

Case When

+ +
CASE WHEN expression THEN term
 
...

 
ELSE term
END
+ + +

Returns the first expression where the condition is true. If no else part is specified, return NULL.

+

Example:

+

CASE WHEN CNT<10 THEN 'Low' ELSE 'High' END

+ +

Name

+ +
A-Z | _
 
A-Z | _
0-9
 
...
quotedName
+ + +

Unquoted names are not case sensitive. There is no maximum name length.

+

Example:

+

my_column

+ +

Quoted Name

+ +
" anything "
+ + +

Quoted names are case sensitive, and can contain spaces. There is no maximum name length. Two double quotes can be used to create a single double quote inside an identifier.

+

Example:

+

"first-name"

+ +

Alias

+ +name + + +

An alias is a name that is only valid in the context of the statement.

+

Example:

+

A

+ +

Null

+ +NULL + + +

NULL is a value without data type and means 'unknown value'.

+

Example:

+

NULL

+ +

Data Type

+ +
charType
varcharType
decimalType
tinyintType
smallintType
integerType
bigintType
floatType
doubleType
timestampType
dateType
timeType
unsignedTinyintType
unsignedSmallintType
unsignedIntType
unsignedLongType
unsignedFloatType
unsignedDoubleType
binaryType
varbinaryType
+ + +

A type name.

+

Example:

+

CHAR(15)
VARCHAR
VARCHAR(1000)
INTEGER
BINARY(200)

+ +

String

+ +
' anything '
+ + +

A string starts and ends with a single quote. Two single quotes can be used to create a single quote inside a string.

+

Example:

+

'John''s car'

+ +

Boolean

+ +
TRUE
FALSE
+ + +

A boolean value.

+

Example:

+

TRUE

+ +

Numeric

+ +
int
long
decimal
+ + +

The data type of a numeric value is always the lowest possible for the given value. If the number contains a dot this is decimal; otherwise it is int, long, or decimal (depending on the value).

+

Example:

+

SELECT -10.05
SELECT 5
SELECT 12345678912345

+ +

Int

+ +
 
-
number
+ + +

The maximum integer number is 2147483647, the minimum is -2147483648.

+

Example:

+

10

+ +

Long

+ +
 
-
number
+ + +

Long numbers are between -9223372036854775808 and 9223372036854775807.

+

Example:

+

100000

+ +

Decimal

+ +
 
-
number
 
. number
+ + +

A decimal number with fixed precision and scale. Internally, java.lang.BigDecimal is used.

+

Example:

+

SELECT -10.5

+ +

Number

+ +
0-9
 
...
+ + +

The maximum length of the number depends on the data type used.

+

Example:

+

100

+ +

Comments

+ +
- - anything
/ / anything
/ * anything * /
+ + +

Comments can be used anywhere in a command and are ignored by the database. Line comments end with a newline. Block comments cannot be nested, but can be multiple lines long.

+

Example:

+

// This is a comment

+ + + + diff --git a/phoenix-core/src/site/markdown/Phoenix-in-15-minutes-or-less.md b/phoenix-core/src/site/markdown/Phoenix-in-15-minutes-or-less.md new file mode 100644 index 00000000..bc01b185 --- /dev/null +++ b/phoenix-core/src/site/markdown/Phoenix-in-15-minutes-or-less.md @@ -0,0 +1,80 @@ +# Phoenix in 15 minutes or less + +*What is this new [Phoenix](index.html) thing I've been hearing about?*
+Phoenix is an open source SQL skin for HBase. You use the standard JDBC APIs instead of the regular HBase client APIs to create tables, insert data, and query your HBase data. + +*Doesn't putting an extra layer between my application and HBase just slow things down?*
+Actually, no. Phoenix achieves as good or likely better [performance](performance.html) than if you hand-coded it yourself (not to mention with a heck of a lot less code) by: +* compiling your SQL queries to native HBase scans +* determining the optimal start and stop for your scan key +* orchestrating the parallel execution of your scans +* bringing the computation to the data by + * pushing the predicates in your where clause to a server-side filter + * executing aggregate queries through server-side hooks (called co-processors) + +In addition to these items, we've got some interesting enhancements in the works to further optimize performance: +* secondary indexes to improve performance for queries on non row key columns +* stats gathering to improve parallelization and guide choices between optimizations +* skip scan filter to optimize IN, LIKE, and OR queries +* optional salting of row keys to evenly distribute write load + +*Ok, so it's fast. But why SQL? It's so 1970s*
+Well, that's kind of the point: give folks something with which they're already familiar. What better way to spur the adoption of HBase? On top of that, using JDBC and SQL: +* Reduces the amount of code users need to write +* Allows for performance optimizations transparent to the user +* Opens the door for leveraging and integrating lots of existing tooling + +*But how can SQL support my favorite HBase technique of x,y,z*
+Didn't make it to the last HBase Meetup did you? SQL is just a way of expressing *what you want to get* not *how you want to get it*. Check out my [presentation](http://files.meetup.com/1350427/IntelPhoenixHBaseMeetup.ppt) for various existing and to-be-done Phoenix features to support your favorite HBase trick. Have ideas of your own? We'd love to hear about them: file an [issue](issues.html) for us and/or join our [mailing list](mailing_list.html). + +*Blah, blah, blah - I just want to get started!*
+Ok, great! Just follow our [install instructions](download.html#Installation): +* [download](download.html) and expand our installation tar +* copy the phoenix jar into the HBase lib directory of every region server +* restart the region servers +* add the phoenix client jar to the classpath of your HBase client +* download and [setup SQuirrel](download.html#SQL-Client) as your SQL client so you can issue adhoc SQL against your HBase cluster + +*I don't want to download and setup anything else!*
+Ok, fair enough - you can create your own SQL scripts and execute them using our command line tool instead. Let's walk through an example now. In the bin directory of your install location: +* Create us_population.sql file +
CREATE TABLE IF NOT EXISTS us_population (
+      state CHAR(2) NOT NULL,
+      city VARCHAR NOT NULL,
+      population BIGINT
+      CONSTRAINT my_pk PRIMARY KEY (state, city));
+* Create us_population.csv file +
NY,New York,8143197
+CA,Los Angeles,3844829
+IL,Chicago,2842518
+TX,Houston,2016582
+PA,Philadelphia,1463281
+AZ,Phoenix,1461575
+TX,San Antonio,1256509
+CA,San Diego,1255540
+TX,Dallas,1213825
+CA,San Jose,912332
+
+* Create us_population_queries.sql file +
SELECT state as "State",count(city) as "City Count",sum(population) as "Population Sum"
+FROM us_population
+GROUP BY state
+ORDER BY sum(population) DESC;
+
+* Execute the following command from a command terminal +
./psql.sh <your_zookeeper_quorum> us_population.sql us_population.csv us_population_queries.sql
+
+ +Congratulations! You've just created your first Phoenix table, inserted data into it, and executed an aggregate query with just a few lines of code in 15 minutes or less! + +*Big deal - 10 rows! What else you got?*
+Ok, ok - tough crowd. Check out our bin/performance.sh script to create as many rows as you want, for any schema you come up with, and run timed queries against it. + +*Why is it called Phoenix anyway? Did some other project crash and burn and this is the next generation?*
+I'm sorry, but we're out of time and space, so we'll have to answer that next time! + +Thanks for your time,
+James Taylor
+http://phoenix-hbase.blogspot.com/ +
+@JamesPlusPlus
diff --git a/phoenix-core/src/site/markdown/building.md b/phoenix-core/src/site/markdown/building.md new file mode 100644 index 00000000..92f9a6f2 --- /dev/null +++ b/phoenix-core/src/site/markdown/building.md @@ -0,0 +1,20 @@ +# Building Phoenix Project + +Phoenix is a fully mavenized project. That means you can build simply by doing: +``` + $ mvn package +``` + +builds, test and package Phoenix and put the resulting jars (phoenix-[version].jar and phoenix-[version]-client.jar) in the generated target/ directory. + +To build, but skip running the tests, you can do: +``` + $ mvn package -DskipTests +``` +To only build the generated parser (i.e. PhoenixSQLLexer and PhoenixSQLParser), you can do: +``` + $ mvn process-sources +``` + +To build an Eclipse project, install the m2e plugin and do an File->Import...->Import Existing Maven Projects selecting the root directory of Phoenix. + diff --git a/phoenix-core/src/site/markdown/download.md b/phoenix-core/src/site/markdown/download.md new file mode 100644 index 00000000..147bc789 --- /dev/null +++ b/phoenix-core/src/site/markdown/download.md @@ -0,0 +1,84 @@ +## Available Phoenix Downloads + +### Download link will be available soon. + +
+ +### Installation ### +To install a pre-built phoenix, use these directions: + +* Download and expand the latest phoenix-[version]-install.tar +* Add the phoenix-[version].jar to the classpath of every HBase region server. An easy way to do this is to copy it into the HBase lib directory. +* Restart all region servers. +* Add the phoenix-[version]-client.jar to the classpath of any Phoenix client. + +### Getting Started ### +Wanted to get started quickly? Take a look at our [FAQs](faq.html) and take our quick start guide [here](Phoenix-in-15-minutes-or-less.html). + +

Command Line

+ +A terminal interface to execute SQL from the command line is now bundled with Phoenix. To start it, execute the following from the bin directory: + + $ sqlline.sh localhost + +To execute SQL scripts from the command line, you can include a SQL file argument like this: + + $ sqlline.sh localhost ../examples/stock_symbol.sql + +![sqlline](images/sqlline.png) + +For more information, see the [manual](http://www.hydromatic.net/sqlline/manual.html). + +
Loading Data
+ +In addition, you can use the bin/psql.sh to load CSV data or execute SQL scripts. For example: + + $ psql.sh localhost ../examples/web_stat.sql ../examples/web_stat.csv ../examples/web_stat_queries.sql + +Other alternatives include: +* Using our [map-reduce based CSV loader](mr_dataload.html) for bigger data sets +* [Mapping an existing HBase table to a Phoenix table](index.html#Mapping-to-an-Existing-HBase-Table) and using the [UPSERT SELECT](language/index.html#upsert_select) command to populate a new table. +* Populating the table through our [UPSERT VALUES](language/index.html#upsert_values) command. + +

SQL Client

+ +If you'd rather use a client GUI to interact with Phoenix, download and install [SQuirrel](http://squirrel-sql.sourceforge.net/). Since Phoenix is a JDBC driver, integration with tools such as this are seamless. Here are the setup steps necessary: + +1. Remove prior phoenix-[version]-client.jar from the lib directory of SQuirrel +2. Copy the phoenix-[version]-client.jar into the lib directory of SQuirrel (Note that on a Mac, this is the *internal* lib directory). +3. Start SQuirrel and add new driver to SQuirrel (Drivers -> New Driver) +4. In Add Driver dialog box, set Name to Phoenix +5. Press List Drivers button and org.apache.phoenix.jdbc.PhoenixDriver should be automatically populated in the Class Name textbox. Press OK to close this dialog. +6. Switch to Alias tab and create the new Alias (Aliases -> New Aliases) +7. In the dialog box, Name: _any name_, Driver: Phoenix, User Name: _anything_, Password: _anything_ +8. Construct URL as follows: jdbc:phoenix: _zookeeper quorum server_. For example, to connect to a local HBase use: jdbc:phoenix:localhost +9. Press Test (which should succeed if everything is setup correctly) and press OK to close. +10. Now double click on your newly created Phoenix alias and click Connect. Now you are ready to run SQL queries against Phoenix. + +Through SQuirrel, you can issue SQL statements in the SQL tab (create tables, insert data, run queries), and inspect table metadata in the Object tab (i.e. list tables, their columns, primary keys, and types). + +![squirrel](images/squirrel.png) + +### Samples ### +The best place to see samples are in our unit tests under src/test/java. The ones in the endToEnd package are tests demonstrating how to use all aspects of the Phoenix JDBC driver. We also have some examples in the examples directory. + +### Phoenix Client - Server Compatibility + +Major and minor version should match between client and server (patch version can mismatch). Following is the list of compatible client and server version(s). It is recommended that same client and server version are used. + +Phoenix Client Version | Compatible Server Versions +-----------------------|--- +1.0.0 | 1.0.0 +1.1.0 | 1.1.0 +1.2.0 | 1.2.0, 1.2.1 +1.2.1 | 1.2.0, 1.2.1 +2.0.0 | 2.0.0, 2.0.1, 2.0.2 +2.0.1 | 2.0.0, 2.0.1, 2.0.2 +2.0.2 | 2.0.0, 2.0.1, 2.0.2 +2.1.0 | 2.1.0, 2.1.1, 2.1.2 +2.1.1 | 2.1.0, 2.1.1, 2.1.2 +2.1.2 | 2.1.0, 2.1.1, 2.1.2 +2.2.0 | 2.2.0, 2.2.1 +2.2.1 | 2.2.0, 2.2.1 + +[![githalytics.com alpha](https://cruel-carlota.pagodabox.com/33878dc7c0522eed32d2d54db9c59f78 "githalytics.com")](http://githalytics.com/forcedotcom/phoenix.git) diff --git a/phoenix-core/src/site/markdown/dynamic_columns.md b/phoenix-core/src/site/markdown/dynamic_columns.md new file mode 100644 index 00000000..0c7d9ce2 --- /dev/null +++ b/phoenix-core/src/site/markdown/dynamic_columns.md @@ -0,0 +1,17 @@ +# Dynamic Columns + +Sometimes defining a static schema up front is not feasible. Instead, a subset of columns may be specified at table [create](language/index.html#create) time while the rest would be specified at [query](language/index.html#select) time. As of Phoenix 1.2, specifying columns dynamically is now supported by allowing column definitions to included in parenthesis after the table in the FROM clause on a SELECT statement. Although this is not standard SQL, it is useful to surface this type of functionality to leverage the late binding ability of HBase. + +For example: + + SELECT eventTime, lastGCTime, usedMemory, maxMemory + FROM EventLog(lastGCTime TIME, usedMemory BIGINT, maxMemory BIGINT) + WHERE eventType = 'OOM' AND lastGCTime < eventTime - 1 + +Where you may have defined only a subset of your event columns at create time, since each event type may have different properties: + + CREATE TABLE EventLog ( + eventId BIGINT NOT NULL, + eventTime TIME NOT NULL, + eventType CHAR(3) NOT NULL + CONSTRAINT pk PRIMARY KEY (eventId, eventTime)) diff --git a/phoenix-core/src/site/markdown/faq.md b/phoenix-core/src/site/markdown/faq.md new file mode 100644 index 00000000..cbcfc0a1 --- /dev/null +++ b/phoenix-core/src/site/markdown/faq.md @@ -0,0 +1,279 @@ +# F.A.Q. + +* [I want to get started. Is there a Phoenix Hello World?](#I_want_to_get_started_Is_there_a_Phoenix_Hello_World) +* [Is there a way to bulk load in Phoenix?](#Is_there_a_way_to_bulk_load_in_Phoenix) +* [How do I create a VIEW in Phoenix? What's the difference between a VIEW and a TABLE?](#How_I_create_Views_in_Phoenix_Whatnulls_the_difference_between_ViewsTables) +* [Are there any tips for optimizing Phoenix?](#Are_there_any_tips_for_optimizing_Phoenix) +* [How do I create Secondary Index on a table?](#How_do_I_create_Secondary_Index_on_a_table) +* [Why isn't my secondary index being used?](#Why_isnnullt_my_secondary_index_being_used) +* [How fast is Phoenix? Why is it so fast?](#How_fast_is_Phoenix_Why_is_it_so_fast) +* [How do I connect to secure HBase cluster?](#How_do_I_connect_to_secure_HBase_cluster) +* [How do I connect with HBase running on Hadoop-2?](#How_do_I_connect_with_HBase_running_on_Hadoop-2) +* [Can phoenix work on tables with arbitrary timestamp as flexible as HBase API?](#Can_phoenix_work_on_tables_with_arbitrary_timestamp_as_flexible_as_HBase_API) +* [Why isn't my query doing a RANGE SCAN?](#Why_isnnullt_my_query_doing_a_RANGE_SCAN) + + +### I want to get started. Is there a Phoenix _Hello World_? + +*Pre-requisite:* Download latest Phoenix from [here](download.html) +and copy phoenix-*.jar to HBase lib folder and restart HBase. + +**1. Using console** + +1. Start Sqlline: `$ sqlline.sh [zookeeper]` +2. Execute the following statements when Sqlline connects: + +``` +create table test (mykey integer not null primary key, mycolumn varchar); +upsert into test values (1,'Hello'); +upsert into test values (2,'World!'); +select * from test; +``` + +3. You should get the following output + +``` ++-------+------------+ +| MYKEY | MYCOLUMN | ++-------+------------+ +| 1 | Hello | +| 2 | World! | ++-------+------------+ +``` + + +**2. Using java** + +Create test.java file with the following content: + +``` +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.sql.PreparedStatement; +import java.sql.Statement; + +public class test { + + public static void main(String[] args) throws SQLException { + Statement stmt = null; + ResultSet rset = null; + + Connection con = DriverManager.getConnection("jdbc:phoenix:[zookeeper]"); + stmt = con.createStatement(); + + stmt.executeUpdate("create table test (mykey integer not null primary key, mycolumn varchar)"); + stmt.executeUpdate("upsert into test values (1,'Hello')"); + stmt.executeUpdate("upsert into test values (2,'World!')"); + con.commit(); + + PreparedStatement statement = con.prepareStatement("select * from test"); + rset = statement.executeQuery(); + while (rset.next()) { + System.out.println(rset.getString("mycolumn")); + } + statement.close(); + con.close(); + } +} +``` +Compile and execute on command line + +`$ javac test.java` + +`$ java -cp "../phoenix-[version]-client.jar:." test` + + +You should get the following output + +`Hello` +`World!` + + + +### Is there a way to bulk load in Phoenix? + +**Map Reduce** + +See the example [here](mr_dataload.html) Credit: Arun Singh + +**CSV** + +CSV data can be bulk loaded with built in utility named psql. Typical upsert rates are 20K - 50K rows per second (depends on how wide are the rows). + +Usage example: +Create table using psql +`$ psql.sh [zookeeper] ../examples/web_stat.sql` + +Upsert CSV bulk data +`$ psql.sh [zookeeper] ../examples/web_stat.csv` + + + +### How I create Views in Phoenix? What's the difference between Views/Tables? + +You can create both a Phoenix table or view through the CREATE TABLE/CREATE VIEW DDL statement on a pre-existing HBase table. In both cases, we'll leave the HBase metadata as-is, except for with a TABLE we turn KEEP_DELETED_CELLS on. For CREATE TABLE, we'll create any metadata (table, column families) that doesn't already exist. We'll also add an empty key value for each row so that queries behave as expected (without requiring all columns to be projected during scans). + +The other caveat is that the way the bytes were serialized must match the way the bytes are serialized by Phoenix. For VARCHAR,CHAR, and UNSIGNED_* types, we use the HBase Bytes methods. The CHAR type expects only single-byte characters and the UNSIGNED types expect values greater than or equal to zero. + +Our composite row keys are formed by simply concatenating the values together, with a zero byte character used as a separator after a variable length type. + +If you create an HBase table like this: + +`create 't1', {NAME => 'f1', VERSIONS => 5}` + +then you have an HBase table with a name of 't1' and a column family with a name of 'f1'. Remember, in HBase, you don't model the possible KeyValues or the structure of the row key. This is the information you specify in Phoenix above and beyond the table and column family. + +So in Phoenix, you'd create a view like this: + +`CREATE VIEW "t1" ( pk VARCHAR PRIMARY KEY, "f1".val VARCHAR )` + +The "pk" column declares that your row key is a VARCHAR (i.e. a string) while the "f1".val column declares that your HBase table will contain KeyValues with a column family and column qualifier of "f1":VAL and that their value will be a VARCHAR. + +Note that you don't need the double quotes if you create your HBase table with all caps names (since this is how Phoenix normalizes strings, by upper casing them). For example, with: + +`create 'T1', {NAME => 'F1', VERSIONS => 5}` + +you could create this Phoenix view: + +`CREATE VIEW t1 ( pk VARCHAR PRIMARY KEY, f1.val VARCHAR )` + +Or if you're creating new HBase tables, just let Phoenix do everything for you like this (No need to use the HBase shell at all.): + +`CREATE TABLE t1 ( pk VARCHAR PRIMARY KEY, val VARCHAR )` + + + +### Are there any tips for optimizing Phoenix? + +* Use **Salting** to increase read/write performance +Salting can significantly increase read/write performance by pre-splitting the data into multiple regions. Although Salting will yield better performance in most scenarios. + +Example: + +` CREATE TABLE TEST (HOST VARCHAR NOT NULL PRIMARY KEY, DESCRIPTION VARCHAR) SALT_BUCKETS=16` + +Note: Ideally for a 16 region server cluster with quad-core CPUs, choose salt buckets between 32-64 for optimal performance. + +* **Per-split** table +Salting does automatic table splitting but in case you want to exactly control where table split occurs with out adding extra byte or change row key order then you can pre-split a table. + +Example: + +` CREATE TABLE TEST (HOST VARCHAR NOT NULL PRIMARY KEY, DESCRIPTION VARCHAR) SPLIT ON ('CS','EU','NA')` + +* Use **multiple column families** + +Column family contains related data in separate files. If you query use selected columns then it make sense to group those columns together in a column family to improve read performance. + +Example: + +Following create table DDL will create two column familes A and B. + +` CREATE TABLE TEST (MYKEY VARCHAR NOT NULL PRIMARY KEY, A.COL1 VARCHAR, A.COL2 VARCHAR, B.COL3 VARCHAR)` + +* Use **compression** +On disk compression improves performance on large tables + +Example: + +` CREATE TABLE TEST (HOST VARCHAR NOT NULL PRIMARY KEY, DESCRIPTION VARCHAR) COMPRESSION='GZ'` + +* Create **indexes** +See [faq.html#/How_do_I_create_Secondary_Index_on_a_table](faq.html#/How_do_I_create_Secondary_Index_on_a_table) + +* **Optimize cluster** parameters +See http://hbase.apache.org/book/performance.html + +* **Optimize Phoenix** parameters +See [tuning.html](tuning.html) + + + +### How do I create Secondary Index on a table? + +Starting with Phoenix version 2.1, Phoenix supports index over mutable and immutable data. Note that Phoenix 2.0.x only supports Index over immutable data. Index write performance index with immutable table is slightly faster than mutable table however data in immutable table cannot be updated. + +Example + +* Create table + +Immutable table: `create table test (mykey varchar primary key, col1 varchar, col2 varchar) IMMUTABLE_ROWS=true;` + +Mutable table: `create table test (mykey varchar primary key, col1 varchar, col2 varchar);` + +* Creating index on col2 + +`create index idx on test (col2)` + +* Creating index on col1 and a covered index on col2 + +`create index idx on test (col1) include (col2)` + +Upsert rows in this test table and Phoenix query optimizer will choose correct index to use. You can see in [explain plan](language/index.html#explain) if Phoenix is using the index table. You can also give a [hint](language/index.html#hint) in Phoenix query to use a specific index. + + + +### Why isn't my secondary index being used? + +The secondary index won't be used unless all columns used in the query are in it ( as indexed or covered columns). All columns making up the primary key of the data table will automatically be included in the index. + +Example: DDL `create table usertable (id varchar primary key, firstname varchar, lastname varchar); create index idx_name on usertable (firstname);` + +Query: DDL `select id, firstname, lastname from usertable where firstname = 'foo';` + +Index would not be used in this case as lastname is not part of indexed or covered column. This can be verified by looking at the explain plan. To fix this create index that has either lastname part of index or covered column. Example: `create idx_name on usertable (firstname) include (lastname);` + + +### How fast is Phoenix? Why is it so fast? + +Phoenix is fast. Full table scan of 100M rows usually completes in 20 seconds (narrow table on a medium sized cluster). This time come down to few milliseconds if query contains filter on key columns. For filters on non-key columns or non-leading key columns, you can add index on these columns which leads to performance equivalent to filtering on key column by making copy of table with indexed column(s) part of key. + +Why is Phoenix fast even when doing full scan: + +1. Phoenix chunks up your query using the region boundaries and runs them in parallel on the client using a configurable number of threads +2. The aggregation will be done in a coprocessor on the server-side, collapsing the amount of data that gets returned back to the client rather than returning it all. + + + +### How do I connect to secure HBase cluster? +Check out excellent post by Anil Gupta +http://bigdatanoob.blogspot.com/2013/09/connect-phoenix-to-secure-hbase-cluster.html + + + +### How do I connect with HBase running on Hadoop-2? +Hadoop-2 profile exists in Phoenix pom.xml. + + +### Can phoenix work on tables with arbitrary timestamp as flexible as HBase API? +By default, Phoenix let's HBase manage the timestamps and just shows you the latest values for everything. However, Phoenix also allows arbitrary timestamps to be supplied by the user. To do that you'd specify a "CurrentSCN" (or PhoenixRuntime.CURRENT_SCN_ATTRIB if you want to use our constant) at connection time, like this: + + Properties props = new Properties(); + props.setProperty(PhoenixRuntime.CURRENT_SCN_ATTRIB, Long.toString(ts)); + Connection conn = DriverManager.connect(myUrl, props); + + conn.createStatement().execute("UPSERT INTO myTable VALUES ('a')"); + conn.commit(); +The above is equivalent to doing this with the HBase API: + + myTable.put(Bytes.toBytes('a'),ts); +By specifying a CurrentSCN, you're telling Phoenix that you want everything for that connection to be done at that timestamp. Note that this applies to queries done on the connection as well - for example, a query over myTable above would not see the data it just upserted, since it only sees data that was created before its CurrentSCN property. This provides a way of doing snapshot, flashback, or point-in-time queries. + +Keep in mind that creating a new connection is *not* an expensive operation. The same underlying HConnection is used for all connections to the same cluster, so it's more or less like instantiating a few objects. + + +### Why isn't my query doing a RANGE SCAN? + +`DDL: CREATE TABLE TEST (pk1 char(1) not null, pk2 char(1) not null, pk3 char(1) not null, non-pk varchar CONSTRAINT PK PRIMARY KEY(pk1, pk2, pk3));` + +RANGE SCAN means that only a subset of the rows in your table will be scanned over. This occurs if you use one or more leading columns from your primary key constraint. Query that is not filtering on leading PK columns ex. `select * from test where pk2='x' and pk3='y';` will result in full scan whereas the following query will result in range scan `select * from test where pk1='x' and pk2='y';`. Note that you can add a secondary index on your "pk2" and "pk3" columns and that would cause a range scan to be done for the first query (over the index table). + +DEGENERATE SCAN means that a query can't possibly return any rows. If we can determine that at compile time, then we don't bother to even run the scan. + +FULL SCAN means that all rows of the table will be scanned over (potentially with a filter applied if you have a WHERE clause) + +SKIP SCAN means that either a subset or all rows in your table will be scanned over, however it will skip large groups of rows depending on the conditions in your filter. See this blog for more detail. We don't do a SKIP SCAN if you have no filter on the leading primary key columns, but you can force a SKIP SCAN by using the /*+ SKIP_SCAN */ hint. Under some conditions, namely when the cardinality of your leading primary key columns is low, it will be more efficient than a FULL SCAN. + + diff --git a/phoenix-core/src/site/markdown/flume.md b/phoenix-core/src/site/markdown/flume.md new file mode 100644 index 00000000..6cc9251a --- /dev/null +++ b/phoenix-core/src/site/markdown/flume.md @@ -0,0 +1,42 @@ +# Apache Flume Plugin + +The plugin enables us to reliably and efficiently stream large amounts of data/logs onto HBase using the Phoenix API. The necessary configuration of the custom Phoenix sink and the Event Serializer has to be configured in the Flume configuration file for the Agent. Currently, the only supported Event serializer is a RegexEventSerializer which primarily breaks the Flume Event body based on the regex specified in the configuration file. + +#### Prerequisites: + +* Phoenix v 3.0.0 SNAPSHOT + +* Flume 1.4.0 + + +#### Installation & Setup: + +1. Download and build Phoenix v 0.3.0 SNAPSHOT +2. Follow the instructions as specified [here](building.html) to build the project as the Flume plugin is still under beta +3. Create a directory plugins.d within $FLUME_HOME directory. Within that, create a sub-directories phoenix-sink/lib +4. Copy the generated phoenix-3.0.0-SNAPSHOT-client.jar onto $FLUME_HOME/plugins.d/phoenix-sink/lib + +#### Configuration: + +Property Name |Default| Description +--------------------------|-------|--- +type | |org.apache.phoenix.flume.sink.PhoenixSink +batchSize |100 |Default number of events per transaction +zookeeperQuorum | |Zookeeper quorum of the HBase cluster +table | |The name of the table in HBase to write to. +ddl | |The CREATE TABLE query for the HBase table where the events will be upserted to. If specified, the query will be executed. Recommended to include the IF NOT EXISTS clause in the ddl. +serializer |regex |Event serializers for processing the Flume Event . Currently , only regex is supported. +serializer.regex |(.*) |The regular expression for parsing the event. +serializer.columns | |The columns that will be extracted from the Flume event for inserting into HBase. +serializer.headers | |Headers of the Flume Events that go as part of the UPSERT query. The data type for these columns are VARCHAR by default. +serializer.rowkeyType | |A custom row key generator . Can be one of timestamp,date,uuid,random and nanotimestamp. This should be configured in cases where we need a custom row key value to be auto generated and set for the primary key column. + + +For an example configuration for ingesting Apache access logs onto Phoenix, see [this](https://github.com/forcedotcom/phoenix/blob/master/src/main/config/apache-access-logs.properties) property file. Here we are using UUID as a row key generator for the primary key. + +#### Starting the agent: + $ bin/flume-ng agent -f conf/flume-conf.properties -c ./conf -n agent + +#### Monitoring: + For monitoring the agent and the sink process , enable JMX via flume-env.sh($FLUME_HOME/conf/flume-env.sh) script. Ensure you have the following line uncommented. + + JAVA_OPTS="-Xms1g -Xmx1g -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=3141 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false" + diff --git a/phoenix-core/src/site/markdown/index.md b/phoenix-core/src/site/markdown/index.md new file mode 100644 index 00000000..8b9f0b04 --- /dev/null +++ b/phoenix-core/src/site/markdown/index.md @@ -0,0 +1,69 @@ +# Overview + +Apache Phoenix is a SQL skin over HBase delivered as a client-embedded JDBC driver targeting low latency queries over HBase data. Apache Phoenix takes your SQL query, compiles it into a series of HBase scans, and orchestrates the running of those scans to produce regular JDBC result sets. The table metadata is stored in an HBase table and versioned, such that snapshot queries over prior versions will automatically use the correct schema. Direct use of the HBase API, along with coprocessors and custom filters, results in [performance](performance.html) on the order of milliseconds for small queries, or seconds for tens of millions of rows. + +## Mission +Become the standard means of accessing HBase data through a well-defined, industry standard API. + +## Quick Start +Tired of reading already and just want to get started? Take a look at our [FAQs](faq.html), listen to the Apache Phoenix talks from [Hadoop Summit 2013](http://www.youtube.com/watch?v=YHsHdQ08trg) and [HBaseConn 2013](http://www.cloudera.com/content/cloudera/en/resources/library/hbasecon/hbasecon-2013--how-and-why-phoenix-puts-the-sql-back-into-nosql-video.html), and jump over to our quick start guide [here](Phoenix-in-15-minutes-or-less.html). + +##SQL Support## +To see what's supported, go to our [language reference](language/index.html). It includes all typical SQL query statement clauses, including `SELECT`, `FROM`, `WHERE`, `GROUP BY`, `HAVING`, `ORDER BY`, etc. It also supports a full set of DML commands as well as table creation and versioned incremental alterations through our DDL commands. We try to follow the SQL standards wherever possible. + +Use JDBC to get a connection to an HBase cluster like this: + + Connection conn = DriverManager.getConnection("jdbc:phoenix:server1,server2:3333"); +where the connection string is composed of: +jdbc:phoenix [ :<zookeeper quorum> [ :<port number> ] [ :<root node> ] ] + +For any omitted part, the relevant property value, hbase.zookeeper.quorum, hbase.zookeeper.property.clientPort, and zookeeper.znode.parent will be used from hbase-site.xml configuration file. + +Here's a list of what is currently **not** supported: + +* **Full Transaction Support**. Although we allow client-side batching and rollback as described [here](#transactions), we do not provide transaction semantics above and beyond what HBase gives you out-of-the-box. +* **Derived tables**. Nested queries are coming soon. +* **Relational operators**. Union, Intersect, Minus. +* **Miscellaneous built-in functions**. These are easy to add - read this [blog](http://phoenix-hbase.blogspot.com/2013/04/how-to-add-your-own-built-in-function.html) for step by step instructions. + +##Schema## + +Apache Phoenix supports table creation and versioned incremental alterations through DDL commands. The table metadata is stored in an HBase table. + +A Phoenix table is created through the [CREATE TABLE](language/index.html#create) DDL command and can either be: + +1. **built from scratch**, in which case the HBase table and column families will be created automatically. +2. **mapped to an existing HBase table**, by creating either a read-write TABLE or a read-only VIEW, with the caveat that the binary representation of the row key and key values must match that of the Phoenix data types (see [Data Types reference](datatypes.html) for the detail on the binary representation). + * For a read-write TABLE, column families will be created automatically if they don't already exist. An empty key value will be added to the first column family of each existing row to minimize the size of the projection for queries. + * For a read-only VIEW, all column families must already exist. The only change made to the HBase table will be the addition of the Phoenix coprocessors used for query processing. The primary use case for a VIEW is to transfer existing data into a Phoenix table, since data modification are not allowed on a VIEW and query performance will likely be less than as with a TABLE. + +All schema is versioned, and prior versions are stored forever. Thus, snapshot queries over older data will pick up and use the correct schema for each row. + +####Salting +A table could also be declared as salted to prevent HBase region hot spotting. You just need to declare how many salt buckets your table has, and Phoenix will transparently manage the salting for you. You'll find more detail on this feature [here](salted.html), along with a nice comparison on write throughput between salted and unsalted tables [here](performance.htm#salting). + +####Schema at Read-time +Another schema-related feature allows columns to be defined dynamically at query time. This is useful in situations where you don't know in advance all of the columns at create time. You'll find more details on this feature [here](dynamic_columns.html). + +####Mapping to an Existing HBase Table +Apache Phoenix supports mapping to an existing HBase table through the [CREATE TABLE](language/index.html#create) and [CREATE VIEW](language/index.html#create) DDL statements. In both cases, the HBase metadata is left as-is, except for with CREATE TABLE the [KEEP_DELETED_CELLS](http://hbase.apache.org/book/cf.keep.deleted.html) option is enabled to allow for flashback queries to work correctly. For CREATE TABLE, any HBase metadata (table, column families) that doesn't already exist will be created. Note that the table and column family names are case sensitive, with Phoenix upper-casing all names. To make a name case sensitive in the DDL statement, surround it with double quotes as shown below: + 
CREATE VIEW "MyTable" ("a".ID VARCHAR PRIMARY KEY)
+ +For CREATE TABLE, an empty key value will also be added for each row so that queries behave as expected (without requiring all columns to be projected during scans). For CREATE VIEW, this will not be done, nor will any HBase metadata be created. Instead the existing HBase metadata must match the metadata specified in the DDL statement or a ERROR 505 (42000): Table is read only will be thrown. + +The other caveat is that the way the bytes were serialized in HBase must match the way the bytes are expected to be serialized by Phoenix. For VARCHAR,CHAR, and UNSIGNED_* types, Phoenix uses the HBase Bytes utility methods to perform serialization. The CHAR type expects only single-byte characters and the UNSIGNED types expect values greater than or equal to zero. + +Our composite row keys are formed by simply concatenating the values together, with a zero byte character used as a separator after a variable length type. For more information on our type system, see the [Data Type](datatypes.html). + +##Transactions## +The DML commands of Apache Phoenix, [UPSERT VALUES](language/index.html#upsert_values), [UPSERT SELECT](language/index.html#upsert_select) and [DELETE](language/index.html#delete), batch pending changes to HBase tables on the client side. The changes are sent to the server when the transaction is committed and discarded when the transaction is rolled back. The only transaction isolation level we support is TRANSACTION_READ_COMMITTED. This includes not being able to see your own uncommitted data as well. Phoenix does not providing any additional transactional semantics beyond what HBase supports when a batch of mutations is submitted to the server. If auto commit is turned on for a connection, then Phoenix will, whenever possible, execute the entire DML command through a coprocessor on the server-side, so performance will improve. + +Most commonly, an application will let HBase manage timestamps. However, under some circumstances, an application needs to control the timestamps itself. In this case, a long-valued "CurrentSCN" property may be specified at connection time to control timestamps for any DDL, DML, or query. This capability may be used to run snapshot queries against prior row values, since Phoenix uses the value of this connection property as the max timestamp of scans. + +## Metadata ## +The catalog of tables, their columns, primary keys, and types may be retrieved via the java.sql metadata interfaces: `DatabaseMetaData`, `ParameterMetaData`, and `ResultSetMetaData`. For retrieving schemas, tables, and columns through the DatabaseMetaData interface, the schema pattern, table pattern, and column pattern are specified as in a LIKE expression (i.e. % and _ are wildcards escaped through the \ character). The table catalog argument to the metadata APIs deviates from a more standard relational database model, and instead is used to specify a column family name (in particular to see all columns in a given column family). + +
+## Disclaimer ## +Apache Phoenix is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the [Apache Incubator PMC](http://incubator.apache.org/). Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. +

diff --git a/phoenix-core/src/site/markdown/issues.md b/phoenix-core/src/site/markdown/issues.md new file mode 100644 index 00000000..64ea3ca4 --- /dev/null +++ b/phoenix-core/src/site/markdown/issues.md @@ -0,0 +1,9 @@ +# Issue Tracking + +This project uses JIRA issue tracking and project management application. Issues, bugs, and feature requests should be submitted to the following: + +
+ +https://issues.apache.org/jira/browse/PHOENIX + +
diff --git a/phoenix-core/src/site/markdown/mailing_list.md b/phoenix-core/src/site/markdown/mailing_list.md new file mode 100644 index 00000000..fbe9e43c --- /dev/null +++ b/phoenix-core/src/site/markdown/mailing_list.md @@ -0,0 +1,14 @@ +# Mailing Lists + +These are the mailing lists that have been established for this project. For each list, there is a subscribe, unsubscribe and post link. + +
+ +Name| Subscribe| Unsubscribe| Post +--------------------------|----|----|---- +User List | [Subscribe](mailto:user-subscribe@phoenix.incubator.apache.org) | [Unsubscribe](mailto:user-unsubscribe@phoenix.incubator.apache.org) | [Post](mailto:user@phoenix.incubator.apache.org) +Developer List | [Subscribe](mailto:dev-subscribe@phoenix.incubator.apache.org) | [Unsubscribe](mailto:dev-unsubscribe@phoenix.incubator.apache.org) | [Post](mailto:dev@phoenix.incubator.apache.org) +Private List | [Subscribe](mailto:private-subscribe@phoenix.incubator.apache.org) | [Unsubscribe](mailto:private-unsubscribe@phoenix.incubator.apache.org) | [Post](mailto:private@phoenix.incubator.apache.org) +Commits List | [Subscribe](mailto:commits-subscribe@phoenix.incubator.apache.org) | [Unsubscribe](mailto:commits-unsubscribe@phoenix.incubator.apache.org) | [Post](mailto:commits@phoenix.incubator.apache.org) + +
diff --git a/phoenix-core/src/site/markdown/mr_dataload.md b/phoenix-core/src/site/markdown/mr_dataload.md new file mode 100644 index 00000000..b0053ac1 --- /dev/null +++ b/phoenix-core/src/site/markdown/mr_dataload.md @@ -0,0 +1,63 @@ +# Bulk CSV Data Load using Map-Reduce + +Phoenix v 2.1 provides support for loading CSV data into a new/existing Phoenix table using Hadoop Map-Reduce. This provides a means of bulk loading CSV data in parallel through map-reduce, yielding better performance in comparison with the existing [psql csv loader](download.html#Loading-Data). + +####Sample input CSV data: + +``` +12345, John, Doe +67890, Mary, Poppins +``` + +####Compatible Phoenix schema to hold the above CSV data: + + CREATE TABLE ns.example ( + my_pk bigint not null, + m.first_name varchar(50), + m.last_name varchar(50) + CONSTRAINT pk PRIMARY KEY (my_pk)) + + + + + + +
Row Key
Column Family (m)
my_pk BIGINTfirst_name VARCHAR(50)last_name VARCHAR(50)
12345JohnDoe
67890MaryPoppins
+ + +####How to run? + +1- Please make sure that Hadoop cluster is working correctly and you are able to run any job like [this](http://wiki.apache.org/hadoop/WordCount). + +2- Copy latest phoenix-[version].jar to hadoop/lib folder on each node or add it to Hadoop classpath. + +3- Run the bulk loader job using the script /bin/csv-bulk-loader.sh as below: + +``` +./csv-bulk-loader.sh