Using the language manager for UDF compiler interface blueprint cmp-tmudf-compile-time-interface
This change includes new CLI calls, to be used in the compiler to invoke routines. Right now, only trusted routines are supported, executed in the same process as the caller, but in the future we may extend this to isolated routines. Using a CLI call allows us to share the language manager between compiler and executor, since language manager resources such as the JVM and loaded DLLs exist only once per process. This change is in preparation for Java UDFs.
Changes in a bit more detail:
- Added 4 new CLI calls to allocate a routine, invoke it, retrieve updated invocation and plan infos and deallocate (put) the routine. The CLI globals now have a C/C++ and a Java language manager that is allocated on demand. - The compiler no longer loads a DLL for the UDF compiler interface, it uses the new CLI calls instead. - DDL syntax is changed to allow TMUDFs in Java (not officially supported, so don't use it quite yet). - TMUDFs in C are no longer supported, only C++ and Java are. Converted remaining TMUDF tests to C++. - C++ TMUDFs now do a basic verification at DDL time, so errors like missing entry points are detected earlier. Validation for Java TMUDFs is also done through the CLI. - Make sure we have no memory or resource leaks: - CmpContext keeps track of UDF-related objects allocated on system heap and in the CLI, cleaned up at the end of a statement - CLI keeps a list of allocated trusted routines, cleaned up when a CLI context is deallocated - Using ExeCliInterface class to make the new CLI calls (4 new calls added). - Removed CmpCli class in the optimizer directory and converted tracking compiler to use ExeCliInterface as well. - Compile-time parameter values are no longer baked into the UDRInvocationInfo. Instead, they are provided as an input row, the same way as they are provided at runtime. - Bug fixes in C++ UDR code, mostly related to serialization and to multiple interactions with the UDF through serialized objects. - Added more info to UDRInvocationInfo (SQL access type, etc.). - Since there are multiple plans per invocation, each of which can have multiple interactions with the UDF, plans need to be numbered so the UDF side can tell them apart to attach the right state (owned by the UDF) to it. - The language manager needs some functions that are provided by the process it's running in. Added those (empty, for now) functions as cli/CliImplLmExtFunc.cpp. - Added a new class for Java TMUDFs, LmRoutineJavaObj. Added methods to allocate such routines and to load their class as well as to create Java objects by invoking the default constructor through JNI. - Java TMUDFs use the new UDR interface (to be provided by Suresh and Pavani). In the language manager, the container is the class of the UDF, the external path is the fully qualified jar name. The Java method name is <init>, the default constructor, with signature "()V". Some code changes were required to do this. - Created a new directory trafodion/core/sql/src for Java sources in the sql engine. Right now, only language manager java sources are in this directory, but I am planning to move the other java sources under sql in a future checkin. Suresh and Pavani will add their UDF-related Java files there as well. - Renamed the udr jar to trafodion-sql-<version>.jar, in anticipation of combining all the sql Java sources into this jar. - Created a maven project file trafodion/core/sql/pom.xml and changed makefiles to invoke maven to build java sources. - More work to separate new UDR interface from older SPInfo object, so that we can get rid of SPInfo if/when we don't support the older style anymore. - Small fix to odb makefile, make clean failed when executed twice.
Patch set 2: Adding a custom filter for test regress/udr/TEST108.
Column-level privileges Support for column-level privileges will be in multiple deliveries.
This delivery add the following portions:
1. Creation of the metadata table COLUMN_PRIVILEGE. This table is created when the INITIALIZE AUTHORIZATION command is run. Existing privileges are preserved, but warnings are issued referring to existing metadata tables. An UPDATE option will be added later.
2. Granting of column-level privileges Full support is present for granting column-level privileges. Privileges can be added and updated for one or more columns on a table or view. Support for WITH GRANT OPTION is coded, though not enabled until WITH GRANT OPTION is enabled at the object level.
3. SHOWDDL The SHOWDDL command displays column-level privileges. Regardless of the order the privileges were granted, SHOWDDL displays them in column order, and within each column, in the order they appear in the bitmap (SELECT, INSERT, UPDATE, REFERENCES).
4. Revoking of column-level privileges Only partially implemented. The basic operation of revoking granted column-level privileges and grant option for is implemented. All relevant security checks are performed. GRANTED BY is not implemented. RESTRICT and CASCADE options are not supported. Hence, any dependent objects remain when column-level privileges are revoked.
In addition to column-level revoke only be partially implemented, here are other items not present in this delivery:
1. Privileges can be granted to roles and revoked from roles, but REVOKE ROLE does not consider column-level privileges when determining if an object depends on a role's granted privileges. 2. Similarly, revoke at the object level does not consider column-level privileges that may allow an object to remain after an object-level privilege is revoked. 3. CREATE VIEW does not consider column-level privileges when determining if the user has authority on the referenced tables and views. 4. Run-time DML operations do not considered column-level when determining if the user has authority to perform the query.