Release notes for 347

[martint]

Dain Sundstrom

• all checked
• 11-12 11:37:36 1b81ae2 Add contains_sequence function
• 11-16 15:35:24 1a60078 Make query result HTTP compression client configurable
• 11-16 15:35:24 50db495 Make query result HTTP compression server configurable
• 11-19 10:58:40 65167a5 Add scan physical size stats to query completed event

David Phillips

• all checked
• 11-11 16:27:44 69b225e Cleanup styling for copy icon
• 11-12 11:01:51 144ae4a Reuse immutable partition fields in Glue partition conversion
• 11-12 18:32:18 548f1ec Update to ZooKeeper 3.5.8
• 11-13 13:40:31 cbd98a8 Require Java 11.0.7+
• 11-13 17:10:41 3acb73b Clarify Java requirement
• 11-13 17:10:41 d88bfc0 Fix indentation for code block to be part of list
• 11-15 11:46:12 126e5e1 Add modernizer rule for String.toUpperCase
• 11-15 11:46:12 ee73167 Specify locale for String.toUpperCase calls
• 11-16 13:05:29 020560f Populate view owner groups for view access control
• 11-19 15:07:56 089bbdb Remove unused variable
• 11-19 15:07:56 46066bc Rename parameters to satisfy UnusedVariable

Grzegorz Kokosiński

• all checked
• 11-12 01:17:31 8508bfd Add boolean to DataMappingSmokeTest
• 11-12 03:24:12 293818b Assume tests container is started immediately
• 11-12 03:24:12 ad51dca Fix warnings in TestRun
• 11-12 11:21:08 0792235 Extract SetAuthorizationStatement class
• 11-12 11:21:08 179e8e3 Add support for ALTER VIEW SET AUTHORIZATION
• 11-12 11:21:08 9f20bfd Add product tests for changing owner of view in hive
• 11-13 22:06:26 a9591a9 Remove unused variables in presto-product-tests
• 11-16 13:19:15 c5fdbe8 Update Postgresql JDBC to 42.2.10
• 11-17 02:42:06 7079a59 Use different table names to prevent a race when running in parallel
• 11-18 12:58:53 0d6da44 Improve formatting
• 11-18 12:58:53 850b086 Use static imports for String.format
• 11-18 12:58:53 b6987f0 Remove redundant String.format invocation
• 11-19 05:32:47 2eb5d31 Update MongoDB docs with tested versions
• 11-19 05:32:47 c483c06 Add Mongo unit test classes for latest 3.6.20 and 4.4.1
• 11-19 05:32:47 db5d02d Extract BaseMongoDistributedQueries

Karol Sobczak

• all checked
• 11-12 02:07:15 be931e7 Parallelize stats update
• 11-13 01:28:10 15405b8 Add TestLongArrayBlockEncoding
• 11-13 01:28:10 2e1e912 Improve serde performance for Long blocks with no nulls
• 11-13 01:28:10 48d115f Static import Scope.Thread
• 11-13 01:28:10 52a5829 Remove unused map
• 11-13 01:28:10 554cb19 Extract BlockTestUtils#assertBlockEquals
• 11-13 01:28:10 7f09027 Benchmark different compression methods in BenchmarkColumnReaders
• 11-13 01:28:10 9f0f600 Add BenchmarkBlockSerde
• 11-13 01:28:10 c741164 Benchmark lineitem data in BenchmarkColumnReaders
• 11-13 01:28:10 caa3a78 Improve serde performance for Long blocks with nulls
• 11-13 04:32:29 f7e4135 Avoid readByte and appendByte in nulls to bits serde
• 11-13 09:00:53 a156380 Rename session property to match session property names
• 11-16 04:32:49 9a68e91 Wait for and build split with dynamic filter in Kudu
• 11-18 02:48:09 235e426 Implement applyTableScanRedirect API in JdbcMetadata
• 11-18 02:48:09 3fea32d Implement table scan redirection in TPCH connector
• 11-18 02:48:09 aa69353 Add applyTableScanRedirect SPI and ApplyTableScanRedirection rule
• 11-19 02:31:22 9c44343 Make RedshiftClientModule#getConnectionFactory static
• 11-19 08:35:50 146caba Make sure redirected table column types match source column types
• 11-19 08:35:50 2830044 Fix formatting
• 11-20 01:09:24 058ea31 Make the type mismatch on redirect message more concise
• 11-20 01:09:24 2c0da69 Use assertThatThrownBy over @Test.expectedExceptions

Martin Traverso

• all checked
• 11-16 11:30:27 19f4914 Recognize ValuesNode with no rows as a node with distinct output
• 11-16 11:30:27 3e9fbd0 Refactor ValuesNode
• 11-16 11:30:27 6989e3f Use proper literal type
• 11-16 11:30:27 c4e81f7 Handle the case of no ValuesNode outputs left in PruneUnreferencedOutputs
• 11-16 11:30:27 cc78247 Fix literal type in a rule
• 11-16 11:30:27 f2f8ddf Support pruning all columns from ValuesNode with non-Row expression
• 11-16 11:30:27 f6ea534 Use proper literal type
• 11-16 15:24:30 23c024c Verify column types in set operation
• 11-16 15:24:30 8c3a424 Return counts from SetOperationTranslator
• 11-16 15:24:30 cae3289 Add support for EXCEPT ALL, INTERSECT ALL
• 11-16 15:24:30 e0c6fc3 Rename rules
• 11-16 15:24:30 e9c238d Remove redundant check
• 11-18 18:29:08 a64f42b Remove redundant list item bullet

Piotr Findeisen

• all checked
• 11-11 08:53:27 0c19198 Use CentOS 8
• 11-11 14:16:38 232b15a Add Druid unit test class against latest 0.20.0
• 11-12 02:36:24 a695095 Make TypeSignature final and @immutable
• 11-12 04:26:47 640c321 Improve logging when FlakyTestRetryAnalyzer not enabled
• 11-12 04:27:16 24a6956 Test Hive VIEW with functions
• 11-12 04:27:16 3e7f19c Update Tempto to 182
• 11-12 04:27:16 5093988 Update and extend Hive VIEW tests
• 11-12 05:22:48 81fd161 Fix FlakyTestRetryAnalyzer in product tests
• 11-12 08:45:53 8d0842d Remove some unnecessary usage of Number
• 11-12 08:45:53 dbc4eaf Remove redundant use of Type#getObjectValue
• 11-12 11:48:41 675dbd2 Return meaningful message when view stale
• 11-13 02:11:26 e1ec17b Add more test cases to TestExtract for DATE type
• 11-13 02:14:39 c0b2fa7 Remove dependency on presto-tests from Druid, BigQuery
• 11-16 05:39:28 8beabd1 Remove obsolete comment
• 11-16 05:39:28 9253948 Use proper logging API to log exception
• 11-16 05:39:28 a33edf5 Log info about failed tests on ERROR
• 11-16 07:35:06 2c1b6cf Remove unnecessary use of map builder
• 11-16 07:35:06 328b8af Remove redundant constructor
• 11-16 07:35:06 5309ce1 Add version check in file metastore
• 11-16 07:35:06 77e03bc Add writing Presto version to table metadata
• 11-16 07:35:06 92e16fc Pass Presto version to FileHiveMetastore
• 11-16 07:35:06 a97a562 Add writing Presto version to database metadata
• 11-16 15:11:59 063f264 Mark shouldFailOnFirstSerializationError as @flaky
• 11-17 00:08:54 129a3ff Simplify QueryExecutor usage in TestIcebergCreateTable
• 11-17 00:08:54 c574d1a Fix retries for TestIcebergCreateTable
• 11-17 00:09:15 4233637 Test Hive VIEW with JOIN and subquery
• 11-17 01:51:58 bcb25ea Adjust product test timeout
• 11-17 05:14:18 ef786ea Remove outdated comment
• 11-17 05:14:18 f400d5e Use 'ignore' for ignored variable
• 11-17 14:28:31 b76f8f9 Remove stats fetcher to test for hangs
• 11-18 01:50:20 e7c36e3 Dump Launcher threads when suite about to exhaust run time limit
• 11-18 03:13:11 1c753db Fix ATDQ.testDataMappingSmokeTest for MemSQL
• 11-18 03:13:56 2a6b8a4 Mark TestHiveCreateTable as @flaky
• 11-18 03:15:02 d19513a Do not push partial aggregations into TableScan
• 11-18 04:54:23 9790127 Record scheduled time in QueryCompletedEvent
• 11-18 07:51:29 ca13e34 Use TEXTFILE format in TestHiveViews
• 11-19 05:59:01 c646a7b Enable unused variable error prone check
• 11-19 13:48:24 1118784 Verify pattern in @flaky
• 11-19 13:48:24 7acb7dc Fix @flaky pattern for "Error committing write to Hive"
• 11-20 03:13:44 dc1e5ef Use PT timeout on CI only
• 11-20 03:14:57 2ee1a03 Fix Launcher failure when suite remaining time is zero

Pratham

• all checked
• 11-11 09:42:03 6145109 Add subnet based topology
• 11-11 10:11:34 5d17655 Refactor and simplify test by adding property argument
• 11-11 10:11:34 d5f1b38 Fix exception type for failures while decoding property values
• 11-12 12:00:02 5e1c0e4 Use ColumnHandle interface in projection adaptation classes
• 11-12 12:00:02 6fdb5fa Rename ReaderProjections class and simplify method names
• 11-12 12:00:02 deb971f Make ReaderPageSource top-level and simplify method names
• 11-13 16:15:20 0293716 Cache hashcode for TypeSignature class
• 11-13 16:41:11 99d702b Allow configuring HDFS replication factor directly in hive.properties

Praveen Krishna

• all checked
• 11-19 00:16:10 285ccce Validate user objects for HiveProcedures
• 11-19 00:16:10 96d7a4b Minor cleanup in checks involving ConnectorSession for Procedure
• 11-19 00:16:10 d7f1f2c Inject ConnectorAccessControl for Procedures

Yuya Ebihara

• all checked
• 11-10 20:24:13 8e33ab9 Extract utility methods in JDBC module
• 11-10 20:24:13 b0ec1bd Add support for batching in JDBC driver
• 11-11 06:42:34 85110dd Add support for limit pushdown in MongoDB
• 11-17 05:59:57 46e0e4c Add main method to MySQL query runner

Łukasz Osipiuk

• all checked
• 11-12 07:29:08 4a776a8 Add toString for Event classes in SPI
• 11-12 09:37:41 d8a164b Fix suite-3 definition
• 11-16 09:36:42 5d07e31 Split TestEventListener test class
• 11-16 09:36:42 7ee7226 Switch actual/expected in assertion
• 11-16 09:36:42 8876534 Rename EventsBuilder to EventsCollector
• 11-16 09:36:42 bf41485 Ignore SplitCompleted events in TestEventListenerBasic
• 11-16 09:36:42 d2a2959 Rename parameter
• 11-16 09:36:42 f2b51c0 Add per-event filtering to EventsCollector
• 11-18 05:41:11 3cab27d Bump hive-apache to 3.1.2-5
• 11-19 03:24:19 d3ee586 Stream container statistics
• 11-19 05:28:07 0397edb Use real executable CLI jar in product tests
• 11-19 05:28:07 14e8dc0 Extract terminal utils
• 11-19 05:28:07 198760d Execute queries from redirected input stream
• 11-19 05:28:07 548b04d Update JLine to 3.17.1
• 11-20 03:54:40 9f5545e Get expected CLI version from resource file

[ebyhr]

## JDBC Driver
* Implement `addBatch()`, `clearBatch()` and `executeBatch()` methods in `PreparedStatement`. ({issue}`5507`)

#5507

[ebyhr]

## MongoDB
* Improve performance of queries containing a `LIMIT` clause. ({issue}`5870`)

#5870

[findepi]

Docker
* Update Presto docker image to use Centos 8 as the base. ({issue}`5920`)

#5920 #5921

[martint]

• Update Presto docker image to use Centos 8 as the base. ({issue}5920)

@findepi, this is an implementation detail. What's the user-visible impact? (security, stability, etc).

[phd3]

# General Changes

* Throw a user error for failing to decode a session property value. ({issue}`5731`)

#5740 #5731

[phd3]

# General Changes

* Add support for configuring subnet based network topology for toplogy-aware scheduling. 

#4862

[findepi]

• Update Presto docker image to use Centos 8 as the base. ({issue}5920)

@findepi, this is an implementation detail. What's the user-visible impact? (security, stability, etc).

It is and it is not.

If you just run the container, there may no be user visible change.
If you build a downstream image based on that, you need to know that.

[sopel39]

Hive
----
* Decrease latency of insert and CTAS queries by updating table and column statistics parallelly. ({issue}`3638`)

#3638

[kokosing]

## General Changes 
--
* Add `ALTER VIEW ... SET AUTHORIZATION` syntax to allow changing the view owner. ({issue}`5789`)

## Hive Connector Changes
--
* Add support for `ALTER VIEW  ... SET AUTHORIZATION` SQL syntax to change the view owner. This supports Presto and Hive views. ({issue}`5789`)

#5789

[dain]

* Add :func:`contains_sequence` function. ({issue}`5593`)

#5593

[sopel39]

General:
* Improve query performance by reducing worker to worker communication overhead. ({issue}`5905`, {issue}`5949`)

#5905, #5949

[sopel39]

General:
* Rename ``rewrite-filtering-semi-join-to-inner-join`` session property to ``rewrite_filtering_semi_join_to_inner_join``. ({issue}`5954`)

#5954

[phd3]

# Hive Connector Changes

* Allow configuring HDFS replication factor through property `hive.dfs.replication`. ({issue}`1829`)

#1829, #4934

[sopel39]

Kudu:

* Add dynamic filtering support for Kudu. It can be enabled using ``kudu.dynamic-filtering.wait-timeout``
  config property or ``dynamic_filtering_wait_timeout`` session property ({issue}`5594`)

#5594

[martint]

* Fix failure when expressions that produce values of type `row` are used in a `VALUES` clause. ({issue}`3398`)

#3398

[martint]

* Add support for `ALL` qualifier for `INTERSECT` and `EXCEPT`. ({issue}`2152`)

#2152

[pettyjamesm]

General:
* Improve result throughput over fast network links by making http response compression configurable. Clients
can disable compressed responses using the `--disable-compression` flag or `disableCompression` driver property.
Compression can be disabled server-wide with `query-results.compression-enabled=false`. ({issue}`5818`)

#5818

[sopel39]

SPI:
* Add support for table scan redirections. ({issue}`5792`)

#5792

[losipiuk]

Hive:
* Fix leaking S3 connections when querying Avro tables. ({issue}`5562`)

#5562
#6001

[losipiuk]

CLI:
* presto-cli now accepts queries passed via shell redirection. ({issue}`5881`)

#5881

[dain]

SPI:
* Add scan physical size stats to query completed event. ({issue}`5872`)

#5872

[Praveen2112]

Hive:
* Add access checks for tables in Hive Procedures. ({issue}`1489`)

#5379

[dain]

Security:
* Add support for multiple LDAP bind patterns. ({issue}`5874`)

#5874

[sopel39]

RDBMS connectors
* Improve query performance by compacting large pushed down predicates.
  Compaction threshold can be changed using the ``domain-compaction-threshold``
  config property or ``domain_compaction_threshold`` session property. ({issues}`6057`)

#6057

[electrum]

# Security Changes

* Include groups for view owner when checking permissions for views. ({issue}`5945`)

#5945

[findepi]

PostgreSQL, MySQL, SQL Server
* Include performance for certain complex queries involving aggregation and predicates by
  by pushing the aggregation and predicates computation into the remote database. ({issue}`4112`)

#4112
#6007