![]() ![]() The UPDATE and DELETE commands now preserve existing clustering information (including Z-ordering) for files that are updated or deleted. Insertion order tags are now preserved for UPDATEs and DELETEs See Low shuffle merge on Azure Databricks. The configuration setting that was previously used to enable this feature has been removed. This behavior improves the performance of the MERGE INTO command significantly for most workloads. The MERGE INTO command now always uses the new low-shuffle implementation. Low Shuffle Merge is now enabled by default Writes will now succeed even if there are concurrent Auto Compaction transactions. Before this release, such writes would often quit, due to concurrent modifications to a table. This release improves the behavior for Delta Lake writes that commit when there are concurrent Auto Compaction transactions. Auto Compaction rollbacks are now enabled by default It does this by using Iceberg native metadata and file manifests. Iceberg to Delta table converter (Public Preview)Ĭonvert to Delta now supports converting an Iceberg table to a Delta table in place. Identity columns support in Delta tables is now generally available.New working directory for High Concurrency clusters.Parameter defaults can now be specified for SQL user-defined functions.Asynchronous state checkpointing is now generally available.Azure Synapse connector now enables the maximum number of allowed reject rows to be set.HikariCP is now the default Hive metastore connection pool.Insertion order tags are now preserved for UPDATEs and DELETEs.Low Shuffle Merge is now enabled by default.Auto Compaction rollbacks are now enabled by default.Iceberg to Delta table converter (Public Preview). ![]() Databricks released these images in March 2022. The following release notes provide information about Databricks Runtime 10.4 and Databricks Runtime 10.4 Photon, powered by Apache Spark 3.2.1. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |