Replies: 1 comment
-
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Overview
This discussion is for communicating to adapter maintainers the scope of work needed to make use of the changes in dbt-core 1.10.0. If you have questions and concerns, please ask them here for posterity.
If there's something missing, please comment below!
Release timeline
The below table gives the milestones between up to and including the final release.
Prior maintainer upgrade guides
TL;DR
There should be no breaking changes -- we hope and believe that things should work by bumping package and dependency versions -- please let us know if otherwise! We recommend adding tests where possible / appropriate and updating implementations within your adapter as needed to pass those tests -- see below for details.
Iceberg data catalogs
Iceberg data catalog support allows users to materialize dbt models into built-in and external catalogs that support table formats like Iceberg. This provides a warehouse-agnostic interface for managing datasets in object storage.
As detailed in the dbt Core catalog integration discussion, this feature centralizes catalog configuration and abstracts differences between catalog providers.
Here is more detail about what adapter maintainers need to implement and how to test it:
Resources
catalogs.yml#11162Sample mode
The
--sampleflag runs dbt in “small-data” mode: instead of materializing full tables, it builds a recent, time-based slice of each model, cutting dev/CI runtimes and warehouse costs while still exercising tests and downstream logic. Adapters must implement this time-window sampling (effectively “an empty table plus a handful of fresh rows”).Note: if an adapter supports microbatch incremental models (introduced in v1.9), then it already has all the technical implementation needed to support sample mode (but not vice-versa).
Resources
dbt-sparkdbt-adapters#897Deprecation warnings
Some parts of dbt Core are overly lenient which can cause confusion and unexpected results. To pave the way for new features, we’re making these areas strict. They are deprecated starting in v1.10 (still functional), and will be removed in a future release.
If your adapter encounters a new deprecation warning in v1.10, plan to update your invalid code soon. Although it’s just a warning for now, in a future version, dbt will enforce stricter validation. Users can autofix many of these by using the
dbt-autofixtool on their dbt project, but any warnings coming from the adapter will need to be addressed by the adapter maintainers.For users, all deprecation warnings must be resolved before upgrading to the new Fusion engine. This includes historic deprecations and new ones as of dbt Core v1.10. While Fusion is in beta, it will raise validation warnings, but these warnings will become errors in Fusion when Fusion goes into Preview.
Below are the deprecation warnings (each link explains what it means and how to fix it):
Resources
1.10Deprecations + Improved Developer Experience #11335Beta Was this translation helpful? Give feedback.
All reactions