Resources in a ModeDefinition that are not required by a pipeline no longer require runtime configuration. This should make it easier to share modes or resources among multiple pipelines.
Dagstermill solids now support retries when a RetryRequested is yielded from a notebook using dagstermill.yield_event.
In Dagit, the asset catalog now supports both a flattened view of all assets as well as a hierarchical directory view.
In Dagit, the asset catalog now supports bulk wiping of assets.
In the Dagit left nav, schedules and sensors accurately reflect the filtered repositories.
When executing a pipeline with a subset of solids, the config for solids not included in the subset is correctly made optional in more cases.
URLs were sometimes not prefixed correctly when running Dagit using the --path-prefix option, leading to failed GraphQL requests and broken pages. This bug was introduced in 0.11.4, and is now fixed.
The update_timestamp column in the runs table is now updated with a UTC timezone, making it consistent with the create_timestamp column.
In Dagit, the main content pane now renders correctly on ultra-wide displays.
The partition run matrix on the pipeline partition tab now shows step results for composite solids and dynamically mapped solids. Previously, the step status was not shown at all for these solids.
Previously, when retrieving the outputs from a run of execute_pipeline, the system would use the io manager that handled each output to perform the retrieval. Now, when using execute_pipeline with the default in-process executor, the system directly captures the outputs of solids for use with the result object returned by execute_pipeline. This may lead to slightly different behavior when retrieving outputs if switching between executors and using custom IO managers.
The K8sRunLauncher and CeleryK8sRunLauncher now add a dagster/image tag to pipeline runs to document the image used. The DockerRunLauncher has also been modified to use this tag (previously it used docker/image).
In Dagit, the left navigation is now collapsible on smaller viewports. You can use the . key shortcut to toggle visibility.
In Dagit, a GraphQL error on partition sets related to missing fragment PartitionGraphFragment has been fixed.
The compute log manager now handles base directories containing spaces in the path.
Fixed a bug where re-execution was not working if the initial execution failed, and execution was delegated to other machines/process (e.g. using the multiprocess executor)
The same solid can now collect over multiple dynamic outputs
Assets are now included in Dagit global search. The search bar has also been moved to the top of the app.
[helm] generatePostgresqlPasswordSecret toggle was added to allow the Helm chart to reference an external secret containing the Postgresql password (thanks @PenguinToast !)
[helm] The Dagster Helm chart is now hosted on Artifact Hub.
[helm] The workspace can now be specified under dagit.workspace, which can be useful if you are managing your user deployments in a separate Helm release.
In Dagit, toggling schedules and sensors on or off will now immediately update the green dot in the left navigation, without requiring a refresh.
When evaluating dict values in run_config targeting Permissive / dict config schemas, the ordering is now preserved.
Integer values for EventMetadataEntry.int greater than 32 bits no longer cause dagit errors.
PresetDefinition.with_additional_config no longer errors if the base config was empty (thanks @esztermarton !)
Fixed limitation on gRPC message size when evaluating run requests for sensors, schedules, and backfills. Previously, a gRPC error would be thrown with status code StatusCode.RESOURCE_EXHAUSTED for a large number of run requests, especially when the requested run configs were large.
Changed backfill job status to reflect the number of successful runs against the number of partitions requested instead of the number of runs requested. Normally these two numbers are the same, but they can differ if a pipeline run initiated by the backfill job is re-executed manually.
dagster new project now scaffolds setup.py using your local dagster pip version (thanks @taljaards!)
Fixed an issue where legacy examples were not ported over to the new documentation site (thanks @keypointt!)
New
If a solid-decorated function has a docstring, and no description is provided to the solid decorator, the docstring will now be used as the solid’s description.
Bugfixes
In 0.11.0, we introduced the ability to auto-generate Dagster Types from PEP 484 type annotations on solid arguments and return values. However, when clicked on in Dagit, these types would show “Type Not Found” instead of rendering a description. This has been fixed.
Fixed an issue where the dagster api execute_step will mistakenly skip a step and output a non-DagsterEvent log. This affected the celery_k8s_job_executor.
Fixed an issue where NaN floats were not properly handled by Dagit metadata entries.
Fixed an issue where Dagit run tags were unclickable.
Fixed an issue where backfills from failures were not able to be scheduled from Dagit.
Integrations
[Helm] A global service account name can now be specified, which will result in the same service account name to be referenced across all parts of the Dagster Kubernetes deployment.
[Helm] Fixed an issue where user deployments did not update, even if their dependent config maps had changed.
Fixed dagster new-project, which broke on the 0.11.0 release (Thank you @saulius!)
Docs fixes (Thanks @michaellynton and @zuik!)
New
The left navigation in Dagit now allows viewing more than one repository at a time. Click “Filter” to choose which repositories to show.
In dagster-celery-k8s, you can now specify a custom container image to use for execution in executor config. This image will take precedence over the image used for the user code deployment.
Bugfixes
Previously, fonts were not served correctly in Dagit when using the --path-prefix option. Custom fonts and their CSS have now been removed, and system fonts are now used for both normal and monospace text.
In Dagit, table borders are now visible in Safari.
Stopping and starting a sensor was preventing future sensor evaluations due to a timezone issue when calculating the minimum interval from the last tick timestamp. This is now fixed.
The blank state for the backfill table is now updated to accurately describe the empty state.
Asset catalog entries were returning an error if they had not been recently materialized since (since 0.11.0). Our asset queries are now backwards compatible to read from old materializations.
Backfills can now successfully be created with step selections even for partitions that did not have an existing run.
Backfill progress were sometimes showing negative counts for the “Skipped” category, when backfill runs were manually re-executed. This has now been amended to adjust the total run counts to include manually re-executed runs.