Changed VersionStrategy.get_solid_version and VersionStrategy.get_resource_version to take in a SolidVersionContext and ResourceVersionContext, respectively. This gives VersionStrategy access to the config (in addition to the definition object) when determining the code version for memoization. (Thanks @RBrossard!).
Note: This is a breaking change for anyone using the experimental VersionStrategy API. Instead of directly being passed solid_def and resource_def, you should access them off of the context object using context.solid_def and context.resource_def respectively.
emr_pyspark_step_launcher to fail when stderr included non-Log4J-formatted lines.applyPerUniqueValue config on the QueuedRunCoordinator to fail Helm schema validation.@asset decorator and build_assets_job APIs to construct asset-based jobs, along with Dagit support.load_assets_from_dbt_project and load_assets_from_dbt_manifest, which enable constructing asset-based jobs from DBT models.[dagstermill] You can now have more precise IO control over the output notebooks by specifying output_notebook_name in define_dagstermill_solid and providing your own IO manager via "output_notebook_io_manager" resource key.
We've deprecated output_notebook argument in define_dagstermill_solid in favor of output_notebook_name.
Previously, the output notebook functionality requires “file_manager“ resource and result in a FileHandle output. Now, when specifying output_notebook_name, it requires "output_notebook_io_manager" resource and results in a bytes output.
You can now customize your own "output_notebook_io_manager" by extending OutputNotebookIOManager. A built-in local_output_notebook_io_manager is provided for handling local output notebook materialization.
See detailed migration guide in https://github.com/dagster-io/dagster/pull/4490.
Dagit fonts have been updated.
context.log.info("foo %s", "bar") would not get formatted as expected.QueuedRunCoordinator’s tag_concurrency_limits to not be respected in some casestags argument of the @graph decorator or GraphDefinition constructor. These tags will be set on any runs of jobs are built from invoking to_job on the graph.k8s_job_executor or celery_k8s_job_executor. Use the key image inside the container_config block of the k8s solid tag.jobs argument. Each RunRequest emitted from a multi-job sensor’s evaluation function must specify a job_name.KubernetesRunLauncher image pull policy is now configurable in a separate field (thanks @yamrzou!).dagster-github package is now usable for GitHub Enterprise users (thanks @metinsenturk!) A hostname can now be provided via config to the dagster-github resource with the key github_hostname:execute_pipeline( github_pipeline, {'resources': {'github': {'config': { "github_app_id": os.getenv('GITHUB_APP_ID'), "github_app_private_rsa_key": os.getenv('GITHUB_PRIVATE_KEY'), "github_installation_id": os.getenv('GITHUB_INSTALLATION_ID'), "github_hostname": os.getenv('GITHUB_HOSTNAME'), }}}} )
pipeline_failure_sensor and run_status_sensor queries. To take advantage of these performance gains, run a schema migration with the CLI command: dagster instance migrate.DockerRunLauncher would raise an exception when no networks were specified in its configuration.dagster-slack has migrated off of deprecated slackclient (deprecated) and now uses [slack_sdk](https://slack.dev/python-slack-sdk/v3-migration/).OpDefinition, the replacement for SolidDefinition which is the type produced by the @op decorator, is now part of the public API.daily_partitioned_config, hourly_partitioned_config, weekly_partitioned_config, and monthly_partitioned_config now accept an end_offset parameter, which allows extending the set of partitions so that the last partition ends after the current time.Previously in Dagit, when a repository location had an error when reloaded, the user could end up on an empty page with no context about the error. Now, we immediately show a dialog with the error and stack trace, with a button to try reloading the location again when the error is fixed.
Dagster is now compatible with Python’s logging module. In your config YAML file, you can configure log handlers and formatters that apply to the entire Dagster instance. Configuration instructions and examples detailed in the docs: https://docs.dagster.io/concepts/logging/python-logging
[helm] The timeout of database statements sent to the Dagster instance can now be configured using .dagit.dbStatementTimeout.
The QueuedRunCoordinator now supports setting separate limits for each unique value with a certain key. In the below example, 5 runs with the tag (backfill: first) could run concurrently with 5 other runs with the tag (backfill: second).
run_coordinator: module: dagster.core.run_coordinator class: QueuedRunCoordinator config: tag_concurrency_limits: - key: backfill value: applyLimitPerUniqueValue: True limit: 5
.migrate.enabled=True.