required_resource_keys and Pythonic resources) are now supported in observable source assets.EcsRunLauncher now allows you to customize volumes and mount points for the launched ECS task. See the API docs for more information.DuckDBPandasIOManager and DuckDBPySparkIOManager follow Pythonic resource system. The existing duckdb_pandas_io_manager and duckdb_pyspark_io_manager remain supported.BigQueryPandasIOManager and BigQueryPySparkIOManager follow Pythonic resource system. The existing bigquery_pandas_io_manager and bigquery_pyspark_io_manager remain supported.gcp_crentials , a temporary file to store the credentials will be created and the GOOGLE_APPLICATION_CREDENTIALS environment variable will be set to the temporary file. When the BigQuery resource is garbage collected, the environment variable will be unset and the temporary file deleted.SnowflakePandasIOManager and SnowflakePySparkIOManager follow Pythonic resource system. The existing snowflake_pandas_io_manager and snowflake_pyspark_io_manager remain supported.dagster dev would periodically emit a harmless but annoying warning every few minutes about a gRPC server being shut down.RunRequest(partition_key=...) object.PartitionsDefinition.run_request_for_partition would incorrectly raise an error for a job with a DynamicPartitionsDefinition that was defined with a function.define_asset_job would raise an error.__ASSET_JOB has no op named ... errors when using automatic run retries.EcsRunLauncher would fail to launch runs if the use_current_ecs_task_config field was set to False but no task_definition field was set.EcsRunLauncher now allows you to set a capacity provider strategy and customize the ephemeral storage used for launched ECS tasks. See the docs for details. Thanks AranVinkItility!AssetsDefinition.from_op. Thanks @tghanken for the fix!minimum_interval_seconds parameter to enable customizing the evaluation interval on the slack run failure sensor, thanks @ldnicolasmay!Resource annotation for Pythonic resource inputs has been renamed to ResourceParam in preparation for the release of the feature in 1.3.materialize and materialize_to_memory now both accept a selection argument that allows specifying a subset of assets to materialize.MultiPartitionsDefinition is no longer marked experimental.MultiPartitionsDefinitions with a time dimension.load_assets_from_package_module and similar methods now accept a freshness_policy, which will be applied to all loaded assets.build_sensor_context and build_multi_asset_sensor_context can now take a Definitions object in place of a RepositoryDefinitions3_resource now accepts use_ssl and verify configurations.selection argument on define_asset_job.MultiPartitionsDefinition.MultiPartitionsMapping (experimental) is loaded.TimeWindowPartitionMappings to not be bubbled up to the UI.pdb stopped working when using dagster dev.AssetMaterialization objects with a null AssetKey.TimeWindowPartitionsDefinition with a non-standard cron schedule, and also provided a minute_of_hour or similar argument in build_schedule_from_partitioned_job. Dagster would silently create the wrong cron expression. It now raises an error.PartitionsDefinition. These partitions are now ignored.PersistentAirflowDatabase where versions of airflow from 2.0.0 till 2.3.0 would not use the correct connection environment variable name.poll_sync_run function ofdagster-census that prevented polling from working correctly (thanks @ldincolasmay!)run_request_for_partition method on JobDefinition and UnresolvedAssetJobDefinition is now deprecated and will be removed in 2.0.0. Instead, directly instantiate a run request with a partition key via RunRequest(partition_key=...).AssetsDefinition.to_source_assets to method convert a set of assets to SourceAsset objects.MultiPartitionsDefinitions.mlflow pin from the dagster-mlflow package.When using build_asset_reconciliation_sensor, in some cases duplicate runs could be produced for the same partition of an asset. This has been fixed.
When using Pythonic configuration for resources, aliased field names would cause an error. This has been fixed.
Fixed an issue where context.asset_partitions_time_window_for_output threw an error when an asset was directly invoked with build_op_context.
[dagster-dbt] In some cases, use of ephemeral dbt models could cause the dagster representation of the dbt dependency graph to become incorrect. This has been fixed.
[celery-k8s] Fixed a bug that caused JSON deserialization errors when an Op or Asset emitted JSON that doesn't represent a DagsterEvent.
Fixed an issue where launching a large backfill while running dagster dev would sometimes fail with a connection error after running for a few minutes.
Fixed an issue where dagster dev would sometimes hang when running Dagster code that attempted to read in input via stdin.
Fixed an issue where runs that take a long time to import code would sometimes continue running even after they were stopped by run monitoring for taking too long to start.
Fixed an issue where AssetSelection.groups() would simultaneously select both source and regular assets and consequently raise an error.
Fixed an issue where BindResourcesToJobs would raise errors encapsulating jobs which had config specified at definition-time.
Fixed Pythonic config objects erroring when omitting optional values rather than specifying None.
Fixed Pythonic config and resources not supporting Enum values.
DagsterInstance.local_temp and DagsterInstance.ephemeral now use object instance scoped local artifact storage temporary directories instead of a shared process scoped one, removing a class of thread safety errors that could manifest on initialization.
Improved direct invocation behavior for ops and assets which specify resource dependencies as parameters, for instance:
class MyResource(ConfigurableResource): pass @op def my_op(x: int, y: int, my_resource: MyResource) -> int: return x + y my_op(4, 5, my_resource=MyResource())
[dagster-azure] Fixed an issue with an AttributeError being thrown when using the async DefaultAzureCredential (thanks @mpicard)
[ui] Fixed an issue introduced in 1.2.3 in which no log levels were selected by default when viewing Run logs, which made it appear as if there were no logs at all.
environment_vars argument to ScheduleDefinition is deprecated (the argument is currently non-functional; environment variables no longer need to be whitelisted for schedules)--path-prefix can now be supplied via Helm chart (thanks @mpicard)define_asset_job now auto-infer their partitions definitions if not explicitly defined.define_asset_job. This allows putting them on a schedule/sensor.instance property to the HookContext object that is passed into Op Hook functions, which can be used to access the current DagsterInstance object for the hook.create_table_schema_metadata_from_dataframe function to generate a TableSchemaMetadataValue from a Pandas DataFrame. Thanks @AndyBys!dag_run configuration on the integration’s database resources.dagster asset wipe-partitions-status-cache to see the partitions displayed.1.1.18, users with a gRPC server that could not access the Dagster instance on user code deployments would see an error when launching backfills as the instance could not instantiate. This has been fixed.build_asset_reconciliation_sensor from kicking off runs of downstream assets. This has been fixed.build_asset_reconciliation_sensor is now much more performant in cases where unpartitioned assets are upstream or downstream of static-partitioned assets with a large number of partitions.jobNamespace field in the Dagster Helm chart when using the CeleryK8sRunLauncher.en-US browser locale. This has been fixed.