dicts.OpDefinition constructor now accept ins and outs arguments, to make direct construction easier.define_dagstermill_op accepts ins and outs in order to make direct construction easier.RunRequests returned from run_status_sensors caused the sensor to error.define_asset_job, an error would occur when selecting most asset subsets. This has been fixed.500 http status codes for GraphQL requests that encountered an unexpected server error.day_offset, hour_offset, or minute_offset parameters were set on the asset’s partitions definition.dagster-k8s/config didn’t actually override the command.acryl-datahub to avoid build error.JobDefinition objects now accept a config argument, and the preset_defs argument has been removed.DagsterPipelineRunMetadataValue has been renamed to DagsterRunMetadataValue. DagsterPipelineRunMetadataValue will be removed in 1.0.MetadataEntry static methods are now marked as deprecated in the docs.PartitionMappings are now included in the API reference.DagsterRun now has a job_name property, which should be used instead of pipeline_name.TimeWindowPartitionsDefinition now has a get_partition_keys_in_range method which returns a sequence of all the partition keys between two partition keys.OpExecutionContext now has asset_partitions_def_for_output and asset_partitions_def_for_input methods.AssetsDefinition objects with the same key are provided to the same repository.build_output_context now accepts a partition_key argument that can be used when testing the handle_output method of an IO manager.asset_key set.@asset and @multi_asset decorators no longer have a top-level “assets” entry in their config schema. This entry was unused.load_assets_from_dbt_manifest with a select parameter that included sources would result in an error. This has been fixed.@repository decorated function expects parameters. Thanks @roeij!examples/hacker_news/, has been removed, because it hadn’t received updates in a long time and had drifted from best practices. The asset version is still there and has an updated README. Check it out heredefault_logger_defs argument has been added to the @repository decorator. Check out the docs on specifying default loggers to learn more.AssetsDefinition.from_graph and AssetsDefinition.from_op now both accept a partition_mappings argument.AssetsDefinition.from_graph and AssetsDefinition.from_op now both accept a metadata_by_output_name argument.define_asset_job now accepts an executor_def argument.gql in dagster-graphql.@multi_asset decorator, either by supplying a group_name argument (which will apply to all of the output assets), or by setting the group_name argument on individual AssetOuts.InputContext and OutputContext now each have an asset_partitions_def property, which returns the PartitionsDefinition of the asset that’s being loaded or stored.build_schedule_from_partitioned_job now raises a more informative error when provided a non-partitioned asset jobPartitionMapping, IdentityPartitionMapping, AllPartitionMapping, and LastPartitionMapping are exposed at the top-level dagster package. They're currently marked experimental.PartitionMapping.PartitionMappings on AssetIn.frame-ancestors restriction.load_assets_from_dbt_project and load_assets_from_dbt_manifest now support a node_info_to_group_name_fn parameter, allowing you to customize which group Dagster will assign each dbt asset to.runtime_metadata_fn when loading dbt assets, this metadata is added to the default metadata that dagster-dbt generates, rather than replacing it entirely.use_build_command=True, seeds and snapshots will now be represented as Dagster assets. Previously, only models would be loaded as assets.DockerRunLauncher would sometimes use Dagit’s Python environment as the entrypoint to launch the run, even if that environment did not exist in the container.Output object is used to type annotate the return of an op, an Output object must be returned or an error will result.PickledObjectS3IOManager in run logs and Asset view. Thanks @danielgafnimaterialize that would cause required resources to not be applied correctly.build_schedule_from_partitioned_job and define_asset_job were used together.FROM_FAILURE strategydefault_executor_def argument on repository where asset jobs that defined executor config would result in errors.databricks_pyspark_step_launcher now allows you to configure the following (Thanks @Phazure!):aws_attributes of the cluster that will be spun up for the step.[dagster-k8s] Added k8s_job_op to launch a Kubernetes Job with an arbitrary image and CLI command. This is in contrast with the k8s_job_executor, which runs each Dagster op in a Dagster job in its own k8s job. This op may be useful when you need to orchestrate a command that isn't a Dagster op (or isn't written in Python). Usage:
from dagster_k8s import k8s_job_op my_k8s_op = k8s_job_op.configured({ "image": "busybox", "command": ["/bin/sh", "-c"], "args": ["echo HELLO"], }, name="my_k8s_op", )
[dagster-dbt] The dbt asset-loading functions now support partitions_def and partition_key_to_vars_fn parameters, adding preliminary support for partitioned dbt assets. To learn more, check out the Github issue!