datahub

api

refactor(ingest): Refactor structured logging to support infos, warnings, and failures structured reporting to UI (#10828 )

2024-07-03 19:20:58 -07:00

cli

feat(open assertion spec): MVP for Snowflake DMF Assertions: update models, add assertions cli with snowflake integration (#10602 )

2024-05-31 12:03:22 -07:00

config

feat(ingest): fetch connections from the backend (#10511 )

2024-05-29 10:32:29 -07:00

data_lake

feat(ingest): Support for JSONL in s3 source with max_rows support (#9921 )

2024-02-28 15:05:30 +01:00

glue

fix(ingestion/glue): Add support for missing config options for profiling in Glue (#10858 )

2024-07-29 16:04:07 +05:30

graph

…

patch

feat(patch): refactor cll patch (#9922 )

2024-02-27 12:00:44 -06:00

reporting

…

s3

fix(ingest/s3): Fixing container creation when there is no folder in path (#10993 )

2024-07-25 23:38:10 +02:00

sagemaker

feat(ingestion/SageMaker): Remove deprecated apis and add stateful ingestion capability (#10573 )

2024-05-28 12:16:28 +02:00

schema

fix(ingest/json-schema): adding support descriptions for array (#9757 )

2024-03-01 13:03:47 -08:00

serde

feat(ui/last-updated): Calculate last updated time as max(properties time, operation time) (#9242 )

2023-12-06 15:21:56 -05:00

sql_parsing

feat(ingest/bigquery): improve handling of information schema in sql parser (#10985 )

2024-07-25 17:58:16 -07:00

stateful_ingestion

feat(ingest): add snowflake-queries source (#10835 )

2024-07-12 15:08:51 -07:00

urns

…

utilities

fix(ingestion/powerbi): fix issue with broken report lineage (#10910 )

2024-07-31 11:40:09 -07:00

__init__.py

…

redshift_query_mocker.py

fix(ingest/redshift): tweak lineage v2 queries (#10045 )

2024-03-15 12:16:18 -07:00

test_apis.py

…

test_athena_source.py

…

test_bigquery_lineage.py

fix(ingest/bigquery): Supporting lineage extraction in case the select query result's target table is set on job (#10191 )

2024-04-02 22:13:05 +02:00

test_bigquery_profiler.py

…

test_bigquery_source.py

fix(ingest/bigquery): handle quota exceeded for project.list requests (#10912 )

2024-07-17 17:17:52 +02:00

test_bigquery_sql_lineage.py

feat(ingest): sql parsing aggregator (#9786 )

2024-02-09 16:27:45 -05:00

test_bigquery_usage.py

fix(ingest/bigquery): fix lineage if multiple sql expression passed in and destination table set (#10212 )

2024-04-09 11:54:02 +02:00

test_bigqueryv2_usage_source.py

fix(ingest/bigquery): changes helper function to decode unicode scape sequences (#10845 )

2024-07-16 15:50:54 -07:00

test_bq_get_partition_range.py

…

test_capability_report.py

…

test_check_upgrade.py

…

test_check.py

…

test_classification.py

feat(classifier): Add support for excluding list of exact column names (#9472 )

2024-01-16 19:12:23 -08:00

test_cli_logging.py

…

test_cli_utils.py

fix(ingest): fix graph config loading (#11002 )

2024-07-26 11:15:46 -07:00

test_clickhouse_source.py

…

test_cockroach_source.py

feat(ingest/cockroachdb): add cockroachdb ingestion (#10226 )

2024-04-09 18:36:51 -07:00

test_codegen.py

…

test_compare_metadata.py

…

test_confluent_schema_registry.py

…

test_csv_enricher_source.py

feat(ingest/csv-enricher): add support for ownership type urn (#9935 )

2024-02-27 17:47:04 +00:00

test_datahub_source.py

…

test_dbt_source.py

feat(ingest/dbt-cloud): update metadata_endpoint inference (#11041 )

2024-07-31 14:16:18 -07:00

test_druid_source.py

…

test_elasticsearch_source.py

feat(ingest): fix validators (#10115 )

2024-03-27 15:20:55 -07:00

test_file_lineage_source.py

feat(ingest): fix validators (#10115 )

2024-03-27 15:20:55 -07:00

test_gcs_source.py

…

test_ge_profiling_config.py

…

test_generic_aspect_transformer.py

…

test_glue_source_stubs.py

fix(ingestion/glue): Add support for missing config options for profiling in Glue (#10858 )

2024-07-29 16:04:07 +05:30

test_glue_source.py

fix(ingestion/glue): Add support for missing config options for profiling in Glue (#10858 )

2024-07-29 16:04:07 +05:30

test_great_expectations_action.py

feat(ingest/great_expectations): support in-memory (Pandas) data assets (#9811 )

2024-04-04 12:46:59 -07:00

test_hana_source.py

…

test_hive_source.py

…

test_iceberg.py

fix(ingest/iceberg): add support for nested dictionaries when configuring pyiceberg (#10762 )

2024-06-21 14:38:01 -07:00

test_kafka_emitter.py

…

test_kafka_sink.py

…

test_kafka_source.py

feat(ingestion/kafka)-Add support for ingesting schemas from schema registry (#10612 )

2024-06-11 14:00:12 +02:00

test_key_value_pattern.py

…

test_ldap_source.py

…

test_ldap_state.py

…

test_mapping.py

feat(ingest): support domains in meta -> "datahub" section (#10967 )

2024-07-25 09:31:19 -07:00

test_mariadb_source.py

…

test_mce_builder.py

fix(ingest): make user_urn and group_urn generation consider user and… (#9026 )

2023-12-14 21:33:36 +05:30

test_mcp_builder.py

…

test_mcp_wrapper.py

…

test_metabase_source.py

feat(metabase): add stateful ingestion (#10360 )

2024-05-02 12:04:18 -07:00

test_mlflow_source.py

feat(ingest): set pipeline name in system metadata (#10190 )

2024-06-27 15:00:35 -07:00

test_nifi_source.py

refactor(ingest): Refactor structured logging to support infos, warnings, and failures structured reporting to UI (#10828 )

2024-07-03 19:20:58 -07:00

test_openapi.py

…

test_oracle_source.py

…

test_ordered_set.py

feat(ingest): sql parsing aggregator (#9786 )

2024-02-09 16:27:45 -05:00

test_packages.py

feat(ingest): check for private import path usages (#10059 )

2024-03-18 14:59:18 -07:00

test_packaging.py

…

test_parsing_util.py

…

test_pipeline.py

fix(ingest): fix graph config loading (#11002 )

2024-07-26 11:15:46 -07:00

test_plugin_system.py

feat(ingest): add and use file system abstraction in file source (#8415 )

2024-07-01 10:47:07 -07:00

test_postgres_source.py

…

test_powerbi_parser.py

…

test_protobuf_util.py

…

test_pulsar_source.py

feat(ingest): fix validators (#10115 )

2024-03-27 15:20:55 -07:00

test_redash_source.py

fix(ingest): only populate audit stamps where accurate (#10604 )

2024-07-11 13:26:57 -07:00

test_redshift_config.py

…

test_redshift_lineage.py

fix(ingest): fix redshift query urns + reduce memory usage (#10691 )

2024-06-13 11:27:06 -07:00

test_redshift_source.py

fix(ingest/redshift): patch instead of replace redshift custom properties (#9293 )

2024-03-11 11:16:44 -07:00

test_report.py

…

test_rest_emitter.py

…

test_rest_sink.py

fix(ingest): set lastObserved in sdk when unset (#11071 )

2024-08-02 15:48:10 -07:00

test_sagemaker_source_stubs.py

feat(ingestion/SageMaker): Remove deprecated apis and add stateful ingestion capability (#10573 )

2024-05-28 12:16:28 +02:00

test_schema_util.py

…

test_serialized_lru_cache.py

feat(ingest/snowflake): refactor + parallel schema extraction (#10653 )

2024-06-14 13:23:07 -07:00

test_snowflake_shares.py

feat(ingest/snowflake): integrate snowflake-queries into main source (#10905 )

2024-07-17 10:22:14 -07:00

test_snowflake_source.py

fix(ingest/snowflake): fix test connection (#10927 )

2024-07-17 11:57:58 -07:00

test_source.py

…

test_sql_common.py

feat(ingestion): Add test_connection methods for important sources (#9334 )

2023-12-14 12:31:51 -05:00

test_sql_utils.py

fix(ingest/profiling): Filter tables early based on profile pattern filter (#10378 )

2024-04-26 17:35:03 +02:00

test_superset_source.py

…

test_tableau_source.py

feat(ingestion/tableau): optionally ingest multiple sites and create site containers (#10498 )

2024-07-09 11:49:41 -07:00

test_topological_sort.py

…

test_transform_dataset.py

fix(ingestion/transformer): replace the externalUrl container (#11013 )

2024-07-30 15:17:04 +05:30

test_unity_catalog_config.py

fix(ingest/databricks): handle and report config parse failure, updat… (#10261 )

2024-04-17 12:14:16 -07:00

test_usage_common.py

feat(ingest): set pipeline name in system metadata (#10190 )

2024-06-27 15:00:35 -07:00

test_utilities.py

fix(ingest): better logging line attribution (#9876 )

2024-02-21 11:37:57 -08:00

test_vertica_source.py

…

test_workunit.py

…