datahub

api

feat: Add support for Data Products (#8039 )

2023-05-17 07:17:25 +00:00

cli

test(sdk): move cli tests into the unit dir (#8028 )

2023-05-19 16:13:39 +02:00

config

feat(cli): delete cli v2 (#8068 )

2023-05-23 14:43:44 -05:00

data_lake

…

glue

feat(ingest): Turn on browse path v2 creation (#8342 )

2023-07-06 16:43:42 -04:00

graph

fix(ingest): emitter bug fixes (#8093 )

2023-05-23 12:04:16 -07:00

patch

test(ingest): Aspect level golden file comparison (#8310 )

2023-07-11 10:39:47 -04:00

reporting

feat(ingestion): send reports of ingestion runs to datahub (#5639 )

2022-08-19 09:08:17 -07:00

s3

fix(ingest/s3): wrong sorting in case of multi-partition key (#8536 )

2023-08-02 09:54:33 +05:30

sagemaker

fix(ingest): Generate browse paths v2 for more sources; properly pass platform_instance (#8501 )

2023-07-25 11:35:34 +05:30

schema

fix(ingest/json-schema): convert non-string enums to strings (#8479 )

2023-08-01 19:35:40 +05:30

serde

feat(sdk): support patches as MCPs in file source (#8220 )

2023-06-14 14:56:27 -07:00

sql_parsing

test(ingest): test case statements with sql parser (#8437 )

2023-08-01 19:34:48 +05:30

stateful_ingestion

fix(ingest): respect max_threads for ingestion reporter (#8521 )

2023-07-28 13:09:32 -07:00

utilities

fix(ingest/build): Fix sagemaker mypy and flake8 issues (#8530 )

2023-07-31 16:13:07 +02:00

__init__.py

…

test_airflow.py

test(ingest/airflow): Fix test for airflow 2.6.3 (#8393 )

2023-07-11 12:55:32 -04:00

test_allow_deny.py

chore(ingest): remove inferred args to MCPW, part 1 (#6819 )

2022-12-30 01:26:47 -05:00

test_apis.py

…

test_athena_source.py

refactor(ingest/athena): Replace s3_staging_dir parameter in Athena source with query_result_location (#7044 )

2023-01-18 09:25:37 -08:00

test_bigquery_lineage.py

feat(ingest/bigquery): support column-level lineage (#8382 )

2023-07-11 11:12:51 -07:00

test_bigquery_profiler.py

fix(ingest/bigquery): Fix for table cache was not cleared (#7323 )

2023-02-13 19:04:19 +01:00

test_bigquery_source.py

feat(ingest/bigquery): support column-level lineage (#8382 )

2023-07-11 11:12:51 -07:00

test_bigquery_sql_lineage.py

refactor(ingest): bigquery-lineage - allow tables and datasets in uppercase (#6739 )

2022-12-14 14:58:03 +01:00

test_bigquery_sql_parser.py

test(ingest/bigquery): Add sql parser xfail test to fix later (#7792 )

2023-04-12 10:51:29 -07:00

test_bigquery_usage.py

test(ingest): Aspect level golden file comparison (#8310 )

2023-07-11 10:39:47 -04:00

test_bigqueryv2_usage_source.py

feat(ingest): Create zero usage aspects (#8205 )

2023-06-22 17:07:50 -04:00

test_bq_get_partition_range.py

deprecate(ingest): bigquery - Removing bigquery-legacy source (#6851 )

2022-12-29 13:19:05 -08:00

test_capability_report.py

…

test_check_upgrade.py

chore(ingest): formatting + cleanup MCPW usages (#7706 )

2023-03-29 11:43:25 -07:00

test_check.py

…

test_classification.py

feat(classification): support for regex based custom infotypes (#8177 )

2023-06-06 14:41:51 +02:00

test_cli_logging.py

fix(ingest): tweak ingestion exit codes (#8418 )

2023-07-14 15:47:16 -07:00

test_cli_utils.py

…

test_clickhouse_source.py

fix(ingest): bigquery - sharded table support improvements (#6789 )

2022-12-19 18:57:37 +01:00

test_codegen.py

test(sdk): better error messages in registry codegen test (#8081 )

2023-05-19 11:18:50 +02:00

test_compare_metadata.py

test(ingest): Aspect level golden file comparison (#8310 )

2023-07-11 10:39:47 -04:00

test_config_clean.py

…

test_confluent_schema_registry.py

…

test_corp_group_urn.py

…

test_corpuser_urn.py

…

test_csv_enricher_source.py

feat(ingest): infer aspect name from type in get_aspect (#6033 )

2022-10-11 13:35:41 -07:00

test_data_flow_urn.py

…

test_data_job_urn.py

…

test_data_process_instance_urn.py

…

test_dataset_urn.py

…

test_dbt_source.py

fix(ingest/dbt): introduce lowercase column urn option (#7418 )

2023-03-20 10:37:19 -07:00

test_domain_urn.py

…

test_druid_source.py

fix(ingest): bigquery - sharded table support improvements (#6789 )

2022-12-19 18:57:37 +01:00

test_elasticsearch_source.py

ingest(elasticsearch): add basic profiling (#8351 )

2023-07-20 08:25:30 +05:30

test_file_lineage_source.py

feat(lineage source): add fine grained lineage support (#7904 )

2023-05-26 17:09:32 +05:30

test_gcs_source.py

feat(ingest): add GCS ingestion source (#7903 )

2023-04-27 19:03:41 +02:00

test_ge_profiling_config.py

fix(ingest/sql-common): Fix profile_table_level_only (#8331 )

2023-07-07 19:05:50 -04:00

test_generic_aspect_transformer.py

test(ingest): Aspect level golden file comparison (#8310 )

2023-07-11 10:39:47 -04:00

test_glue_source_stubs.py

feat(glue): allow resource links to be ignored (#7639 )

2023-04-21 10:42:32 -07:00

test_glue_source.py

fix(ingest/build): Fix sagemaker mypy and flake8 issues (#8530 )

2023-07-31 16:13:07 +02:00

test_great_expectations_action.py

chore(ci): upgrade GE version (#7290 )

2023-03-02 10:47:38 -08:00

test_hana_source.py

…

test_hive_source.py

fix(ingest): bigquery - sharded table support improvements (#6789 )

2022-12-19 18:57:37 +01:00

test_iceberg.py

chore(ingest): drop python 3.6 support (#5521 )

2022-08-10 15:00:31 -07:00

test_kafka_emitter.py

refactor(ingest): move common host_port validation (#6009 )

2022-09-22 16:32:07 -07:00

test_kafka_sink.py

chore(ingest): remove inferred args to MCPW, part 2 (#6905 )

2023-01-04 23:29:56 -05:00

test_kafka_source.py

fix(ingest/build): Fix sagemaker mypy and flake8 issues (#8530 )

2023-07-31 16:13:07 +02:00

test_key_value_pattern.py

feat(ingest): add fallthrough support to KeyValuePattern (#6302 )

2022-10-28 11:07:47 +02:00

test_ldap_source.py

fix(ldap): properly handle escaped characters in LDAP DNs (#7928 )

2023-05-03 13:57:52 -07:00

test_ldap_state.py

refactor(ingest/stateful): remove most remaining state classes (#6791 )

2022-12-19 13:40:48 -05:00

test_mapping.py

…

test_mariadb_source.py

…

test_mce_builder.py

fix(ingest): encode reserved characters when creating dataset urn (#5977 )

2022-09-20 16:59:02 -07:00

test_mcp_builder.py

feat(ingest): add env to container properties (#8027 )

2023-05-22 12:07:16 -07:00

test_mcp_wrapper.py

feat(ingest): avoid embedding serialized json in metadata files (#6742 )

2022-12-28 19:28:38 -05:00

test_metabase_source.py

feat(ingest): Add metabase database id to platform instance mapping (#8359 )

2023-08-02 09:53:48 +05:30

test_nifi_source.py

feat(ingest/nifi): add support for basic auth in nifi (#8457 )

2023-07-20 14:12:33 -04:00

test_notebook_urn.py

…

test_openapi.py

feat(ingest): add ability to read other method types than GET for OAS ingest recipes (#8303 )

2023-08-02 09:54:09 +05:30

test_oracle_source.py

…

test_packages.py

test(cli): add check for missing init files (#7378 )

2023-02-20 18:41:12 +05:30

test_packaging.py

ci(ingest/kafka): improve kafka integration test reliability (#8085 )

2023-05-25 15:40:56 -07:00

test_parsing_util.py

…

test_pipeline.py

fix(ingest): tweak ingestion exit codes (#8418 )

2023-07-14 15:47:16 -07:00

test_plugin_system.py

chore(ingest): enable flake8 bugbear linting (#7763 )

2023-04-10 14:14:42 -07:00

test_postgres_source.py

fix(ingest/postgres): Allow specification of initial engine database; set default database to postgres (#7915 )

2023-05-09 11:11:43 -07:00

test_protobuf_util.py

fix(ingest): bigquery - sharded table support improvements (#6789 )

2022-12-19 18:57:37 +01:00

test_pulsar_source.py

…

test_pydantic_validators.py

fix(ingest): update pydantic helpers to address unique name issue (#8324 )

2023-07-06 13:16:07 -07:00

test_redash_source.py

fix(ingest): redash - fix redash dashboard url bug (#5500 )

2022-08-16 13:39:27 -07:00

test_redshift_lineage.py

fix(ingest/redshift): Enabling autocommit for Redshift connection (#7983 )

2023-05-08 10:24:40 +02:00

test_report.py

feat(ingest): reporting revamp, part 1 (#7031 )

2023-01-18 13:34:32 -08:00

test_rest_emitter.py

docs(): generate docs for our Python SDK (#7612 )

2023-03-28 20:23:20 -07:00

test_rest_sink.py

feat(ingest): fix handling of unions with aliases in post restli conversion (#7058 )

2023-01-18 09:29:46 -08:00

test_sagemaker_source_stubs.py

…

test_schema_util.py

fix(ingest): kafka - properly picking doc from union type (#6472 )

2022-11-23 20:42:21 +01:00

test_snowflake_source.py

fix(ingest/snowflake): fix azure cloud region ids in external url (#8376 )

2023-07-12 17:11:33 +05:30

test_source_helpers.py

feat(ingest): Create zero usage aspects (#8205 )

2023-06-22 17:07:50 -04:00

test_source.py

feat(ingest): Create Browse Paths V2 under flag (#8120 )

2023-06-02 12:50:38 -07:00

test_sql_common.py

refactor(ingest/containers): Refactoring container creation to common place (#6877 )

2023-01-21 00:14:31 +01:00

test_sql_utils.py

refactor(ingest/containers): Refactoring container creation to common place (#6877 )

2023-01-21 00:14:31 +01:00

test_superset_source.py

fix(ingest/superset): support superset v2 (#7588 )

2023-03-20 19:49:32 -07:00

test_tag_urn.py

…

test_transform_dataset.py

ingest(elasticsearch): add basic profiling (#8351 )

2023-07-20 08:25:30 +05:30

test_unity_catalog_config.py

feat(ingest/unity): Set external url for containers and datasets (#8238 )

2023-06-15 09:05:49 +02:00

test_urn.py

…

test_usage_common.py

chore(ingest): bump sqllineage and sqlparse (#8481 )

2023-07-28 13:10:19 -07:00

test_utilities.py

fix(ingest): bigquery - sharded table support improvements (#6789 )

2022-12-19 18:57:37 +01:00

test_vertica_source.py

fix(ingest): bigquery - sharded table support improvements (#6789 )

2022-12-19 18:57:37 +01:00

test_workunit.py

test(ingest): Aspect level golden file comparison (#8310 )

2023-07-11 10:39:47 -04:00