datahub

api

feat: Add support for Data Products (#8039 )

2023-05-17 07:17:25 +00:00

cli

test(sdk): move cli tests into the unit dir (#8028 )

2023-05-19 16:13:39 +02:00

config

feat(ingest): support writing configs to files (#8696 )

2023-08-28 09:55:50 -07:00

data_lake

…

glue

feat(ingest): Turn on browse path v2 creation (#8342 )

2023-07-06 16:43:42 -04:00

graph

fix(ingest): emitter bug fixes (#8093 )

2023-05-23 12:04:16 -07:00

patch

test(ingest): Aspect level golden file comparison (#8310 )

2023-07-11 10:39:47 -04:00

reporting

feat(ingestion): send reports of ingestion runs to datahub (#5639 )

2022-08-19 09:08:17 -07:00

s3

fix(ingest/s3): wrong sorting in case of multi-partition key (#8536 )

2023-08-02 09:54:33 +05:30

sagemaker

fix(ingest): Generate browse paths v2 for more sources; properly pass platform_instance (#8501 )

2023-07-25 11:35:34 +05:30

schema

fix(ingest/json-schema): convert non-string enums to strings (#8479 )

2023-08-01 19:35:40 +05:30

serde

feat(sdk): support patches as MCPs in file source (#8220 )

2023-06-14 14:56:27 -07:00

sql_parsing

feat(ingest/airflow): airflow plugin v2 (#8853 )

2023-10-04 16:23:15 +05:30

stateful_ingestion

feat(ingest/datahub): Improvements, bug fixes, and docs (#8735 )

2023-08-29 14:33:40 -04:00

utilities

feat(bigquery): add better timers around every API call (#8626 )

2023-09-15 11:55:39 -07:00

__init__.py

…

test_allow_deny.py

chore(ingest): remove inferred args to MCPW, part 1 (#6819 )

2022-12-30 01:26:47 -05:00

test_apis.py

…

test_athena_source.py

refactor(ingest/athena): Replace s3_staging_dir parameter in Athena source with query_result_location (#7044 )

2023-01-18 09:25:37 -08:00

test_bigquery_lineage.py

feat(bigquery): add better timers around every API call (#8626 )

2023-09-15 11:55:39 -07:00

test_bigquery_profiler.py

fix(ingest/bigquery): fix partition and median queries for profiling (#8778 )

2023-09-06 12:48:11 -07:00

test_bigquery_source.py

feat(bigquery): excluding projects without any datasets from ingestion (#8535 )

2023-10-04 09:20:51 -04:00

test_bigquery_sql_lineage.py

refactor(ingest): bigquery-lineage - allow tables and datasets in uppercase (#6739 )

2022-12-14 14:58:03 +01:00

test_bigquery_sql_parser.py

test(ingest/bigquery): Add sql parser xfail test to fix later (#7792 )

2023-04-12 10:51:29 -07:00

test_bigquery_usage.py

test(ingest/unity): Add Unity Catalog memory performance testing (#8932 )

2023-10-04 10:23:13 -04:00

test_bigqueryv2_usage_source.py

feat(bigquery): add better timers around every API call (#8626 )

2023-09-15 11:55:39 -07:00

test_bq_get_partition_range.py

deprecate(ingest): bigquery - Removing bigquery-legacy source (#6851 )

2022-12-29 13:19:05 -08:00

test_capability_report.py

fix(ingest): fix serialization of report to handle nesting (#5455 )

2022-07-20 18:25:07 -07:00

test_check_upgrade.py

chore(ingest): formatting + cleanup MCPW usages (#7706 )

2023-03-29 11:43:25 -07:00

test_check.py

…

test_classification.py

feat(classification): support for regex based custom infotypes (#8177 )

2023-06-06 14:41:51 +02:00

test_cli_logging.py

fix(ingest): tweak ingestion exit codes (#8418 )

2023-07-14 15:47:16 -07:00

test_cli_utils.py

feat(cli,build): remove deprecated variables GMS_HOST/_PORT (#5451 )

2022-07-20 20:54:43 +05:30

test_clickhouse_source.py

Fix(ingestion/clickhouse) move to two tier sqlalchemy (#8300 )

2023-08-11 16:11:40 -04:00

test_codegen.py

test(sdk): better error messages in registry codegen test (#8081 )

2023-05-19 11:18:50 +02:00

test_compare_metadata.py

test(ingest): Aspect level golden file comparison (#8310 )

2023-07-11 10:39:47 -04:00

test_config_clean.py

…

test_confluent_schema_registry.py

fix(ingest/kafka): use SchemaReference properties instead of dict access (#8615 )

2023-08-16 10:33:09 +05:30

test_corp_group_urn.py

…

test_corpuser_urn.py

…

test_csv_enricher_source.py

feat(ingest): infer aspect name from type in get_aspect (#6033 )

2022-10-11 13:35:41 -07:00

test_data_flow_urn.py

…

test_data_job_urn.py

…

test_data_process_instance_urn.py

…

test_dataset_urn.py

…

test_dbt_source.py

fix(ingest/dbt): introduce lowercase column urn option (#7418 )

2023-03-20 10:37:19 -07:00

test_domain_urn.py

…

test_druid_source.py

fix(ingest): bigquery - sharded table support improvements (#6789 )

2022-12-19 18:57:37 +01:00

test_elasticsearch_source.py

feat(ingest): allow lower freq profiling based on date of month/day of week (#8489 )

2023-08-04 10:13:48 +05:30

test_file_lineage_source.py

feat(lineage source): add fine grained lineage support (#7904 )

2023-05-26 17:09:32 +05:30

test_gcs_source.py

feat(ingest): add GCS ingestion source (#7903 )

2023-04-27 19:03:41 +02:00

test_ge_profiling_config.py

fix(ingest/sql-common): Fix profile_table_level_only (#8331 )

2023-07-07 19:05:50 -04:00

test_generic_aspect_transformer.py

test(ingest): Aspect level golden file comparison (#8310 )

2023-07-11 10:39:47 -04:00

test_glue_source_stubs.py

feat(glue): allow resource links to be ignored (#7639 )

2023-04-21 10:42:32 -07:00

test_glue_source.py

fix(ingest/build): Fix sagemaker mypy and flake8 issues (#8530 )

2023-07-31 16:13:07 +02:00

test_great_expectations_action.py

chore(ci): upgrade GE version (#7290 )

2023-03-02 10:47:38 -08:00

test_hana_source.py

…

test_hive_source.py

fix(ingest): bigquery - sharded table support improvements (#6789 )

2022-12-19 18:57:37 +01:00

test_iceberg.py

feat(iceberg): Upgrade Iceberg ingestion source to pyiceberg 0.4.0 (#8357 )

2023-08-31 13:01:05 -04:00

test_kafka_emitter.py

refactor(ingest): move common host_port validation (#6009 )

2022-09-22 16:32:07 -07:00

test_kafka_sink.py

chore(ingest): remove inferred args to MCPW, part 2 (#6905 )

2023-01-04 23:29:56 -05:00

test_kafka_source.py

feat(ingest/kafka): support metadata mapping from kafka avro schemas (#8825 )

2023-09-22 17:11:42 -07:00

test_key_value_pattern.py

feat(ingest): add fallthrough support to KeyValuePattern (#6302 )

2022-10-28 11:07:47 +02:00

test_ldap_source.py

fix(ldap): properly handle escaped characters in LDAP DNs (#7928 )

2023-05-03 13:57:52 -07:00

test_ldap_state.py

refactor(ingest/stateful): remove most remaining state classes (#6791 )

2022-12-19 13:40:48 -05:00

test_mapping.py

feat(dbt-ingestion): add documentation link from dbt source to institutionalMemory (#8686 )

2023-10-04 20:38:58 +00:00

test_mariadb_source.py

…

test_mce_builder.py

fix(ingest): encode reserved characters when creating dataset urn (#5977 )

2022-09-20 16:59:02 -07:00

test_mcp_builder.py

feat: data contracts models + CLI (#8923 )

2023-10-04 20:11:06 -07:00

test_mcp_wrapper.py

feat(ingest): avoid embedding serialized json in metadata files (#6742 )

2022-12-28 19:28:38 -05:00

test_metabase_source.py

feat(ingest): Add metabase database id to platform instance mapping (#8359 )

2023-08-02 09:53:48 +05:30

test_mlflow_source.py

feat(metadata-ingestion): implement mlflow source (#7971 )

2023-09-26 13:51:30 -04:00

test_nifi_source.py

feat(ingest/nifi): add support for basic auth in nifi (#8457 )

2023-07-20 14:12:33 -04:00

test_notebook_urn.py

…

test_openapi.py

feat(ingest): add ability to read other method types than GET for OAS ingest recipes (#8303 )

2023-08-02 09:54:09 +05:30

test_oracle_source.py

…

test_packages.py

test(cli): add check for missing init files (#7378 )

2023-02-20 18:41:12 +05:30

test_packaging.py

ci(ingest/kafka): improve kafka integration test reliability (#8085 )

2023-05-25 15:40:56 -07:00

test_parsing_util.py

…

test_pipeline.py

fix(ingest): tweak ingestion exit codes (#8418 )

2023-07-14 15:47:16 -07:00

test_plugin_system.py

chore(ingest): enable flake8 bugbear linting (#7763 )

2023-04-10 14:14:42 -07:00

test_postgres_source.py

fix(ingest/postgres): Allow specification of initial engine database; set default database to postgres (#7915 )

2023-05-09 11:11:43 -07:00

test_protobuf_util.py

fix(ingest): bigquery - sharded table support improvements (#6789 )

2022-12-19 18:57:37 +01:00

test_pulsar_source.py

…

test_pydantic_validators.py

feat(ingest/dbt): support use_compiled_code and test_warnings_are_errors (#8956 )

2023-10-05 10:29:47 -07:00

test_redash_source.py

fix(ingest): redash - fix redash dashboard url bug (#5500 )

2022-08-16 13:39:27 -07:00

test_redshift_lineage.py

fix(ingest/redshift): Enabling autocommit for Redshift connection (#7983 )

2023-05-08 10:24:40 +02:00

test_report.py

feat(ingest): reporting revamp, part 1 (#7031 )

2023-01-18 13:34:32 -08:00

test_rest_emitter.py

docs(): generate docs for our Python SDK (#7612 )

2023-03-28 20:23:20 -07:00

test_rest_sink.py

feat(ingest): fix handling of unions with aliases in post restli conversion (#7058 )

2023-01-18 09:29:46 -08:00

test_sagemaker_source_stubs.py

…

test_schema_util.py

feat(ingest/kafka): support metadata mapping from kafka avro schemas (#8825 )

2023-09-22 17:11:42 -07:00

test_snowflake_shares.py

feat(ingest/snowflake): allow shares config without platform instance (#8803 )

2023-09-25 14:04:05 +05:30

test_snowflake_source.py

fix(ingest/snowflake): fix azure cloud region ids in external url (#8376 )

2023-07-12 17:11:33 +05:30

test_source_helpers.py

feat(ingest): Create zero usage aspects (#8205 )

2023-06-22 17:07:50 -04:00

test_source.py

feat(ingest): Create Browse Paths V2 under flag (#8120 )

2023-06-02 12:50:38 -07:00

test_sql_common.py

feat(ingest/airflow): airflow plugin v2 (#8853 )

2023-10-04 16:23:15 +05:30

test_sql_utils.py

refactor(ingest/containers): Refactoring container creation to common place (#6877 )

2023-01-21 00:14:31 +01:00

test_superset_source.py

fix(ingest/superset): support superset v2 (#7588 )

2023-03-20 19:49:32 -07:00

test_tag_urn.py

…

test_time_window_config.py

fix(ingest): stateful redundant run skip handler (#8467 )

2023-08-28 15:03:31 +05:30

test_transform_dataset.py

feat(transfomer): add transformer to get ownership from tags (#8748 )

2023-10-04 14:06:03 +05:30

test_unity_catalog_config.py

feat(ingest/unity): Set external url for containers and datasets (#8238 )

2023-06-15 09:05:49 +02:00

test_urn.py

…

test_usage_common.py

feat(ingest/usage): Make cumulative query character limit configurable (#8751 )

2023-08-30 15:53:08 -04:00

test_utilities.py

feat(ingest): drop sql_metadata parser (#8765 )

2023-09-07 11:32:28 -07:00

test_vertica_source.py

fix(ingest): bigquery - sharded table support improvements (#6789 )

2022-12-19 18:57:37 +01:00

test_workunit.py

test(ingest): Aspect level golden file comparison (#8310 )

2023-07-11 10:39:47 -04:00