244 Commits

Author SHA1 Message Date
Harshal Sheth
9b64b117c7
docs(ingest/dbt): clarify dbt ingestion docs (#11312) 2024-09-09 09:01:46 +02:00
Aseem Bansal
efc42df3d9
ci: update outdated actions for java and python setup (#11261) 2024-08-29 14:27:35 +05:30
Aseem Bansal
15c1cfc386
ci(build): update outdated action & pin deepdiff lib (#11260) 2024-08-28 19:53:42 +05:30
Felix Lüdin
ce99bc4f22
feat(ingest): add ingestion source for SAP Analytics Cloud (#10958)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-26 11:29:15 -07:00
Mayuri Nehate
223650dd7a
feat(ingest): add bigquery-queries source (#10994)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-25 22:51:00 -07:00
Harshal Sheth
e0c13fda27
feat(ingest/dbt): add support for urns in add_owner directive (#11221) 2024-08-23 13:52:21 -04:00
Tamas Nemeth
ef6a410091
feat(ingest/s3): Partition support improvements (#11083)
- Partition autodetection
- Option to find min/max/min-max partition of a dataset
- Generating Partition aspects
2024-08-22 17:55:43 +02:00
sagar-salvi-apptware
608c5cfc33
fix(ingestion/transformer): extend dataset_to_data_product_urns_pattern to support containers (#11124) 2024-08-16 17:18:35 +05:30
sid-acryl
d5eda0de7e
doc(ingestion/tableau): doc update for derived permission (#11054)
Co-authored-by: Pedro Silva <pedro.cls93@gmail.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-01 10:26:47 -07:00
sagar-salvi-apptware
da72ba2113
fix(ingestion/transformer): replace the externalUrl container (#11013) 2024-07-30 15:17:04 +05:30
Tamas Nemeth
f4fb89e799
feat(ingest/spark): Promote beta plugin (#10881)
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-07-25 14:46:32 +02:00
sid-acryl
c30f4c7fb5
doc(ingest/looker): fix doc for sql parsing documentation (#10883)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-07-19 18:09:37 +05:30
Joel Pinto Mata (KPN-DSH-DEX team)
13b6febce9
feat(ingest/abs): Adding azure blob storage ingestion source (#10813) 2024-07-17 11:06:05 +02:00
Mayuri Nehate
ff1c6b895e
feat(ingest/BigQuery): refactor+parallelize dataset metadata extraction (#10884) 2024-07-16 11:46:42 -07:00
Tamas Nemeth
d204d5654a
fix(ingestion/spark): Platform instance and column level lineage fix (#10843)
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-07-09 13:54:06 +02:00
sagar-salvi-apptware
640d42dc65
feat(ingest/transformer): tags to terms transformer (#10758)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-07-02 15:30:05 +05:30
Mayuri Nehate
b6ec52bf5c
docs(classification): correct the casing for full name infotype (#10782) 2024-07-01 10:45:25 -07:00
Teppo Naakka
b223281305
feat(ingest/powerbi): powerbi dataset profiling (#9355) 2024-06-28 14:50:08 -07:00
Harshal Sheth
f4be88d0a9
feat(ingest): set pipeline name in system metadata (#10190)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-06-27 15:00:35 -07:00
Alex Waldron
edb9cf61f7
fix(docs): formatting of transformers code blocks (#10670) 2024-06-14 17:02:34 -07:00
Davi Arnaut
d29d2c3b90
docs(ingest): Add Oracle prerequisites (#10712) 2024-06-14 08:46:57 -10:00
sagar-salvi-apptware
d69966074a
fix(ingest/bigquery): Map BigQuery policy tags to datahub column-level tags (#10669) 2024-06-14 16:43:12 +05:30
Andrew Sikowitz
46dbb10940
docs(ingest): Rename csv / s3 / file source and sink (#10675) 2024-06-11 11:44:13 -07:00
sid-acryl
2c3943d723
fix(doc): Fix doc typo in transformer (#10658) 2024-06-07 09:16:28 -07:00
Woong Seok Kang
5b4fc9612f
docs(ingest): remove trailing comma on athena permission (#10634) 2024-06-04 21:59:43 -07:00
Eric L (CCCS)
c04b3bc2e4
fix(ingest/iceberg): update iceberg source to support newer versions of pyiceberg at runtime (#10614) 2024-06-04 09:45:29 -07:00
sid-acryl
666de9e4e6
fix(ingestion/powerbi): Databricks support for table lineage (#10416) 2024-05-22 18:57:45 +05:30
sagar-salvi-apptware
5fbf781558
fix(ingest/transformer): Add dataset domains based on tags using transformer (#10458) 2024-05-15 14:13:03 +05:30
Hyejin Yoon
10cbbd7619
docs: fix docs utms & slack footer (#10481) 2024-05-12 16:39:33 -07:00
Davi Arnaut
94d424b7d4
fix(docs): adjust new requirements for DynamoDB ingestion (#10470) 2024-05-09 10:22:33 -05:00
dushayntAW
c00ddb2a0d
fix(ingest/transformer): new transformer to clean user URN for datasetUsageStatistics aspect (#10398) 2024-05-03 13:24:48 +05:30
Ellie O'Neil
d82750b891
DynamoDB IAM auth (#10419) 2024-05-02 16:02:40 -07:00
Hyejin Yoon
704ca650ad
docs: add slack utm component in docs (#10214) 2024-04-30 12:54:54 +09:00
Harshal Sheth
e64229b036
feat(ingest/dbt): handle complex dbt sql + improve docs (#10323) 2024-04-24 11:13:32 -07:00
Raj Tekal
996e5b0130
chore(metadata) Addressing vulnerabilities (#10296) 2024-04-19 12:53:50 -07:00
Pablo Osinaga
62c7ac706b
feat(ingest/metabase): add ability to exclude other users collections (#10330) 2024-04-19 12:53:17 -07:00
Harshal Sheth
8ecafbe802
fix(ingest/kafka): clarify meta-mapping docs (#10320) 2024-04-18 10:47:43 -07:00
Mayuri Nehate
529710ab9d
fix(ingest/databricks): handle and report config parse failure, updat… (#10261) 2024-04-17 12:14:16 -07:00
Tamas Nemeth
d463a16b49
chore(ingest/presto-on-hive): Renaming presto-on-hive to hive-metastore source (#10278) 2024-04-16 23:35:16 +02:00
Shubham Jagtap
90c1249e7d
feat(ingest/sigma): Sigma connector integration (#10037)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-04-15 20:18:31 -07:00
dushayntAW
f860f7907d
fix(ingest/transformer): replace externalUrl in dataset properties (#10281) 2024-04-15 20:14:42 +05:30
Dotan Mor
fa0c1b3fa9
feat(ingest/cockroachdb): add cockroachdb ingestion (#10226) 2024-04-09 18:36:51 -07:00
Mayuri Nehate
6997abd42e
feat(ingest/nifi): ingest process group as browse path v2, incremental lineage (#10202)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-04-09 17:44:20 +05:30
dushayntAW
e82e8ba715
fix(ingestion/datahub): moved urn_pattern config to source config (#10215) 2024-04-08 15:54:07 +05:30
Shubham Jagtap
fa139a582e
feat(ingestion/transformer): Handle overlapping while mapping in extract ownership from tags transformer (#10201) 2024-04-04 12:19:11 -07:00
dushayntAW
8c70aa15f2
fix(ingestion/datahub): add allow/deny URN option (#10174) 2024-04-03 17:33:33 +05:30
Shubham Jagtap
9f2c5d36f3
feat(ingestion/bigquery): BigQuery Owner Label to Datahub Ownership (#10047) 2024-03-28 15:50:25 -07:00
Mayuri Nehate
9928d70c1d
fix(ingest/databricks): support hive metastore schemas with special char (#10049) 2024-03-27 09:41:46 -07:00
dushayntAW
dd502ae662
fix(ingest): added new transformer to cleanup suffix/prefix in owner URN (#10067) 2024-03-22 15:23:03 +05:30
Harshal Sheth
6c3834b38c
feat(ingest/dbt): add option to skip sources (#10077) 2024-03-21 09:10:24 +01:00