1604 Commits

Author SHA1 Message Date
Shirshanka Das
f3a348a231
sdk(platform-resource): add entity type for ease of use (#11541) 2024-10-07 20:30:14 -07:00
sid-acryl
6b961133e9
fix(ingestion/powerbi): fix for databricks lineage m-query pattern (#11462) 2024-10-07 14:42:44 -07:00
Mayuri Nehate
364496c3d7
fix(ingest/sigma): handle members api paginated response (#11535) 2024-10-04 11:32:57 -07:00
skrydal
134ad21afe
fix(ingestion/nifi): Fix for incremental lineage ingestion for nifi (#11517) 2024-10-04 17:29:06 +05:30
skrydal
e1514d5e8e
fix(ingestion/nifi): Improve nifi lineage extraction performance (#11490) 2024-10-01 21:51:00 +02:00
sagar-salvi-apptware
660fbf8e57
fix(ingestion/transformer): Add container support for ownership and domains (#11375) 2024-10-01 11:39:07 -07:00
sid-acryl
67d7116055
fix(ingest/lookml): missing lineage for looker template -- if prod (#11426) 2024-10-01 11:26:00 -07:00
Harshal Sheth
07034caf09
feat(ingest): support DATAHUB_INCLUDE_ENV_IN_CONTAINER_PROPERTIES (#11476) 2024-09-27 10:24:22 -07:00
Jonny Dixon
a8e6a06ea4
docs(ingest/dbt): update run result paths examples (#11138) 2024-09-27 10:11:06 -07:00
haeniya
99bfcefb72
feat(ingestion/tableau): introduce project_path_pattern (#10855)
Co-authored-by: Yanik Häni <Yanik.Haeni1@swisscom.com>
2024-09-27 09:46:39 -07:00
Mayuri Nehate
1a73c664f0
feat(ingest/databricks): add usage perf report (#11480) 2024-09-27 09:46:22 -07:00
Mayuri Nehate
6a58493011
fix(ingest/bq): do not query PARTITIONS for biglake tables (#11463) 2024-09-27 16:46:37 +05:30
Harshal Sheth
d754650875
feat(ingest/dbt): produce multiple assertions for multi-table dbt tests (#11451) 2024-09-25 13:10:11 -07:00
sid-acryl
59b21ac24f
fix(ingestion/looker): skip personal folder independent looks (#11415) 2024-09-24 20:31:06 +05:30
sagar-salvi-apptware
d696dbef10
fix(ingest/mode): add connection timeouts to avoid RemoteDisconnected errors (#11245) 2024-09-23 12:28:30 -07:00
sid-acryl
9fb2df11f3
fix(ingest): sort by last modified not working in the UI (#11343) 2024-09-23 10:06:05 -07:00
Harshal Sheth
aec5e1b249
fix(ingest/dbt): handle null index values (#11433) 2024-09-19 16:05:44 -07:00
Sergio Gómez Villamor
31edb46dbc
feat(ingestion): adds env property in ContainerProperties (#11214)
Co-authored-by: siladitya2 <siladitya2@gmail.com>
2024-09-18 14:56:52 +05:30
Harshal Sheth
38bcd9c381
feat(ingest): default to ASYNC_BATCH mode in datahub-rest sink (#11369) 2024-09-17 07:11:58 +01:00
Harshal Sheth
3755731f0e
chore(ingest): improve code formatting (#11326) 2024-09-11 10:48:57 -07:00
Harshal Sheth
311ea10833
feat(ingest): maintain ordering in file-backed dict (#11346) 2024-09-10 13:53:38 -07:00
Mayuri Nehate
837d00d391
fix(ingest/bq): fix ordering of queries for use_queries_v2 (#11333) 2024-09-10 12:17:23 -07:00
sagar-salvi-apptware
852a23b845
feat(mode/ingest): Add support for missing Mode datasets in lineage (#11290) 2024-09-10 10:24:55 -07:00
Harshal Sheth
f4033707d4
chore(ingest): bump acryl-sqlglot (#11331) 2024-09-09 21:09:44 -07:00
Mayuri Nehate
cf49f80e77
feat(ingest/sql): auto extract and use mode query user metadata (#11307)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-09-09 12:38:24 -07:00
Harshal Sheth
28310bb64b
feat(ingest): support full urns without owner_type in meta mapping (#11298)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-09-09 11:18:07 -07:00
sid-acryl
3150d90bd1
fix(ingestion/tableau): restructure the tableau graphql datasource query (#11230)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-09-09 10:45:06 -07:00
Harshal Sheth
9589adebb3
feat(ingest): enable query usage stats by default (#11281) 2024-09-06 10:02:40 -07:00
david-leifker
ede9520b87
feat(schemaField): populate schemaFields with side effects (#10928) 2024-09-04 15:36:12 -05:00
Harshal Sheth
401b787427
fix(ingest): add custom StrEnum type (#11270) 2024-09-04 08:09:36 -07:00
Mayuri Nehate
08a5956e26
fix(bigquery): followups on bigquery queries v2 integration (#11291) 2024-09-04 07:42:35 -07:00
sleeperdeep
e34a628afe
fix(ingest/mssql): remove lower() method from sql_common get_db_name (#10773) 2024-09-03 12:41:23 -07:00
Mayuri Nehate
1f3688a1ed
feat(ingest/databricks): include metadata for browse only tables (#10766)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-09-02 19:09:45 +05:30
Mayuri Nehate
a7fc7f519a
feat(ingest/bq): integrate bigquery-queries into main source (#11247) 2024-08-30 18:16:45 -07:00
matthew-coudert-cko
c513e17dbb
feat(ingest/fivetran): support filtering on destination ids (#11277) 2024-08-30 13:25:18 -07:00
Andrew Sikowitz
fa1164aa63
feat(ingest/s3): Support reading S3 file type (#11177)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-08-30 12:15:12 +02:00
Alisa Aylward
675b225c5b
feat(ingest/snowflake): Add cluster formula to dataset properties (#11254) 2024-08-29 16:11:35 -07:00
Alisa Aylward
ca2e134d34
fix(ingest/snowflake): exclude snowflake excluded tags (#11250) 2024-08-27 08:48:59 -07:00
Felix Lüdin
ce99bc4f22
feat(ingest): add ingestion source for SAP Analytics Cloud (#10958)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-26 11:29:15 -07:00
Mayuri Nehate
223650dd7a
feat(ingest): add bigquery-queries source (#10994)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-25 22:51:00 -07:00
Harshal Sheth
e0c13fda27
feat(ingest/dbt): add support for urns in add_owner directive (#11221) 2024-08-23 13:52:21 -04:00
Tamas Nemeth
ef6a410091
feat(ingest/s3): Partition support improvements (#11083)
- Partition autodetection
- Option to find min/max/min-max partition of a dataset
- Generating Partition aspects
2024-08-22 17:55:43 +02:00
sagar-salvi-apptware
50ed448861
fix(ingest/sagemaker): ensure consistent STS token usage with refresh mechanism (#11170)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-08-22 15:42:13 +05:30
sid-acryl
dc30c0a0b7
feat(ingestion/looker): filter Looker dashboards by folder (#11205)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-22 14:10:00 +05:30
Alisa Aylward
88b3893b5c
feat(ingest/snowflake): allow iceberg tables in lineage and access metadata (#10961) 2024-08-22 01:20:51 +00:00
Andrew Sikowitz
30c4fa91ce
feat(ingest/mongodb): Ingest databases as containers (#11178)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-08-21 14:53:47 -04:00
sid-acryl
4e35016696
fix(ingest/lookml): support view inheritance for fields (#11148) 2024-08-21 12:32:46 -04:00
Mayuri Nehate
9568a4254d
feat: separate great-expectations action package (#11096)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-21 12:13:36 -04:00
sid-acryl
6b3c06ae8c
fix(ingestion/lookml): drop hive. from CLL (#11210) 2024-08-21 14:56:37 +05:30
sid-acryl
627c5abfd6
feat(ingestion/bigquery): Add ability to filter GCP project ingestion based on project labels (#11169)
Co-authored-by: Alice Naghshineh <alice.naghshineh@nytimes.com>
Co-authored-by: Alice Naghshineh <45885699+anaghshineh@users.noreply.github.com>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-08-20 14:42:00 -04:00