3975 Commits

Author SHA1 Message Date
Aseem Bansal
efc42df3d9
ci: update outdated actions for java and python setup (#11261) 2024-08-29 14:27:35 +05:30
Aseem Bansal
15c1cfc386
ci(build): update outdated action & pin deepdiff lib (#11260) 2024-08-28 19:53:42 +05:30
Tamas Nemeth
1893b7b2dc
feat(ingest/dagster): Add automatic snowflake_pandas_io_manager asset capture (#11189)
Co-authored-by: shubhamjagtap639 <shubham.jagtap@gslab.com>
Co-authored-by: John Joyce <john@acryl.io>
2024-08-28 09:22:11 +02:00
Mayuri Nehate
dc88d11fa3
docs: fix great-expectations doc module_name (#11253) 2024-08-27 20:31:29 -07:00
Harshal Sheth
8f88858eef
fix(ingest/kafka): update warning reporting for kafka (#11171) 2024-08-27 09:42:26 -07:00
Alisa Aylward
ca2e134d34
fix(ingest/snowflake): exclude snowflake excluded tags (#11250) 2024-08-27 08:48:59 -07:00
Jay Feldman
56a563b8c5
feat(ingest/bigquery): Add query job retries for transient errors (#11162)
Co-authored-by: Gabe Lyons <gabe.lyons@acryl.io>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2024-08-26 21:19:32 -07:00
Peter Esselius
414dc54b13
feat(ingest/superset): clickhousedb -> clickhouse mapping in superset ingestion (#11201) 2024-08-26 16:38:01 -07:00
Felix Lüdin
ce99bc4f22
feat(ingest): add ingestion source for SAP Analytics Cloud (#10958)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-26 11:29:15 -07:00
Aseem Bansal
e2e6d105d3
fix(cli/delete): change filter to include env (#11246) 2024-08-26 09:56:02 -07:00
david-leifker
94e7706e3b
chore(bump): bump azure-identity (#11235)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-26 09:44:08 -05:00
Mayuri Nehate
223650dd7a
feat(ingest): add bigquery-queries source (#10994)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-25 22:51:00 -07:00
Harshal Sheth
1a09cb2c2a
fix(ingest/snowflake): propagate table list from main to query extractor (#11222) 2024-08-23 16:42:32 -04:00
Harshal Sheth
e0c13fda27
feat(ingest/dbt): add support for urns in add_owner directive (#11221) 2024-08-23 13:52:21 -04:00
Harshal Sheth
2729925cb8
docs(ingest): update config docs on platform instances (#11206) 2024-08-23 13:50:17 -04:00
Tamas Nemeth
ef6a410091
feat(ingest/s3): Partition support improvements (#11083)
- Partition autodetection
- Option to find min/max/min-max partition of a dataset
- Generating Partition aspects
2024-08-22 17:55:43 +02:00
sagar-salvi-apptware
50ed448861
fix(ingest/sagemaker): ensure consistent STS token usage with refresh mechanism (#11170)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2024-08-22 15:42:13 +05:30
sid-acryl
dc30c0a0b7
feat(ingestion/looker): filter Looker dashboards by folder (#11205)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-22 14:10:00 +05:30
Alisa Aylward
88b3893b5c
feat(ingest/snowflake): allow iceberg tables in lineage and access metadata (#10961) 2024-08-22 01:20:51 +00:00
Harshal Sheth
0927c6331c
fix(ingest/redshift): avoid asserts in redshift schemas (#11219) 2024-08-21 17:08:13 -04:00
Andrew Sikowitz
30c4fa91ce
feat(ingest/mongodb): Ingest databases as containers (#11178)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-08-21 14:53:47 -04:00
sid-acryl
4e35016696
fix(ingest/lookml): support view inheritance for fields (#11148) 2024-08-21 12:32:46 -04:00
Mayuri Nehate
9568a4254d
feat: separate great-expectations action package (#11096)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-21 12:13:36 -04:00
sid-acryl
6b3c06ae8c
fix(ingestion/lookml): drop hive. from CLL (#11210) 2024-08-21 14:56:37 +05:30
sid-acryl
627c5abfd6
feat(ingestion/bigquery): Add ability to filter GCP project ingestion based on project labels (#11169)
Co-authored-by: Alice Naghshineh <alice.naghshineh@nytimes.com>
Co-authored-by: Alice Naghshineh <45885699+anaghshineh@users.noreply.github.com>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-08-20 14:42:00 -04:00
skrydal
b46a9632cb
fix(tests): Bump databricks-sdk dependency to >=0.30.0 (#11209) 2024-08-20 15:07:30 +01:00
skrydal
8f7642b910
fix(ingestion/tableau): Tableau field type parsing (#11202) 2024-08-20 13:24:16 +02:00
Harshal Sheth
bb8cf97ea3
feat(ingest/dbt): skip CLL on sources with skip_sources_in_lineage (#11195) 2024-08-19 15:38:27 -07:00
sid-acryl
cb33c0fef7
feat(ingestion/lookml): support looker -- if comments (#11113) 2024-08-16 15:27:59 -04:00
sagar-salvi-apptware
608c5cfc33
fix(ingestion/transformer): extend dataset_to_data_product_urns_pattern to support containers (#11124) 2024-08-16 17:18:35 +05:30
skrydal
11890e5445
feat(ingest): allow custom SF API version (#11145) 2024-08-16 10:46:42 +02:00
RyanHolstien
3b895d9062
feat(docs): update docs for new release (#11164) 2024-08-15 15:59:22 -05:00
Harshal Sheth
897173f270
feat(dbt): support prefer_sql_parser_lineage with sources enabled (#11168) 2024-08-13 13:54:50 -07:00
Tamas Nemeth
5e9188ca2c
fix(ingest/databricks): Updating code to work with Databricks sdk 0.30 (#11158) 2024-08-13 16:57:31 +02:00
sid-acryl
b1f16f9b11
fix(ingestion/lookml): fix for sql parsing error (#11079)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-09 15:06:42 -07:00
Harshal Sheth
17868cb069
feat(ingest): various logging improvements (#11126) 2024-08-09 14:48:53 -07:00
Mayuri Nehate
78336c9f58
fix(ingest/bigquery): use small batch size if use_tables_list_query_v2 is set (#11121) 2024-08-08 20:48:51 -07:00
Harshal Sheth
840b15083a
fix(sql-parser): prevent bad urns from alter table lineage (#11092) 2024-08-08 14:05:55 -07:00
AndreasHegerNuritas
a4a887c866
fix(ingest/redshift): replace r'\n' with '\n' to avoid token error redshift serverless… (#11111) 2024-08-08 08:38:16 -07:00
Harshal Sheth
d6e46b9bcf
fix(ingest): downgrade column type mapping warning to info (#11115) 2024-08-07 14:57:05 -07:00
Harshal Sheth
a25df8e6a0
fix(snowflake): avoid reporting warnings/info for sys tables (#11114) 2024-08-07 14:04:18 -07:00
Harshal Sheth
edb0f19f1b
fix(ingest/tableau): prevent empty site content urls (#11057)
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2024-08-07 13:50:08 -07:00
sid-acryl
8bea5d2a3d
feat(ingest/lookml): CLL support for sql set in sql_table_name attribute of lookml view (#11069) 2024-08-06 21:03:14 -07:00
sid-acryl
2755cf3559
fix(ingest/powerbi): fix broken lineage between chart and dataset (#11080) 2024-08-06 21:02:21 -07:00
Harshal Sheth
832093ad75
feat(ingest): tweak stale entity removal messaging (#11064) 2024-08-06 14:00:03 -07:00
Felix Lüdin
9619553e2d
fix(ingest): use correct native data type in all SQLAlchemy sources by compiling data type using dialect (#10898)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-06 12:52:20 -07:00
Tamas Nemeth
41fbae5afd
fix(ingest/mode): Regexp in mode name matcher didn't allow underscore (#11098) 2024-08-05 22:03:21 +02:00
Pinaki Bhattacharjee
210b31135c
chore(vulnerability): Overly permissive regex range (#11061)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-03 08:09:40 -05:00
Tamas Nemeth
6704d44470
doc(ingest): Update capabilities (#11072) 2024-08-02 20:58:20 -07:00
Harshal Sheth
50139431be
fix(ingest): set lastObserved in sdk when unset (#11071) 2024-08-02 15:48:10 -07:00