272 Commits

Author SHA1 Message Date
Mayuri Nehate
9928d70c1d
fix(ingest/databricks): support hive metastore schemas with special char (#10049) 2024-03-27 09:41:46 -07:00
Harshal Sheth
6c3834b38c
feat(ingest/dbt): add option to skip sources (#10077) 2024-03-21 09:10:24 +01:00
Harshal Sheth
1f9ce9d7d8
feat(ingest/dbt): point dbt assertions at dbt nodes (#10055) 2024-03-18 18:13:01 -07:00
dushayntAW
3a4bdef44a
fix(ingest/unity): generate sibling and lineage (#9894) 2024-03-18 20:22:17 +05:30
Harshal Sheth
0c7ca4cd3e
docs(ingest/lookml): update known discrepancy list (#9941)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2024-02-29 12:39:32 -08:00
Aditya Malik
92b1cfa194
feat(ingest): Support for JSONL in s3 source with max_rows support (#9921)
Co-authored-by: Aditya <aditya.malik@quillbot.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-02-28 15:05:30 +01:00
Shubham Jagtap
a1f2216da7
feat(ingest/qlik): Qlik cloud connector integration (#9682)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-02-26 11:27:37 -08:00
Shubham Jagtap
5921a334d3
feat(ingestion/trino): Add sibling support in ingestion (#9853) 2024-02-26 13:55:11 +05:30
tom
f7f0b14f37
fix(ingestion/metabase): Fetch Dashboards through Collections (#9631)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-25 17:51:41 -08:00
Harshal Sheth
f14dc4ded2
feat(ingest): improve sql tests + dbt reporting (#9649) 2024-01-23 08:37:23 +01:00
Pedro Silva
0906ce832d
fix(docs): Correct Kafka Connect sink documentation (#9672) 2024-01-19 16:45:22 -08:00
Mayuri Nehate
f2e78db92e
feat(ingest/databricks): ingest hive metastore by default, more docs (#9601)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-18 14:26:33 -08:00
Harshal Sheth
cf200a32ae
fix(docs-website): update copy around the video on index (#9661) 2024-01-18 12:33:19 -08:00
Shubham Jagtap
0486319bc8
feat(ingestion/fivetran): Add fivetran bigquery destination support (#9531)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-10 14:18:36 -05:00
Harshal Sheth
aa28837eed
docs(slack): add note about private Slack channels (#9589) 2024-01-09 22:07:14 +01:00
Andrew Sikowitz
e4d8dcbc02
docs(ingest/sql-queries): Rearrange sections (#9426) 2023-12-11 09:43:23 -05:00
Andrew Sikowitz
a6726c12dd
docs(ingest/sql-queries): Add documentation (#9406) 2023-12-07 17:03:37 +01:00
Andrew Sikowitz
27f23ecdd5
feat(ingest/unity): GE Profiling (#8951) 2023-12-06 13:59:23 -05:00
Tamas Nemeth
3ee82590cd
fix(ingest/redshift): Identify materialized views properly + fix connection args support (#9368) 2023-12-05 17:42:29 +01:00
Harshal Sheth
82f375ded6
docs(ingest/lookml): clarify that ssh key has no passphrase (#9348) 2023-12-01 14:33:20 +05:30
Harshal Sheth
4d9eb12cba
feat(ingest/dbt): support custom ownership types in dbt meta (#9332) 2023-12-01 14:33:10 +05:30
terratrue-daniel
ab10e6bc58
feat(ingest/mssql): enable TLS encryption for SQLServer using pytds (#9256) 2023-11-29 13:32:26 +05:30
Shubham Jagtap
e73e926999
feat(integration/fivetran): Fivetran connector integration (#9018)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-11-07 23:02:41 -08:00
sachinsaju
dae320c9bc
docs(teradata): teradata doc external link 404 fix (#9152) 2023-10-31 15:58:39 -07:00
Tamas Nemeth
1007204cda
feat(ingest/teradata): view parsing (#9005) 2023-10-13 15:07:19 -04:00
Tamas Nemeth
a8f0080c08
feat(ingest/teradata): Teradata source (#8977) 2023-10-12 15:14:45 -07:00
Jinlin Yang
6310e51eb0
feat(ingestion/dynamodb): implement pagination for list_tables (#8910) 2023-10-05 09:33:31 +05:30
Mayuri Nehate
2fcced6db9
docs(ingest): add permissions required for athena ingestion (#8948) 2023-10-05 09:31:57 +05:30
ethan-cartwright
e2afd44bfe
feat(dbt-ingestion): add documentation link from dbt source to institutionalMemory (#8686)
Co-authored-by: Ethan Cartwright <ethan.cartwright@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-10-04 20:38:58 +00:00
Andrew Sikowitz
d3346a04e4
feat(ingest/unity): Ingest notebooks and their lineage (#8940) 2023-10-04 10:22:45 -04:00
siddiquebagwan-gslab
c415d63dda
feat(ingestion/powerbi): column level lineage extraction for M-Query (#8796) 2023-10-04 16:22:51 +05:30
hariishaa
622816dcb8
feat(metadata-ingestion): implement mlflow source (#7971)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-09-26 13:51:30 -04:00
Shubham Jagtap
501522d891
feat(ingest/kafka-connect): Lineage for Kafka Connect > Snowflake (#8811)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-09-22 17:12:48 -07:00
Mayuri Nehate
5c40390a92
feat(ingest/kafka): support metadata mapping from kafka avro schemas (#8825)
Co-authored-by: Daniel Messias <danielcmessias@gmail.com>
Co-authored-by: Deepankarkr <deepankar.kumar@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-09-22 17:11:42 -07:00
Harshal Sheth
5bb9f30895
docs(ingest/lookml): add guide on debugging lkml parse errors (#8890) 2023-09-22 16:55:15 -07:00
Tony Ouyang
f4da93988e
feat(ingestion/dynamodb): Add DynamoDB as new metadata ingestion source (#8768)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-09-15 13:26:17 -07:00
Hyejin Yoon
31abf383d1
ci: add markdown-link-check (#8771) 2023-09-14 11:34:21 +09:00
cccs-eric
6fe60a274e
feat(iceberg): Upgrade Iceberg ingestion source to pyiceberg 0.4.0 (#8357)
Co-authored-by: cccs-Dustin <96579982+cccs-Dustin@users.noreply.github.com>
Co-authored-by: Fokko Driesprong <fokko@apache.org>
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-08-31 13:01:05 -04:00
Andrew Sikowitz
40d17f00ea
feat(ingest/datahub): Improvements, bug fixes, and docs (#8735) 2023-08-29 14:33:40 -04:00
Tamas Nemeth
d86b336e70
chore(ingest/s3) Bump Deequ and Pyspark version (#8638)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-08-29 18:11:37 +02:00
Hyejin Yoon
04ecf4f75a
docs(docs): add native versioning (#8714) 2023-08-25 14:10:13 -07:00
Mayuri Nehate
e285da3e75
feat(ingest/snowflake): tables from snowflake shares as siblings (#8531) 2023-08-24 10:23:07 -04:00
RChygir
43d48ddde4
feat(ingest/mssql): load jobs and stored procedures (#5363) 2023-08-24 14:48:03 +05:30
Mayuri Nehate
3681e1a128
docs(ingest/kafka-connect): add details on platform instance mapping (#8654) 2023-08-18 18:51:14 +05:30
alplatonov
11fdfcf956
Fix(ingestion/clickhouse) move to two tier sqlalchemy (#8300)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-08-11 16:11:40 -04:00
Mayuri Nehate
7bb4e7b90d
docs(ingest): update s3 and gcs doc with concept mapping (#8575) 2023-08-11 11:01:15 -07:00
Jarod Smilkstein
f51bd01a70
feat(ingest): add ability to read other method types than GET for OAS ingest recipes (#8303) 2023-08-02 09:54:09 +05:30
Kirill Popov
eec89a884a
feat(ingest): Add metabase database id to platform instance mapping (#8359) 2023-08-02 09:53:48 +05:30
VISHAL KUMAR
ef3b9489aa
feat(ingest/vertica): performance improvement and bug fixes (#8328)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-08-01 19:34:35 +05:30
Harshal Sheth
99f1624ce7
docs(ingest/lookml): clarify connection map config (#8508) 2023-07-27 17:06:04 +05:30