3904 Commits

Author SHA1 Message Date
Aseem Bansal
53d1379a85
fix(cli): telemetry - failures should not cause the CLI to fail (#4302) 2022-03-03 09:40:01 -08:00
Ravindra Lanka
520968b247
fix(ingest): bigquery - fix stateful ingestion checkpoint reconstruction. (#4295) 2022-03-02 13:10:15 -08:00
Kevin Hu
b2b8826118
fix(ingest): clarify s3/s3a requirements and platform defaults (#4263) 2022-03-02 08:58:27 -08:00
Tamas Nemeth
2a5cf3dd07
feat(ingest): bigquery - ability to disable partition profiling (#4228) 2022-03-01 22:29:48 -08:00
Aseem Bansal
d52638a252
fix(ingest): lib - revert positional arg change, create new method (#4266)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-03-01 22:06:37 -08:00
Ravindra Lanka
ba0c912c88
fix(ingest): snowflake - fix view upstream lineages to eliminate false edges. (#4284) 2022-03-01 16:53:26 -08:00
MugdhaHardikar-GSLab
ec26b0d52a
fix(ingest): fix config parsing for $ (#4258) 2022-03-01 16:11:06 -08:00
Tamas Nemeth
329b0b34b2
feat(ingest): bigquery - supporting v2 audit metadata on Bigquery (#4233) 2022-03-01 16:09:19 -08:00
Aditya Radhakrishnan
1275379d7d
feat(ingest) - cli - always display CLI version (#4282) 2022-03-01 16:07:58 -08:00
mohdsiddique
e2f8db7926
feat: powerbi - add new source (#4201) 2022-02-28 17:37:22 -08:00
Ravindra Lanka
f12f653366
fix(ingest): snowflake - lineage + logging & reporting improvements. (#4276) 2022-02-28 17:34:48 -08:00
RyanHolstien
77a27359f2
fix(platform): timeline - fix semantic version ordering in API response and cli improvements (#4271) 2022-02-28 15:14:13 -08:00
Vishal Shah
93ff09517b
feat(ingest): add lineage_client_project_id field to the BigQuery config (#4138)
* feat(ingest): add lineage_client_project_id field to the bigquery config

* fix linting issues

* add type annotation for arguments
2022-02-28 11:19:23 -08:00
Aseem Bansal
c8a3e68202
fix(docs): doc build failing (#4267) 2022-02-28 10:53:22 -08:00
ksrinath
60c12d04be
docs(lineage): add fine-grained lineage docs and examples (#4260) 2022-02-25 07:35:48 -08:00
Vincenzo Lavorini
a113e4357e
fix(ingest): openapi - add support for user, password auth (#4086) 2022-02-24 23:29:01 -08:00
Kevin Hu
02fe05eb8f
feat(ingest): data-lake - remove spark requirement if not profiling (#4131) 2022-02-24 23:26:06 -08:00
Kevin Hu
3d6db93aa6
fix(ingest): fix mixpanel profile emits (#4253) 2022-02-24 21:26:41 -08:00
Ravindra Lanka
1ac20ee764
feat(platform): add timeline api for dataset entity (#4252) 2022-02-24 19:40:55 -08:00
Sunil Patil
16f3d4683a
feat(ingest): elasticsearch - add support for url_prefix in configuration (#4214)
Co-authored-by: Sunil Patil <spatil@twilio.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-02-24 17:06:38 -08:00
Edward Vaisman
6ff551cbcd
feat(ingest): lineage-file - add ability to provide lineage manually through a file (#4116) 2022-02-24 17:02:38 -08:00
Aseem Bansal
6248a55edb
fix(docs): fix broken link (#4242) 2022-02-24 15:45:52 -08:00
David Sanchez
8f75c278a1
fix(ingest): tableau - fix sheets external url ingestion (#4231) 2022-02-24 15:44:31 -08:00
Ravindra Lanka
232f7c5b21
feat(model): Column level lineage model (#4248)
co-authored-by: ksrinath <k.srinath@gmail.com>
2022-02-24 14:36:19 -08:00
Arun Vasudevan
a3940962ba
fix(ingest): kafka-connect - pass the env variable as part of making dataset (#4244) 2022-02-24 13:58:52 -08:00
Kevin Hu
46701319dc
feat(ingest): switch telemetry endpoint to Mixpanel (#4238) 2022-02-24 12:35:48 -08:00
Harshal Sheth
49a8ece02a
fix(ingestion): enable compat with avro 1.11 (#4205) 2022-02-22 22:13:50 -08:00
Xu Wang
aa3363bcc2
feat(ingest): lib - add better support for working with urns (#4172)
Co-authored-by: Xu Wang <xu.wang@grandrounds.com>
2022-02-22 19:39:24 -08:00
Ravindra Lanka
7f4cb87c57
Revert "fix(ingest): Use lower-case dataset names in the dataset urns for all SQL-styled datasets. (#4140)" (#4218)
This reverts commit 6c75185445bbb23974932ff64cb142ee6bf5b51b.
2022-02-22 16:21:40 -08:00
Ravindra Lanka
84005d3848
feat(ingest): kafka - add support for non-default schema registry subject name strategies (#4215) 2022-02-22 16:05:46 -08:00
Gabe Lyons
a9c1b7e6a3
fix(ingest): dbt - dont produce key aspects if the entity has no other aspects (#4217) 2022-02-22 16:02:58 -08:00
Jie Qiu
c372b93804
Fix config typo in stateful ingestion README (#4202) 2022-02-22 15:20:53 -08:00
Swaroop Jagadish
c6cb549918
fix(ingest): superset - adding missing greenlet dep (#4203) 2022-02-21 09:07:44 -08:00
Alexander Chashnikov
c2065bd7fe
feat(ingest): clickhouse - add initial support (#4057)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-02-21 07:36:08 -08:00
Guido Turtu
3a6d1bde64
feat(ingest): airflow - capture airflow task description (#4147)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-02-21 00:07:08 -08:00
Kevin Hu
6b5fba882e
fix(ingest): sql-sources: add mapping for postgres types (#4179) 2022-02-20 19:32:15 -08:00
Tamas Nemeth
a4dc4137b7
feat(ingest): sql-sources - prevent hard failure on table/view ingestion exceptions (#4185) 2022-02-20 14:32:59 -08:00
Tamas Nemeth
3d02b5bec8
feat(ingest): bigquery - ignore temporary tables from lineage and connect edges directly (#4160) 2022-02-20 14:23:23 -08:00
Kevin Hu
c3f531fbf9
feat(ingest): profiling - record bucketed profiling runtimes (#4068) 2022-02-20 13:39:25 -08:00
Tamas Nemeth
c7ea6f0db2
fix(ingest): bigquery - profiling works with limit and offset using temp tables (#4161) 2022-02-19 19:20:26 -08:00
Harshal Sheth
1b60fae014
test(airflow): fix airflow version parsing (#4142)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-02-19 18:13:01 -08:00
Sunil Patil
be64092c89
fix(ingest): elasticsearch - pinning down elasticsearch dependency to less than 8.0.0 (#4182)
Co-authored-by: Sunil Patil <spatil@twilio.com>
2022-02-19 11:44:32 -08:00
abiwill
8bbc66b3e6
fix(ingest): elasticsearch - http/https host config support (#4191)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-02-19 11:42:01 -08:00
Ravindra Lanka
ab633d0047
fix(ingest): snowflake - handle external S3 bucket lineage for "External Tables". (#4192) 2022-02-18 15:52:13 -08:00
Swaroop Jagadish
eaf7b02b2a
docs(model): auto-generated docs and hand-written docs for the metadata model (#4189) 2022-02-18 09:45:45 -08:00
Tamas Nemeth
8cb312c4be
fix(ci): pinning markup_safe (#4188) 2022-02-18 08:53:28 -08:00
Tamas Nemeth
585aad1aac
Fix for wrong containers on Athena (#4167) 2022-02-17 01:34:02 -08:00
Tamas Nemeth
2e7f3ae6f3
Removing sql_common dep from glue (#4168) 2022-02-17 01:33:29 -08:00
Harshal Sheth
75d2ec2a39
ci(ingestion): fix airflow 1 deps for tox (#4083) 2022-02-17 00:33:28 -08:00
Harshal Sheth
6f37ddfee2
chore(ingestion): upgrade mypy (#4141) 2022-02-17 00:31:20 -08:00