102 Commits

Author SHA1 Message Date
Kevin Hu
9bdc9af7b9
fix(ingest): postgres - ignore information_schema tables by default (#4069) 2022-02-09 23:20:25 -08:00
Aseem Bansal
dbcfe9e50e
docs(kafka): add example for using domains, change for clarity (#4100) 2022-02-09 08:56:27 -08:00
John Joyce
2a9a076fc1
feat(ingest): Adding Tableau Source Connector [BETA] (#4063) 2022-02-08 14:26:44 -08:00
Tamas Nemeth
2de29dc623
feat(ingest) - bigquery: More verbose and faster lineage generation and option to set partition datetime for profiling (#4079) 2022-02-08 10:09:30 -08:00
Gabe Lyons
ec062b6787
feat(glue): make ownership configurable in glue source (#4078) 2022-02-07 14:29:51 -08:00
iasoon
782e66f5cf
docs(ingestion) glue: document required IAM permissions (#3929) 2022-02-07 10:10:35 -08:00
Tamas Nemeth
63bc830cfe
Data domain containers ingestion (#4051) 2022-02-07 09:51:49 -08:00
Pedro Silva
ef617784b1
Mark data lake metadata source as Beta (#4061) 2022-02-04 16:27:44 -08:00
Ravindra Lanka
f20382f956
feat(ingest): framework - client side changes for monitoring and reporting (#3807) 2022-02-02 13:19:15 -08:00
Tamas Nemeth
928ab74f33
feat(ingest): bigquery - profile only the latest partition/shard (#3930) 2022-02-01 10:05:35 -08:00
Tamas Nemeth
68711222d4
feat(ingest): usage-stats - add ability to ignore users from top users calculation (#3735) 2022-02-01 00:11:23 -08:00
mayurinehate
faac911df8
feat(ingest): use trino python client with sqlalchemy, supports python3.6 (#3888) 2022-01-31 09:22:55 -08:00
Aseem Bansal
f32643148e
fix(docs): fix business glossary docs (#3916) 2022-01-30 17:25:11 -08:00
Michael A. Schlosser
c36662f837
feat(ingest): snowflake - support for additional auth mechanisms (#4009) 2022-01-30 11:47:53 -08:00
Swaroop Jagadish
9a9f448ebf
feat(ingest): platform-instance - changes to support platform instances in sources (#3996) 2022-01-27 15:31:25 -08:00
John Joyce
cf38b61a7f
feat(ingest): looker - optional external_base_url config for chart and dashboard urls (#3985)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-26 15:06:34 -08:00
Kevin Hu
62f7d05557
feat(ingest): data-lake - support for Avro files (#3913) 2022-01-26 10:29:00 -08:00
iasoon
5dac30c93e
fix(ingestion): match default username for Azure OIDC and Azure ingestion source (#3926) 2022-01-25 18:57:18 -08:00
Ravindra Lanka
715a2e5f8e
feat(ingest): snowflake, glue - add support for lineage to s3 (#3946) 2022-01-21 17:19:21 -08:00
Aseem Bansal
6c3441ccdc
fix(docs): fix openapi docs (#3940) 2022-01-21 07:55:50 -08:00
Kevin Hu
d4075b972c
docs(ingest): data lake - describe path-spec configuration (#3905) 2022-01-17 15:41:02 -08:00
Kevin Hu
9e30b42638
feat(ingest): data lake - support for profiling data lake assets (#3656) 2022-01-17 15:01:23 -08:00
Jiwei
f804184884
docs(elasticsearch): fix typo in config example (#3901) 2022-01-17 14:56:17 -08:00
Ravindra Lanka
a44b48a6b8
feat(ingest): elasticsearch - add Elasticsearch Source (#3893)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-14 13:10:12 -08:00
iasoon
285aa1b9ac
docs(ingest): azure-ad - specify required permission type (#3886) 2022-01-13 12:30:16 -08:00
Aseem Bansal
7498f4bf54
doc(kafka): add permissions required for confluent cloud (#3850) 2022-01-12 14:50:58 -08:00
iasoon
988baeb467
feat(ingest): metabase - allow configuring how database engines get mapped to platforms (#3869) 2022-01-12 00:52:55 -08:00
iasoon
130d5ff635
docs(ingest): azure-ad - correct default value for username attr (#3861) 2022-01-11 19:37:28 -08:00
Aditya Radhakrishnan
c3e98645a2
feat(ingest): operational stats - show last updated for sql usage sources (#3845) 2022-01-07 17:27:09 -08:00
Swaroop Jagadish
480ad24282
feat(ingest): dbt - add support for dbt meta mapping (#3832) 2022-01-05 16:32:05 -08:00
Kevin Hu
5b369447f4
fix(ingest): glue - ignore custom connectors (#3805) 2022-01-05 15:12:50 -08:00
Tamas Nemeth
e5d8313788
feat(ingest): sql parsing - Replacing sqlmetadata sql parser lib with sqlineage parser lib (#3806) 2022-01-05 12:22:59 -08:00
Tamas Nemeth
450cdc1739
feat(snowflake-usage): Generate email address if not exists (#3791) 2021-12-29 08:14:18 -08:00
Aseem Bansal
895af09af2
docs(kafka): add example for ingestion from confluent cloud (#3789) 2021-12-28 07:17:03 -08:00
Tamas Nemeth
5df5150e51
feat(ingest): Add config option to set Bigquery credential in source config (#3786) 2021-12-27 05:48:45 -08:00
Tamas Nemeth
c2459056e2
docs(redshift): Updating Redshift permission prerequisites in doc (#3777) 2021-12-22 14:15:09 -08:00
John Joyce
110efa68b9
docs(snowflake): Adding documentation about required Snowflake Privileges (#3770) 2021-12-19 12:01:53 -08:00
Tamas Nemeth
599edd22ae
fix(ingest): profiling - disable expensive profilers by default (#3759) 2021-12-17 17:17:25 -08:00
Hyun Min Choi
e73a30dc81
feat(ingest): bigquery - add support for parsing exported audit logs (#3680) 2021-12-17 17:05:21 -08:00
Ravindra Lanka
bd69e736ba
feat(Stateful Ingestion-2/3): Client side changes for checkpointing a source job state. (#3763)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-12-16 20:06:33 -08:00
jawadqu
b4eaa9e09e
feat(ingestion): Mode retry wait logic to avoid hitting Mode API rate limit (#3761) 2021-12-16 19:17:12 -08:00
Sergio Gómez Villamor
c59c63e90d
feat: enables dbt metadata files to be loaded from URIs (#3739) 2021-12-15 09:11:39 -08:00
varunbharill
70d068892e
fix(ingest): snowflake honor allow/deny pattern for lineage and usage. (#3748) 2021-12-14 20:42:29 -08:00
Gabe Lyons
2e2ed34250
feat(ingest): snowflake-usage add knob for direct objects accessed vs base objects accessed (#3744) 2021-12-14 18:07:55 -08:00
jawadqu
578590e795
feat(ingestion) : Add Metabase Source Connector (#3602)
Co-authored-by: Jawad Qureshi <jqureshi@petabloc.com>
2021-12-13 23:02:47 -08:00
Tamas Nemeth
0c8c29a6a2
docs(redshift): adding svv_table privilege requirement to redshift source (#3708) 2021-12-10 17:58:42 -08:00
Gabe Lyons
5d8c813684
fix(mode): support definitions in mode query (#3721)
Co-authored-by: Jawad Qureshi <jqureshi@petabloc.com>
2021-12-10 17:56:39 -08:00
Gabe Lyons
8394fc62b0
feat(mode): add mode analytics ingestion source (#3710) 2021-12-09 16:10:08 -08:00
mayurinehate
bd4ecbc7b9
fix(nifi): add env in nifi config, add unit tests, fix nifi doc (#3703) 2021-12-09 13:34:13 -08:00
Tamas Nemeth
eef26fe8ef
docs(redshift): Adding requirements for redshift permissions (#3707) 2021-12-09 13:32:15 -08:00