68 Commits

Author SHA1 Message Date
Tamas Nemeth
5df5150e51
feat(ingest): Add config option to set Bigquery credential in source config (#3786) 2021-12-27 05:48:45 -08:00
Tamas Nemeth
c2459056e2
docs(redshift): Updating Redshift permission prerequisites in doc (#3777) 2021-12-22 14:15:09 -08:00
John Joyce
110efa68b9
docs(snowflake): Adding documentation about required Snowflake Privileges (#3770) 2021-12-19 12:01:53 -08:00
Tamas Nemeth
599edd22ae
fix(ingest): profiling - disable expensive profilers by default (#3759) 2021-12-17 17:17:25 -08:00
Hyun Min Choi
e73a30dc81
feat(ingest): bigquery - add support for parsing exported audit logs (#3680) 2021-12-17 17:05:21 -08:00
Ravindra Lanka
bd69e736ba
feat(Stateful Ingestion-2/3): Client side changes for checkpointing a source job state. (#3763)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-12-16 20:06:33 -08:00
jawadqu
b4eaa9e09e
feat(ingestion): Mode retry wait logic to avoid hitting Mode API rate limit (#3761) 2021-12-16 19:17:12 -08:00
Sergio Gómez Villamor
c59c63e90d
feat: enables dbt metadata files to be loaded from URIs (#3739) 2021-12-15 09:11:39 -08:00
varunbharill
70d068892e
fix(ingest): snowflake honor allow/deny pattern for lineage and usage. (#3748) 2021-12-14 20:42:29 -08:00
Gabe Lyons
2e2ed34250
feat(ingest): snowflake-usage add knob for direct objects accessed vs base objects accessed (#3744) 2021-12-14 18:07:55 -08:00
jawadqu
578590e795
feat(ingestion) : Add Metabase Source Connector (#3602)
Co-authored-by: Jawad Qureshi <jqureshi@petabloc.com>
2021-12-13 23:02:47 -08:00
Tamas Nemeth
0c8c29a6a2
docs(redshift): adding svv_table privilege requirement to redshift source (#3708) 2021-12-10 17:58:42 -08:00
Gabe Lyons
5d8c813684
fix(mode): support definitions in mode query (#3721)
Co-authored-by: Jawad Qureshi <jqureshi@petabloc.com>
2021-12-10 17:56:39 -08:00
Gabe Lyons
8394fc62b0
feat(mode): add mode analytics ingestion source (#3710) 2021-12-09 16:10:08 -08:00
mayurinehate
bd4ecbc7b9
fix(nifi): add env in nifi config, add unit tests, fix nifi doc (#3703) 2021-12-09 13:34:13 -08:00
Tamas Nemeth
eef26fe8ef
docs(redshift): Adding requirements for redshift permissions (#3707) 2021-12-09 13:32:15 -08:00
Gabe Lyons
0fdd3352bd
feat(ingestion): Add lineage support for Redshift source (#3697)
Co-authored-by: treff7es <treff7es@gmail.com>
2021-12-08 23:41:18 -08:00
mayurinehate
1d7ec8dba8
feat(ingest): add nifi source (#3681) 2021-12-08 14:56:31 -08:00
Aseem Bansal
aba060a04c
docs(business glossary): fix specification of the file (#3679) 2021-12-07 17:14:27 -08:00
Gabe Lyons
996b0491f6
docs(business glossary): document the business glossary file format (#3639) 2021-11-29 23:53:08 -08:00
varunbharill
c22192a3d4
fix(redshift-usage): Add docs for redshift usage ingestion. (#3617) 2021-11-28 16:04:14 -08:00
Harshal Sheth
cb2a3e6927
feat(profiler): dynamically combine queries (#3572) 2021-11-24 10:19:03 -08:00
Tamas Nemeth
a36fefaa33
feat(ingest): Starburst Trino usage (#3558) 2021-11-18 09:56:24 -08:00
mayurinehate
69036acfca
feat(ingest): add bigquery sink connector lineage in kafka connect source (#3590) 2021-11-17 17:18:37 -08:00
Ravindra Lanka
7d64fc755b
fix(docs): snowflake documentation (#3585) 2021-11-16 10:58:14 -08:00
Zach Bluhm
42419ec9fa
feat(ingest): allow logs to be filtered in Bigquery Usage plugin (#3567) 2021-11-16 08:11:17 -08:00
Kevin Hu
af366711b7
fix(ingest): Glue lineage compatibility docs and more resilience (#3555) 2021-11-13 08:15:59 -08:00
Vincenzo Lavorini
6cd31b5bdc
feat(ingest): adding openapi ingestion (#2706) 2021-11-09 21:22:15 -08:00
varunbharill
417b555fef
feat(dbt): dbt enhancements - dbt nodes, lineage, subtype, etc. (#3519) 2021-11-05 09:27:41 -07:00
Andrew Maguire
8ef9330fa3
docs: fix typo in bigquery source docs (#3484) 2021-11-02 18:05:46 -07:00
Ravindra Lanka
a1bf95307b
feat(ingest): user configurable dataset profiling. (#3453) 2021-10-27 19:49:40 -07:00
AndreasTA-AW
561c04bcf8
feat(dbt-ingestion): add ability to skip specific models (#3340) 2021-10-25 22:31:19 -07:00
Swaroop Jagadish
6704765d41
feat(ingest): add parallelism to looker source and datahub rest sink (#3431) 2021-10-21 11:27:27 -07:00
aseembansal-gogo
9dff1b7a1c
fix(ingest): allow database alias, remove extra removal from connect_… (#3352) 2021-10-19 22:29:38 -07:00
Harshal Sheth
9c09b78147
feat(ingest): support profiling tables in parallel (#3369) 2021-10-19 22:24:20 -07:00
John Joyce
c6a657939a
docs(ingest): add required permissions to Azure AD source doc (#3414) 2021-10-18 09:09:11 -07:00
varunbharill
e717d6a937
feat(bigquery): Ingest lineage metadata from GCP logs. (#3389) 2021-10-18 08:48:04 -07:00
aseembansal-gogo
ebe0b3f7c1
feat(ingest): add mariadb as possible source (#3245) 2021-10-12 23:27:47 -07:00
aseembansal-gogo
558aba9917
feat(ingest): allow disabling sample values in profiling (#3355) 2021-10-11 22:46:15 -07:00
mayurinehate
9590c29ea3
docs(ingest): fix trino doc (#3339) 2021-10-08 12:04:12 -07:00
mayurinehate
d22d1c2795
feat(ingest): add trino source to datahub (#3307) 2021-10-06 11:57:06 -07:00
Remi
3d8e22ea34
fix(ingest): make dbt tag prefix configurable (#3334) 2021-10-06 11:55:59 -07:00
Swaroop Jagadish
cfc97107e8
feat(ingest): add support for github and looker links in looker views and explores (#3332) 2021-10-06 07:13:38 -07:00
taufiqibrahim
ed38168f54
feat(ingest): add support for parsing lineage for redash connector (#3267)
Co-authored-by: tibrahim <taufiq.ibrahim@bizzy.co.id>
2021-10-05 23:08:02 -07:00
Vincenzo Lavorini
9cb71d974e
feat(ingest): support user group filtering for Azure AD (#3312) 2021-10-05 23:03:30 -07:00
Ravindra Lanka
b4c0e20c68
feat(ingest): add lineage support to snowflake connector (#3331) 2021-10-05 22:58:07 -07:00
Seref Acet
fe531b9537
docs: Fix topic_pattern typo in kafka ingestion docs (#3317) 2021-10-03 17:52:25 -07:00
Swaroop Jagadish
9db7a34cfa
feat(ingest): looker and lookml improvements - connection, explores, folders (#3255) 2021-09-16 23:09:45 -07:00
Karolis Kascenas
792a08c283
feat(ingest): optionally ingest deleted looker dashboards (#3158) 2021-09-07 19:41:39 -07:00
Nick Wu
f833a1aa14
docs: fix "data platforms" link in dbt.md (#3188) 2021-09-02 18:53:28 -07:00