3904 Commits

Author SHA1 Message Date
Swaroop Jagadish
9f9aaaac23
feat(cli): adding a put command and docs (#3614) 2021-11-24 00:21:44 -08:00
varunbharill
192cf65c4d
fix(bigquery-usage): fix filters and event joining logic. (#3610) 2021-11-23 15:40:11 -08:00
Ravindra Lanka
f1045f817c
docs(ingestion): Emitter api examples + Documentation (#3599) 2021-11-18 23:30:25 -08:00
Tamas Nemeth
a36fefaa33
feat(ingest): Starburst Trino usage (#3558) 2021-11-18 09:56:24 -08:00
Swaroop Jagadish
ebdd30bb73
feat(model): adding a field to capture unmodeled field level properties (#3593) 2021-11-17 17:34:20 -08:00
mayurinehate
69036acfca
feat(ingest): add bigquery sink connector lineage in kafka connect source (#3590) 2021-11-17 17:18:37 -08:00
Remi
a9754ebe83
fix(ingest): fix dbt source platform when disable_dbt_node_creation is False (#3592) 2021-11-17 16:49:48 -08:00
Ravindra Lanka
3a9ef61147
feat(ingestion): Adds --dry-run and --preview options to datahub ingest command. (#3584) 2021-11-17 09:24:58 -08:00
Swaroop Jagadish
806b28e697
fix(ci): SKIP_RELEASE_UPLOAD flag was not being respected by python release script (#3588) 2021-11-16 15:11:25 -08:00
Swaroop Jagadish
9fdcbdabb0
feat(ci): adding support for env variables in python release script (#3587) 2021-11-16 12:15:52 -08:00
Ravindra Lanka
7d64fc755b
fix(docs): snowflake documentation (#3585) 2021-11-16 10:58:14 -08:00
Zach Bluhm
42419ec9fa
feat(ingest): allow logs to be filtered in Bigquery Usage plugin (#3567) 2021-11-16 08:11:17 -08:00
Swaroop Jagadish
4b4c972e26
fix(cli): m1 check should not fail on windows machines (#3579) 2021-11-15 16:50:53 -08:00
Swaroop Jagadish
6a00cfb4a1
feat(ingest): adding snowflake app name to connection string (#3574) 2021-11-15 13:08:07 -08:00
Ravindra Lanka
87478a1658
fix(ingestion): redshift-usage should not append email domain to the username if already an email. (#3569) 2021-11-13 19:47:39 -08:00
Swaroop Jagadish
482dac6d2e
fix(ingest): fix bigquery-usage regex for both partitioned and sharded tables (#3571) 2021-11-13 19:43:59 -08:00
Courtney Lang
d33d9cbb17
fix(ingestion): bigquery add better support for partitioned and snapshot table names (#3533) 2021-11-13 15:02:19 -08:00
Kevin Hu
af366711b7
fix(ingest): Glue lineage compatibility docs and more resilience (#3555) 2021-11-13 08:15:59 -08:00
Vincenzo Lavorini
6cd31b5bdc
feat(ingest): adding openapi ingestion (#2706) 2021-11-09 21:22:15 -08:00
Noé López
cde1ce043d
feat(ingest): add transformer to add properties (#3480) 2021-11-09 20:03:21 -08:00
Harshal Sheth
3a432b7154
feat(profiler): streamline profiler (#3510) 2021-11-09 20:00:47 -08:00
Ravindra Lanka
dcb8789f17
feat(ingestion): Support for converting AVRO schemas with logical types to MCE fields. (#3546) 2021-11-09 19:59:01 -08:00
Swaroop Jagadish
94e9459586
feat(cli): add support for m1 laptops during quickstart (#3547) 2021-11-09 19:18:46 -08:00
Ravindra Lanka
c3f1bf0534
fix(ingestion): Make AVRO schema parsing robust to exceptions. (#3541) 2021-11-09 18:01:18 -08:00
aseembansal-gogo
0079101740
fix(cli): fix dry-run cli arg name to match documentation (#3536) 2021-11-09 06:42:27 -08:00
Harshal Sheth
8a432898ba
fix(ingest): denylist botocore 1.23.0 (#3534) 2021-11-08 16:23:25 -08:00
Swaroop Jagadish
3984506f51
feat(cli): add support for deletion based on filters, soft deletes an… (#3527) 2021-11-07 22:13:50 -08:00
Harshal Sheth
b0b275ee34
test(profiler): make profiling tests more comprehensive (#3513) 2021-11-07 18:53:53 -08:00
Ravindra Lanka
fcbf2977a8
fix(ingestion): Enhance logging and error-handling in bigquery usage connector. (#3521) 2021-11-07 18:38:10 -08:00
varunbharill
417b555fef
feat(dbt): dbt enhancements - dbt nodes, lineage, subtype, etc. (#3519) 2021-11-05 09:27:41 -07:00
Swaroop Jagadish
ec406c7928
feat(ingest): read-modify-write capabilities during ingestion, adding ownership patch transformer (#3506) 2021-11-03 21:39:52 -07:00
mayurinehate
192f0d33a2
feat(ingest): kafka connect source improvements (#3481) 2021-11-03 15:03:05 -07:00
Danny L
d8ea39b6e1
Docs: nits (#3493) 2021-11-02 22:59:54 -07:00
Kevin Hu
de41134a33
fix(ingestion): fix incorrect glue job names (#3503) 2021-11-02 22:54:47 -07:00
Andrew Maguire
8ef9330fa3
docs: fix typo in bigquery source docs (#3484) 2021-11-02 18:05:46 -07:00
Harshal Sheth
6d36b0a00c
refactor(profiler): add PerfTimer class and fix typos (#3497) 2021-11-02 18:04:52 -07:00
Gabe Lyons
4a0ed0710d
tests(smoke): introducing first isolated smoke test: updating tags & terms (#3496) 2021-11-02 12:42:53 -07:00
Gabe Lyons
758631287b
fix(lineage backend): fix ownership timestamps (#3498) 2021-11-01 18:54:48 -07:00
Gabe Lyons
cad9be6b1d
fix(ingest): fix whitespace in foreign key urns (#3486) 2021-10-28 23:14:42 -07:00
Harshal Sheth
1fec105b24
fix(ingest): correctly support multiple snowflake databases (#3482) 2021-10-28 11:44:56 -07:00
Ravindra Lanka
a1bf95307b
feat(ingest): user configurable dataset profiling. (#3453) 2021-10-27 19:49:40 -07:00
Tamas Nemeth
cdaa957014
fix(ingest): redshift source gets external table types properly (#3371) 2021-10-27 12:36:55 -07:00
aseembansal-gogo
d27ae61f15
fix(docs): fix transformer doc example (#3469) 2021-10-27 11:43:35 -07:00
John Joyce
2f03ad854b
feat(recommendations): Recommendations infra P1 (#3455) 2021-10-26 21:23:08 -07:00
Ravindra Lanka
34a5d5cf1b
fix(ingestion): workaround for Python 3.8/3.9 mypy invalid syntax issue with airflow 2.2.0 (#3460) 2021-10-26 11:25:04 -07:00
varunbharill
8d8cdc44e8
fix(ingest): Ensure mongodb source doesn't croak on large documents. (#3456) 2021-10-26 11:21:12 -07:00
AndreasTA-AW
561c04bcf8
feat(dbt-ingestion): add ability to skip specific models (#3340) 2021-10-25 22:31:19 -07:00
David Schmidt
43f1aeb538
docs: add guide for using custom sources (#3324) 2021-10-25 22:23:48 -07:00
WaStCo
bd1a5505d7
docs(transformers): add exemple of custom tag function (#3354)
Co-authored-by: STEUER COSTA <wagner.steuer-costa@lhind.dlh.de>
2021-10-25 22:23:19 -07:00
Enrico Minack
b0435fd808
test(CI): show test results on GitHub (#3362) 2021-10-25 22:21:56 -07:00