3089 Commits

Author SHA1 Message Date
Swaroop Jagadish
6a00cfb4a1
feat(ingest): adding snowflake app name to connection string (#3574) 2021-11-15 13:08:07 -08:00
Ravindra Lanka
87478a1658
fix(ingestion): redshift-usage should not append email domain to the username if already an email. (#3569) 2021-11-13 19:47:39 -08:00
Swaroop Jagadish
482dac6d2e
fix(ingest): fix bigquery-usage regex for both partitioned and sharded tables (#3571) 2021-11-13 19:43:59 -08:00
Courtney Lang
d33d9cbb17
fix(ingestion): bigquery add better support for partitioned and snapshot table names (#3533) 2021-11-13 15:02:19 -08:00
Kevin Hu
af366711b7
fix(ingest): Glue lineage compatibility docs and more resilience (#3555) 2021-11-13 08:15:59 -08:00
Vincenzo Lavorini
6cd31b5bdc
feat(ingest): adding openapi ingestion (#2706) 2021-11-09 21:22:15 -08:00
Noé López
cde1ce043d
feat(ingest): add transformer to add properties (#3480) 2021-11-09 20:03:21 -08:00
Harshal Sheth
3a432b7154
feat(profiler): streamline profiler (#3510) 2021-11-09 20:00:47 -08:00
Ravindra Lanka
dcb8789f17
feat(ingestion): Support for converting AVRO schemas with logical types to MCE fields. (#3546) 2021-11-09 19:59:01 -08:00
Swaroop Jagadish
94e9459586
feat(cli): add support for m1 laptops during quickstart (#3547) 2021-11-09 19:18:46 -08:00
Ravindra Lanka
c3f1bf0534
fix(ingestion): Make AVRO schema parsing robust to exceptions. (#3541) 2021-11-09 18:01:18 -08:00
aseembansal-gogo
0079101740
fix(cli): fix dry-run cli arg name to match documentation (#3536) 2021-11-09 06:42:27 -08:00
Swaroop Jagadish
3984506f51
feat(cli): add support for deletion based on filters, soft deletes an… (#3527) 2021-11-07 22:13:50 -08:00
Harshal Sheth
b0b275ee34
test(profiler): make profiling tests more comprehensive (#3513) 2021-11-07 18:53:53 -08:00
Ravindra Lanka
fcbf2977a8
fix(ingestion): Enhance logging and error-handling in bigquery usage connector. (#3521) 2021-11-07 18:38:10 -08:00
varunbharill
417b555fef
feat(dbt): dbt enhancements - dbt nodes, lineage, subtype, etc. (#3519) 2021-11-05 09:27:41 -07:00
Swaroop Jagadish
ec406c7928
feat(ingest): read-modify-write capabilities during ingestion, adding ownership patch transformer (#3506) 2021-11-03 21:39:52 -07:00
mayurinehate
192f0d33a2
feat(ingest): kafka connect source improvements (#3481) 2021-11-03 15:03:05 -07:00
Kevin Hu
de41134a33
fix(ingestion): fix incorrect glue job names (#3503) 2021-11-02 22:54:47 -07:00
Harshal Sheth
6d36b0a00c
refactor(profiler): add PerfTimer class and fix typos (#3497) 2021-11-02 18:04:52 -07:00
Gabe Lyons
4a0ed0710d
tests(smoke): introducing first isolated smoke test: updating tags & terms (#3496) 2021-11-02 12:42:53 -07:00
Gabe Lyons
758631287b
fix(lineage backend): fix ownership timestamps (#3498) 2021-11-01 18:54:48 -07:00
Gabe Lyons
cad9be6b1d
fix(ingest): fix whitespace in foreign key urns (#3486) 2021-10-28 23:14:42 -07:00
Harshal Sheth
1fec105b24
fix(ingest): correctly support multiple snowflake databases (#3482) 2021-10-28 11:44:56 -07:00
Ravindra Lanka
a1bf95307b
feat(ingest): user configurable dataset profiling. (#3453) 2021-10-27 19:49:40 -07:00
Tamas Nemeth
cdaa957014
fix(ingest): redshift source gets external table types properly (#3371) 2021-10-27 12:36:55 -07:00
varunbharill
8d8cdc44e8
fix(ingest): Ensure mongodb source doesn't croak on large documents. (#3456) 2021-10-26 11:21:12 -07:00
AndreasTA-AW
561c04bcf8
feat(dbt-ingestion): add ability to skip specific models (#3340) 2021-10-25 22:31:19 -07:00
Kevin Hu
3fe5ef7812
fix(ingest): lineage for Airflow subdags (#3351) 2021-10-25 16:34:45 -07:00
Harshal Sheth
dccef30daf
perf(ingest): lazy load ingestion plugins (#3430) 2021-10-25 14:10:24 -07:00
Remi
91d20e41c0
fix(ingest): fix lookml derived tables parsing (#3443) 2021-10-25 14:08:30 -07:00
Gabe Lyons
1fa49e2b79
feat(ingestion): support custom properties to be ingested via business glossary yaml (#3438) 2021-10-21 22:21:23 -07:00
Swaroop Jagadish
6704765d41
feat(ingest): add parallelism to looker source and datahub rest sink (#3431) 2021-10-21 11:27:27 -07:00
Harshal Sheth
a121aff6eb
refactor(ingest): resolve click-default-group deprecation warning (#3427) 2021-10-20 22:46:48 -07:00
varunbharill
328a204d44
fix(ingest): fix error when extracting lineage in bigquery is turned off. (#3428) 2021-10-20 14:42:07 -07:00
aseembansal-gogo
9dff1b7a1c
fix(ingest): allow database alias, remove extra removal from connect_… (#3352) 2021-10-19 22:29:38 -07:00
Harshal Sheth
9c09b78147
feat(ingest): support profiling tables in parallel (#3369) 2021-10-19 22:24:20 -07:00
mayurinehate
f5d37c0621
feat(ingest): add complex types support in hive and trino source (#3375) 2021-10-18 22:53:51 -07:00
Harshal Sheth
6e5d1fe42f
fix(ingest): switch to avro from deprecated avro-python3 (#3412) 2021-10-18 15:35:27 -07:00
varunbharill
e717d6a937
feat(bigquery): Ingest lineage metadata from GCP logs. (#3389) 2021-10-18 08:48:04 -07:00
Rukesh Kapuluru
aa52d98994
fix(ingest): set athena dataset name with 2 parts in redash (#3406) 2021-10-17 08:50:17 -07:00
Swaroop Jagadish
0cf157e991
fix(ingest): lookml view file resolution and looker spurious aspect issues (#3397) 2021-10-14 01:16:35 -07:00
Swaroop Jagadish
37e7141989
feat(ingest): teaching lookml source that athena has 2 parts in its names (#3393) 2021-10-13 19:33:04 -07:00
aseembansal-gogo
ebe0b3f7c1
feat(ingest): add mariadb as possible source (#3245) 2021-10-12 23:27:47 -07:00
Swaroop Jagadish
146e79b9c3
fix(ingest): fixing support for kafka key schemas when only key schemas are present (#3379) 2021-10-12 18:50:39 -07:00
John Joyce
d7a6ca5423
feat(subtypes): support visualization for "view" subtypes (#3376) 2021-10-12 15:47:29 -07:00
Tamas Nemeth
4c8105e710
fix(ingest): redshift usage properly counting sqls (#3370) 2021-10-12 15:41:54 -07:00
aseembansal-gogo
558aba9917
feat(ingest): allow disabling sample values in profiling (#3355) 2021-10-11 22:46:15 -07:00
varunbharill
3c3e31262b
fix(bigquery_usage): gracefully fail while parsing GCP log events (#3367) 2021-10-11 22:36:24 -07:00
Dexter Lee
2580f71474
fix(gitignore): move metadata-ingestion ignores to its own folder (#3365) 2021-10-11 15:30:52 -07:00