3089 Commits

Author SHA1 Message Date
Harshal Sheth
f99f73841a
feat(ingest/profiling): allow unique count queries to be combined (#10322) 2024-04-18 10:47:27 -07:00
Aseem Bansal
d3fb698d8d
fix(ingest): make gms url configuration resilient in rest emitter (#10316) 2024-04-18 14:46:32 +05:30
Andrew Sikowitz
a041a2ee52
fix(ingest/transformers): Use set to store tags in AddDatasetTags (#10317) 2024-04-18 13:11:18 +05:30
Harshal Sheth
77f1a0c60e
fix(ingest/profiling): compute sample row count correctly (#10319) 2024-04-18 08:40:11 +02:00
Harshal Sheth
4e2cec86b3
feat(ingest/sigma): fix stateful ingestion (#10321) 2024-04-17 20:09:30 -07:00
Mayuri Nehate
529710ab9d
fix(ingest/databricks): handle and report config parse failure, updat… (#10261) 2024-04-17 12:14:16 -07:00
Mayuri Nehate
0613234d2a
fix(ingest/tableau): handle very large filter queries (#10295) 2024-04-17 12:14:02 -07:00
Harshal Sheth
3cdc462a7b
fix(ingest): disallow src.* imports, fix powerbi/sigma (#10292) 2024-04-16 15:04:51 -07:00
Tamas Nemeth
d463a16b49
chore(ingest/presto-on-hive): Renaming presto-on-hive to hive-metastore source (#10278) 2024-04-16 23:35:16 +02:00
Felix Lüdin
9eb6b2d68d
fix(ingest): improve performance of get_allowed_list in AllowDenyPattern when dealing with large lists (#10219) 2024-04-16 12:48:48 -07:00
Shubham Jagtap
90c1249e7d
feat(ingest/sigma): Sigma connector integration (#10037)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-04-15 20:18:31 -07:00
dushayntAW
20e2cc7eca
fix(ingest/csv): add support multiple ownership type for the same dataset (#10287) 2024-04-15 20:15:08 +05:30
dushayntAW
f860f7907d
fix(ingest/transformer): replace externalUrl in dataset properties (#10281) 2024-04-15 20:14:42 +05:30
Mayuri Nehate
8b79461bd5
feat(ingest/looker): browse path followups (#10217) 2024-04-12 10:21:06 -07:00
jonasHanhan
223b72f0cd
fix(ingestion/lite): An index with the name aspect_idxalready exists … (#10267) 2024-04-12 09:00:45 -07:00
dushayntAW
5497393096
fix(ingest/powerbi): patch lineage for powerbi report (#10270) 2024-04-12 15:21:06 +05:30
Tamas Nemeth
8ed87d6a90
feat(ingest/mode): Mode improvements (#10273) 2024-04-12 09:01:16 +02:00
Tamas Nemeth
e19b1fef62
fix(ingest/bigquery): Adding way to change api's batch size on schema init (#10255)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-04-11 22:09:54 +02:00
Harshal Sheth
3d94388edf
feat(ingest): show custom model info (#10259) 2024-04-11 21:38:55 +02:00
sid-acryl
d546f65071
fix(ingest/mongodb): schema_metadata referenced before assignment (#10169) 2024-04-10 17:17:09 -07:00
Marcin Szymański
47bf1f9858
fix(ingestion/airflow-plugin): replace deprecated calls (#10238) 2024-04-10 10:16:44 -07:00
Harshal Sheth
f5417f6829
fix(ingest): support pydantic v2 with properties subcommand (#10256) 2024-04-09 18:40:56 -07:00
Ellie O'Neil
6bc5b167ab
feat(cli): Make yaml loaders compatible with pydantic v2 (#10257) 2024-04-09 18:40:26 -07:00
Shubham Jagtap
d4120ce3f7
feat(ingest/fivetran): use emails in owner user urns (#10229) 2024-04-09 18:40:12 -07:00
olgapenedo
5d560a8e8f
feat(ingestion/bigquery): support patterns for label -> tag capture (#10146)
Co-authored-by: Olga Penedo <psolga1@mapfre.net>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-04-09 18:37:42 -07:00
Dotan Mor
fa0c1b3fa9
feat(ingest/cockroachdb): add cockroachdb ingestion (#10226) 2024-04-09 18:36:51 -07:00
Andrew Sikowitz
bffefd5735
fix(ingest/unity): Fix bug around unity notebook ingestion (#10253) 2024-04-09 11:39:00 -07:00
Mayuri Nehate
6997abd42e
feat(ingest/nifi): ingest process group as browse path v2, incremental lineage (#10202)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-04-09 17:44:20 +05:30
Tamas Nemeth
00a890f84f
fix(ingest/bigquery): fix lineage if multiple sql expression passed in and destination table set (#10212)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-04-09 11:54:02 +02:00
dushayntAW
278a39d3df
fix(ingest/salesforce): add null check for description (#10239) 2024-04-09 15:13:39 +05:30
Gabe Lyons
45e0460050
docs(init): Update entrypoints.py to be more clear about acryl init (#10248) 2024-04-08 18:47:45 -07:00
Harshal Sheth
9c8f8a5192
feat(gql): support operationName (#10210) 2024-04-08 18:41:03 -05:00
Harshal Sheth
b3aa4d5c93
feat(ingest/redshift): filter out system queries from usage (#10247) 2024-04-08 18:37:57 -05:00
Harshal Sheth
29bf0e96c6
fix(ingest): avoid requiring sqlalchemy for dynamodb classification (#10213) 2024-04-08 15:13:25 -07:00
Harshal Sheth
deaeabff21
fix(ingest): suppress all column-level parsing errors (#10211) 2024-04-08 07:31:59 -07:00
dushayntAW
e82e8ba715
fix(ingestion/datahub): moved urn_pattern config to source config (#10215) 2024-04-08 15:54:07 +05:30
Harshal Sheth
bdf2c9a5c1
feat(ingest/sql): normalize bigquery partitioned tables when parsing (#10224) 2024-04-07 17:17:28 +02:00
Harshal Sheth
e74347812b
fix(ingest/dbt): better dbt timestamp parsing (#10223) 2024-04-05 16:13:18 -07:00
Harshal Sheth
c1b489f7ef
feat(ingest/bigquery): fix support for incremental column lineage (#10222) 2024-04-05 14:26:18 -07:00
jonasHanhan
294b6d4dae
fix(ingestion/mongodb): MongoDB source unable to parse datetimes with years > 9999 (#10110)
Co-authored-by: JonasHan <zengqh12>
2024-04-04 15:52:02 -07:00
ACHRAF BOUAOUDA
b5615fac54
feat(ingest/great_expectations): support in-memory (Pandas) data assets (#9811)
Co-authored-by: Achraf BOUAOUDA <achraf_bouada@carrefour.com>
2024-04-04 12:46:59 -07:00
Shubham Jagtap
fa139a582e
feat(ingestion/transformer): Handle overlapping while mapping in extract ownership from tags transformer (#10201) 2024-04-04 12:19:11 -07:00
dushayntAW
bad96ed824
fix(ingestion/hive): ignore sampling for tagged column/table (#10096) 2024-04-04 13:56:05 +05:30
Harshal Sheth
786c776802
feat(ingest/looker): cleanup usage generation code (#10153) 2024-04-03 14:44:38 +02:00
dushayntAW
3c7c3ec904
fix(ingestion/glue): fix to ingest the comment for partition key as description (#10189) 2024-04-03 17:34:02 +05:30
dushayntAW
8c70aa15f2
fix(ingestion/datahub): add allow/deny URN option (#10174) 2024-04-03 17:33:33 +05:30
Tamas Nemeth
5c06f7a245
fix(ingest/bigquery): Supporting lineage extraction in case the select query result's target table is set on job (#10191)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-04-02 22:13:05 +02:00
dushayntAW
2873736eac
fix(ingestion/salesforce): fixed the issue by escaping the markdown string (#10157) 2024-04-02 23:05:47 +05:30
Aseem Bansal
e0b20e159b
feat(ingest/gc): add index truncation logic (#10099) 2024-04-02 21:34:22 +05:30
david-leifker
77c4629ccf
refactor(docker): move to acryldata repo for all images (#9459) 2024-04-02 09:36:44 -05:00