Harshal Sheth
2eb9fe408a
docs(): generate docs for our Python SDK ( #7612 )
2023-03-28 20:23:20 -07:00
Mayuri Nehate
fc238c2513
feat(ingest/postgres): support extracting metadata from all databases in single recipe ( #7581 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-03-28 14:16:12 -07:00
Harshal Sheth
b2689b7514
test(ingest/dbt): add test for column meta match ( #7673 )
2023-03-28 21:51:31 +05:30
Andrew Sikowitz
c7d35ffd66
perf(ingest): Improve FileBackedDict iteration performance; minor refactoring ( #7689 )
...
- Adds dirty bit to cache, only writes data if dirty
- Refactors __iter__
- Adds sql_query_iterator
- Adds items_snapshot, more performant `items()` that allows for filtering
- Renames connection -> shared_connection
- Removes unnecessary flush during close if connection is not shared
- Adds Closeable mixin
2023-03-27 17:20:34 -04:00
Mayuri Nehate
279f38a2fd
fix(ingest/bigquery): quote string constants in query ( #7694 )
2023-03-27 09:32:20 -07:00
Aseem Bansal
a62889b39a
chore(lint): fix lint in looker ( #7695 )
2023-03-27 14:08:51 +02:00
Harshal Sheth
d1bab5616c
feat(ingest/looker): enable looker usage ingestion by default ( #7684 )
2023-03-27 00:02:25 +00:00
Harshal Sheth
6d04511949
fix(ingest/looker): correct looker/lookml capability reports ( #7683 )
2023-03-26 23:36:12 +00:00
Andrew Sikowitz
419bee8614
fix(ingest/bigquery): Fix BigQueryTableType enum accesses ( #7685 )
2023-03-25 00:08:11 +00:00
Harshal Sheth
c8abf9a1d4
fix(ingest/dbt): enable incremental lineage by default ( #7674 )
2023-03-24 18:14:19 -04:00
Harshal Sheth
d71463041f
fix(ingest/looker): skip empty user ids for usage ( #7686 )
2023-03-24 21:40:11 +00:00
Mayuri Nehate
301c8616ed
refactor(ingest/bigquery): add inline comments + refactor in table name parsing ( #7609 )
2023-03-24 14:44:30 -04:00
Aseem Bansal
1324231e70
chore(ci): add coverage code for python ( #7681 )
2023-03-24 07:10:45 +00:00
Hyejin Yoon
918718e7d0
feat(docs-website): add docs on creating users and groups ( #7574 )
...
Co-authored-by: Hyejin Yoon <yoonhyejin@Hyejins-MacBook-Pro.local>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-03-24 11:52:12 +09:00
Harshal Sheth
39bc8a3e93
fix(cli): allow usage without kafka ( #7677 )
2023-03-24 02:36:56 +00:00
Hyejin Yoon
864ac2da9f
docs: add guides on deleting entities ( #7636 )
...
Co-authored-by: Hyejin Yoon <yoonhyejin@Hyejins-MacBook-Pro.local>
Co-authored-by: Hyejin Yoon <yoonhyejin@ip-192-168-0-10.us-west-2.compute.internal>
2023-03-24 08:49:34 +09:00
Andrew Sikowitz
589d354a57
perf(ingest): Increase default rest sink parallelism ( #7675 )
2023-03-23 19:43:20 -04:00
Shirshanka Das
3d81539c7e
fix(ingest): json-schema - nullability handling ( #7667 )
2023-03-23 23:07:30 +00:00
Mayuri Nehate
4ec34fc73b
feat(ingest/bigquery): emit create operation event for view ( #7656 )
2023-03-23 11:04:41 -07:00
Harshal Sheth
cf40aecb84
fix(cli/delete): include aspect name in dry-run message ( #7664 )
2023-03-22 22:25:33 -07:00
Harshal Sheth
3b519924e8
fix(cli): protect against timeseries get_aspects ( #7665 )
2023-03-22 22:04:11 -07:00
Shirshanka Das
f872ca9cba
feat(cli): add user and group commands to create users and groups ( #7595 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-22 18:44:01 -07:00
rtol5
1938b843cf
fix(ingest/mode): prevent null last_saved_at values from breaking runs ( #7630 )
2023-03-22 16:12:12 -07:00
Harshal Sheth
d9231addd2
feat(docs-website): add vercel preview environment ( #7644 )
2023-03-22 12:28:52 -07:00
Andrew Sikowitz
95f99198af
fix(ingest/bigquery): Pass whether view is materialized; pass last_altered correctly ( #7660 )
2023-03-22 13:40:57 -04:00
Harshal Sheth
2722ad54d4
feat(cli): build and upload Python wheels in CI ( #7537 )
2023-03-22 21:31:32 +05:30
mohdsiddique
6d6d59141e
feat(ingestion): powerbi # uniquly identify the multiple instance of same platform ( #7632 )
...
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: John Joyce <john@acryl.io>
2023-03-21 09:27:29 -07:00
mohdsiddique
7efac2215d
feat(ingestion): powerbi # support platform instance ( #7583 )
...
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: John Joyce <john@acryl.io>
2023-03-21 09:07:31 -07:00
david-leifker
697e8e2647
fix(misc): misc fixes ( #7633 )
2023-03-21 19:42:50 +05:30
Harshal Sheth
482431bcf4
fix(ingest/superset): support superset v2 ( #7588 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-03-20 19:49:32 -07:00
Harshal Sheth
d54ff061a4
fix(ingest): remove get_platform_instance_id from stateful ingestion ( #7572 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-03-20 17:35:10 -07:00
Harshal Sheth
cbd8e14b78
feat(ingest): add auto_materialize_referenced_tags helper ( #7626 )
2023-03-20 16:34:22 -07:00
alex-magno
6ab606b748
fix(ingest/dbt): introduce lowercase column urn option ( #7418 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-20 10:37:19 -07:00
mohdsiddique
fc8757d25e
feat(ingestion): powerbi # Amazon Redshift lineage support ( #7562 )
...
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-20 10:24:34 -07:00
Shirshanka Das
104c9811f5
fix(ingest/docs): improve matcher to include types with spaces in them ( #7631 )
2023-03-18 12:59:43 -07:00
Harshal Sheth
14ecba1e62
fix(docker): fix gradle quickstart version parsing ( #7614 )
2023-03-17 12:10:05 -04:00
Shirshanka Das
41d4c0b074
feat(ingest/docs): json-schema fixes, improvements to ingestion doc generation ( #7615 )
2023-03-17 15:58:14 +01:00
Aseem Bansal
c43d7dd9d2
chore(ci): fix flakiness, misc improvements ( #7605 )
2023-03-17 09:34:00 +05:30
Hyejin Yoon
38555385e1
docs: add new code examples on creating entities & fix minor typos ( #7613 )
...
Co-authored-by: Hyejin Yoon <yoonhyejin@Hyejins-MacBook-Pro.local>
Co-authored-by: Hyejin Yoon <yoonhyejin@ip-192-168-0-10.us-west-2.compute.internal>
2023-03-17 10:07:24 +09:00
Harshal Sheth
89734587f7
feat(ingest): add urn modification helper ( #7440 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-03-16 13:27:08 -07:00
Andrew Sikowitz
8c1fa04c87
fix(ingest/snowflake): Allow SnowflakeObjectAccessEntry.objectId to be None ( #7601 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-03-16 12:55:52 +01:00
Harshal Sheth
bcda510656
feat(cli): show image pull progress in quickstart ( #7593 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-03-16 12:28:51 +05:30
mohdsiddique
0a36826407
fix(ingest/powerbi): support each expression in m-query function invocation ( #7541 )
...
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-15 09:06:29 -07:00
Aseem Bansal
b2ecd7f968
chore(ci): update dependencies, fix smoke image build ( #7580 )
2023-03-15 21:07:30 +05:30
Harshal Sheth
b87e4c312a
fix(ingest): pin typeguard version for feast ( #7591 )
2023-03-15 13:27:20 +05:30
Harsha Mandadi
bf36c935fa
feat(ingest/s3): support path_specs of different S3 buckets in the same recipe ( #7514 )
2023-03-14 21:55:57 -07:00
Andrew Sikowitz
8dd7a85533
refactor(ingest): Use shared connection wrapper over connection cache ( #7570 )
2023-03-14 15:09:37 -07:00
Aseem Bansal
776806d348
fix(gms): convert obj to string, fix wrong setup ( #7582 )
2023-03-15 03:02:30 +05:30
Mayuri Nehate
70c9b20c6d
feat(ingest/kafka-connect): add stateful ingestion and platform instance support ( #7526 )
2023-03-14 13:59:54 -07:00
Harshal Sheth
cb6c3dc8dd
feat(ingest/snowflake): use auto_workunit_reporter helper ( #7568 )
2023-03-14 13:54:57 -07:00
