Harshal Sheth
e06117af66
fix(ingest/demo-data): fix bug in path type ( #7749 )
2023-04-04 23:16:15 -07:00
Mayuri Nehate
20504aae70
fix(ingest/bigquery): fix and refractor exported audit logs query ( #7699 )
2023-04-05 11:17:25 +05:30
Thosan Girisona
062e8ae323
fix(TestResults UI):show non-failing TestResult ( #7747 )
2023-04-05 11:16:44 +05:30
Harshal Sheth
e71c0d3490
feat(sdk): fix ownership emission for groups ( #7751 )
2023-04-05 11:15:06 +05:30
Aseem Bansal
a11a7fa9d0
feat(snowflake): better error message on key pair authentication ( #7734 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-04-05 00:46:07 +00:00
Harshal Sheth
8d99babf75
feat(ingest/dbt): update subtypes for dbt ( #7750 )
2023-04-04 17:11:23 -07:00
Indy Prentice
ae9326c0db
docs(search): Add example search for finding tables without the name field ( #7647 )
...
Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
2023-04-04 17:16:27 -05:00
mohdsiddique
6a1abc832e
doc(ingestion): tableau quick ingestion guide ( #7682 )
...
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: John Joyce <john@acryl.io>
2023-04-04 13:12:42 -07:00
Andrew Sikowitz
06bc1c32e0
refactor(ingest/bigquery): Standardize audit log parsing and make TopKDict a DefaultDict ( #7738 )
...
- Moves get_sanitized_table_ref calls to ReadEvent / QueryEvent creation
- Standardizes how the audit log is read and parsed, unifying code when reading from gcp logging vs audit metadata (exported logs)
- Adds error handling around the parsing of each event, to catch errors from the new get_sanitized_table_ref calls
- Makes TopKDict inherit from DefaultDict and cleans up calls around that.
2023-04-04 11:58:48 -07:00
Andrew Sikowitz
ce1ac7fa12
refactor(ingest): Use sqlite.Row row_factory for FileBackedCollections ( #7739 )
2023-04-04 11:53:56 -07:00
Tim
23e57fffa2
fix(sdk): remove rest emitter to graph cache in CorpGroup ( #7743 )
2023-04-04 10:32:15 -07:00
Harshal Sheth
f860ce95c0
feat(ingest): emit state payloads as soft-deleted ( #7714 )
2023-04-04 17:06:21 +00:00
Harshal Sheth
82dc2b6393
feat(docs): clear up source configs ( #7720 )
2023-04-04 18:40:19 +05:30
Harshal Sheth
8394dcb538
chore(ingest): change kafka connect mapped ports ( #7728 )
2023-04-04 18:38:30 +05:30
Aseem Bansal
0e04e0c80f
docs(airflow): update with information for new plugin ( #7732 )
2023-04-04 15:20:02 +05:30
Harshal Sheth
1634edaf25
feat(ingest/dbt): include dbt unique_id in properties ( #7737 )
2023-04-04 13:37:13 +05:30
John Joyce
3609167086
fix(ci): Fixing broken Domains Test ( #7746 )
2023-04-04 13:36:34 +05:30
Harshal Sheth
f780da4c0a
feat(ingest/lookml): support views with derived_table.explore_source ( #7704 )
2023-04-03 16:18:39 -07:00
Andrew Sikowitz
de587b2bfe
refactor(ingest): Minor cleanup of File, CsvEnricher, BusinessGlossary, and FileLineage sources ( #7718 )
...
- Adds auto_workunit_reporter to each source
- Standardizes comments around remote paths
- Adds back AuditStamp to FileLineage source
- Some generic refactoring
2023-03-31 15:49:24 -07:00
Tamas Nemeth
86960adf98
doc(redshift) - Adding Redshift ingestion quickstart guide ( #7700 )
2023-03-31 12:57:51 -07:00
Andrew Sikowitz
a2f8c76388
feat(ingest/bigquery): Capture all operation types when ingesting operational stats ( #7723 )
2023-03-31 16:01:28 +05:30
Harshal Sheth
f6d7e1a325
feat(ingest/snowflake): hide host_port from snowflake docs ( #7717 )
2023-03-31 15:58:52 +05:30
Aseem Bansal
f0a675f9aa
docs(okta): add how to use email in urns ( #7708 )
2023-03-31 15:55:22 +05:30
Hyejin Yoon
92ad7202c9
feat: add pre-commit ( #7680 )
...
Co-authored-by: Hyejin Yoon <yoonhyejin@Hyejins-MacBook-Pro.local>
2023-03-31 15:34:21 +09:00
xiphl
7d240c600a
feat(ingestion) Allow for ingestion to read files remotely ( #7552 )
...
Co-authored-by: xiphl <xiphlerl9@gmail.com>
Allows the CsvEnricher, BusinessGlossary, File, and LineageFile sources to read from URLs.
2023-03-29 18:10:46 -07:00
Harshal Sheth
575909e41c
feat(docs): support inlining code snippets from files ( #7712 )
2023-03-30 00:02:21 +00:00
Sergio Gómez Villamor
25808478cb
fix(ingestion): fix AssertionError in base_transformer ( #7702 )
...
Co-authored-by: Sergio Gomez Villamor <sergio.gomez.villamor@adevinta.com>
2023-03-29 16:15:57 -07:00
sz
d7ea1629af
feat(jwt authenticator): JWT authenticator with asymmetric PublicKey verification for JWT token. ( #6495 )
...
Co-authored-by: syed.javed <syed.javed@precisely.com>
2023-03-29 14:38:07 -07:00
Hyejin Yoon
11ce32ac34
docs(): add concept section ( #7655 )
...
Co-authored-by: Hyejin Yoon <yoonhyejin@Hyejins-MacBook-Pro.local>
2023-03-29 14:34:48 -07:00
John Joyce
cbfe887609
fix(docs): Fixing timeseries delete doc until code path is fixed ( #7711 )
2023-03-29 14:33:17 -07:00
Andrew Sikowitz
54a372795b
test(ingest/bigquery): Add performance testing framework for bigquery usage ( #7690 )
...
- Creates metadata-ingestion/tests/performance directory
- Excludes metadata-ingestion/tests from docs generation
- Updates bigquery reporting around project state
2023-03-29 14:13:43 -07:00
Harshal Sheth
94fa62d431
chore(ingest): formatting + cleanup MCPW usages ( #7706 )
2023-03-29 11:43:25 -07:00
mohdsiddique
c0f7ba2f85
feat(ingestion): azure-ad stateful ingestion ( #7701 )
2023-03-29 21:50:31 +05:30
Tamas Nemeth
f348113b38
fix(ingest/redshift): Lineage query fix to work with the latest redshift ( #7698 )
2023-03-29 09:32:09 +02:00
Harshal Sheth
2eb9fe408a
docs(): generate docs for our Python SDK ( #7612 )
2023-03-28 20:23:20 -07:00
Mayuri Nehate
fc238c2513
feat(ingest/postgres): support extracting metadata from all databases in single recipe ( #7581 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-03-28 14:16:12 -07:00
Harshal Sheth
b2689b7514
test(ingest/dbt): add test for column meta match ( #7673 )
2023-03-28 21:51:31 +05:30
Harshal Sheth
7cf3d378a8
feat(quickstart): move quickstart back to master ( #7697 )
2023-03-28 15:10:57 +02:00
Andrew Sikowitz
c7d35ffd66
perf(ingest): Improve FileBackedDict iteration performance; minor refactoring ( #7689 )
...
- Adds dirty bit to cache, only writes data if dirty
- Refactors __iter__
- Adds sql_query_iterator
- Adds items_snapshot, more performant `items()` that allows for filtering
- Renames connection -> shared_connection
- Removes unnecessary flush during close if connection is not shared
- Adds Closeable mixin
2023-03-27 17:20:34 -04:00
Chris Collins
04f1b86d54
fix(ui) Update Looker form to set client id as Secret ( #7479 )
2023-03-27 15:09:40 -04:00
Chris Collins
ba6426ef61
feat(ui) Update auto-complete functionality and design ( #7515 )
...
Adds in new styles for auto-complete as well as the new functionality around quick filters.
2023-03-27 12:45:38 -04:00
Mayuri Nehate
279f38a2fd
fix(ingest/bigquery): quote string constants in query ( #7694 )
2023-03-27 09:32:20 -07:00
Aseem Bansal
a62889b39a
chore(lint): fix lint in looker ( #7695 )
2023-03-27 14:08:51 +02:00
Aseem Bansal
ed2afb10a4
doc(freshness): add faq for dataset freshness ( #7693 )
2023-03-27 15:42:23 +05:30
Harshal Sheth
d1bab5616c
feat(ingest/looker): enable looker usage ingestion by default ( #7684 )
2023-03-27 00:02:25 +00:00
Harshal Sheth
6d04511949
fix(ingest/looker): correct looker/lookml capability reports ( #7683 )
2023-03-26 23:36:12 +00:00
Andrew Sikowitz
419bee8614
fix(ingest/bigquery): Fix BigQueryTableType enum accesses ( #7685 )
2023-03-25 00:08:11 +00:00
Harshal Sheth
c8abf9a1d4
fix(ingest/dbt): enable incremental lineage by default ( #7674 )
2023-03-24 18:14:19 -04:00
Harshal Sheth
d71463041f
fix(ingest/looker): skip empty user ids for usage ( #7686 )
2023-03-24 21:40:11 +00:00
Mayuri Nehate
301c8616ed
refactor(ingest/bigquery): add inline comments + refactor in table name parsing ( #7609 )
2023-03-24 14:44:30 -04:00
