yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-07-31 05:18:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
11,753 Commits 205 Branches 123 Tags
Commit Graph

11753 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Hyejin Yoon
92ad7202c9
feat: add pre-commit (#7680) 
Co-authored-by: Hyejin Yoon <yoonhyejin@Hyejins-MacBook-Pro.local>
 2023-03-31 15:34:21 +09:00
xiphl
7d240c600a
feat(ingestion) Allow for ingestion to read files remotely (#7552) 
Co-authored-by: xiphl <xiphlerl9@gmail.com>
Allows the CsvEnricher, BusinessGlossary, File, and LineageFile sources to read from URLs.
 2023-03-29 18:10:46 -07:00
Harshal Sheth
575909e41c
feat(docs): support inlining code snippets from files (#7712) 2023-03-30 00:02:21 +00:00
Sergio Gómez Villamor
25808478cb
fix(ingestion): fix AssertionError in base_transformer (#7702) 
Co-authored-by: Sergio Gomez Villamor <sergio.gomez.villamor@adevinta.com>
 2023-03-29 16:15:57 -07:00
sz
d7ea1629af
feat(jwt authenticator): JWT authenticator with asymmetric PublicKey verification for JWT token. (#6495) 
Co-authored-by: syed.javed <syed.javed@precisely.com>
 2023-03-29 14:38:07 -07:00
Hyejin Yoon
11ce32ac34
docs(): add concept section (#7655) 
Co-authored-by: Hyejin Yoon <yoonhyejin@Hyejins-MacBook-Pro.local>
 2023-03-29 14:34:48 -07:00
John Joyce
cbfe887609
fix(docs): Fixing timeseries delete doc until code path is fixed (#7711) 2023-03-29 14:33:17 -07:00
Andrew Sikowitz
54a372795b
test(ingest/bigquery): Add performance testing framework for bigquery usage (#7690) 
- Creates metadata-ingestion/tests/performance directory
- Excludes metadata-ingestion/tests from docs generation
- Updates bigquery reporting around project state
 2023-03-29 14:13:43 -07:00
Harshal Sheth
94fa62d431
chore(ingest): formatting + cleanup MCPW usages (#7706) 2023-03-29 11:43:25 -07:00
mohdsiddique
c0f7ba2f85
feat(ingestion): azure-ad stateful ingestion (#7701) 2023-03-29 21:50:31 +05:30
Tamas Nemeth
f348113b38
fix(ingest/redshift): Lineage query fix to work with the latest redshift (#7698) 2023-03-29 09:32:09 +02:00
Harshal Sheth
2eb9fe408a
docs(): generate docs for our Python SDK (#7612) 2023-03-28 20:23:20 -07:00
Mayuri Nehate
fc238c2513
feat(ingest/postgres): support extracting metadata from all databases in single recipe (#7581) 
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
 2023-03-28 14:16:12 -07:00
Harshal Sheth
b2689b7514
test(ingest/dbt): add test for column meta match (#7673) 2023-03-28 21:51:31 +05:30
Harshal Sheth
7cf3d378a8
feat(quickstart): move quickstart back to master (#7697) 2023-03-28 15:10:57 +02:00
Andrew Sikowitz
c7d35ffd66
perf(ingest): Improve FileBackedDict iteration performance; minor refactoring (#7689) 
- Adds dirty bit to cache, only writes data if dirty
- Refactors __iter__
- Adds sql_query_iterator
- Adds items_snapshot, more performant `items()` that allows for filtering
- Renames connection -> shared_connection
- Removes unnecessary flush during close if connection is not shared
- Adds Closeable mixin
 2023-03-27 17:20:34 -04:00
Chris Collins
04f1b86d54
fix(ui) Update Looker form to set client id as Secret (#7479) 2023-03-27 15:09:40 -04:00
Chris Collins
ba6426ef61
feat(ui) Update auto-complete functionality and design (#7515) 
Adds in new styles for auto-complete as well as the new functionality around quick filters.
 2023-03-27 12:45:38 -04:00
Mayuri Nehate
279f38a2fd
fix(ingest/bigquery): quote string constants in query (#7694) 2023-03-27 09:32:20 -07:00
Aseem Bansal
a62889b39a
chore(lint): fix lint in looker (#7695) 2023-03-27 14:08:51 +02:00
Aseem Bansal
ed2afb10a4
doc(freshness): add faq for dataset freshness (#7693) 2023-03-27 15:42:23 +05:30
Harshal Sheth
d1bab5616c
feat(ingest/looker): enable looker usage ingestion by default (#7684) 2023-03-27 00:02:25 +00:00
Harshal Sheth
6d04511949
fix(ingest/looker): correct looker/lookml capability reports (#7683) 2023-03-26 23:36:12 +00:00
Andrew Sikowitz
419bee8614
fix(ingest/bigquery): Fix BigQueryTableType enum accesses (#7685) 2023-03-25 00:08:11 +00:00
Harshal Sheth
c8abf9a1d4
fix(ingest/dbt): enable incremental lineage by default (#7674) 2023-03-24 18:14:19 -04:00
Harshal Sheth
d71463041f
fix(ingest/looker): skip empty user ids for usage (#7686) 2023-03-24 21:40:11 +00:00
Mayuri Nehate
301c8616ed
refactor(ingest/bigquery): add inline comments + refactor in table name parsing (#7609) 2023-03-24 14:44:30 -04:00
Aseem Bansal
c5059a3d53
doc(release): managed datahub v0.2.4 release notes (#7679) 2023-03-24 13:16:01 +05:30
Aseem Bansal
1324231e70
chore(ci): add coverage code for python (#7681) 2023-03-24 07:10:45 +00:00
Hyejin Yoon
918718e7d0
feat(docs-website): add docs on creating users and groups (#7574) 
Co-authored-by: Hyejin Yoon <yoonhyejin@Hyejins-MacBook-Pro.local>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
 2023-03-24 11:52:12 +09:00
Indy Prentice
2099c42213
test(elasticsearch): Add unit test for timestamp-based lineage feature (#7661) 
Co-authored-by: Indy Prentice <indy@Indys-MacBook-Pro.local>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
 2023-03-23 19:41:38 -07:00
Harshal Sheth
39bc8a3e93
fix(cli): allow usage without kafka (#7677) 2023-03-24 02:36:56 +00:00
Hyejin Yoon
864ac2da9f
docs: add guides on deleting entities (#7636) 
Co-authored-by: Hyejin Yoon <yoonhyejin@Hyejins-MacBook-Pro.local>
Co-authored-by: Hyejin Yoon <yoonhyejin@ip-192-168-0-10.us-west-2.compute.internal>
v0.10.1 		2023-03-24 08:49:34 +09:00
Andrew Sikowitz
589d354a57
perf(ingest): Increase default rest sink parallelism (#7675) 2023-03-23 19:43:20 -04:00
Shirshanka Das
3d81539c7e
fix(ingest): json-schema - nullability handling (#7667) 2023-03-23 23:07:30 +00:00
david-leifker
90ecdb5417
fix(lineage): add cache key truncation for search lineage key (#7671) 2023-03-23 17:52:16 -05:00
Mayuri Nehate
4ec34fc73b
feat(ingest/bigquery): emit create operation event for view (#7656) 2023-03-23 11:04:41 -07:00
Aditya Radhakrishnan
660d477457
fix(lineage): change default lineage time window to All Time (#7662) 2023-03-23 10:47:47 -07:00
Harshal Sheth
cf40aecb84
fix(cli/delete): include aspect name in dry-run message (#7664) 2023-03-22 22:25:33 -07:00
Harshal Sheth
3b519924e8
fix(cli): protect against timeseries get_aspects (#7665) 2023-03-22 22:04:11 -07:00
Shirshanka Das
f872ca9cba
feat(cli): add user and group commands to create users and groups (#7595) 
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
 2023-03-22 18:44:01 -07:00
rtol5
1938b843cf
fix(ingest/mode): prevent null last_saved_at values from breaking runs (#7630) 2023-03-22 16:12:12 -07:00
Harshal Sheth
d9231addd2
feat(docs-website): add vercel preview environment (#7644) 2023-03-22 12:28:52 -07:00
Andrew Sikowitz
95f99198af
fix(ingest/bigquery): Pass whether view is materialized; pass last_altered correctly (#7660) 2023-03-22 13:40:57 -04:00
Harshal Sheth
2722ad54d4
feat(cli): build and upload Python wheels in CI (#7537) 2023-03-22 21:31:32 +05:30
John Joyce
7fe0171754
refactor(lineage): Refactor getAndUpdatePaths inside of ESGraphQueryDao (#7556) 
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
 2023-03-21 21:50:14 -05:00
John Joyce
4da49fb1b4
refactor(ui): Make Navigating DataHub UI easier, fix duplicate tracking, duplicate networks calls, + misc optimizations (#7592) 
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
 2023-03-21 18:36:21 -07:00
John Joyce
bd76183b00
fix(ui): Address regression in column usage stats + add unit test (#7645) 
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
 2023-03-21 18:27:58 -07:00
John Joyce
ce33db74fd
refactor(ui): Loading schema dynamically for dataset profile (#7558) 
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
 2023-03-21 18:27:31 -07:00
david-leifker
1b15dd484f
lint(test): remove unused imports, other test fixes (#7659) 2023-03-21 16:53:04 -05:00