Aseem Bansal
861015d57a
fix: change for repository change ( #4464 )
2022-03-22 09:21:09 -07:00
Kevin Neville
d8e6f890a9
fix: Replace old repository link with new link ( #4446 )
2022-03-18 14:12:19 -07:00
Dexter Lee
93acefd771
fix(ci): fix model generation workflow ( #4352 )
2022-03-08 15:41:58 -08:00
Tamas Nemeth
3158e49171
fix(ingest): Removing python 3.9.9 restriction ( #4312 )
2022-03-04 08:21:33 -08:00
Kevin Hu
02fe05eb8f
feat(ingest): data-lake - remove spark requirement if not profiling ( #4131 )
2022-02-24 23:26:06 -08:00
finian
0fd4cb5f8f
feat(docker): add multiplatform docker support for arm64 (m1) ( #4221 )
2022-02-24 14:37:28 -08:00
MugdhaHardikar-GSLab
ede31c4177
feat(ci): add spark smoke test ( #4158 )
2022-02-22 19:33:21 -08:00
Swaroop Jagadish
eaf7b02b2a
docs(model): auto-generated docs and hand-written docs for the metadata model ( #4189 )
2022-02-18 09:45:45 -08:00
Swaroop Jagadish
4dba8fe6e7
fix(ci): fix formatting for action yaml ( #4122 )
2022-02-10 23:09:23 -08:00
Swaroop Jagadish
8716d7345c
fix(ci): fix fomatting in doc generation action yaml ( #4121 )
2022-02-10 23:01:24 -08:00
Swaroop Jagadish
d1a14abb53
fix(docs): fixing metadata model doc generation script and updating png ( #4120 )
2022-02-10 22:56:28 -08:00
Swaroop Jagadish
4b7f2b7d1a
fix(docker): datahub-gms - build in native, copy to target ( #3992 )
2022-01-27 09:22:41 -08:00
John Joyce
9366a47f88
fix(quickstart): Assign correct mysql-setup container for M1 and remove "head" default version. ( #3987 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-26 18:15:54 -08:00
Swaroop Jagadish
8dbde4bf0d
feat(build): remove requirement for git directory for builds ( #3977 )
2022-01-25 21:52:47 -08:00
Tamas Nemeth
cec541d827
fix(ci): ingestion - Restrict python to <=3.9.9 due to upstream looker sdk issues ( #3961 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-25 19:11:53 -08:00
Swaroop Jagadish
d23a5bb725
feat(spark-lineage): simplified jars, config, auto publish to maven ( #3924 )
2022-01-20 00:48:09 -08:00
Kevin Hu
9e30b42638
feat(ingest): data lake - support for profiling data lake assets ( #3656 )
2022-01-17 15:01:23 -08:00
Dexter Lee
6a85d451db
fix(ci): docker-ingestion version calculation on release ( #3896 )
2022-01-14 16:13:51 -08:00
Dexter Lee
f72c754dca
fix(ci): fix kafka setup workflow ( #3876 )
2022-01-11 19:36:44 -08:00
Dexter Lee
2b54475c16
feat(api): Add git-based server version to the UI and config endpoint ( #3866 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-11 15:00:57 -08:00
Swaroop Jagadish
adce0dea39
fix(datahub-client): fix shadow jar build, correct spark-lineage url parsing ( #3871 )
2022-01-11 14:55:21 -08:00
Harshal Sheth
0171735ae0
feat(build): include correct version in metadata-ingestion docker image ( #3857 )
2022-01-10 15:38:19 -08:00
Swaroop Jagadish
94fc93fe7c
feat(ci): split metadata-ingestion ci to streamline build ( #3854 )
2022-01-07 11:27:58 -08:00
Swaroop Jagadish
3e234a992c
feat(java-emitter): improvements to builder API-s, moving spark-lineage under metadata-integration/java ( #3819 )
2022-01-04 01:41:09 -08:00
mayurinehate
1d7ec8dba8
feat(ingest): add nifi source ( #3681 )
2021-12-08 14:56:31 -08:00
Tamas Nemeth
001075b3c8
[fix] create multiplatform docker build from elasticsearch-setup ( #3562 )
2021-11-12 07:17:15 -08:00
Tamas Nemeth
504a72ac8c
[fix] Build frontend docker on build platform instead on target platform ( #3552 )
2021-11-10 13:09:06 -08:00
Tamas Nemeth
b2f59e8745
fix: disable arm64 frontend build temporarily ( #3551 )
2021-11-10 10:08:13 -08:00
Tamas Nemeth
d41c5eba35
feat(docker): add multiplatform docker support for arm64 (m1) ( #3537 )
2021-11-09 11:50:06 -08:00
Enrico Minack
e7376cd33d
fix(CI): stabilize gradle build ( #3413 )
2021-10-26 16:07:25 -07:00
Ravindra Lanka
34a5d5cf1b
fix(ingestion): workaround for Python 3.8/3.9 mypy invalid syntax issue with airflow 2.2.0 ( #3460 )
2021-10-26 11:25:04 -07:00
Enrico Minack
b0435fd808
test(CI): show test results on GitHub ( #3362 )
2021-10-25 22:21:56 -07:00
Gabe Lyons
aeba0e9afe
Revert "revert "test(ElasticSearch): Retry ES requests" ( #3385 )" ( #3392 )
...
This reverts commit b16b090f2bf78e0a751490310b95db3dfc142bbf.
2021-10-13 18:53:47 -07:00
Gabe Lyons
b16b090f2b
revert "test(ElasticSearch): Retry ES requests" ( #3385 )
2021-10-13 14:37:51 -07:00
Swaroop Jagadish
146e79b9c3
fix(ingest): fixing support for kafka key schemas when only key schemas are present ( #3379 )
2021-10-12 18:50:39 -07:00
Enrico Minack
104b331a2e
test(ci): timeout build job ( #3364 )
2021-10-11 22:50:27 -07:00
Dexter Lee
4c038d7cfe
fix(git-ignore): Git ignore generated python and avro artifacts ( #3320 )
2021-10-06 11:54:30 -07:00
Enrico Minack
25e1bb1b5a
ci: upload test reports ( #3201 )
2021-09-07 09:30:13 -07:00
John Joyce
2c5edd88ab
feat(graphql): migrating GraphQL API to metadata-service (nee GMS) ( #3131 )
2021-08-20 10:58:07 -07:00
Dexter Lee
801e39bc40
feat(workflow): Add mysql/postgres setup workflow ( #3090 )
2021-08-12 15:56:48 -07:00
Harshal Sheth
328b098d01
text(ingestion): test multiple python versions in CI ( #2952 )
2021-07-26 13:25:58 -07:00
Harshal Sheth
d7248fd08d
build(ingest): separate metadata-ingestion build workflow fully ( #2862 )
2021-07-13 09:13:13 -07:00
Harshal Sheth
1b4788537d
feat(ci): separate metadata-ingestion into a separate workflow ( #2828 )
2021-07-02 20:26:00 -07:00
John Joyce
7591c8994a
feat(datahub cli): DataHub CLI Quickstart ( #2689 )
2021-06-14 17:15:24 -07:00
Kevin Hu
ebdaa0e359
feat(ingest): Feast ingestion integration ( #2605 )
...
* Add feast testing setup
* Init Feast test script
* Add feast to dependencies
* Update feast descriptors
* Sort integrations
* Working feast pytest
* Clean up feast docker-compose file
* Expand Feast tests
* Setup feast classes
* Add continuous and bytes data to feature types
* Update field type mapping
* Add PDLs
* Add MLFeatureSetUrn.java
* Comment out feast setup
* Add snapshot file and update inits
* Init Feast golden files generation
* Clean up Feast ingest
* Feast testing comments
* Yield Feature snapshots
* Fix Feature URN naming
* Update feast MCE
* Update Feature URN prefix
* Add MLEntity
* Update golden files with entities
* Specify feast sources
* Add feast source configs
* Working feast docker ingestion
* List entities and features before adding tables
* Add featureset names
* Remove unused
* Rename feast image
* Update README
* Add env to feast URNs
* Fix URN naming
* Remove redundant URN names
* Fix enum backcompatibility
* Move feast testing to docker
* Move URN generators to mce_builder
* Add source for features
* Switch TypeClass -> enum_type
* Rename source -> sourceDataset
* Add local Feast ingest image builds
* Rename Entity -> MLPrimaryKey
* Restore features and keys for each featureset
* Do not json encode source configs
* Remove old source properties from feature sets
* Regenerate golden file
* Fix race condition with Feast tests
* Exclude unknown source
* Update feature datatype enum
* Update README and fix typos
* Fix Entity typo
* Fix path to local docker image
* Specify feast config and version
* Fix feast env variables
* PR fixes
* Refactor feast ingest constants
* Make feature sources optional for back-compatibility
* Remove unused GCP files
* adding docker publish workflow
* Simplify name+namespace in PrimaryKeys
* adding docker publish workflow
* debug
* final attempt
* final final attempt
* final final final commit
* Switch to published ingestion image
* Update name and namespace in java files
* Rename FeatureSet -> FeatureTable
* Regenerate codegen
* Fix initial generation errors
* Update snapshot jsons
* Regenerated schemas
* Fix URN formats
* Revise builds
* Clean up feast URN builders
* Fix naming typos
* Fix Feature Set -> Feature Table
* Fix comments
* PR fixes
* All you need is Urn
* Regenerate snapshots and update validation
* Add UNKNOWN data type
* URNs for source types
* Add note on docker requirement
* Fix typo
* Reorder aspect unions
* Refactor feast ingest functions
* Update snapshot jsons
* Rebuild
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-06-09 15:07:04 -07:00
Harshal Sheth
2f7b4f9def
build(docker): test docker builds in pull request CI ( #2661 )
2021-06-08 09:47:32 -07:00
John Joyce
97e9660037
feat: No Code Metadata Modeling ( #2629 )
...
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-06-03 13:24:33 -07:00
Dexter Lee
9bff4e1cc4
fix(ci): update trigger to always generate docker images ( #2636 )
2021-06-03 11:02:59 -07:00
Harshal Sheth
a8ced6525a
ci(docker): disable GitHub Docker registry ( #2547 )
2021-05-12 17:53:20 -07:00
Harshal Sheth
d0ca3191c9
build(ingest): add metadata-ingestion to gradle build ( #2510 )
2021-05-06 22:10:49 -07:00
