29 Commits

Author SHA1 Message Date
david-leifker
16ef1ac174
feat(validation): enable validation trim options (#12712) 2025-02-24 06:30:45 -06:00
david-leifker
7326bb9470
feat(urn-validation): Add UrnValidation PDL annotation (#12572) 2025-02-13 05:47:25 -06:00
david-leifker
412600a163
feat(telemetry): cross-component async write tracing (#12405) 2025-01-29 11:30:44 -06:00
pankajmahato-visa
f6c86df1f5
chore(deps): Migrate EOL vulnerability of javax.mail to jakarta.mail (#12282) 2025-01-09 13:15:09 -06:00
Chakru
85b42e3ea5
build(coverage): enable code coverage for java and python (#11992)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-12-02 19:27:43 -06:00
david-leifker
264278861b
feat(build): support jdk8 spark lineage (#9697) 2024-01-23 17:49:23 -06:00
david-leifker
824df5a6a3
feat(build): gradle 8, jdk17, neo4j 5 (#9458) 2023-12-15 13:28:33 -06:00
RyanHolstien
1b737243b2
feat(avro): upgrade avro to 1.11 (#9031) 2023-10-18 13:45:46 -05:00
david-leifker
21eb4dfc12
feat(search): update to support OpenSearch 2.x (#8852) 2023-09-21 13:01:55 -05:00
david-leifker
7dd6e09ac5
refactor(build): upgrade to gradle 7 & guava update (#8745) 2023-09-01 19:36:01 +05:30
Aseem Bansal
0f8f186bd0
chore(logs): clean logs scanning location (#7261) 2023-02-07 16:41:39 +05:30
Chris Collins
2ef2ad05d0
fix(ingestion) Inject pipeline_name into recipes at runtime (#6833) 2022-12-22 13:16:51 -08:00
david-leifker
2de9d3d5bf
fix(logging): Remove lombok as source of slf4j-api, convert to compileOnly where possible (#6616) 2022-12-04 19:57:47 -08:00
david-leifker
4ca3327d89
fix(security): update ranger commons & dependencies for security vulns (#6577)
* fix(security): update ranger commons & dependencies for security vulns
2022-11-30 17:05:01 -06:00
John Joyce
da74943485
feat(ui): Edit glossary term descriptions via UI (#4156) 2022-02-16 15:01:22 -08:00
RyanHolstien
34c27f076b
feat(removeGMA): remove all dependencies on gma libraries (#3835) 2022-01-05 17:32:31 -08:00
xiphl
8cd1e91072
Upgrade to 3rd Apache patch for log4j (#3772) 2021-12-20 06:55:22 -08:00
John Joyce
5b5135be0b
fix(vuln): log4j vulnerability - bumping to 2.16.0 (#3755) 2021-12-15 11:07:45 -08:00
Fredrik Sannholm
d651040c85
Fix vulnderability (#3716) 2021-12-10 10:07:55 -08:00
Dexter Lee
8747fbe43c
feat(perf): Add perf testing and monitoring framework (#3195) 2021-09-07 23:06:15 -07:00
John Joyce
ccb09a6966
feat(access control): Fine-Grained Access Control M1 (#3182) 2021-09-02 19:05:13 -07:00
Dexter Lee
15c0c4dfb3
fix(browse): Fix browse pagination and multi-browse path issue (#2984) 2021-07-30 10:48:32 -07:00
John Joyce
97e9660037
feat: No Code Metadata Modeling (#2629)
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-06-03 13:24:33 -07:00
John Plaisted
25b663cc18
refactor: move code to linkedin/datahub-gma. (#1955)
Move code to linkedin/datahub-gma.

"GMA" (Generalized Metadata Architecture) is the backend of DataHub, and has been moved to its own repository.

This deletes the code that was moved and uses jars that GMA publishes to bintray to load it.

Note that not all of GMA was moved, but most of it. We may still move more things to the other repository in the future.
2020-10-23 15:14:57 -07:00
Kerem Sahin
1168501083 Enable tests for all modules by using global gradle config 2020-02-21 11:53:45 -08:00
Kerem Sahin
b17b91f24a Bump gradle to 5.6.4 and pegasus to 27.7.18 2020-02-12 17:10:49 -08:00
Kerem Sahin
7e2e6b5c93 Enable datahub-mae-consumer job to build graph as well 2019-11-26 22:19:46 -08:00
Kerem Sahin
5bf797b216 corp-identity-gms 1.0.0 -> 1.0.25:
1.0.24: Corp user search across teams and skills
    1.0.21: Make /corpGroups /gridUsers /gridGroups extend BaseEntityResource
    1.0.17: Use correct util function to load resource file
    1.0.16: Add ingest, backfill & getSnapshot action methods to all top-level resources in corp-identity-gms
    1.0.13: Onboard search query templates on corp-identity-gms
     1.0.9: Fix batch get and add client for batch get
     1.0.8: Change package name for corpuser search config
     1.0.7: Use search config to get autocomplete field
     1.0.6: Implement searchable client
     1.0.5: Auto-complete backend support
     1.0.3: Add search API

metadata-models 24.0.0 -> 38.1.6:
    38.1.3: Index active status to corp user search index
    38.1.2: Change update response to create response for create API
   38.0.10: Mark BaseAspectResource as deprecated
    38.0.9: Allow TYPEREF items which have primitive types for arrays in models
    38.0.7: Add get-and-set-if-absent function to Local DAO
    38.0.6: Add find entities with one relationship in query dao
    38.0.4: Fix the inconsistency use of constants and urn params in query dao
    38.0.2: Parse source map to obtain the urn
    38.0.1: Search document validator in Index Builder
    38.0.0: Add urns to search result metadata
    37.0.7: Refactor the query dao
    37.0.6: Use test models in neo4j dao
    37.0.5: Drop metadata model structural assumptions made in neo4j DAOs
    37.0.2: Return empty list from getBrowsePaths if browsePaths field doesn't exist
    36.0.3: Drop elasticsearch-dao's metadata-models dependency
   35.0.10: ES Search DAO to handle null values
    35.0.5: Ebean local Dao query string match
    35.0.4: Drop all search & browse configs that have been moved to individual GMS
    35.0.3: Add ReportTo relationship model
    35.0.0: Load resource file properly
    34.0.9: Make RestliAuditor injectable
    34.0.8: Use encoded query in the test resource
    34.0.4: Handle empty aspects param correctly for backfill & getSnapshot actions
    34.0.1: Remove corp user specific files from metadata-models
    34.0.0: Add backfill & getSnapshot actions to BaseEntityResource
   32.0.16: Merge data template classes into metadata-dao's main artifact
   32.0.14: Replace "update" method with "ingest" action in BaseEntityResource
   32.0.13: Make fliter & sortCriteria parameters optional as they should have been
   32.0.12: Move AspectVersion & SnapshotKey back to their original namespaces
   32.0.11: Break metadata-dao's dependency on metadata-models
   32.0.10: Move model validators to a separate module
    32.0.9: Extract principal from the request context for user AuditStamp
    32.0.8: Fixing nullability annotations for search/autocomplete/browse resources & daos
    32.0.7: Move DAO-specific models to metadata-dao module
    32.0.4: Fix search finder not returning total search results count
    32.0.3: Implement get_all using search index
    32.0.2: Add missing nullability annotation
    32.0.1: Use more consistent naming for the test models
    31.0.1: Use test-specific metadata models in metadata-dao
    31.0.0: Add sort order to Search Dao
    30.0.2: Rename Aspect test model to AspectUnion to avoid confusion
    30.0.1: Committing migration for metadata-models.
    30.0.0: Add default autocomplete field in search config
   29.0.16: Modify testcase to account for empty filters
   29.0.15: Add searchable interface that clients can use
   29.0.14: Use test-specific metadata models in ebean-dao
   29.0.12: Move TestUtils to metadata-test-models module
   29.0.11: Refactor all tests in metadata-restli to use test models
   29.0.10: Move li-metadata-test-utils, metadata-test-models, metadata-test-utils into a new metadata-testing directory to improve code organization.
            Note that this is a backward compatible change as this doesn't alter the produced artifacts.
    29.0.9: Move test-specific models to a stand-alone module
    29.0.2: Refactor the rest of validators
    28.0.3: Refactor validateSchema for aspect
    28.0.2: Implement searchDao for CorpUserInfo.
   27.0.16: Refactor for ModelValidation tests
   27.0.10: Add new relationship union to model utils
    27.0.9: Add plugin to rest client factory
    27.0.6: Add rest high level factory
    27.0.5: Fix a test bug when reviewing the code
    27.0.4: Add create via lambda API to BaseVersionedAspectResource
    27.0.2: Change return type of search finder to capture search result metadata in BaseSearchableEntityResource
    27.0.1: Drop the unnecessary get method from BaseEntityResource
    27.0.0: Add BaseBrowsableEntityResource
   26.0.15: Add autocomplete action to BaseSearchableEntityResource
   26.0.14: Add BaseSearchableEntityResource
   26.0.13: Add getUrnFromDocument & urnClassForDocument util methods that are needed in future RBs
   26.0.11: Add BaseVersionedAspectResource
    26.0.9: Index signals associated with dataset relevance
    26.0.4: Support namespace for ID generation
    26.0.1: Fix inconsistent instance variable naming in SearchResult
    25.0.6: Add entity-snapshot conversion
    25.0.5: Use test-specific metadata models in metadata-restli
    25.0.3: Add aspect filtering to BaseEntityResource
    24.0.9: Add update method to BaseEntityResource
    24.0.7: Fix for parameter types of getBrowsePaths action method

MP_VERSION=corp-identity-gms:1.0.25
MP_VERSION=metadata-models:38.1.6
MP_VERSION=wherehows-samza:1.0.29
2019-10-02 11:13:44 -07:00
Kerem Sahin
23339df23a Initial commit for Data Hub 2019-08-31 20:51:14 -07:00