This section describes how metadata is served in GMA. In particular, it demonstrates how GMA can efficiently service different types of queries, including key-value, complex queries, and full text search.
Below shows a high-level system diagram for the metadata serving architecture.
There are four types of Data Access Object ([DAO]) that standardize the way metadata is accessed.
This section describes each type of DAO, its purpose, and the interface.
These DAOs rely heavily on [Java Generics](https://docs.oracle.com/javase/tutorial/extra/generics/index.html) so that the core logics can remain type-neutral.
However, as there’s no inheritance in [Pegasus], the generics often fallback to extending [RecordTemplate] instead of the desired types (i.e. [entity], [relationship], metadata [aspect] etc). Additional runtime type checking has been added to the DAOs to avoid binding to unexpected types. We also cache the type checking result to minimize runtime overhead.
public abstract <URNextendsUrn,METADATAextendsRecordTemplate>
Optional<METADATA> get(Class<METADATA> type, URN urn, int version);
public abstract <URNextendsUrn,METADATAextendsRecordTemplate>
ListResult<Integer> listVersions(Class<METADATA> type, URN urn, int start,
int pageSize);
public abstract <METADATAextendsRecordTemplate> ListResult<Urn> listUrns(
Class<METADATA> type, int start, int pageSize);
public abstract <URNextendsUrn,METADATAextendsRecordTemplate>
ListResult<METADATA> list(Class<METADATA> type, URN urn, int start, int pageSize);
}
```
Another important function of [Local DAO] is to automatically emit [MAE]s whenever the metadata is updated.
This is doable because MAE effectively use the same [Pegasus] models so [RecordTemplate] can be easily converted into the corresponding [GenericRecord].
## Search DAO
Search DAO is also a generic class that can be bound to a specific type of search document.
The DAO provides 3 APIs:
* A `search` API that takes the search input, a [Filter], a [SortCriterion], some pagination parameters, and returns a [SearchResult].
* An `autoComplete` API which allows typeahead-style autocomplete based on the current input and a [Filter], and returns [AutocompleteResult].
* A `filter` API which allows for filtering only without a search input. It takes a a [Filter] and a [SortCriterion] as input and returns [SearchResult].
```java
public abstract class BaseSearchDAO<DOCUMENTextendsRecordTemplate> {
public abstract SearchResult<DOCUMENT> search(String input, Filter filter,
SortCriterion sortCriterion, int from, int size);
public abstract AutoCompleteResult autoComplete(String input, String field,
Filter filter, int limit);
public abstract SearchResult<DOCUMENT> filter(Filter filter, SortCriterion sortCriterion,
int from, int size);
}
```
## Query DAO
Query DAO allows clients, e.g. [GMS](../what/gms.md), [MAE Consumer Job](metadata-ingestion.md#mae-consumer-job) etc, to perform both graph & non-graph queries against the metadata graph.
For instance, a GMS can use the Query DAO to find out “all the dataset owned by the users who is part of the group `foo` and report to `bar`,” which naturally translates to a graph query.
Alternatively, a client may wish to retrieve “all the datasets that stored under /jobs/metrics”, which doesn’t involve any graph traversal.
Below is the base class for Query DAOs, which contains the `findEntities` and `findRelationships` methods.
Both methods also have two versions, one involves graph traversal, and the other doesn’t.
You can use `findMixedTypesEntities` and `findMixedTypesRelationships` for queries that return a mixture of different types of entities or relationships.
As these methods return a list of [RecordTemplate], callers will need to manually cast them back to the specific entity type using [isInstance()](https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#isInstance-java.lang.Object-) or reflection.
Note that the generics (ENTITY, RELATIONSHIP) are purposely left untyped, as these types are native to the underlying graph DB and will most likely differ from one implementation to another.
```java
public abstract class BaseQueryDAO<ENTITY,RELATIONSHIP> {
public abstract <ENTITYextendsRecordTemplate> List<ENTITY> findEntities(
Class<ENTITY> type, Filter filter, int offset, int count);
public abstract <ENTITYextendsRecordTemplate> List<ENTITY> findEntities(
Class<ENTITY> type, Statement function);
public abstract List<RecordTemplate> findMixedTypesEntities(Statement function);
public abstract <ENTITYextendsRecordTemplate,RELATIONSHIPextendsRecordTemplate> List<RELATIONSHIP>
findRelationships(Class<ENTITY> entityType, Class<RELATIONSHIP> relationshipType, Filter filter, int offset, int count);
public abstract <RELATIONSHIPextendsRecordTemplate> List<RELATIONSHIP>
