feat(docs): Add example documentation for scrollAcrossEntities (#8014)

This commit is contained in:
Pedro Silva 2023-05-11 18:54:41 +01:00 committed by GitHub
parent afcf462cb1
commit 6f491c00e9
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -178,6 +178,62 @@ for integrations and programmatic use-cases.
}
```
### Searching at Scale
For queries that return more than 10k entities we recommend using the [scrollAcrossEntities](https://datahubproject.io/docs/graphql/queries/#scrollacrossentities) GraphQL API:
```
# Example query
{
scrollAcrossEntities(input: { types: [DATASET], query: "*", count: 10}) {
nextScrollId
count
searchResults {
entity {
type
... on Dataset {
urn
type
platform {
name
}
name
}
}
}
}
}
```
This will return a response containing a `nextScrollId` value which must be used in subsequent queries to retrieve more data, i.e:
```
{
scrollAcrossEntities(input:
{ types: [DATASET], query: "*", count: 10,
scrollId: "eyJzb3J0IjpbMy4wLCJ1cm46bGk6ZGF0YXNldDoodXJuOmxpOmRhdGFQbGF0Zm9ybTpiaWdxdWVyeSxiaWdxdWVyeS1wdWJsaWMtZGF0YS5jb3ZpZDE5X2dlb3RhYl9tb2JpbGl0eV9pbXBhY3QucG9ydF90cmFmZmljLFBST0QpIl0sInBpdElkIjpudWxsLCJleHBpcmF0aW9uVGltZSI6MH0="}
) {
nextScrollId
count
searchResults {
entity {
type
... on Dataset {
urn
type
platform {
name
}
name
}
}
}
}
}
```
In order to complete scrolling through all of the results, continue to request data in batches until the `nextScrollId` returned is null or undefined.
### DataHub Blog
* [Using DataHub for Search & Discovery](https://blog.datahubproject.io/using-datahub-for-search-discovery-fa309089be22)