yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-09-23 08:03:17 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(azure_ad): make redirect and graph_url optional parameters and update docs (#4754)

Browse Source
This commit is contained in:
Aditya Radhakrishnan 2022-04-27 15:19:49 -07:00 committed by GitHub
parent 2a52632a2f
commit a7d76e43b5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
4 changed files with 16 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
metadata-ingestion/source_docs/azure-ad.md
View File

@ -78,6 +78,10 @@ to read your organization's Users and Groups. The following permissions are requ
- `GroupMember.Read.All` - `GroupMember.Read.All`
- `User.Read.All` - `User.Read.All`
You can add a permission by navigating to the permissions tab in your DataHub application on the Azure AD portal. ![Azure AD API Permissions](./images/azure_ad_api_permissions.png)
You can view the necessary endpoints to configure by clicking on the Endpoints button in the Overview tab. ![Azure AD Endpoints](./images/azure_ad_endpoints.png)
You can use the following recipe to get started with Azure ingestion! See [below](#config-details) for full configuration options. You can use the following recipe to get started with Azure ingestion! See [below](#config-details) for full configuration options.
```yml ```yml
@ -88,9 +92,10 @@ source:
client_id: "00000000-0000-0000-0000-000000000000" client_id: "00000000-0000-0000-0000-000000000000"
tenant_id: "00000000-0000-0000-0000-000000000000" tenant_id: "00000000-0000-0000-0000-000000000000"
client_secret: "xxxxx" client_secret: "xxxxx"
redirect: "https://login.microsoftonline.com/common/oauth2/nativeclient"
authority: "https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000" authority: "https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000"
token_url: "https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/token" token_url: "https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/token"
# All of the below parameters are optional.
redirect: "https://login.microsoftonline.com/common/oauth2/nativeclient"
graph_url: "https://graph.microsoft.com/v1.0" graph_url: "https://graph.microsoft.com/v1.0"
ingest_users: True ingest_users: True
ingest_groups: True ingest_groups: True
@ -117,10 +122,10 @@ Note that a `.` is used to denote nested fields in the YAML configuration block.
| `client_id` | string | ✅ | | Application ID. Found in your app registration on Azure AD Portal | | `client_id` | string | ✅ | | Application ID. Found in your app registration on Azure AD Portal |
| `tenant_id` | string | ✅ | | Directory ID. Found in your app registration on Azure AD Portal | | `tenant_id` | string | ✅ | | Directory ID. Found in your app registration on Azure AD Portal |
| `client_secret` | string | ✅ | | Client secret. Found in your app registration on Azure AD Portal | | `client_secret` | string | ✅ | | Client secret. Found in your app registration on Azure AD Portal |
| `redirect` | string | ✅ | | Redirect URI. Found in your app registration on Azure AD Portal |
| `authority` | string | ✅ | | The [authority](https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-client-application-configuration) is a URL that indicates a directory that MSAL can request tokens from. | | `authority` | string | ✅ | | The [authority](https://docs.microsoft.com/en-us/azure/active-directory/develop/msal-client-application-configuration) is a URL that indicates a directory that MSAL can request tokens from. |
| `token_url` | string | ✅ | | The token URL that acquires a token from Azure AD for authorizing requests. This source will only work with v1.0 endpoint. | | `token_url` | string | ✅ | | The token URL that acquires a token from Azure AD for authorizing requests. This source will only work with v1.0 endpoint. |
| `graph_url` | string | ✅ | | [Microsoft Graph API endpoint](https://docs.microsoft.com/en-us/graph/use-the-api) | | `redirect` | string | | | Redirect URI. Found in your app registration on Azure AD Portal. Defaults to https://login.microsoftonline.com/common/oauth2/nativeclient. |
| `graph_url` | string | | | [Microsoft Graph API endpoint](https://docs.microsoft.com/en-us/graph/use-the-api). Defaults to https://graph.microsoft.com/v1.0. |
| `ingest_users` | bool | | `True` | Whether users should be ingested into DataHub. | | `ingest_users` | bool | | `True` | Whether users should be ingested into DataHub. |
| `ingest_groups` | bool | | `True` | Whether groups should be ingested into DataHub. | | `ingest_groups` | bool | | `True` | Whether groups should be ingested into DataHub. |
| `ingest_group_membership` | bool | | `True` | Whether group membership should be ingested into DataHub. ingest_groups must be True if this is True. | | `ingest_group_membership` | bool | | `True` | Whether group membership should be ingested into DataHub. ingest_groups must be True if this is True. |

BIN
metadata-ingestion/source_docs/images/azure_ad_api_permissions.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 297 KiB

BIN
metadata-ingestion/source_docs/images/azure_ad_endpoints.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 88 KiB

10
metadata-ingestion/src/datahub/ingestion/source/identity/azure_ad.py
View File

@ -36,10 +36,12 @@ class AzureADConfig(ConfigModel):
client_id: str client_id: str
tenant_id: str tenant_id: str
client_secret: str client_secret: str
redirect: str
authority: str authority: str
token_url: str token_url: str
graph_url: str
# Optional: URLs for redirect and hitting the Graph API
redirect: str = "https://login.microsoftonline.com/common/oauth2/nativeclient"
graph_url: str = "https://graph.microsoft.com/v1.0"
# Optional: Customize the mapping to DataHub Username from an attribute in the REST API response # Optional: Customize the mapping to DataHub Username from an attribute in the REST API response
# Reference: https://docs.microsoft.com/en-us/graph/api/user-list?view=graph-rest-1.0&tabs=http#response-1 # Reference: https://docs.microsoft.com/en-us/graph/api/user-list?view=graph-rest-1.0&tabs=http#response-1
@ -228,7 +230,9 @@ class AzureADSource(Source):
) )
else: else:
# Unless told otherwise, we only care about users and groups. Silently skip other object types. # Unless told otherwise, we only care about users and groups. Silently skip other object types.
pass logger.warning(
f"Unsupported @odata.type '{odata_type}' found in Azure group member. Skipping...."
)
def _add_user_to_group_membership( def _add_user_to_group_membership(
self, self,