unstructured/test_unstructured_ingest/clean-permissions-files.sh
Ahmet Melek 94836cfad4
feat: add file-based access permissions for SharePoint ingest (#1628)
This PR:

- defines rbac_data as a SourceMetadata field,
- manages connections to an external api for obtaining rbac data with
ConnectorRBAC class,
- serializes rbac data and saves it to the disk,
- matches the rbac_data in the disk to each IngestDoc, using a common
field,
- forwards rbac data to Elements, via the partition() function

To test the changes, run `examples/ingest/sharepoint/ingest.sh` with the
relevant rbac & connector credentials

---------

Co-authored-by: ryannikolaidis <1208590+ryannikolaidis@users.noreply.github.com>
Co-authored-by: ahmetmeleq <ahmetmeleq@users.noreply.github.com>
2023-10-13 00:38:08 +00:00

28 lines
641 B
Bash
Executable File

#!/usr/bin/env bash
# Description: Delete (cleanup) permissions files in a folder, so that they are not included in
# text diff tests.
#
# Arguments:
# - $1: Name of the folder to do the cleanup operation in.
set +e
if [ "$#" -ne 1 ]; then
echo "Please provide a folder to clean the files in: $0 <folder_path>"
exit 1
fi
folder_path="$1"
if [ ! -d "$folder_path" ]; then
echo "'$folder_path' is not a directory. Please provide a folder / directory."
exit 1
fi
for file in "$folder_path"/*_SEP_*; do
if [ -e "$file" ]; then
rm "$file"
fi
done
echo "Completed cleanup for permissions files"