Quick readme updates

metadata-ingestion/README.md

@@ -1,7 +1,9 @@
 # Dev
-## Set up dev environment
+## Dependencies
 - On MacOS: `brew install librdkafka`
 - On Debian/Ubuntu: `sudo apt install librdkafka-dev`
+
+## Set up dev environment
 - python3 -m venv venv
 - source venv/bin/activate
 - pip install -e .
@@ -13,8 +15,32 @@
 # Sanity check code before checkin (currently broken)
 - flake8 src test && mypy -p gometa && black --check -l 120 src test && isort --check-only src test && pytest
 
-# Run recipe
-- ./recipes/kafka_to_console.sh
+# Recipes
+
+A recipe is a configuration that tells our ingestion scripts where to pull data from (source) and where to put it (sink).
+Here's a simple example that pulls metadata from MSSQL and puts it into datahub.
+
+```yaml
+source:
+  type: mssql
+  mssql:
+    username: sa
+    password: test!Password
+    database: DemoData
+
+sink:
+  type: "datahub-rest"
+  datahub-rest:
+    server: 'http://localhost:8080'
+```
+
+Running a recipe is quite easy.
+
+```bash
+gometa-ingest -c ./recipes/kafka_to_datahub_rest.yaml
+```
+
+A number of recipes are included in the recipes directory.
 
 # Using Docker
 ## Build the image

metadata-ingestion/recipes/kafka_to_console.sh

@@ -1,2 +0,0 @@
-#gometa-ingest -c recipes/kafka_to_console.yaml
-gometa-ingest -c recipes/kafka_to_console.toml

metadata-ingestion/src/gometa/ingestion/source/ldap.py

@@ -150,7 +150,7 @@ class LDAPSource(Source):
                 sys.stdout.write('LDAP search failed: %s' % e)
                 continue
 
-                        pctrls = get_pctrls(serverctrls)
+        pctrls = get_pctrls(serverctrls)
         if not pctrls:
                 print >> sys.stderr, 'Warning: Server ignores RFC 2696 control.'
                 break

metadata-ingestion/src/gometa/metadata/schema_classes.py

@@ -1,4 +1,3 @@
-from __future__ import annotations
 import json
 import os.path
 import decimal