Harshal Sheth
66985335d0
fix(ingest): add trailing newline to ssh keys ( #5947 )
2022-09-14 20:53:26 -07:00
Shirshanka Das
14873775bd
feat(docs): lookml - add video for ui ingestion, remove secret step until we support ssh-keys as secrets ( #5948 )
2022-09-14 20:16:49 -07:00
Harshal Sheth
9384a52b23
feat(ingest): skip ssh known_hosts verification for git clone ( #5945 )
2022-09-14 16:43:21 -07:00
Ravindra Lanka
cf2f59f85f
fix(ingest): Fix snowflake usage stateful ingestion backward compatibility check. ( #5943 )
2022-09-14 13:25:50 -07:00
skrydal
f61a040555
feat(ingestion) Add more info to glue entities ( #5874 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-14 12:25:09 -07:00
Harshal Sheth
e23523a781
fix(ingest): fix type annotations on some pydantic fields ( #5795 )
2022-09-14 11:05:31 -07:00
Mayuri Nehate
8ead62dae5
fix(transformer): fix invalid lastModified.actor entry in transformers ( #5906 )
2022-09-14 09:59:17 -07:00
Harshal Sheth
a1e1d2fd0a
feat(ingest): add ConfigEnum type ( #5734 )
2022-09-14 09:57:42 -07:00
Shirshanka Das
c5f8fc1907
fix(docs): lookml - clean up links ( #5938 )
2022-09-14 09:43:59 -07:00
Ravindra Lanka
ee68f09624
feat(ingestion): Refactor standard state-handling tasks into a common handler that are common across all stateful ingestion sources. ( #5766 )
2022-09-14 09:30:42 -07:00
Harshal Sheth
8f5b773025
fix(airflow): prioritize conn_type over id-based type inference ( #5911 )
2022-09-14 18:30:03 +02:00
firas omrane
c606abdb40
fix(docker) Add platform to docker-compose command ( #5683 )
2022-09-14 17:56:42 +05:30
Harshal Sheth
d5d8a38caa
fix(ingest): move git requirement into lookml deps ( #5932 )
2022-09-14 14:47:07 +05:30
Harshal Sheth
68ea926770
fix(ingest): add dbt redshift type mappings ( #5933 )
2022-09-14 14:46:21 +05:30
Harshal Sheth
e3f8f29a31
feat(ingest): support version option in datahub client get_aspect_v2 ( #5934 )
2022-09-14 14:45:01 +05:30
Harshal Sheth
a8ac5f9e33
refactor(ingest): simplify DefaultSQLParser alias ( #5935 )
2022-09-14 14:44:22 +05:30
Shirshanka Das
15a33fadc8
feat(ingest): lookml - support for git checkout ( #5924 )
2022-09-13 16:40:59 -07:00
Danilo Peixoto
d1cfdf1894
fix(docs): feast recipe documentation ( #5917 )
2022-09-13 10:46:47 -07:00
Chris Collins
0be5c39802
feat(ingestion-ui) Display ingestion sources in UI more dynamically ( #5789 )
2022-09-13 13:27:12 -04:00
Harshal Sheth
dfeced8eee
fix(ingest): hide internal profiler.allow_deny_patterns from config ( #5619 )
2022-09-13 16:09:10 +05:30
Harshal Sheth
45979edd12
fix(ingest): handle lookml directory being in a symlink'd folder ( #5916 )
2022-09-13 13:42:44 +05:30
Harshal Sheth
b6d3fe88e5
fix(ingest): lookml - support relative paths in base_folder ( #5914 )
2022-09-12 13:59:48 -07:00
Harry Nguyen
46bc1fe367
feat(ingestion): schema inference for jsonlines in S3 ( #5725 )
...
Co-authored-by: Nguyen Tri Hieu <nhieu@fossil.com>
2022-09-12 10:49:46 -07:00
Mayuri Nehate
aedf1522fb
feat(ingest): snowflake-beta - minor changes, tests ( #5910 )
2022-09-12 10:42:52 -07:00
Ravindra Lanka
386719f15a
feat(platform): timeseries - Server & Client side changes to support timeseries aspect deletion & rollback. ( #4756 )
2022-09-11 11:27:46 -07:00
Harshal Sheth
e556bcb306
feat(ingest): add entity type inference to mcpw ( #5880 )
2022-09-10 20:36:10 -07:00
Harshal Sheth
220ae0b6c9
feat(ingest): make sink use type annotations ( #5899 )
2022-09-10 19:46:20 -07:00
Harshal Sheth
8f72de5d8b
refactor(ingest): move aspect maps to dedicated file ( #5821 )
2022-09-10 18:43:11 -07:00
Harshal Sheth
3cd8ea8316
test(ingest): use pytest parameterization for dbt integration tests ( #5879 )
2022-09-10 18:35:58 -07:00
Harshal Sheth
954397a078
fix(ingest): remove dbt disable_dbt_node_creation and load_schema options ( #5877 )
2022-09-09 14:07:55 -07:00
Marcin Szymański
57996049fb
fix(ingest): sql - add missing trino types ( #5870 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-09 14:07:00 -07:00
Shirshanka Das
5074035c4d
feat(ingest): looker - improve defaults for usage extraction ( #5893 )
...
Co-authored-by: Ravindra Lanka <rslanka@gmail.com>
2022-09-09 13:35:49 -07:00
Shirshanka Das
056add128d
fix(ingest): datahub-api - move instantiation to the right config class ( #5878 )
2022-09-09 13:34:21 -07:00
Pedro Silva
e7b33c595b
feat(cli): make docker compose quiet ( #5869 )
2022-09-08 14:55:03 -07:00
Harshal Sheth
da46dcdc22
refactor(ingest): add ALL_ENV_TYPES constant ( #5866 )
2022-09-08 14:07:22 -07:00
Harshal Sheth
6063484714
fix(ingest): avrogen handling for missing fields with default values ( #5844 )
2022-09-08 14:05:28 -07:00
Harshal Sheth
2115d5bf1d
fix(ingest): remove dbt delete_tests_as_datasets option ( #5865 )
...
This option was added for backwards compat and can be dropped now.
2022-09-08 11:22:25 -07:00
Harshal Sheth
39fa9321c1
feat(ingest): support reading config file from stdin ( #5847 )
2022-09-08 11:04:20 -07:00
Harshal Sheth
eb57bebf48
fix(ingest): include urns in rest sink failure logs ( #5848 )
2022-09-08 10:29:40 -07:00
Shirshanka Das
b9068ffd2e
fix(m1): preflight works with brew postgres@14 ( #5868 )
2022-09-08 15:02:30 +05:30
Shirshanka Das
d911b15da5
docs(ingest): improve doc gen, docs for snowflake, looker ( #5867 )
2022-09-08 00:38:10 -07:00
Harshal Sheth
6f09f1025e
ci(ingest): test with python 3.10 ( #5863 )
2022-09-07 17:08:49 -07:00
Harshal Sheth
1e8a1038d8
chore(ingest): remove outdated GE compatibility hack ( #5862 )
2022-09-07 17:06:02 -07:00
Tamas Nemeth
f9917ea5fb
Fixing sharded regexp pattern config ( #5861 )
2022-09-07 15:36:37 -07:00
Shirshanka Das
2fea3d26df
fix(ingest): reporting should work with timestamps ( #5860 )
2022-09-07 21:09:08 +02:00
Harshal Sheth
abddc01877
fix(ingest): fix doc generation import ordering issue with postgres ( #5846 )
...
Relying on the correct import directly, rather than going through
SQLAlchemy's import wrapper (in their dialect.py) allows us to bypass
this strange error in doc generation.
2022-09-07 21:44:33 +05:30
Harshal Sheth
08622f25ef
feat(ingest): add utility for converting MCEs to MCPs ( #5812 )
2022-09-06 15:25:48 -07:00
mohdsiddique
2f65e2f226
feat(transformers): Add semantics & transform_aspect support in transformers ( #5514 )
...
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-06 14:44:14 -07:00
Pedro Silva
20138a32e5
fix(standalone-consumers): Exclude Solr from spring boot application config & make them run on M1 ( #5827 )
...
* fix(standalone-consumers) Removes Solr from spring boot application config
* Adds standalone consumer option to datahub quickstart cli
* Rename files
* Make dockerize platform agnostic & change docker compose utility to work with M1
* Fix MAE/MCE dockerfiles for arm & make smoke tests use standalone consumers
* Fix identation
* Use master code
* Adds ARM image publishing for consumers
* Fix linter
* Fix lint
2022-09-06 13:55:31 -05:00
Shirshanka Das
f4eea5dfd5
fix(ingest): profiling - memory usage reduction ( #5830 )
2022-09-05 23:37:19 -07:00
