33 Commits

Author SHA1 Message Date
Harshal Sheth
1c7836dce8
fix(cli): avoid click 8.2.0 due to bugs (#13518) 2025-05-14 23:26:18 -07:00
Chakru
cf40116680
CI speedup (#13057) 2025-04-07 10:13:07 +05:30
david-leifker
fa4ff7bced
feat(openapi-ingestion): implement openapi ingestion (#12757) 2025-03-11 08:25:53 -05:00
Aseem Bansal
436b74cd3a
dev: move from flake8,isort to ruff (#12375) 2025-01-17 23:50:13 +05:30
david-leifker
4296373c5f
config(rest-api): enable authentication and api authorization by default (#11484) 2024-09-27 11:31:25 -05:00
Harshal Sheth
f816a14a98
fix(ingest): fix graph config loading (#11002)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2024-07-26 11:15:46 -07:00
Pedro Silva
dd732d0d46
feat(cli): Make consistent use of DataHubGraphClientConfig (#10466)
Deprecates get_url_and_token() in favor of a more complete option: load_graph_config() that returns a full DatahubClientConfig.
This change was then propagated across previous usages of get_url_and_token so that connections to DataHub server from the client respect the full breadth of configuration specified by DatahubClientConfig.

I.e: You can now specify disable_ssl_verification: true in your ~/.datahubenv file so that all cli functions to the server work when ssl certification is disabled.

Fixes #9705
2024-07-25 19:06:14 +00:00
Aseem Bansal
3f9490d901
feat(cli): option to init via username password, add lint for smoke-test (#9675)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-01-31 14:42:40 +05:30
Aseem Bansal
bb39d5418f
fix(lint): run black, isort (#8978) 2023-10-10 16:08:34 +05:30
Shirshanka Das
8e6e4e8962
refactor(ci): move from sleep to kafka lag based testing (#8094) 2023-06-06 14:44:52 -07:00
Harshal Sheth
afd65e16fb
feat(cli): delete cli v2 (#8068) 2023-05-23 14:43:44 -05:00
Shirshanka Das
b3c790aab6
feat: Add support for Data Products (#8039)
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
2023-05-17 07:17:25 +00:00
david-leifker
cd05f5b174
feat(schema-registry): replace confluent schema registry (#7930)
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Ryan Holstien <ryan@acryl.io>
2023-05-01 13:18:41 -05:00
Aditya Radhakrishnan
04a32d487f
tests(cypress): add improved Cypress tests for timeline lineage (#7464) 2023-03-02 15:02:06 -08:00
Aseem Bansal
cb7f1c6dc3
feat(tests): allow use of system auth for test session (#7445) 2023-02-27 22:41:06 +05:30
Aseem Bansal
c12f3491a2
refactor(tests): extract common code (#7441) 2023-02-27 19:06:16 +05:30
david-leifker
c845866521
test(misc): misc test updates (#6890) 2022-12-29 11:26:42 -06:00
Aseem Bansal
1d0bb1cd8a
refactor(smoke): use env variables (#6866) 2022-12-29 17:16:43 +05:30
Aseem Bansal
1bec1d87bd
fix(ci): reduce smoke test run time (#6841) 2022-12-27 00:08:01 +05:30
Aditya Radhakrishnan
f0f0355a83
feat(onboarding): adds framework and some steps for onboarding steps UI (#6462)
* feat(onboarding): adds models and API for onboarding steps feature

* feat(onboarding): adds backend for onboarding steps feature

* feat(onboarding): adds framework and some steps for onboarding steps UI
2022-12-07 16:21:55 -08:00
david-leifker
a2dc229c62
fix(tests): Misc updates for tests, auth log level, and quickstart (#6491) 2022-11-29 08:44:55 -08:00
david-leifker
33fd876421
feat(elasticsearch): Updates to elasticsearch configuration, dao, and tests (#6269) 2022-11-15 18:03:11 -08:00
firas omrane
c606abdb40
fix(docker) Add platform to docker-compose command (#5683) 2022-09-14 17:56:42 +05:30
Ravindra Lanka
386719f15a
feat(platform): timeseries - Server & Client side changes to support timeseries aspect deletion & rollback. (#4756) 2022-09-11 11:27:46 -07:00
Aseem Bansal
39c919407f
fix(ci): add env variable for creds smoke test (#5609) 2022-08-10 17:04:28 +05:30
Aseem Bansal
d20071ad91
fix(ci): smoke test less flaky, add src, dev dep in smoke image (#5594) 2022-08-09 19:07:12 +05:30
Aseem Bansal
edc31f453a
feat(test): add read-only smoke tests (#5558) 2022-08-05 16:35:23 -07:00
Pedro Silva
6e07ec5924
fix(tests): Removes duplicate policies tests & makes DataHub user configurable (#5365) 2022-07-15 19:14:19 +05:30
Navin Sharma
ee7c5f75a2
Add Changes to support smoke test on Datahub deployed on kubernetes Cluster (#5334)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2022-07-14 22:04:06 +05:30
Aseem Bansal
c95dc926a3
fix(test): add cleanup in tests, make urls configurable (#5287) 2022-06-30 16:00:50 +05:30
Pedro Silva
e8f6c4cabd
feat(cli) Changes rollback behaviour to apply soft deletes by default (#4358)
* Changes rollback behaviour to apply soft deletes by default

Summary:
Addresses feature request: Flag in delete command to only delete aspects touched by an ingestion run; add flag to nuke everything by modifying the default behaviour of a rollback operation which will not by default delete an entity if a keyAspect is being rolled-back.

Instead the key aspect is kept and a StatusAspect is upserted with removed=true, effectively making a soft delete.
Another PR will follow to perform garbage collection on these soft deleted entities.

To keep old behaviour, a new parameter to the cli ingest rollback endpoint: --hard-delete was added.

* Adds restli specs

* Fixes deleteAspect endpoint & adds support for nested transactions

* Enable regression test & fix docker-compose for local development

* Add generated quickstart

* Fix quickstart generation script

* Adds missing var env to docker-compose-without-neo4j

* Sets status removed=true when ingesting resources

* Adds soft deletes for ElasticSearch + soft delete flags across ingestion sub-commands

* Makes elastic search consistent

* Update tests with new behaviour

* apply review comments

* apply review comment

* Forces Elastic search to add documents with status removed false when ingesting

* Reset gradle properties to default

* Fix tests
2022-03-15 12:05:52 -07:00
Ravindra Lanka
bd69e736ba
feat(Stateful Ingestion-2/3): Client side changes for checkpointing a source job state. (#3763)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-12-16 20:06:33 -08:00
Gabe Lyons
4a0ed0710d
tests(smoke): introducing first isolated smoke test: updating tags & terms (#3496) 2021-11-02 12:42:53 -07:00