31 Commits

Author SHA1 Message Date
Teddy
77b94f9ebb
fix: rename tests endpoint to dataQuality/<specificity> (#10970) 2023-04-14 00:14:49 -07:00
Teddy
9b4e9132ae
fixed #9656 - Add support for date type to column values to be between (#10890)
* fix: renamed  to  submodule

* fix: linting

* fix: columnValuesToBeBetween test for date column type
2023-04-04 17:16:44 +02:00
Suresh Srinivas
c8b640674b
10041 part2 - Refactor and cleanup APIs (#10900)
* Use @Tag annotation to group APIs in the swagger documentation.

* Hide internal APIs

* Change API path events/subscription to events/subscriptions

* Change API path from automations/workflow to automations/workflows

* Change API path v1/testCase to v1/testCases

* Change API path v1/testDefinition to v1/testDefinitions

* Change API path v1/testSuite to v1/testSuites

* Rename Kpi and kpi in the documentation to KPI

* Change API path v1/testConnectionDefinition to v1/testConnectionDefinitions

* Update API section in the API documentation

* Fix test failures

* Correctly capitalize Test Cases and Test Suites in API docs
2023-04-03 13:03:48 -07:00
Mayur Singal
69729ac8eb
Fix #10320: Add Athena Usage & Lineage (#10879) 2023-04-03 11:21:55 +00:00
Onkar Ravgan
5d6e18dc28
Fix 10642: Mark delete entities and tags toggle (#10695)
* Added mark delete logic

* Final test and optimization

* After merge fixes

* Added include tags for dash pipelines dbt

* added docs and fixed test

* Fixed py tests

* Added UI changes for following newly added fields:
- markDeletedDashboards
- markDeletedMlModels
- markDeletedPipelines
- markDeletedTopics
- includeTags

* Fixed failing unit tests

* updated json files of localization for other languages

* Improved localization changes

* added localization changes for other languages

* Updated mark deleted desc

* updated the ingestion fields descriptions in the ingestion form for UI

* automated localization changes for other languages

* updated descriptions for includeTags field for dbtPipeline and databaseServiceMetadataPipeline json

* fixed issue where includeTags field was being sent in the dbtConfigSource

* Added flow to input taxonomy while adding BigQuery service.

---------

Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
2023-03-29 12:41:44 +05:30
Teddy
b2e1eed842
doc: added best practice section to profiler (#10712)
* doc: added best practice section to profiler

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

* Update openmetadata-docs/content/connectors/ingestion/workflows/profiler/index.md

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>

---------

Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-03-22 19:36:42 +01:00
Onkar Ravgan
aeec285901
Added s3 and gcs examples for dbt (#10639) 2023-03-17 12:19:34 +05:30
Pere Miquel Brull
47585e035a
Update MWAA docs (#10588) 2023-03-15 09:22:04 +01:00
Nahuel
f2e1a87b5a
Fix#10377: service connection not overwritten as expected (#10445) 2023-03-06 16:32:10 +01:00
NiharDoshi99
1ff76f5e65
pii tagging using spacy (#10256)
* WIP: pii tagging using spacy

* added test cases and changes as per comment

* fix python checkstyle

* fix python checkstyle

* added score, test_cases and docs update

* solved merge conflict

* fix python checkstyle

* remove pii tagging using regex

* fix python test

* lib changes and added some test case

* changed as per comment

* fix: python test

* fix: changes to get source_config

* fix: changes as per comment
2023-03-03 18:33:18 +05:30
Teddy
b91611b33c
doc(DQ): Added summary of failure/success behavior for each test (#10106) 2023-02-03 15:34:51 +00:00
Teddy
3bd613e51a
doc: added more details regarding freshness metrics (#10098) 2023-02-03 15:03:28 +01:00
Teddy
0a77f8a3f2
Fixes issue #10082 -- Unsupported field types for bigquery (#10091)
* fix(profile): Raise nicer error for BQ unsupported interval type for timestamp

* fix(profiler): run checkstyle
2023-02-02 19:00:56 +01:00
Pere Miquel Brull
23ba96a271
Docs - Upgrade and breaking changes for 0.13.2 (#9882)
* Prep breaking changes docs

* Prep breaking changes docs

* highlights

* Superset Changes

* Metabase and Domo

* Add missing task_id param

* Update upgrade docs for 0.13.2

* Version alignments

* What's new

* doc: Added breaking change around entityName and entityLink

* Update openmetadata-docs/content/overview/roadmap/index.md

---------

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
Co-authored-by: Teddy Crepineau <teddy.crepineau@gmail.com>
2023-02-01 14:41:38 +01:00
Onkar Ravgan
949989fb1c
Added dbt parser (#9982)
* Added dbt parser

* Added library dependency

* format and final fixes

* Addressed review comments

* Fixed typo
2023-01-29 20:47:39 +01:00
Onkar Ravgan
579d61275d
Added dbt lineage and troubleshooting docs (#9892) 2023-01-24 18:26:42 +05:30
Akash Jain
0d74cf7029
Fix/docs-misc (#9666)
* use latest version 0.13.1 in examples

* fix docs for #9654

* fix for #9607
2023-01-19 14:54:08 +05:30
Ayush Shah
f11cdbfab1
Profile sample rows + Randomization of chunks for datalake (#9595) 2023-01-16 22:17:46 +05:30
Onkar Ravgan
14fa96958f
Added dbt workflow docs (#9493)
* Added dbt workflow docs

* added dbt small case

* Fixed review comments
2022-12-22 13:11:18 +00:00
Teddy
9bc4b08960
Add freshness documentation (#9449) 2022-12-21 14:54:45 +00:00
Ayush Shah
ba94f78aaa
Add docs for data quality (#9457) 2022-12-21 13:41:37 +00:00
Pere Miquel Brull
83ef3315c3
Fix query log parser (#8920) 2022-11-21 12:55:55 +00:00
Teddy
88c0ecb579
Remove count logic in custom sql test as 1) create complex logic and 2) multiple users got confused with expected tests results (#8819) 2022-11-16 18:10:55 +01:00
Onkar Ravgan
39afe9cd4a
Added lineage workflow docs (#8710) 2022-11-14 15:43:45 +05:30
Pere Miquel Brull
8d15985117
Docs - Run connectors from MWAA (#8548)
* Run connectors from MWAA

* Run connectors from MWAA

* Update url

* Add backend conn

* Add MWAA extraction
2022-11-07 09:43:42 +01:00
Pere Miquel Brull
b23d8353fc
Docs best practices (#8541) 2022-11-07 09:27:53 +01:00
clueless-bot
68c3e0b8fe
FIX: Add postgres usage and lineage details (#8456) 2022-11-01 00:18:05 +05:30
Milan Bariya
23e4967eda
FIX: Add Databricks usage and lineage details (#8130) 2022-10-14 06:37:57 +02:00
Pere Miquel Brull
1edd41e28a
Fix #7898 - DockerOperator and GCS K8s operator docs (#8137) 2022-10-13 23:00:15 +02:00
Sriharsha Chintalapani
795e7ae265
Fix Doc Links (#7998) 2022-10-07 08:49:46 -07:00
Sriharsha Chintalapani
538e827f5f
Fix Menu , Connectors should've its own section after deployment (#7950)
* Fix Menu

* Fix broken links

* Fix config values

* Fix config values
2022-10-06 06:54:02 +02:00