Onkar Ravgan
4ade6fd39f
Remove col displayname from patch req ( #14279 )
2023-12-06 10:12:04 -08:00
Mayur Singal
d6a01ca5b1
Minor: Remove retension period from patch request ( #14276 )
...
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
2023-12-06 20:29:44 +05:30
Onkar Ravgan
c7e32c8abf
Fixes: 14240 and Fixes 13365: Bulk patch descriptions from dbt ( #13999 )
2023-12-06 19:00:31 +05:30
chyueyi
070ff1f605
feat: add profile for doris connector ( #14214 )
2023-12-06 08:59:56 +05:30
Ayush Shah
b8d0b3b6e4
#14168 : Mark SP and Data model deletion ( #14219 )
2023-12-06 08:57:41 +05:30
Onkar Ravgan
b3578cd496
Fix 14040: Part 2 Add Patch Request when updating the entities from create request ( #14224 )
2023-12-05 17:56:40 +05:30
Teddy
3bbf55fcda
FIXES #14049 - Split test case resolution status from test case result ( #14204 )
...
* refactor: entityFQN as ListFilter condition
* feat: implement resolution entity timeseries
* fix: rename to testCaseResolutionStatus
* ref: extracted ES query builder into private method
* ref: extract OS query builder in its own method
* ref: remove ingestion logic for test case resolution
* fix: reorganize json schemas to fix circular import in Python
* ref: object names in typescript code
* feat: added indexing of test case resolution
* feat: added test case resolution sample data
* fix: test case resolution api logic
* fix: audit logger for entityTimeSeriesInterface
* fix: DDL generation
* style: python linting
* fix: skip UI test case resolution tests
* fix: remove extension field
* fix: renamed testCaseFailureStatus to testCaseResolutionStatus
* fix: remove reviewer
* fix: rename sequenceId to stateId
* fix: re adjust search weights
* fix: removed InReview status
* style: ran python linting
2023-12-04 23:18:01 -08:00
Teddy
f571f67eda
fix: DI skip on failure when listing entities ( #14226 )
2023-12-04 16:59:09 +00:00
Mayur Singal
5965d4c63b
Minor: Fix airflow empty status ( #14216 )
2023-12-04 07:54:13 -08:00
Mayur Singal
4843d85bf9
Minor: Fix kafka Topic Entity Import ( #14227 )
2023-12-04 21:08:33 +05:30
Mayur Singal
b57e6710b6
Minor: Fix 'str' object has no attribute 'serviceType' for SP Lineage ( #14221 )
2023-12-04 19:52:02 +05:30
Mayur Singal
389ae79d3c
#14115 : Separate Unity Catalog From Databricks ( #14138 )
2023-12-04 11:22:46 +05:30
Pere Miquel Brull
7fcdf08ca4
#11626 & #14131 - Lineage with other Entities & attr-based xlets ( #14191 )
...
* Add OMEntity model
* Test OMEntity
* Update repr
* Fix __str__
* Add entity ref map
* Test serializer for backend
* Fix tests
* Fix serializer
* Test runner
* Add runner tests
* Update docs
* Format
2023-12-01 06:29:44 +01:00
Onkar Ravgan
b403c2b250
Added SP lineage as edge ( #14180 )
2023-11-30 16:36:52 +05:30
Mayur Singal
2092a52048
Minor: Fix E2E Tests failure ( #14171 )
2023-11-29 12:30:37 -08:00
Teddy
8ff70e31fe
MINOR: fixes ingestion of sample data for custom metrics ( #14170 )
...
* fix: updated sample data ingestoon for custom metrics
* style: fix python linting
* style: fix java linting
2023-11-29 17:28:19 +01:00
Mayur Singal
49a5557e44
#14099 : fix bigquery test connection & SP query ( #14106 )
...
* Fix #14099 : fix bigquery test connection & sp query
* fix unquote condition
* pyformat
2023-11-28 14:02:46 +01:00
chyueyi
b6b337e09a
feat: add support for doris datasource ( #14087 )
...
* feat: add support for doris datasource
* fix: fix python style check
* fix: add pydoris dependency
* fix: add pydoris dependency
* fix: py_format_check
* fix: parse error when doris view column is VARCHAR(*), check data length if not digit then return 1
---------
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
2023-11-28 13:57:52 +05:30
Onkar Ravgan
b56b67bf67
Fixed overwriting data for service entites ( #14135 )
2023-11-28 08:49:29 +01:00
karanh37
853a83f8b9
fix: allow config update for external app ( #14034 )
...
* fix: allow config update for external app
* Prep config
* Add configs
* fix: call configure app on saving of configurations
* fix: add autocomplete widget for json schema forms
* ORganize configuration handling
* Allow config
* Format
* Fix test
* Prep migrations for allowConfiguration
* TODO
* Add pg migrations
* fix: add allowConfiguration support
* Autotagger
* Update install flow
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2023-11-27 16:55:33 -08:00
Onkar Ravgan
ebb2317cf0
Fix 14040: Part 1 Remove get_by_name calls from topology ( #14098 )
...
* Changed for database
* Added changes for dashboard_service
* Changed for messaging service
* Changed for mlmodel service
* Changed for pipeline service
* Changed for search service
* Changed for objectstore service
* fixed wrong import
* fixed lint
* fixes
* fixed pytests
* fixed domo db pytest
* Fixed review comments
2023-11-27 16:15:47 +05:30
Mayur Singal
aa00802f6d
Fix #14100 : mssql fix usage query, make table name lowercase ( #14101 )
2023-11-27 09:37:37 +01:00
Mayur Singal
4bce6a9aaf
Athena: Add support for work groups in usage & lineage ( #14072 )
2023-11-23 11:13:35 +05:30
Teddy
131eea32f8
Add Custom metrics sample data ( #14069 )
...
* feat: add sample data for custom metrics
* feat: update project.scripts to `metadata`
2023-11-22 11:57:19 +01:00
Mayur Singal
9f14ef7fab
Fix #13954 : Fix ParseException for older version of databricks ( #14015 )
2023-11-20 13:14:40 +05:30
Teddy
c7ac28f2c2
Fixes #11357 - Implement profiler custom metric processing ( #14021 )
...
* feat: add backend support for custom metrics
* feat: fix python test
* feat: support custom metrics computation
* feat: updated tests for custom metrics
* feat: added dl support for min max of datetime
* feat: added is safe query check for query sampler
* feat: added support for custom metric computation in dl
* feat: added explicit addProper for pydantic model import fo Extra
* feat: added custom metric to returned obj
* feat: wrapped trino import in __init__
* feat: fix python linting
* feat: fix typing in 3.8
2023-11-17 17:51:39 +01:00
Onkar Ravgan
9f1f830775
Convert query duration to milliseconds for usage ( #14011 )
...
* fixed athena query duration
* change second to millisecond from ui side
* added cypress support
* converted duration to millis
* supported milli second duration and change cypress
* minor change
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
2023-11-17 19:59:47 +05:30
Pere Miquel Brull
a8a1fd26b1
Snowflake SP don't filter by warehouse ( #14018 )
2023-11-17 14:18:15 +01:00
Pere Miquel Brull
f847c1cf10
Fix #14012 - Snowflake procedure with empty definition ( #14014 )
...
* Fix #14012 - Snowflake procedure with empty definition
* Add debugs
2023-11-17 11:29:27 +01:00
Pere Miquel Brull
6fb3f12675
App Resource - Add /configure endpoint & WAII app definition ( #13983 )
...
* App Configuration
* TODO
* Configure App with OM server info
* Remove init
* Rename configs
* Remove extra source
2023-11-17 09:28:11 +01:00
Suresh Srinivas
5f34bd02d3
Fixes #13595 Consolidate changes by a user in a single session to a single change ( #13617 )
...
* getChangeDescription to use entity and update type to decide previous version
* Fixes #13595 - Consolidate changes by a user in a single session to a single change
2023-11-16 18:14:27 -08:00
Teddy
a1a1d9f4d1
fix: GX integration ts ms ( #13984 )
2023-11-15 16:11:52 +01:00
Mohit Yadav
10d8ec84fe
Logs added for Search Indexing and Stats issue fixed ( #13956 )
...
* Logs added for Search Indexing and Stats issue fixed
* Fix uninstall error
* Add error handling
* fix lint
* Push Job Level Exception on top
* disable flaky tests
* Fix Logs not visible in Search
---------
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2023-11-13 23:39:56 +05:30
Onkar Ravgan
a1b967b857
Nifi wrong password fix ( #13950 )
2023-11-13 14:48:07 +05:30
Mayur Singal
b34111ea00
ElasticSearch: Fix ES connection ( #13919 )
2023-11-13 14:10:11 +05:30
Pere Miquel Brull
c742835766
Auto Tagger Application - Preparing the Ingestion Framework ( #13862 )
...
* Prepare the skeleton for generic app registration
* Prepare the skeleton for generic app registration
* Handle app runner
* Prepare the skeleton for generic app registration
* Prepare the skeleton for generic app registration
* Allow deployment
* Fix PII APP
* Fix lint
* Fix PII APP
* Fix PII APP
* Prepare config-based external apps
* Prepare config-based external apps
* Fix lint
* Prepare config-based external apps
* Fix DI errors
* Amend comments
2023-11-13 08:58:38 +01:00
Mayur Singal
367bac9064
Fix #13787 : Add support for ES data types ( #13916 )
...
* Fix #13787 : Add support for ES data types
* fixed tests
---------
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
2023-11-10 20:14:42 +05:30
Pere Miquel Brull
ae969aa4f3
Fix #13660 - Metadata Backup Memory Exhaustion ( #13935 )
2023-11-10 12:21:48 +01:00
Onkar Ravgan
b7e7625071
Convert dbt test result timestamps to millis ( #13903 )
...
* Fixed dbt test res time to millis
* added postgres migration
2023-11-10 16:15:16 +05:30
Mayur Singal
71a67d6bbe
Fix ES search during lineage ingestion ( #13932 )
2023-11-10 15:50:37 +05:30
Pere Miquel Brull
0eacc829a4
Fix #13794 - Add domain support to the Python SDK ( #13931 )
2023-11-10 11:00:06 +01:00
Pere Miquel Brull
7c06116b53
Add deprecation warnings ( #13927 )
2023-11-10 15:17:07 +05:30
Pere Miquel Brull
8891a9a410
Fix #13906 - Fix add_mlmodel_lineage description field ( #13920 )
2023-11-10 15:16:09 +05:30
Pere Miquel Brull
b250cd8808
Fix #13699 - Add separator for Storage Container manifest ( #13924 )
...
* Fix #13699 - Add separator for Storage Container manifest
* Fix #13906 - Fix add_mlmodel_lineage description field
* Add separator
* Add separator
2023-11-10 10:44:47 +01:00
Mayur Singal
4b625f7ba5
Move pandas top level import ( #13926 )
2023-11-10 14:15:14 +05:30
Hung Duong
575a4a0ebc
put correct dataset only ( #13369 )
...
* put correct dataset only
* update code
---------
Co-authored-by: hung.duong <hung.duong@be.com.vn>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2023-11-09 10:30:43 -08:00
Mayur Singal
4c19bb5a1d
Add support for external profiler workflow ( #13887 )
...
* Add support for external profiler workflow
* pylint
* resolve comments
* resolve comments
* pyformat
* fix code smell
2023-11-09 18:50:47 +05:30
Mayur Singal
a8145a82fa
Fix #13603 : Configurable Sample Data Rows for Profiler ( #13807 )
...
* Fix #13603 : Configurable Sample Data Rows
* Fix #13603 : Configurable Sample Data Rows for Profiler
* fix table config
* support configurable overwriting of sample data
* add support for schema and database profiler configuration
* chore(ui): put sampleDataStorageConfig under advanced config
* fix tests
* py format
* chore(ui): add sampleDataCount in table profiler config
* fix tests
* pylint & tests
* feat(ui): add profiler settings tab in database and database schema page
* chore(ui): show different inputs for profile sample type
* schema changes to make default storange config null
* add unit test
* schema changes to fix api
* update profiler setting schema
* move profiler settings to manage button
* sync locals
* fix(ui): unit tests
* fix tests
* py format
* fix lint
* minor improvements
* chore(ui): update profiler settings schema
* resolve review comments
* pytest
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-11-09 18:49:42 +05:30
Teddy
7f151ca5f7
fix: remove RESULT_SCAN dep. ( #13904 )
2023-11-09 08:13:59 +01:00
Onkar Ravgan
bdfc9226d5
Added cost analysis producer in DI ( #13872 )
2023-11-08 12:04:48 +05:30
