* feat: add list from search service logic
* fix: added test and enable ES testing
* style: ran java linting
* fix: ES container startup timeout
* style: ran java linting
* fix: removed array definition as attributes of filter class
* style: ran java linting
* fix: update list from search
* feat: added q param to list from search endpoint
* style: ran java linting
* fix: enable ES test for list from search
* fix: added include/exclude from source
* style: ran java linting
* fix: test case
* fix: exception type
* Add entityReferences and entityReferenceList as custom properties
* ui: allow to add custom properties for all available property types.
* make entity type as an array
* Fix label typo and import order in AddCustomProperty component
* Remove unnecessary property types with format
* Refactor CustomPropertyTable.tsx to handle array config and enum config
* Add entity reference options and update custom property form
* Add EntityReference and EntityReferenceList as a custom property
* add validation for date format
* Add date and dateTime input support to PropertyValue component
* Fix import order and add search functionality to type dropdown
* Fix custom property cypress tests
* add input for number, email, timestamp, timeInterval, duration, time, sqlQuery,
* add input support for entityReference and entityReferenceList
* Add placeholders for email, timestamp, start time, end time, and duration inputs
* Refactor PropertyValue component to use destructuring for timeInterval object
* Add minWidth style to PropertyValue component and include SQL query editor
* Add entityReference and entityReferenceList as a generic types and not reference to the existing types
* Remove services from entity reference types list
* handle property values for different property type
* Update ExtensionTable to handle object values in CustomPropertyTable
* Add entity reference list rendering and styling
* Fix file paths in complexTypes.json
* Add regex constant for UNIX timestamp in milliseconds and update language files
* Refactor custom property configuration and add new options
* Fix import order and update property value handling
* add unit tests for different input types for different property type
* add unit test for different property type values
* fix cypress tests
* add cypress test for CRUD for different types
* add cypress test for entity ref and entity ref list property type
* add cypress tests for all the 11 entities which support custom properties
* fix cypress test for createCustomPropertyForEntity
* Add new interfaces and types for CustomPropertyTable
* add cypress test for custom property to create via APIs for entities
* add cypress tests for string,integer, markdown, number, duration, email
* add cypress test for enum property for all entities
* add cypress test for sqlQuery, timestamp and timeInterval property
* fix cypress test for sqlQuery input
* fix flaky cypress test for sqlQuery input
* address comments
* Refactor import statements and update ENTITY_PATH enum
* Update import statement in CustomProperty.ts
* change TimeInterval to Time Interval
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* Implement Initial MultiThread suggestion
* Update all the ingestion sources to use the new ContextManager
* Fix missing wraps on decorator
* Fix Unittests
* Fix linters
* Fix linters
* Fix BigQuery UnitTests
* Add UnitTests to the newly created code
* Fix unittest
* change the threads from table to schemas
* Update README.md
* Small change suggested by Sonar
* Slight change to test a different way to multithread over tables
* Debug changes
* More multithread tests
* Remove uneeded wait time
* Testing
* refactor code based on removal of time.sleep
* Fix wrong paste
* Improve ExecutionTimeContextManager
* Fix missing .get() and unit tests
* Fix conflicting changes
* Update Multithread logic with the incremental extraction
* Fix linters
* Fix unittest
* Remove commented code
* Fix Unittests
* Fix checkstyle
* Change default to threads = 1
* - Fix App Run Type
- Remove Init on App Start
- Migrate stats setup for SearchIndexing to execution
* fix the AppLogsViewer unit test failing due to appRunType type changes
---------
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
* Application cannot be installed with None as schedule type
* MINOR: Add / Fix GCS and ADLS - docs, bugs (#15502)
Add GCS and ADLS docs
* Add stack trace while throwing an error to debug (#15522)
* Fix#15533: Fix name & display name for kafka json schema parser (#15534)
* MINOR: postgres add ssl options in yaml (#15538)
* Small change to fix the Glossary TErm Tasks not fetching Owner automatically (#15535)
* remove the DataInsightAlert spec as the alerts test are covered in ObservabilityAlerts spec (#15531)
* MINOR: fix help dropdown item font and icon sizes (#15511)
* fix help dropdown item font and icon sizes
* added unit test for navbarUtils
* fix sonar failure
* changes as per comments
* minor fixes
* Minor: fixed DQ edit test case issue and searchIndexDetails typescript issue (#15528)
* Minor: fixed DQ edit test case issue and searchIndexDetails typescript issue
* fixed failing unit test
* fixed unit test for Data quality test
* reverting e2eLabeler changes
* chore(ui): separate routes as per categories (#15512)
* chore(ui): seprate routes as per categories
* add tests
* fix test
* domain path fix
* [MINOR] GX logging hierarchy (#15542)
* fix: GX module logging hierarchy
* style: ran python linting
* - Use Entity Type (#15546)
* removed docs 1.4.x (#15550)
* Suggestions Alert new design (#15532)
* update ux of suggestions alert
* locales
* minor changes
* fix descriptions
* minor css fixes
* Fix#11868: Duplicated queries cannot be created (#15519)
* Fix#11868: Duplicate query should throw an error of entityExists
* Fix#11868: Duplicate query should throw an error of entityExists
* fix test
* fix test
* Fix uniquee constraint for checksum in Postgres
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
* added OpenMetadataApplication.getDao (#15549)
* Add the entityUrl to the header for ThreadMessages (#15552)
* feat: AppResource (#15555)
forbid modification of system app
* MINOR: Skip source hash generation for service (#15516)
* sync the documentation roadmap page with the getcollate roadmap (#15551)
* Application cannot be installed with None as schedule type
* None type should not register in scheduler
* scheduleTimeline fix
* fix payload of schedule type none and change it to scheduleTimeline
* fix unit test
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: IceS2 <pjt1991@gmail.com>
Co-authored-by: Aniket Katkar <aniketkatkar97@gmail.com>
Co-authored-by: Shailesh Parmar <shailesh.parmar.webdev@gmail.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: Imri Paran <imri.paran@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
* - Fix Poll Interval
* make poll interval as 1
* - Add random tests
- Disable Tests for Subscription for Query
* - Checkstyle Fix
* - Remove ondemand jobs as well
- On update remove already init instance
* - use alias
* - typo
* - Search Index Fix
* Add 'Mentions' enum for subscriptionCategory
* fix the user list selection field not showing for 'Mentioned Users' filter
* update the cypress for new changes
* fix teams not showing in the mention suggestions
* Add Custom Propety Config to store format, enum values, entity types
* Fix import statements and remove unused code
* Add Custom Propety Config to store format, enum values, entity types
* Add support for enum field type in custom properties
* update name in customPropertyConfigTypeValueField
* add custom property config column in custom property table
* Update padding-left in block-editor.less
* Add enum value translation for multiple languages
* update placeholder of config
* fixed python sdk
* add enum type in property value
* add unit tests
* Add Custom Propety Config to store format, enum values, entity types
* update ui to handle the enum config and validation
* Fix enum value handling in EditCustomPropertyModal and PropertyValue
* Update CustomProperty.md with enum values and multi-select option
* add cypress test
* add cypress for multiselect enum value
* Add tests for enum props
* add cypress for editing the enum property
* Add validations to enum
* Fix dependency issue
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
* add "Assignees" option in subscription category enum
* add Assignee option for task alert source destination options
* change alert form section names from "Trigger" to "Source" and "Actions" to "Trigger"
* localization changes for other languages
* rename the trigger item components
* rename observability alert form actions item component to a proper name
* only show "Owners" and "Assignees" options for "Task" source
* change all alert trigger references to source
* change all alert action item references to trigger
* fix failing unit test
* add cypress coverage for task source
* convert alert specs to .ts files
* update the changed key value for other languages
* made ui enhancement in setting page
* ui fixes
* fix ui render in application and other setting page
* fix unit test and user profile page issues
* added loader in team selectable in user profile page
* changes as per comments
---------
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
* feat(nosql-profiler): row count
1. Implemented the NoSQLProfilerInterface as an entrypoint for the nosql profiler.
2. Added the NoSQLMetric as an abstract class.
3. Implemented the interface for the MongoDB database source.
4. Implemented an e2e test using testcontainers.
* added profiler support for mongodb connection
* doc
* use int_admin_ometa in test setup
* - fixed linting issue in gx
- removed unused inheritance
* moved the nosql function into the metric class
* formatting
* validate_compose: raise exception for bad status code.
* fixed import
* format
* feat(connector): add BigTable
* bigtable work
1. docstrings
2. tests
3. created a Row BaseModel
4. implemented a ClassConverter
* docs moved to separate PR
* format files
* minor cosmetic
- removed TODO
- changed headers' year to 2024 for new files
- fixed typos
* format
* formatting and comments
1. added missing docstrings.
2. abstracted the _find_instance method.
3. aliased the IDs used in the BigTable connection
* added comment regarding private key
* added comments regarding column families
* enclose get_schema_name_list in `try/except/else`
* format
* streamlined get_schema_name_list to include all logic in the try block
* linting: fix python linting
* fix: get column types from parquet schema for parquet files
* style: python linting
* fix: remove displayType check in test as variation depending on OS
* Fix#14786: Suggestions API
* Handle suggestions in ometa
* Minor: Optimise Databricks Client (#14776)
* MINOR - Fix SP topology context & Looker usage context (#14816)
* MINOR - Fix SP topology context & Looker usage context
* MINOR - Fix SP topology context & Looker usage context
* Fix tests
* Fixes#14598: Fix Tags / Labels ingestion on includeTags as False (#14782)
* fix(ui): password error message for char limits (#14808)
* fix(ui): password error message for char limits
* fix java side code
* Fixes#13556: Support for Salesforce table description ingestion (#14733)
* ISSUE-13556: Add suport for Salesforce table description ingestion
* ISSUE-13556: Remove unnecessary blank line
* ISSUE-13556: Fix to get description for each table
---------
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
* MINOR - Better handling of Ingestion Pipeline Status (#14792)
* MINOR - Better handling of Ingestion Pipeline Status
* format
* format
* MINOR: Added table validation for cost analysis data (#14793)
* Added validation for cost analysis source
* centralized life cycle logic
* CYPRESS: simplify side navigation click in cypress (#14818)
* simplify side navigation click in cypress
* make sidbar item uses common enum
* fix cypress failure of outside import
* fix(#14326): tier dropdown is not working in advance search (#14780)
* improvement in advance search based on custom property
* fix a reading undefined property issue
* wip: advance search based on tier
* some code cleanup and improvement
* some fixes
* fix: ui flicker when advanceSearched is apply and refresh the page
* some cleanup
* no need to call customproperty api call, if entity not suppport customProperties
* minor change
* fix: autocomplete not working in tier search option in advance search modal
* added unit test for advance search provider component
* some cleanup
* added testcase for open modal
* added testcase for resetAllFilters method
* removed unwanted code
* added e2e test for testing tier advance search
* fix: e2e search flow for single field
* fix: string field not working after giving listValues in TierSearch
* fix: group query e2e test fix
* used asyncFetch way to get the tierOptions synchronously
* some cleanup
* remove unwanted lines
* some cleanup
* fix: selected option show option value instead of option title
* fix(minor): update skip icon for executions (#14809)
* Fixes#14803: ignore capitalization when confirming deletes (#14804)
* ignore case when confirming deletes
* Test confirmation of deletes works when case differs
Added test case for 'delete' as the confirmation text.
* minor(config): update openmetadata-ui code reviewers (#14823)
* Add Tests
* Add list/accept/reject apis
* initial ui changes
* localisation
* show suggestion for empty description
* ui feedbacks
* Fix permission check for entities without owner
* Fix entityLink and add tests
* Add update suggestion WIP
* Fix test
* Fix PUT and Pagination
* Fix styling
* update test
* Update status
* add OM server connection in apps
* add permissions check
* Fix CI
* Remove TODO
* Fix feedResourceTest
* fix unit tests
* add private configs for apps
* add private configs for apps
* fix update application icons
* minor center align icon
* add private configs for apps
* Format
* Fix pydantic gen
* Remove token
* Update name
* Rework private conf
* Fix apps
* Fix apps
* Format
* Format
* show metapilot only if its installed
---------
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: kwgdaig <18678754+kwgdaig@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
Co-authored-by: Abhishek Porwal <80886271+Abhishek332@users.noreply.github.com>
Co-authored-by: Carlo Q <carlo@machina.bio>
Co-authored-by: karanh37 <karanh37@gmail.com>
* Fix(#14262): Format the ES reindexing logs for better readability
* Refactor formatJsonString function in StringsUtils.ts
* Fix Error Schema for Search Indexing Application
* Refactor AppLogsViewer.component.tsx
* Fix import order and add default value for formatJsonString
* Refactor formatJsonString function in StringsUtils.ts
* Fix Schema
---------
Co-authored-by: Mohit Yadav <105265192+mohityadav766@users.noreply.github.com>
Co-authored-by: mohitdeuex <mohit.y@deuexsolutions.com>
* Create the iceberg connection schema
* Link the IcebergConnection configuration with the forms on the UI
* Add the pyiceberg dependency on the ingestion package
* Create the get_connection and test_connection functions
* First iteration on the iceberg ingestion logic
* Add A more comprehensive implementation of the Iceberg Source
* Add UnitTests
* Update icebergConnection definition
* Update the iceberg souce code based on new schema
* Updated icebergConnecgtion schema for simplicity and to be able to configure Converters
* Updated setup dependencies to be more flexible
* Updated get_owner_ref logic
* Fix formatting
* Changed the icebergConnection json schema structure to enable the ClassConverters
* Add the IcebergCatalog and IcebergFileSystem ClassConverters
* Refactor the code to take into account the new jsonSchema structure
* Fix formatting
* Add Documentation for the Iceberg Connector
* Fix Menu order for Iceberg
* ui: add Iceberg service icon and constant
* Fix DynamoDb Catalog issue due to how PyIceberg instantes it
* Changed uri title to URI
* Fix ClassConverter for Iceberg
* Fix GetSecretValue for password types
* Fix formatting
* Fix formatting
* Add Iceberg Connector Images for the docs
* Add pylint disable for Hacky super() call
* Add Iceberg.md for the UI docs
* Fix pylint complaint
* Fix pylint complaint
* Fix UnitTests
* fix type error and unit tests
* update pipeline type checks
* Fix Sonar Cloud complaints
---------
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
* Fix#11944: API support to fetch count for mentions
* Add closedTask/totalTask
* fetch count as per new count api for entity and user page and remove mentions tab from entity
---------
Co-authored-by: Ashish Gupta <ashish@getcollate.io>
