Mayur Singal
9f2b10f6e2
Fix #10858 : Fix dashbaord & pipeline url ( #10957 )
2023-04-10 09:48:49 +00:00
Teddy
fc9b64b52a
Fix histogram labels ( #10797 )
...
* fix: added logic to format bin labels for histogram metric
* fix: histogram labels
* fix: added types for arguments
2023-03-28 12:59:45 +02:00
Nahuel
dd225d1463
Fix: Clean Source and SourceStatus inheritance ( #10687 )
...
* Fix: clean source and sourcestatus inheritance
* Remove auth parameters from mysql.yaml
* Fix failing test
* Fix pylint checks
* Add missing docstrings
2023-03-21 12:57:48 +01:00
Teddy
b44972ef60
Fixes #8470 - Implements refinement functions for web analytics events ( #8528 )
...
* Moved webanalytics type in its own folder
* Added data insight chart api endpoint
* Jave formatting
* Added resource descriptor
* Added metadata entity endpoint
* Added aggregation endpoint for dataInsight
* Fix tag name
* Added logic to ingestion pipeline resource to add ES config info if pipeline type is dataInsight
* added domo to test subpackage
* cleaned up branch by removing commit from issue-8353 that were not merged in main
* Added web analytics data refinement
* Added get_status function
* Added from __futur__ for typing
* Fixed typos brought up during reviews
2022-11-07 17:08:20 +01:00
Ayush Shah
2d7d89754c
Remove Click and Add ArgParse ( #8182 )
2022-10-31 18:12:26 +05:30
Teddy
83b048c07c
Fixes #8206 -- Implement data insight refinement for entity report data ( #8351 )
...
* added data insight module to refine and ingest data
* Added json schema for data insight workflow
* Added mixin for data insight
* Added tests for data insight workflow and methods
* Fixed python style
* Fixed python linting
* Fixed unit test
* Added data insight worflow + workflow logging
* Fix python style
* Renamed analytic -> data_insight + added test to make + fixed style
* Fixed java style
* Fixed code smells + python style
* Ignore ES linting for now
* Fixed comments from review
* fix python formatting
* Fixe PipelineType attribute for data insight
2022-10-26 11:18:08 +02:00
Nahuel
36b12bd6f1
Fix lineage issues with merge_into and copy grants queries ( #8335 )
2022-10-24 17:22:22 +02:00
Pere Miquel Brull
d48fd468d4
Tag PATCH + Cleanup of helpers methods ( #8150 )
...
* cleanup
* lint
* Add tag patch
* Fix rename
* Dont kill tests
2022-10-15 14:56:30 +02:00
Teddy
f883863b8a
Fixes #7490 - Split Profiler and TestSuite Interface ( #8032 )
...
* Clean up test suite workflow and interface
* Fixed tests
* Split profiler and testSuite interfaces
* Cleaned up workflows and runners
* Fixed code formatting
* - remove old code
- remove `table` attribute used for testing and used mock instead
* Fixed execution bugs from refactor
* Fixed static type checking for profiler/api/workflow.py
* Fixed linting
* Added __init__ files
2022-10-11 15:57:25 +02:00
Onkar Ravgan
107eeef8c7
Added fixes according to pylint ( #8009 )
...
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-10-10 12:53:47 +02:00
Pere Miquel Brull
783557e6da
Fix Pylint for ingestion/sink ( #7961 )
...
* pylint on sink
* Add missing registration
2022-10-07 07:02:27 +02:00
Teddy
e244a241c3
added logic to clean up quotes from table constraints ( #7959 )
2022-10-05 16:09:33 +02:00
Mayur Singal
66a934c837
Fix #4694 : Refactor Source Status ( #6480 )
...
* Fix #4694 : Refactor Source Status
* Using Calculate Time Decorator
* Fixed Decorator
* Fixed Decorator
* perf_counter
* Move Calculate Sucesss to Soruce
* make py_format
* Fixed Sample Data
* Fixed Pipeline, Messaging, Dashboard & MlModel Service
* Fix ZeroDivisionError
2022-08-17 14:39:50 +05:30
Pere Miquel Brull
6701ae6416
Fix #6489 - Update backup CLI ( #6488 )
...
Fix #6489 - Update backup CLI (#6488 )
2022-08-03 12:01:57 +02:00
Nahuel
a878aa911c
Fix#6212: Retrieve connection params from secret manager in CLI commands ( #6441 )
...
* Retrieve connection params from secret manager for database connectors
* Retrieve connection params from secret manager for all services except database connectors
* Stop retrieving connection from SM in Airflow rest plugin
* Retrieve connection params from secret manager for dashboard services
* Retrieve connection params when initializing Workflow/ProfilerWorkflow objects
* Align services topologies + comment changes in topology runner
* Address SonarCloud bug detected
* Update database service topology
* Address PR comments
* Address PR comments
* Address PR comments
2022-08-02 09:13:46 +02:00
Pere Miquel Brull
53930a9253
Fix #6283 #6281 #6279 - Send compressed log, backend logs and Airflow fix ( #6288 )
...
Fix #6283 #6281 #6279 - Send compressed log, backend logs and Airflow fix (#6288 )
2022-07-27 07:47:25 +02:00
Pere Miquel Brull
9fa5d5cab2
Fix #6154 - Python Client to PATCH descriptions ( #6165 )
...
* Prepare patch description utilities for python client
* Improvements
2022-07-19 12:58:58 +02:00
Pere Miquel Brull
0ecc9f0da6
Fix #5459 - Remove sql-metadata in favor of sqllineage ( #5494 )
...
Fix #5459 - Remove sql-metadata in favor of sqllineage (#5494 )
2022-06-21 18:02:50 +02:00
Mayur Singal
b15ce6f52b
Fix #5338 : Dashboard Connector's to return create entities ( #5341 )
...
Fix #5338 : Dashboard Connector's to return create entities (#5341 )
2022-06-08 11:03:48 +02:00
Pere Miquel Brull
9138c70dee
Fix #4696 - Only store URL suffixes when extracting metadata ( #5281 )
...
* Only store URL suffix
* Fix test service name
2022-06-03 11:43:40 +02:00
Mayur Singal
b3428771a3
Fix #4767 : Refactor Usage & Lineage ( #5190 )
...
Fix #4767 : Refactor Usage & Lineage (#5190 )
2022-06-03 10:12:28 +02:00
Pere Miquel Brull
02c2dd9494
Fix #4689 - Rename FQDN to FQN and clean unused code ( #5165 )
2022-05-26 21:00:18 +02:00
Pere Miquel Brull
8e121fb306
Fix #4096 - PipelineService JSON refactor & Airflow metadata source connector ( #5125 )
...
Fix #4096 - PipelineService JSON refactor & Airflow metadata source connector (#5125 )
2022-05-25 08:35:16 +02:00
Pere Miquel Brull
0c51ecde63
Fix #2830 - Centralise loggers and update format ( #4570 )
...
Fix #2830 - Centralise loggers and update format (#4570 )
2022-04-29 06:54:30 +02:00
Onkar Ravgan
148d7f47ec
Refactor DBT for schema changes ( #4497 )
...
Refactor DBT for schema changes (#4497 )
2022-04-26 14:10:47 +02:00
Onkar Ravgan
c4aa07858d
ISSUE-2925: Added support to get dbt files via s3, gcs and http sources ( #3736 )
2022-04-19 19:36:24 +05:30
Pere Miquel Brull
2aecebd595
Fix #4189 - Fix airflow lineage callback ( #4191 )
...
Fix #4189 - Fix airflow lineage callback (#4191 )
2022-04-18 15:13:26 +02:00
Pere Miquel Brull
43be01b92e
Fix #4091 & #3948 - Create services correctly ( #4101 )
2022-04-13 18:04:52 +02:00
Sriharsha Chintalapani
4a572713a7
Fix #3865 : Service Connection & Schema Name - Fix ES indexes
2022-04-10 09:47:36 -07:00
Mayur Singal
42dcb383c0
Lineage Revamp for Schema changes ( #3995 )
...
* Fix #3592 : Refractored Lineage Ingestion
2022-04-10 08:44:30 -07:00
Ayush Shah
3840eb77c0
Refactoring Connectors - Metabase, MariaDB, Looker ( #3977 )
...
* Metabase Fix
* Looker, Metabase and Mariadb
* modified metabase
2022-04-09 16:56:12 +05:30
Pere Miquel Brull
bd4071bd64
Fix #3826 & #3886 - Profiler workflow & filter pattern ( #3893 )
...
Fix #3826 & #3886 - Profiler workflow & filter pattern (#3893 )
2022-04-06 17:05:00 +02:00
Mayur Singal
7292695bd3
Sample Data Fix ( #3888 )
...
* Sample Data Fix
2022-04-06 18:26:54 +05:30
Pere Miquel Brull
63533eb388
Fix for connectors based on refactoring of schemas V2 ( #3870 )
...
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-04-05 18:33:25 -07:00
Pere Miquel Brull
b3480693e4
Fix #3824 - OMeta Schema and JSON Connections ( #3861 )
...
Fix #3824 - OMeta Schema and JSON Connections (#3861 )
2022-04-05 21:20:39 +02:00
Pere Miquel Brull
e2539c5e83
Fix #3844 - First iteration for deprecating MetadataServerConfig ( #3853 )
...
* Style
* deprecate MetadataServerConfig
* Remove audience from Okta
2022-04-05 18:02:45 +05:30
Sriharsha Chintalapani
7b3e459eb3
Fix #3659 Refactor Service Connection String to be specific to per service ( #3804 )
...
* Fix #3659 Refactor Service Connection String to be specific to per service
* Simplify and centralize Airflow Pipeline info for REST (#3740 )
* Remove code
* Modified Configs based on refactoring schema (#3816 )
* Clean WorkflowContext
Co-authored-by: pmbrull <peremiquelbrull@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-04-04 12:46:09 -07:00
Mayur Singal
a38b08882a
Fix #3566 : ingesting sample data with space in name ( #3586 )
2022-03-22 18:29:03 +05:30
codingwithabhi
55c65b5c53
issue-3420: debug-logger-added ( #3434 )
2022-03-15 19:13:31 +05:30
codingwithabhi
2ef2edbf17
Bigquery lineage mapping ( #2849 )
...
* lineage-boilerplate-code-added
* bigquery-lineage-completed
* lineage-code-optimised
* code-cleaned
* code-smell-fixed
* code-refined
* added-database-name-in-fqdn
* code-smell-removed
* exception-handled
* removed-print-statement
* formatted-setup.py
* added-sqllineage-version
* included-valid-type-none
* debug-added-in-logger
* lineage-logic-updated
* updated-lineage-params
* global-config-variables-removed
* return-type-updated-in-entity
* updated-conditionin-from-entity
* imported-sqllineage-inside-method
* code-smell-removed
* code-formatted
2022-02-24 11:59:00 +05:30
Mayur Singal
4739aaa096
Fix #2905 : Added MSSQL Usage Connector ( #2948 )
...
* Fix #2509 : added mssql usage connector
* removed comment
* removed unused variables
* fixed code smell
* renamed sql variable
2022-02-24 01:01:03 +05:30
Mayur Singal
f9182a6244
Fix #2901 : fixed error message ( #2917 )
...
* Fix #2901 : fixed error message
* moved error message to helper.py
* replaced error to http
2022-02-23 00:20:36 +05:30
Pere Miquel Brull
76f4ccd590
Fix #2735 - Simplify Airflow properties extraction ( #2749 )
...
* Simplify lineage properties extraction
* Add network name
* Format
2022-02-14 16:53:42 +01:00
Pere Miquel Brull
1b2ea54d4f
Fix #2316 - Append Tasks & Add Status in Airflow ( #2738 )
...
* Add license
* Add date to timestamp helper
* Prepare pipeline status operation
* Update and clean tasks from client
* Update tasks from client
* Check if pipeline is empty
* Keep all current pipeline info
* Append and clean tasks
* format
* Add status information
* Test pipelineStatus update
* Update task on clear
* Log status on callback
* Update lineage and status docs
* Update lineage docs
* Format
* Logic to handle DAG status
* Lint and format
* Update lineage tests
2022-02-13 17:51:25 +01:00
Robert Pack
67a1c23482
Fix #2348 : Harmonize request titles in json schemas
2022-01-31 16:29:56 -08:00
Ayush Shah
917ccd7147
Bigquery and Helpers fix - Ingestion ( #2408 )
...
* Bigquery Ingestion fix - error handling and keyerror
* Update helpers.py and column_type_parser
* Update helpers.py
2022-01-25 14:30:09 +05:30
Ayush Shah
b689c2d632
Multiple fixes - Looker, Ingestion fix, pyarrow dependency ( #2385 )
...
* Added pyarrow dependency
* Fix #2378
* Fix Looker
* Fix Looker, secretstr to str
2022-01-24 12:29:54 -08:00
Sriharsha Chintalapani
10e71a7e09
Fix #2340 : Ingestion pipeline deployment refactor ( #2341 )
...
* Fix #2340 : Ingestion pipeline deployment refactor
2022-01-21 22:06:14 -08:00
Alberto Miorin
f9436c65f0
Fix #1306 : Evolve Location entity integration ( #1442 )
2021-12-03 02:13:13 +01:00
parthp2107
a9e70e6a89
License header update ( #1498 )
...
* updated liscense header
* updated license header
* updated license header
* updated license header
* addressing stylecheck findings
2021-12-01 12:46:28 +05:30
