Sriharsha Chintalapani
e3cfb4dc65
For hive's complex data types parse raw type ( #560 )
...
* For hive's complex data types parse raw type
* Complex Data type logic modification
* Complex Data Type parsing implemented
* Raw Data type helper modification
* handling unnamed/anonymous struct
* Complex Nested structure implementation
* print statements removed and reverted to raw_data_type
* Complex Structure Array & MAP logic implemented
* Raw Data Type Logic revamped
* Redshift Integration
* MAP and UnionType support added
* Redshift Pypi package updated
* dataLength validationError fix
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-10-04 23:36:35 +05:30
Sriharsha Chintalapani
1361bace11
sample entity generation ( #646 )
...
* 1M entities to test scalability
* 1M entities to test scalability
* 1M entities test
* column name modification
* Fix service add and tags
* Add sample entity generation
Co-authored-by: parthp2107 <parth@getcollate.io>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-10-04 22:56:28 +05:30
Sriharsha Chintalapani
37796c655b
Fix #650 : Add support for pipelines in sample data & search integration ( #651 )
2021-10-04 17:48:45 +05:30
Sriharsha Chintalapani
08cefee56a
Fix #636 : Pipeline/TaskEntity capture more metadata like start, end time, type etc.. ( #637 )
2021-10-02 17:12:18 -07:00
Sriharsha Chintalapani
ebfbd39cc2
Fix #624 : Ingestion: Fix REST API calls to get service and create service ( #625 )
...
* Fix metadata usage listing of tables
* Fix #624 : Ingestion: Fix REST API calls to get service and create service
2021-09-30 20:35:35 +05:30
Sriharsha Chintalapani
7e16e39ca0
Fix metadata usage listing of tables ( #620 )
2021-09-29 22:54:52 -07:00
Sriharsha Chintalapani
8f5c5021d7
Fix redshift complex types ( #619 )
2021-09-30 11:02:52 +05:30
Sriharsha Chintalapani
e84de1a310
Fix #611 : ElasticSearch matches are case-sensitive ( #617 )
2021-09-29 17:43:09 -07:00
Julia Valenti
a514934f16
Fixes docs that reference messaging service instead of dashboard or pipeline ( #613 ) ( #614 )
2021-09-29 16:21:31 -07:00
Sriharsha Chintalapani
bfec0bfbed
Ingestion: Airflow integration to ingest metadata about pipelines and tasks ( #609 )
...
* [WIP] Airlfow integration
* [WIP] Airlfow integration
* [WIP] airflow integration
* [WIP] Airflow
* [WIP] Airflow
* Fix #608 : Ingestion: Airflow integration to ingest metadata about pipelines and tasks
* Fix #608 : Ingestion: Airflow integration to ingest metadata about pipelines and tasks
* Update DashboardServiceResource.java
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-09-29 11:32:09 -07:00
Sriharsha Chintalapani
f7da8045b4
Fix #605 : Ingestion: metadata list apis should paginate ( #606 )
...
* Fix #605 : Ingestion: metadata list apis should paginate
* Fix #605 : Ingestion: metadata list apis should paginate, Addressing review comments
2021-09-29 00:06:08 +05:30
Sriharsha Chintalapani
0983ec7f1f
Add nested types to sample data ( #601 )
...
* Add nested types to sample data
* Add nested types to sample data
2021-09-28 12:23:34 +05:30
Sriharsha Chintalapani
3d10347aec
Fix #596 : Update ingestion to new OpenMetadata column entity changes ( #597 )
2021-09-27 23:11:19 +05:30
Sriharsha Chintalapani
eb2717b0e3
Fix #587 : Ingestion: Add standalone report process to generate datasets, usage & profile and serve from standalone server ( #588 )
...
* Fix #587 : Ingestion: Add standalone report process to generate datasets, usage & profile and serve from standalone server
* add localhost
2021-09-27 08:43:38 -07:00
Ayush Shah
627481f181
Status record Json encoding bug fixed and pandas not found fixed ( #584 )
2021-09-25 13:54:04 -07:00
Sriharsha Chintalapani
745ae0c253
Fix #577 : Users API should support put op ( #578 )
...
* Fix #577 : Users API should support put op
2021-09-24 17:55:26 -07:00
parthp2107
06810cdec1
Fix #432:Added Redash Connector ( #444 )
...
* added redash connector
* added redash connector
* Added Redash Connector
* minor changes
Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
Co-authored-by: parthp2107 <parth@getcollate.io>
2021-09-22 15:09:24 +05:30
Ayush Shah
7bc2b14178
Added and Modified Redshift Pipeline ( #528 )
...
* Pylint build failure fixed
* Redshift workflow added
2021-09-19 21:16:50 -07:00
Ayush Shah
7652baa00d
Setup.py Refactored, ES port fix ( #521 )
...
* Pylint build failure fixed
* Setup & dependency modified, Data profiler default to False, ES port fix
* Profiler requirements refactored
* Setup.py requirement fix
* openmetadata-ingestion version upgrade
2021-09-19 13:59:14 +05:30
Sriharsha Chintalapani
4c6c8fd446
Fix #515 : Ingestion: Add ES configuration to allow port ( #516 )
2021-09-17 08:57:41 -07:00
Sriharsha Chintalapani
8c103bd2ad
Profiler ( #496 )
...
* profiler code
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-09-15 15:49:26 -07:00
Ayush Shah
9b04d781e0
Fixed Bigquery to support data profiling ( #494 )
2021-09-14 22:07:51 -07:00
Sriharsha Chintalapani
20a98aca81
Fix #396 : Elastic search is not indexing usage for dashboard ( #492 )
2021-09-15 09:47:14 +05:30
Sriharsha Chintalapani
b7adb5dc6b
Fix #469 : Add Vertica Connector ( #470 )
2021-09-12 21:59:31 -07:00
parthp2107
191ecb7332
added unit tests to helper and workflow classes in ingestion ( #391 )
...
* added unit tests to helper and workflow classes in ingestion
* Addressing comments
* Adressing comments
* Adressing comments
* Failing Unit tests fixed
* Workflow test and config file modified
Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-09-12 01:07:13 +05:30
Ayush Shah
d2df40cf2b
Fix #355 : Tableau Implemented ( #468 )
...
* Fix #355 : Tableau Implemented
* Tableau pipeline location modification
2021-09-11 11:46:10 -07:00
Sriharsha Chintalapani
ed698dc40f
Fix #466 : Fix airflow example dag ( #467 )
2021-09-10 22:28:23 -07:00
Sriharsha Chintalapani
1c80dc246e
Fix #456 : Make PII-Processor optional and independent install ( #457 )
2021-09-10 10:41:19 +05:30
Suresh Srinivas
30c1e5a14c
Merge pull request #447 from open-metadata/Issue-446
...
Fix #446 : Add DataProfiler to ingestion and APIs
2021-09-09 11:25:56 -07:00
Sriharsha Chintalapani
4998d49167
Fix #446 : Add DataProfiler to ingestion and APIs
2021-09-09 10:51:07 -07:00
Sriharsha Chintalapani
2369ddc858
[WIP] Fix #446 : Add DataProfiler to ingestion and APIs
2021-09-08 23:55:48 -07:00
Ayush Shah
bdad904e82
Sample Data ingestion fix ( #437 )
2021-09-08 16:52:54 +05:30
Suresh Srinivas
328658ebea
[WIP] profiler
2021-09-07 22:03:57 -07:00
Ayush Shah
2b61c8ea8c
fix Issue-404 replicationFactor and Message Maximum Byte added ( #419 )
...
* fix Issue-404 replicationFactor and Message Maximum Byte added
* Cleanup Policy values changed in sample topic data
2021-09-07 12:22:51 -07:00
parthp2107
3965b030a9
Added Salesforce Connector ( #423 )
...
* Added salesforce-connector
* minor changes
* Added Salesforce-connector
* Added Salesforce-connector
* Salesforce sample data implemented
Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-09-08 00:37:13 +05:30
Ayush Shah
657962bc4f
MSSQL sample-data query fix ( #375 )
...
* MSSQL sample-data query fix
* Query Format as per Database implemented
2021-09-06 21:03:04 -07:00
Sriharsha Chintalapani
072c2034ad
Fix #208 : Add Global Suggest ElasticSearch index to capture all the entities ( #406 )
2021-09-05 20:56:54 -07:00
Sriharsha Chintalapani
d0dbcc19b7
Fix #401 : Merge sample data generation into single connector ( #402 )
...
* Fix #401 : Merge sample data generation into single connector
* Path for datasets modified
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-09-05 22:35:02 +05:30
Sriharsha Chintalapani
59c74fffcd
Fix #393 : ingestion support for indexing dashboard/charts entities ( #394 )
2021-09-03 14:54:05 +05:30
Sriharsha Chintalapani
eff1de87ef
Fix #381 : SQLAlchemy URL is causing issues in creating service ( #383 )
2021-09-02 20:36:18 -07:00
Ayush Shah
c9ada4ca1a
Looker Dashboard Connecter Added ( #351 )
...
* Looker Dashboard Connecter Added
* Dashboard yield fixed
* Looker Connector Method modifications
2021-09-02 20:32:03 -07:00
Sriharsha Chintalapani
044b785d15
Fix #384 : Check if the elastic search index exists and make sure we have document mappings ( #386 )
2021-09-03 08:58:26 +05:30
parthp2107
d3d0ac19d0
Removed serviceType as a configuration option from workflow configs ( #371 )
...
* Removed serviceType as a configuration option from workflow configs
* Removed serviceType as a configuration option from workflow configs
Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
2021-09-02 19:45:41 +05:30
Sriharsha Chintalapani
8dcbcbc171
Fix #353 : TableData should accept any type. ( #354 )
2021-09-01 20:57:17 -07:00
Sriharsha Chintalapani
193c6a782a
Fix #346 : Add displayName to Chart and Dashboard entities and use the dashboard and chart ids/unique identifiers in name field
2021-08-31 23:49:01 -07:00
Ayush Shah
150f70353c
Merge pull request #340 from open-metadata/ISSUE-339
...
Fix #339 : Add sample dashboards & charts
2021-08-31 11:01:49 +05:30
Suresh Srinivas
fd99dbdb1a
Fix #339 : Add sample dashboards & charts
2021-08-30 22:13:45 -07:00
parthp2107
f9bda790c2
added auth0 integration and auth0 integration docs ( #249 )
...
* added auth0 integration and auth0 integration docs
* added client_id check and minor changes
* Update JwtFilter.java
* UI implementation for Auth0
* added auth0 integration and auth0 integration docs
* added client_id check and minor changes
* Update JwtFilter.java
* UI implementation for Auth0
* added exception in JwtFilter
* Addressing comments
* Update JwtFilter.java
* Addressing comments
* Addressing Comments
* Addressig comments
* post merge fixes
Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
Co-authored-by: darth-coder00 <aashit@getcollate.io>
2021-08-31 10:17:23 +05:30
Suresh Srinivas
bd0c65f3e4
Fix #331 : Integrate superset connector to OpenMetadata APIs
2021-08-27 21:33:42 -07:00
parthp2107
5fe66b8535
modification in postgres source and cleaning of status reports ( #324 )
...
Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
2021-08-27 19:16:11 +05:30
