1489 Commits

Author SHA1 Message Date
Sriharsha Chintalapani
e84de1a310
Fix #611: ElasticSearch matches are case-sensitive (#617) 2021-09-29 17:43:09 -07:00
Julia Valenti
a514934f16
Fixes docs that reference messaging service instead of dashboard or pipeline (#613) (#614) 2021-09-29 16:21:31 -07:00
Sriharsha Chintalapani
bfec0bfbed
Ingestion: Airflow integration to ingest metadata about pipelines and tasks (#609)
* [WIP] Airlfow integration

* [WIP] Airlfow integration

* [WIP] airflow integration

* [WIP] Airflow

* [WIP] Airflow

* Fix #608: Ingestion: Airflow integration to ingest metadata about pipelines and tasks

* Fix #608: Ingestion: Airflow integration to ingest metadata about pipelines and tasks

* Update DashboardServiceResource.java

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-09-29 11:32:09 -07:00
Sriharsha Chintalapani
f7da8045b4
Fix #605: Ingestion: metadata list apis should paginate (#606)
* Fix #605: Ingestion: metadata list apis should paginate

* Fix #605: Ingestion: metadata list apis should paginate, Addressing review comments
2021-09-29 00:06:08 +05:30
Sriharsha Chintalapani
0983ec7f1f
Add nested types to sample data (#601)
* Add nested types to sample data

* Add nested types to sample data
2021-09-28 12:23:34 +05:30
Sriharsha Chintalapani
3d10347aec
Fix #596: Update ingestion to new OpenMetadata column entity changes (#597) 2021-09-27 23:11:19 +05:30
Sriharsha Chintalapani
eb2717b0e3
Fix #587: Ingestion: Add standalone report process to generate datasets, usage & profile and serve from standalone server (#588)
* Fix #587: Ingestion: Add standalone report process to generate datasets, usage & profile and serve from standalone server

* add localhost
2021-09-27 08:43:38 -07:00
Ayush Shah
627481f181
Status record Json encoding bug fixed and pandas not found fixed (#584) 2021-09-25 13:54:04 -07:00
Sriharsha Chintalapani
745ae0c253
Fix #577: Users API should support put op (#578)
* Fix #577: Users API should support put op
2021-09-24 17:55:26 -07:00
parthp2107
06810cdec1
Fix #432:Added Redash Connector (#444)
* added redash connector

* added redash connector

* Added Redash Connector

* minor changes

Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
Co-authored-by: parthp2107 <parth@getcollate.io>
2021-09-22 15:09:24 +05:30
Ayush Shah
7bc2b14178
Added and Modified Redshift Pipeline (#528)
* Pylint build failure fixed

* Redshift workflow added
2021-09-19 21:16:50 -07:00
Ayush Shah
7652baa00d
Setup.py Refactored, ES port fix (#521)
* Pylint build failure fixed

* Setup & dependency modified, Data profiler default to False, ES port fix

* Profiler requirements refactored

* Setup.py requirement fix

* openmetadata-ingestion version upgrade
2021-09-19 13:59:14 +05:30
Sriharsha Chintalapani
4c6c8fd446
Fix #515: Ingestion: Add ES configuration to allow port (#516) 2021-09-17 08:57:41 -07:00
Sriharsha Chintalapani
8c103bd2ad
Profiler (#496)
* profiler code

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-09-15 15:49:26 -07:00
Ayush Shah
9b04d781e0
Fixed Bigquery to support data profiling (#494) 2021-09-14 22:07:51 -07:00
Sriharsha Chintalapani
20a98aca81
Fix #396: Elastic search is not indexing usage for dashboard (#492) 2021-09-15 09:47:14 +05:30
Sriharsha Chintalapani
b7adb5dc6b
Fix #469: Add Vertica Connector (#470) 2021-09-12 21:59:31 -07:00
parthp2107
191ecb7332
added unit tests to helper and workflow classes in ingestion (#391)
* added unit tests to helper and workflow classes in ingestion

* Addressing comments

* Adressing comments

* Adressing comments

* Failing Unit tests fixed

* Workflow test and config file modified

Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-09-12 01:07:13 +05:30
Ayush Shah
d2df40cf2b
Fix #355: Tableau Implemented (#468)
* Fix #355: Tableau Implemented

* Tableau pipeline location modification
2021-09-11 11:46:10 -07:00
Sriharsha Chintalapani
ed698dc40f
Fix #466: Fix airflow example dag (#467) 2021-09-10 22:28:23 -07:00
Sriharsha Chintalapani
1c80dc246e
Fix #456: Make PII-Processor optional and independent install (#457) 2021-09-10 10:41:19 +05:30
Suresh Srinivas
30c1e5a14c
Merge pull request #447 from open-metadata/Issue-446
Fix #446: Add DataProfiler to ingestion and APIs
2021-09-09 11:25:56 -07:00
Sriharsha Chintalapani
4998d49167 Fix #446: Add DataProfiler to ingestion and APIs 2021-09-09 10:51:07 -07:00
Sriharsha Chintalapani
2369ddc858 [WIP] Fix #446: Add DataProfiler to ingestion and APIs 2021-09-08 23:55:48 -07:00
Ayush Shah
bdad904e82
Sample Data ingestion fix (#437) 2021-09-08 16:52:54 +05:30
Suresh Srinivas
328658ebea [WIP] profiler 2021-09-07 22:03:57 -07:00
Ayush Shah
2b61c8ea8c
fix Issue-404 replicationFactor and Message Maximum Byte added (#419)
* fix Issue-404 replicationFactor and Message Maximum Byte added

* Cleanup Policy values changed in sample topic data
2021-09-07 12:22:51 -07:00
parthp2107
3965b030a9
Added Salesforce Connector (#423)
* Added salesforce-connector

* minor changes

* Added Salesforce-connector

* Added Salesforce-connector

* Salesforce sample data implemented

Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-09-08 00:37:13 +05:30
Ayush Shah
657962bc4f
MSSQL sample-data query fix (#375)
* MSSQL sample-data query fix

* Query Format as per Database implemented
2021-09-06 21:03:04 -07:00
Sriharsha Chintalapani
072c2034ad
Fix #208: Add Global Suggest ElasticSearch index to capture all the entities (#406) 2021-09-05 20:56:54 -07:00
Sriharsha Chintalapani
d0dbcc19b7
Fix #401: Merge sample data generation into single connector (#402)
* Fix #401: Merge sample data generation into single connector

* Path for datasets modified

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-09-05 22:35:02 +05:30
Sriharsha Chintalapani
59c74fffcd
Fix #393: ingestion support for indexing dashboard/charts entities (#394) 2021-09-03 14:54:05 +05:30
Sriharsha Chintalapani
eff1de87ef
Fix #381: SQLAlchemy URL is causing issues in creating service (#383) 2021-09-02 20:36:18 -07:00
Ayush Shah
c9ada4ca1a
Looker Dashboard Connecter Added (#351)
* Looker Dashboard Connecter Added

* Dashboard yield fixed

* Looker Connector Method modifications
2021-09-02 20:32:03 -07:00
Sriharsha Chintalapani
044b785d15
Fix #384: Check if the elastic search index exists and make sure we have document mappings (#386) 2021-09-03 08:58:26 +05:30
parthp2107
d3d0ac19d0
Removed serviceType as a configuration option from workflow configs (#371)
* Removed serviceType as a configuration option from workflow configs

* Removed serviceType as a configuration option from workflow configs

Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
2021-09-02 19:45:41 +05:30
Sriharsha Chintalapani
8dcbcbc171
Fix #353: TableData should accept any type. (#354) 2021-09-01 20:57:17 -07:00
Sriharsha Chintalapani
193c6a782a Fix #346: Add displayName to Chart and Dashboard entities and use the dashboard and chart ids/unique identifiers in name field 2021-08-31 23:49:01 -07:00
Ayush Shah
150f70353c
Merge pull request #340 from open-metadata/ISSUE-339
Fix #339: Add sample dashboards & charts
2021-08-31 11:01:49 +05:30
Suresh Srinivas
fd99dbdb1a Fix #339: Add sample dashboards & charts 2021-08-30 22:13:45 -07:00
parthp2107
f9bda790c2
added auth0 integration and auth0 integration docs (#249)
* added auth0 integration and auth0 integration docs

* added client_id check and minor changes

* Update JwtFilter.java

* UI implementation for Auth0

* added auth0 integration and auth0 integration docs

* added client_id check and minor changes

* Update JwtFilter.java

* UI implementation for Auth0

* added exception in JwtFilter

* Addressing comments

* Update JwtFilter.java

* Addressing comments

* Addressing Comments

* Addressig comments

* post merge fixes

Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
Co-authored-by: darth-coder00 <aashit@getcollate.io>
2021-08-31 10:17:23 +05:30
Suresh Srinivas
bd0c65f3e4 Fix #331: Integrate superset connector to OpenMetadata APIs 2021-08-27 21:33:42 -07:00
parthp2107
5fe66b8535
modification in postgres source and cleaning of status reports (#324)
Co-authored-by: parthp2107 <parth.panchal@deuexsoultions.com>
2021-08-27 19:16:11 +05:30
Ayush Shah
5703a69e8c
Sample User pipeline fixed and modified imports (#322) 2021-08-27 14:18:33 +05:30
Suresh Srinivas
c36727aaf9
Merge pull request #307 from open-metadata/bigqueryusage-pipeline
Bigquery Usage Pipeline added
2021-08-26 22:24:09 -07:00
Suresh Srinivas
aa0ab362e6
Merge pull request #289 from open-metadata/issue-288
Ingestion of Sample Data fixed - #288
2021-08-26 22:23:22 -07:00
Suresh Srinivas
c790d4fc75
Merge pull request #291 from open-metadata/bigquery-optimize
Added credentials path to the bigquery config
2021-08-26 22:22:57 -07:00
Suresh Srinivas
9bff3ccba9 Fix #317: Ingestion bug introduced due to auth_token 2021-08-26 17:39:35 -07:00
Suresh Srinivas
afb0d724eb Fix #317: Ingestion bug introduced due to auth_token 2021-08-26 16:26:34 -07:00
Ayush Shah
3f4b938b20 Timestamp according to duration added 2021-08-27 02:21:41 +05:30