Mayur Singal
7760663b22
MINOR: Change ingestion licence header ( #20549 )
2025-04-03 10:39:47 +05:30
Pere Miquel Brull
2b32808011
MINOR - Upgrade Airflow to 2.10.5 ( #19840 )
...
* MINOR - Bump Ingestion versions
* MINOR - Airflow bump
* MINOR - Set Airflow 2.10.5
2025-02-20 17:11:38 +01:00
Pere Miquel Brull
69c9102da1
MINOR - Bump Ingestion versions ( #19836 )
...
* MINOR - Bump Ingestion versions
* MINOR - Bump Ingestion versions
* fix
* fix db_scheme for airflow +2.9.1
* fix
2025-02-18 07:56:46 +01:00
Pere Miquel Brull
d8e2187980
#15243 - Pydantic V2 & Airflow 2.9 ( #16480 )
...
* pydantic v2
* pydanticv2
* fix parser
* fix annotated
* fix model dumping
* mysql ingestion
* clean root models
* clean root models
* bump airflow
* bump airflow
* bump airflow
* optionals
* optionals
* optionals
* jdk
* airflow migrate
* fab provider
* fab provider
* fab provider
* some more fixes
* fixing tests and imports
* model_dump and model_validate
* model_dump and model_validate
* model_dump and model_validate
* union
* pylint
* pylint
* integration tests
* fix CostAnalysisReportData
* integration tests
* tests
* missing defaults
* missing defaults
2024-06-05 21:18:37 +02:00
Pere Miquel Brull
4b4a7d70aa
MINOR - Add default to ingestion dependency sh ( #16021 )
...
* MINOR - Add default to ingestion dependency sh
* MINOR - Add default to ingestion dependency sh
2024-04-25 14:18:41 +02:00
Ayush Shah
9c6d202555
Add Sample data, modify regex pattern ( #14467 )
2024-01-11 14:23:33 +05:30
Akash Jain
6eefd51180
fix: AIRFLOW_DB_PROPERTIES default value in docker compose files ( #14274 )
2023-12-06 19:56:19 +05:30
Preet Shah
20cf8f63ab
Update ingestion_dependency.sh ( #14139 )
...
* Update ingestion_dependency.sh
* Add parameter for DB connection properties
---------
Co-authored-by: Akash Jain <15995028+akash-jain-10@users.noreply.github.com>
2023-12-04 18:09:42 +05:30
Harpreet
b50f6f8939
#14028 : fix the sed command issue in ingestion dep script ( #14029 )
...
* fix the sed command issue in ingestion dep script
* remove sql_alchemy_conn from airflow.cfg and use env variable
* try execute ingestion dep script without source
2023-12-03 11:27:36 -08:00
Pere Miquel Brull
339abc5bf3
Prepare Ingestion Base Docker image ( #8065 )
...
Prepare Ingestion Base Docker image (#8065 )
2022-10-11 07:50:49 +02:00
Nahuel
301433e0d0
Airflow container can not be restarted ( #7615 )
2022-09-20 14:37:39 -07:00
Akash Jain
fc8312c5fb
feat: Refactor openmetadata/ingestion docker image ( #7456 )
...
* centralize openmetadata/ingestion docker image
* update volume mappings; fix ingestion tag!
* fix PR comments
* fix cypress mysql tests!
2022-09-19 09:20:54 +05:30
Nahuel
db41be4c4a
Fix#6929: docker-compose services should wait until OpenMetadata DB is created ( #7025 )
...
* Added the docker compose for postgres
* Added changes as per the comments on PR
* Airflow Connectivity to Postgres
* Changes are done as per the comments
* Replocate changes to local dockerfile
* Fixed Airflow Dockerfile
* docker-compose services should wait until OpenMetadata DB is created
* remove unnecesary wait for db service
* Add dependencies to airflow image
* Add missing dependencies to airflow Dockerfile
Co-authored-by: “Vijay” <“vijay.l@deuexsolutions.com”>
Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2022-08-30 23:55:46 +02:00
Onkar Ravgan
87c36c094f
Fix #3259 : Added option in CLI to ingest sample data ( #3616 )
...
* ISSUE-3588: Added option in cli to ingest sample data
* ISSUE-3259: Added flag in CLI to skip sample data ingestion
* Fixed security hotspots
* changed lines in dependency file
* Using datetime instead of random for dag id
* Changed variable names
Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-03-24 19:26:04 +05:30
Ayush Shah
9a620c380d
Fixing Metadata indexing issue ( #3580 )
2022-03-22 14:37:41 +05:30
Sriharsha Chintalapani
3f91ae33c5
feat(ingestion): make ingestion image runtime configurable ( #3087 )
...
Co-authored-by: Jorgen Evens <jorgen@evens.eu>
2022-03-16 22:54:14 -07:00
Ayush Shah
116f8d333d
Airflow Auth fix + Version Downgrade + Increased Docker Min Memory Limit ( #3233 )
2022-03-08 13:51:07 -08:00
Ayush Shah
0e205d93dd
Added DBT to airflow, modified tables.json - added profile data, fixed user's displayName ( #1501 )
...
* User's Name fixed, added profile data, added dbt pipeline under airflow
* service name changed
2021-12-01 08:32:29 -08:00
parthp2107
a9e70e6a89
License header update ( #1498 )
...
* updated liscense header
* updated license header
* updated license header
* updated license header
* addressing stylecheck findings
2021-12-01 12:46:28 +05:30
Ayush Shah
47a97e7be0
FQN changed from bigquery to bigquery_gcp
2021-11-29 16:57:39 +05:30
Ayush Shah
2111f6c938
Docker Commands Modified ( #1199 )
...
* Docker ingestion modified
* Command line commands updated
2021-11-16 00:25:40 -08:00
parthp2107
4a5f5e69b3
uncommenting add-user test ( #1097 )
...
* uncommenting add-user test
* quick-fix
* Ingestion script modified
* Airflow sample_usage modified, added sleep in curl commands
* adding chrome options for failing tests
* addressing reviewdog finding
* testing
* addressing reviewdog findings
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-11-14 20:16:39 -08:00
Ayush Shah
5dc3bb9297
Docker Support from Python added ( #1158 )
...
* Script modified - supports running from different locations
* Docker support from Python CLI
* Docker plugin setup.py
* Paths for latest and local dockers updated
* Resolved Comments - Docker CLI optimized, timestamp added
* help attribute added to options
* Docker clean code refactored
2021-11-12 10:30:28 -08:00
Ayush Shah
c95f1542eb
Docker fix ( #1101 )
...
* Docker Fix, User Ingestion Fix
* Docker file optimized
* ES docker version rolledback
* Healthcheck removed - incompatible with 3.9
* Docker & Airflow Config files updated with - resolved comments
* Openmetadata managed Airflow apis support added
* Dockerfile updated
* Docker file Updated with release package
* Elasticsearch allocated memory increased
2021-11-10 21:22:32 -08:00
Akash Jain
06b2f70e3e
kubernetes docker image compatibility for #1026 ( #1042 )
2021-11-03 08:18:52 -07:00
Ayush Shah
93921814af
Docker fix - latest release changes ( #983 )
...
* Docker fix - Architecture, MySQL
* Docker Airflow API Dagrun support
* Docker latest changes modified
2021-10-30 09:05:30 -07:00
Ayush Shah
85b6b72848
Airflow docker ( #762 )
...
* Airflow Docker implementation - Ingestion
* Dockerfiles modified
2021-10-14 07:46:24 -07:00
Ayush Shah
627481f181
Status record Json encoding bug fixed and pandas not found fixed ( #584 )
2021-09-25 13:54:04 -07:00
Ayush Shah
19f7904258
Pip install git path modified pylint - simplescheduler
2021-08-18 01:01:20 +05:30
Ayush Shah
042ef45ca4
Registry Cleanup
2021-08-13 20:35:17 +05:30
Ayush Shah
d9cd3e4856
Connector Dependency Cleanup and Docker Modification
2021-08-13 02:29:36 +05:30
Ayush Shah
32c7c0e4e0
Permission changes- sh file
2021-08-05 09:33:56 +05:30
Ayush Shah
982ca639b5
Docker Scripts and Commands modified
2021-08-03 01:26:20 +05:30
