39 Commits

Author SHA1 Message Date
Mayur Singal
c758d83276
Fix #9058: Add time elapsed in seconds instead of miliseconds (#9310) 2022-12-16 08:11:30 +00:00
Mayur Singal
5476879c51
Fix #8651: Fix Hive Table Descriptions (#8652)
* Fix #8651: Fix Hive Table Descriptions

* Removed pandas
2022-11-11 09:57:32 +01:00
Onkar Ravgan
ddc66c8392
vertica query syntax issue fixed (#8463) 2022-11-01 00:16:45 +05:30
Mayur Singal
b55a46f4de
Fix #5492: Fix Trino Comments (#8113) 2022-10-14 06:39:31 +02:00
Pere Miquel Brull
4b3ca065c4
Fix #3394 - Pylint fixes and add it to CI (#8084)
Fix #3394 - Pylint fixes and add it to CI (#8084)
2022-10-11 17:15:02 +02:00
Abhishek Pandey
d40be7a9aa
postgres-sql-query-updated (#7421) 2022-09-21 07:04:06 +05:30
Teddy
45c457280b
Fix Amundsen migration & added logic for table entity owners (#7288)
* Fixed minor bugs with Amundsen ingestion and added logic to add table entity owners

* Updated version requirement for amundsen doc

* Fixed python style
2022-09-07 09:20:06 +02:00
Mayur Singal
791245c0e8
Redshift Mark Partitioned Tables (#7063) 2022-08-30 18:40:51 +02:00
Mayur Singal
5a3fcafb3b
Snowflake Mark Partitioned Tables (#7021)
Snowflake Mark Partitioned Tables (#7021)
2022-08-30 14:35:09 +02:00
Mayur Singal
b775649ba8
Postgres Mark Paertitioned Tables (#7017) 2022-08-29 17:00:22 +02:00
Abhishek Pandey
fde53809eb
added-query-log-parser-in-postgres (#6750)
added-query-log-parser-in-postgres (#6750)
2022-08-29 06:48:29 +02:00
Onkar Ravgan
8d56cb99b7
Optimize usage queries (#6837)
* Snowflake usage query optimized

* Added docs for snowflake query

* Updated docs

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-08-23 12:54:34 +05:30
Onkar Ravgan
424239840f
Added result limits to usage queries (#6748)
* Added result limits to queries

* Fixed formatting

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-08-17 08:07:39 +02:00
Pere Miquel Brull
99f0c96df4
Fix #6278 - Lineage workflow + Usage workflow (#6568)
* Fixing conflicts

* Fixing Conflicts

* Fix snowflake

* Prepare snowflake and redshift

* snowflake use a single query

* Add bigquery

* Add clickhouse

* format

* Add query log lineage source

* Add filters to core

* Prepare mssql usage

* Update schemas

* Add example workflows

* Prepare lineage deployment

* Remove lineage

* Use filters

* Prep lineage DAG

* Prepare Lineage Ingestion

* Test lineage workflow

* format

* Comments

* fixed tests

* Revert "fixed tests"

This reverts commit 0e846c7c4806098e4adf7c071d133d1df68aedf5.

* fixed get connection method

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
2022-08-12 13:24:58 +05:30
Mayur Singal
5754b1424a
Bigquery Docs Min Requirements (#6561) 2022-08-04 10:29:45 +02:00
Pere Miquel Brull
b5daf1737b
Add braces as literals (#6377) 2022-07-27 16:04:32 +02:00
Francisco J. Jurado Moreno
78dd44ed32
[TASK-6140] Add OMeta header to SQLAlchemy-based queries (#6293)
[TASK-6140] Add OMeta header to SQLAlchemy-based queries (#6293)
2022-07-23 12:31:26 +02:00
Francisco J. Jurado Moreno
0cfe6e6cb7
[ISSUE-6162] Escape wildcards on revised RS query (#6200)
[ISSUE-6162] Escape wildcards on revised RS query (#6200)
2022-07-20 06:34:26 +02:00
Francisco J. Jurado Moreno
710591f106
[ISSUE-6162] Upgrade RS SQL statement query (#6188)
* Upgrade RS SQL statement query

* Fix query, remove dupes
2022-07-19 17:46:12 +02:00
Milan Bariya
954635b6dc
FIX 5907: Ingest Trino complex type (#5931)
* FIX 5907: Ingest Trino complex type

* Included Row & Array Datatype

* Resolved review comments

Co-authored-by: ulixius9 <mayursingal9@gmail.com>
2022-07-14 11:28:50 +05:30
Mayur Singal
9da62be6f4
Fix #5806: Fix Clickhouse Usage (#6010) 2022-07-12 06:21:23 +02:00
Pere Miquel Brull
286fe5bdd7
lineage fixes (#5749) 2022-06-29 17:59:50 +02:00
Pere Miquel Brull
e3223f6a20
Simplify bigquery usage (#5728) 2022-06-29 13:58:41 +02:00
Mayur Singal
71bf32e438
Fix #5562: Handle Analysis Date for Usage (#5582)
Fix #5562: Handle Analysis Date for Usage (#5582)
2022-06-27 15:16:12 +02:00
Pere Miquel Brull
0ecc9f0da6
Fix #5459 - Remove sql-metadata in favor of sqllineage (#5494)
Fix #5459 - Remove sql-metadata in favor of sqllineage (#5494)
2022-06-21 18:02:50 +02:00
Pere Miquel Brull
0a9e36f1b4
Fix Snowflake get comments query (#5451) 2022-06-14 18:02:30 -07:00
Pere Miquel Brull
02fece727b
Fix #5307 - Define a better topology for Ingestion Sources (#5366)
Fix #5307 - Define a better topology for Ingestion Sources (#5366)
2022-06-13 12:17:54 +02:00
Mayur Singal
8ac83509a1
Fix #4549: Added Snowflake Session Query Tag (#5404)
Co-authored-by: Pere Miquel Brull <peremiquelbrull@gmail.com>
2022-06-13 10:19:48 +02:00
Onkar Ravgan
7c387d6d54
Optimized Snowflake fetch tags (#5376)
* optimized snowflake fetch tags

* fixed same named entities issue

Co-authored-by: Onkar Ravgan <onkarravgan@Onkars-MacBook-Pro.local>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
2022-06-09 22:24:40 +05:30
Ayush Shah
ed0d65efd5
Optimize and fix snowflake + usage (#4862)
Optimize and fix snowflake + usage (#4862)
2022-05-11 14:05:35 +02:00
Ayush Shah
a4212ec356
Revamp Snowflake (#4653)
Revamp Snowflake (#4653)
2022-05-09 15:54:36 +02:00
Mayur Singal
fd521763d4
Fix #4590: Usage through query log file (#4719)
* Usage through query log file

* Pick fileds if available in file

* Created Usage Souce
2022-05-09 11:37:52 +02:00
Mayur Singal
98421f2d56
fix #3423: excluded unwanted query types (#3484) 2022-03-17 09:40:44 -07:00
Ayush Shah
7c84f063e6
Minor Fixes - Ingestion - Datetime format, Looker fix (#3365)
* Minor Fixes - UTC timezone for usage, tag method modification, looker fix

* Modified core version

* Snowflake Usage Result limit and Makefile ignore env fix
2022-03-11 13:31:42 +05:30
Mayur Singal
e79be68bea
Fix #2937: added clickhouse usage connector (#2987) 2022-02-26 08:18:49 -08:00
Mayur Singal
4739aaa096
Fix #2905: Added MSSQL Usage Connector (#2948)
* Fix #2509: added mssql usage connector

* removed comment

* removed unused variables

* fixed code smell

* renamed sql variable
2022-02-24 01:01:03 +05:30
Mayur Singal
b4ebf5ce87
Fix #2591: added get_view_definition for vertica (#2650)
Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
2022-02-07 19:19:44 +05:30
Mayur Singal
bc13e343a1
issue-2572: added comments for vertica columns (#2596)
* issue-2572: added comments for vertica columns

* moved sql queries to sql_queries.py

* fixed sonar bug and sql query

Co-authored-by: Mayur SIngal <mayursingal@Mayurs-MacBook-Pro.local>
2022-02-04 15:43:27 +05:30
codingwithabhi
5714c4122b
Move sql queries (#2183)
* sql-queries-moved-separate-file

* sql-queries-moved-separate-file

* redshift-and-neo4j-queries-added

* redshift-and-neo4j-queries-added

* neo4j-query-imported

* code-formatted

Co-authored-by: = <=>
2022-01-13 14:16:54 +05:30