16 Commits

Author SHA1 Message Date
Pere Miquel Brull
a9135e8db5
Mask queries, Table sample data and Topic sample data (#12139)
* Filter out PII queries

* Mask topic PII

* Mask topic PII

* Update sample data test

* Format

* Moved logic to repository or PIIMasker

* Fix test

* chore(ui): update sample data api endpoint for topic

---------

Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2023-06-26 11:47:39 +02:00
Onkar Ravgan
4d11db4220
Added doc in avro array and tests (#10473) 2023-03-08 20:16:50 +05:30
Sriharsha Chintalapani
b9242c1390
Fix topic indexes and migrations (#9439)
* Fix topic indexing

* Fix topic db migrations

* Fix topic db migrations

* Fix topic db migrations

* Fix test; add missing topic indexes on python side; fix ranking for tables using usage count
2022-12-21 10:33:12 +01:00
Onkar Ravgan
b539b299ee
Integrated schema parsers (#9305)
* Integrated schema parsers

* Addressed review comments

* fixed pytests
2022-12-15 16:54:55 +05:30
Sriharsha Chintalapani
38074f763b
Fix #8509: Add schema fields (#9209)
* Fix #8509: Add schema fields

* Fix #4675: Ingestion deployment from UI is broken

* Added sample data for topics

* Fixed pytests

* Address comments

* Refactored sampledata according to new schema

* Added return type

* Feat(ui)  : Add support for Avro editor (#9224)

* Feat(ui)  : Add support for Avro editor

* chore : add mock data for nested fields

* style: add group and opacity css

* add schema fields component to topic details and unit test

* Add locale keys

* Add support for edit description.

* refactor: expandableConfig

* test: add unit test for util method

* chore : make changes according to the schema change

* test : fix unit test and add new mock data

* chore : rename files

* Add row key

* chore : add default value for tags

* chore : update util method

* chore : add support for editing field tags

* chore : rename util files

* test : add unit test

* add comments

* addressing comments

* Address comments

* Added avro requirments

* Added requirement versions

* fixed versions

* protobuf version fix

* chore : rename util test file

* Fixed Dataype

* test: add unit test for schema component

Co-authored-by: Onkar Ravgan <onkar.10r@gmail.com>
Co-authored-by: Sachin Chaurasiya <sachinchaurasiyachotey87@gmail.com>
2022-12-14 19:56:37 +05:30
Mayur Singal
bafca3b7b6
Fix #6387: Clean Ingestion (#6405)
* Fix #6387: Clean Ingestion

* postgres fix

* Fixed Location in Sample Data
2022-07-29 08:51:58 +02:00
Sriharsha Chintalapani
34858f3b32
Topics: Add Sample data (#4787)
* Topics: Add Sample data

* Topics: Add Sample data

* Topics: Add Sample data

* Topics: Add Sample data

* Topics: Add Sample data
2022-05-09 12:50:54 +02:00
Pere Miquel Brull
a22ff4627f
Fix #4728 - Fix Kafka test connection (#4730)
* Fix Kafka connection

* Update defaults and required values

* Update test connection

* Add filter tests

* Fix kafka tests
2022-05-06 08:59:05 +02:00
Pere Miquel Brull
43be01b92e
Fix #4091 & #3948 - Create services correctly (#4101) 2022-04-13 18:04:52 +02:00
Sriharsha Chintalapani
7b3e459eb3
Fix #3659 Refactor Service Connection String to be specific to per service (#3804)
* Fix #3659 Refactor Service Connection String to be specific to per service

* Simplify and centralize Airflow Pipeline info for REST (#3740)

* Remove code

* Modified Configs based on refactoring schema (#3816)

* Clean WorkflowContext


Co-authored-by: pmbrull <peremiquelbrull@gmail.com>
Co-authored-by: Ayush Shah <ayush@getcollate.io>
2022-04-04 12:46:09 -07:00
Pere Miquel Brull
4d928bab80
Fix #2965 - Additional Properties set to False (#2976) 2022-02-24 08:38:56 -08:00
Ayush Shah
6175053dd4
Sample Topics fixed (#2005) 2022-01-03 11:00:04 -08:00
Sriharsha Chintalapani
eb2717b0e3
Fix #587: Ingestion: Add standalone report process to generate datasets, usage & profile and serve from standalone server (#588)
* Fix #587: Ingestion: Add standalone report process to generate datasets, usage & profile and serve from standalone server

* add localhost
2021-09-27 08:43:38 -07:00
Ayush Shah
bdad904e82
Sample Data ingestion fix (#437) 2021-09-08 16:52:54 +05:30
Ayush Shah
2b61c8ea8c
fix Issue-404 replicationFactor and Message Maximum Byte added (#419)
* fix Issue-404 replicationFactor and Message Maximum Byte added

* Cleanup Policy values changed in sample topic data
2021-09-07 12:22:51 -07:00
Sriharsha Chintalapani
d0dbcc19b7
Fix #401: Merge sample data generation into single connector (#402)
* Fix #401: Merge sample data generation into single connector

* Path for datasets modified

Co-authored-by: Ayush Shah <ayush@getcollate.io>
2021-09-05 22:35:02 +05:30