Roy
7807379bee
Remove unused ids parameter from _build_query_context function
2025-03-10 09:18:22 +00:00
yangdx
3cca18c59c
Refactor pipeline status updates and entity extraction.
...
- Let all parrallel jobs using one pipe_status objects
- Improved thread safety with pipeline_status_lock
- Only pipeline jobs can add message to pipe_status
- Marked insert_custom_chunks as deprecated
2025-03-10 16:48:59 +08:00
yangdx
adca27fae9
Merge branch 'main' into neo4j-add-min-degree
2025-03-10 02:13:49 +08:00
yangdx
c938989920
Fix llm cache save problem in json_kv storage
2025-03-09 23:33:03 +08:00
yangdx
bc42afe7b6
Unify llm_response_cache and hashing_kv, prevent creating an independent hashing_kv.
2025-03-09 22:15:26 +08:00
yangdx
c854aabde0
Add process ID to log messages for better multi-process debugging clarity
...
- Add PID to KV and Neo4j storage logs
- Add PID to query context logs
- Improve KV data count logging for llm cache
2025-03-09 15:25:10 +08:00
Roy
04fdc617bb
main_merge
2025-03-08 20:34:29 +00:00
Roy
e31c0c8f6c
Update vector query methods to support ID filtering in PostgreSQL
...
- Modified `mix_kg_vector_query` in operate.py to pass optional IDs to vector search
- Updated PostgreSQL SQL template to filter results using document IDs instead of chunk_id
- Improved query flexibility by allowing precise document selection during vector search
2025-03-08 20:25:20 +00:00
zrguo
548f9a8234
Update prompts
2025-03-09 01:21:39 +08:00
yangdx
6a969e8de4
Disable logging for graph database lock acquisition and release
2025-03-09 01:14:24 +08:00
yangdx
c5d0962872
Fix linting
2025-03-09 01:00:42 +08:00
yangdx
18c0770409
fix: duplicate nodes for same entity(label) problem in Neo4j
...
- Add entity_id field as key in Neo4j nodes
- Use entity_id for nodes retrival and upsert
2025-03-09 00:24:55 +08:00
Roy
528fb11364
Refactor vector query methods to support optional ID filtering
...
- Updated BaseVectorStorage query method signature to accept optional IDs
- Modified operate.py to pass query parameter IDs to vector storage queries
- Updated PostgreSQL vector storage SQL templates to filter results by document IDs
- Removed unused parameters and simplified query logic across multiple files
2025-03-08 15:43:17 +00:00
yangdx
73452e63fa
Add async lock for atomic graph database operations
...
• Introduced graph_db_lock mechanism
• Ensured atomic node/edge merge and insert operation
2025-03-08 22:48:12 +08:00
Roy
0ec61d6407
Update project dependencies and example test files
...
- Updated requirements.txt with latest package versions
- Added support for filtering query results by IDs in base and operate modules
- Modified PostgreSQL vector storage to include document and chunk ID fields
2025-03-07 18:45:28 +00:00
Lukas Selch
bad3781f51
Fixed entites_section_list comma error
2025-03-07 12:04:10 +01:00
zrguo
5e7ef39998
Update operate.py
2025-03-05 15:12:01 +08:00
yangdx
c0b22a8ae2
Merge branch 'main' into add-multi-worker-support
2025-03-02 02:54:57 +08:00
zrguo
4219454fab
fix format
2025-03-01 17:45:06 +08:00
yangdx
3507e894d9
Merge branch 'main' into add-multi-worker-support
2025-03-01 15:55:37 +08:00
yangdx
d704512139
Refactor shared storage module to improve async handling and naming consistency
...
• Add async support for get_namespace_data
• Rename get_update_flags to get_update_flag
• Rename set_update_flag to set_all_update_flags
• Update docstrings for clarity
• Fix typos in log messages
2025-03-01 05:01:26 +08:00
yangdx
731d820bcc
Remove redundancy set_logger function and related calls
2025-02-28 21:46:45 +08:00
yangdx
c973498c34
Fix linting
2025-02-28 21:35:04 +08:00
yangdx
8cd45161f2
feat: add history_messages to track pipeline processing progress
...
• Add shared history_messages list
• Track pipeline progress with messages
2025-02-28 13:53:40 +08:00
Huỳnh Triệu Vĩ
2f7fe5e4b6
feat: fix delete by document id
2025-02-27 23:34:57 +07:00
Zhichun Wu
d79a9d7acc
consistent format
2025-02-26 23:04:21 +08:00
Yannick Stephan
2ba3447a56
Merge branch 'HKUDS:main' into main
2025-02-26 12:10:43 +01:00
yangdx
2ebab84b72
Fix typo
2025-02-23 19:54:12 +08:00
yangdx
9546be326a
Improve entity extraction logging and metrics
2025-02-23 19:47:43 +08:00
yangdx
bf13bfcab8
Merge branch 'clear-text-before-insert' into simplify-cli-arguments
2025-02-23 18:50:48 +08:00
yangdx
4202ce8d2f
Fix entity_type string formatting(Fix PostgreSQL indexing)
...
- Remove redundant quotes
- Use consistent double quotes
2025-02-23 18:47:10 +08:00
yangdx
e935fed50e
Add automatic comment handling in .env files
2025-02-22 13:25:12 +08:00
yangdx
c95656ca87
feat: improve debug message handling with better truncation and formatting
2025-02-21 16:28:08 +08:00
yangdx
2d8a262ac3
Improve entity extraction logging with cleaner summary and verbose debug output
...
• Add count-based summary log message
• Move detailed data to verbose debug logs
2025-02-21 16:26:56 +08:00
Yannick Stephan
db5159fc24
added missing type
2025-02-20 14:47:31 +01:00
Yannick Stephan
c4562f71b9
cleanup extraction
2025-02-20 14:17:26 +01:00
Yannick Stephan
98d005dc1c
updated paralle
2025-02-19 23:32:21 +01:00
Yannick Stephan
8958046b74
cleaned code
2025-02-19 22:07:25 +01:00
Yannick Stephan
2524e02428
remove tqdm and cleaned readme and ollama
2025-02-18 19:58:03 +01:00
Yannick Stephan
613b748dfa
cleaned code
2025-02-18 09:09:12 +01:00
Yannick Stephan
53384e9a46
fixed keywords
2025-02-18 09:06:25 +01:00
Yannick Stephan
0ab8eeed14
cleaned code
2025-02-17 23:29:25 +01:00
Yannick Stephan
ba65329898
cleanup
2025-02-17 23:23:30 +01:00
Yannick Stephan
80272cbf16
fixed edge
2025-02-17 23:20:10 +01:00
Yannick Stephan
efb28c8003
fixed already edge
2025-02-17 23:16:37 +01:00
MdNazishArmanShorthillsAI
c2ff17d343
Added system prompt support in all modes
2025-02-17 16:45:00 +05:30
zrguo
fc6dff04ff
fix logger level
2025-02-17 15:10:15 +08:00
yangdx
ae37454c42
Replace verbose_debug with logger.debug for token logging.
...
- Removed unused verbose_debug import
- Updated debug logging in kg_query
- Updated debug logging in mix_kg_vector_query
- Updated debug logging in kg_query_with_keywords
2025-02-17 12:28:49 +08:00
yangdx
806eadf5dc
Add verbose debug option to control detailed debug output level
...
• Added VERBOSE env var & CLI flag
• Implemented verbose_debug() function
• Added verbose option to splash screen
• Reduced default debug output length
• Modified LLM debug logging behavior
2025-02-17 01:38:18 +08:00
yangdx
b450430109
Change log level from info to debug for token count logging
2025-02-16 22:42:53 +08:00
