Tuan Anh Nguyen Dang (Tadashi_Cin)
3bd3830b8d
feat: sso login, demo mode & new mindmap support ( #644 ) bump:minor
...
* fix: update .env.example
* feat: add SSO login
* fix: update flowsetting
* fix: add requirement
* fix: refine UI
* fix: update group id-based operation
* fix: improve citation logics
* fix: UI enhancement
* fix: user_id to string in models
* fix: improve chat suggestion UI and flow
* fix: improve group id handling
* fix: improve chat suggestion
* fix: secure download for single file
* fix: file limiting in docstore
* fix: improve chat suggestion logics & language conform
* feat: add markmap and select text to highlight function
* fix: update Dockerfile
* fix: user id auto generate
* fix: default user id
* feat: add demo mode
* fix: update flowsetting
* fix: revise default params for demo
* feat: sso_app alternative
* feat: sso login demo
* feat: demo specific customization
* feat: add login using API key
* fix: disable key-based login
* fix: optimize duplicate upload
* fix: gradio routing
* fix: disable arm build for demo
* fix: revise full-text search js logic
* feat: add rate limit
* fix: update Dockerfile with new launch script
* fix: update Dockerfile
* fix: update Dockerignore
* fix: update ratelimit logic
* fix: user_id in user management page
* fix: rename conv logic
* feat: update demo hint
* fix: minor fix
* fix: highlight on long PDF load
* feat: add HF paper list
* fix: update HF papers load logic
* feat: fly config
* fix: update fly config
* fix: update paper list pull api
* fix: minor update root routing
* fix: minor update root routing
* fix: simplify login flow & paper list UI
* feat: add paper recommendation
* fix: update Dockerfile
* fix: update Dockerfile
* fix: update default model
* feat: add long context Ollama through LCOllama
* feat: espose Gradio share to env
* fix: revert customized changes
* fix: list group at app load
* fix: relocate share conv button
* fix: update launch script
* fix: update Docker CI
* feat: add Ollama model selection at first setup
* docs: update README
2025-02-02 15:19:48 +07:00
Tadashi
3006402d7e
docs: update README #none
2025-01-22 15:21:55 +07:00
Tadashi
1f9b52cab2
docs: update README #none
2025-01-22 12:58:40 +07:00
Tadashi
06d3bd4386
docs: update README #none
2025-01-21 10:50:33 +07:00
Tuan Anh Nguyen Dang (Tadashi_Cin)
1c5849c711
docs: improve installation instructions ( #620 ) #none
...
* docs: update instructions
* fix: typo in CONTRIBUTING.md (#614 )
* fix: typo in usage.md (#598 )
minor typo fix (tt -> it)
---------
Co-authored-by: Varun Sharma <contactvarun27@gmail.com>
Co-authored-by: natowi <natowi@users.noreply.github.com>
2025-01-13 07:42:47 +07:00
ian
7d8f40e841
docs: add bibtex ( #551 ) #none
2024-12-05 15:13:16 +07:00
Tadashi
d42b4773cb
docs: update mkdocs develop guide #none
2024-11-16 11:09:23 +07:00
Quang (Albert)
56c40f1c05
feat: integrate with docling ( #471 ) bump:patch
...
* feat: add docling reader implementation
* feat: expose docling to UI
* fix: improve docling output parsing
* docs: update README
---------
Co-authored-by: Tadashi <tadashi@cinnamon.is>
2024-11-16 10:04:57 +07:00
Tuan Anh Nguyen Dang (Tadashi_Cin)
6a81c77dd8
fix: update Docker file for graphrag ( #477 ) #none
2024-11-08 17:00:51 +07:00
Tuan Anh Nguyen Dang (Tadashi_Cin)
aeace96c48
feat: add lightrag support ( #474 ) bump:patch
...
* feat: add lightrag support
* docs: update README
2024-11-07 17:22:41 +07:00
Tadashi
f38dd0fe8b
docs: update README #none
2024-10-30 16:26:46 +07:00
cin-klein
66e565649e
feat: integrate nano-graphrag ( #433 )
...
* add nano graph-rag
* ignore entities for relevant context reference
* refactor and add local model as default nano-graphrag
* feat: add kotaemon llm & embedding integration with nanographrag
* fix: add env var for nano GraphRAG
---------
Co-authored-by: Tadashi <tadashi@cinnamon.is>
2024-10-30 15:32:30 +07:00
cin-jimmy
2bc1b01876
docs: update README.md ( #404 ) #none
...
* docs: update README.md
* feat: prettify commit
* docs: prettify commit
* fix: trim trailing whitespace and prettier
* docs: minor update
* docs: update resources
* docs: minor update
---------
Co-authored-by: Tadashi <tadashi@cinnamon.is>
2024-10-17 15:40:58 +07:00
Tadashi
c793856d91
docs: update README bump:patch
2024-10-15 22:45:08 +07:00
Tadashi
41966fcd5b
docs: update README for GraphRAG instructions #none
2024-10-15 08:33:30 +07:00
Tadashi
710e3cce59
docs: update README for GraphRAG instructions #none
2024-10-14 21:46:30 +07:00
Tadashi
b01fc217b2
docs: update README #none
2024-10-02 05:47:44 +07:00
Ben Dykstra
f7b6f313b5
fix: update setup instructions ( #144 ) #none
...
* activate directory to gitignore
* add my custom env to gitignore, will have to change that
* add unstructured to kotaemon pyproject.toml
* add .env to gitignore
* remove .env from tracking
* make changes to the run_macos script, update readme with more detailed instructions
* remove my personal changes from gitignore
* remove line from run_macos script
* remove option for not installing miniconda for non technical users, mark docker dependency as optional
* docs: update demo URL
* gitignore changes
* merge .env.example
* revert changes to run_macos.sh
* unstructured to advanced dependencies
* add link to unstructured system dependencies
* remove api key
* fix: skip tests when unstructured pdf not installed
* chore: loosen unstructured package version in pyproject.toml
* chore: correct syntax
---------
Co-authored-by: Tadashi <tadashi@cinnamon.is>
Co-authored-by: cin-albert <albert@cinnamon.is>
2024-09-29 22:26:02 +07:00
削微寒
0bdb9a32f2
fix HelloGitHub Badge code ( #313 )
2024-09-22 16:31:12 +07:00
Anush
e2bd78e9c4
feat: Qdrant vectorstore support ( #260 )
...
* feat: Qdrant vectorstore support
* chore: review changes
* docs: Updated README.md
2024-09-16 04:17:36 +07:00
Tadashi
cbe45a4395
docs: update README
2024-09-13 10:55:16 +07:00
Tadashi
463890745c
docs: update README
2024-09-12 21:23:12 +07:00
kan_cin
d3fd75297f
feat: add multi-stages docker and support platform arm ( #274 )
...
* feat: add multi-stages docker and support platform arm
* refactor: pre-commit
* fix: raise ImportError (fastembed) instead of auto install
* feat: add dependencies for local llm
* feat: free disk
* feat: update README
* feat: update README
* chore: fix typo
---------
Co-authored-by: cin-niko <niko@cinnamon.is>
2024-09-12 20:25:03 +07:00
Tadashi
cd85c4935c
docs: update badge #none
2024-09-10 15:28:26 +07:00
Tadashi
9bfb5ef778
docs: fix typos
2024-09-08 21:31:05 +07:00
Tadashi
2d6c02ebea
fix: update README bump:patch
2024-09-08 21:29:40 +07:00
taprosoft
4d7f16475f
docs: update default Docker image instruction
2024-09-06 03:07:42 +00:00
Tadashi
92f6b8e1bf
fix: update README (bump:patch)
2024-09-04 08:05:21 +07:00
Tadashi
7daa9eb149
docs: update demo URL (bump:minor)
2024-08-30 23:46:18 +07:00
Tuan Anh Nguyen Dang (Tadashi_Cin)
09f8f91510
docs: update README ( #157 )
2024-08-30 23:29:31 +07:00
Tuan Anh Nguyen Dang (Tadashi_Cin)
9354ad8241
fix: update default settings and local model guide ( #156 )
2024-08-30 23:18:31 +07:00
ian
971ffcc9d0
add github star history ( #137 )
2024-08-28 17:19:20 +07:00
Tuan Anh Nguyen Dang (Tadashi_Cin)
2570e11501
feat: merge develop ( #123 )
...
* Support hybrid vector retrieval
* Enable figures and table reading in Azure DI
* Retrieve with multi-modal
* Fix mixing up table
* Add txt loader
* Add Anthropic Chat
* Raising error when retrieving help file
* Allow same filename for different people if private is True
* Allow declaring extra LLM vendors
* Show chunks on the File page
* Allow elasticsearch to get more docs
* Fix Cohere response (#86 )
* Fix Cohere response
* Remove Adobe pdfservice from dependency
kotaemon doesn't rely more pdfservice for its core functionality,
and pdfservice uses very out-dated dependency that causes conflict.
---------
Co-authored-by: trducng <trungduc1992@gmail.com>
* Add confidence score (#87 )
* Save question answering data as a log file
* Save the original information besides the rewritten info
* Export Cohere relevance score as confidence score
* Fix style check
* Upgrade the confidence score appearance (#90 )
* Highlight the relevance score
* Round relevance score. Get key from config instead of env
* Cohere return all scores
* Display relevance score for image
* Remove columns and rows in Excel loader which contains all NaN (#91 )
* remove columns and rows which contains all NaN
* back to multiple joiner options
* Fix style
---------
Co-authored-by: linhnguyen-cinnamon <cinmc0019@CINMC0019-LinhNguyen.local>
Co-authored-by: trducng <trungduc1992@gmail.com>
* Track retriever state
* Bump llama-index version 0.10
* feat/save-azuredi-mhtml-to-markdown (#93 )
* feat/save-azuredi-mhtml-to-markdown
* fix: replace os.path to pathlib change theflow.settings
* refactor: base on pre-commit
* chore: move the func of saving content markdown above removed_spans
---------
Co-authored-by: jacky0218 <jacky0218@github.com>
* fix: losing first chunk (#94 )
* fix: losing first chunk.
* fix: update the method of preventing losing chunks
---------
Co-authored-by: jacky0218 <jacky0218@github.com>
* fix: adding the base64 image in markdown (#95 )
* feat: more chunk info on UI
* fix: error when reindexing files
* refactor: allow more information exception trace when using gpt4v
* feat: add excel reader that treats each worksheet as a document
* Persist loader information when indexing file
* feat: allow hiding unneeded setting panels
* feat: allow specific timezone when creating conversation
* feat: add more confidence score (#96 )
* Allow a list of rerankers
* Export llm reranking score instead of filter with boolean
* Get logprobs from LLMs
* Rename cohere reranking score
* Call 2 rerankers at once
* Run QA pipeline for each chunk to get qa_score
* Display more relevance scores
* Define another LLMScoring instead of editing the original one
* Export logprobs instead of probs
* Call LLMScoring
* Get qa_score only in the final answer
* feat: replace text length with token in file list
* ui: show index name instead of id in the settings
* feat(ai): restrict the vision temperature
* fix(ui): remove the misleading message about non-retrieved evidences
* feat(ui): show the reasoning name and description in the reasoning setting page
* feat(ui): show version on the main windows
* feat(ui): show default llm name in the setting page
* fix(conf): append the result of doc in llm_scoring (#97 )
* fix: constraint maximum number of images
* feat(ui): allow filter file by name in file list page
* Fix exceeding token length error for OpenAI embeddings by chunking then averaging (#99 )
* Average embeddings in case the text exceeds max size
* Add docstring
* fix: Allow empty string when calling embedding
* fix: update trulens LLM ranking score for retrieval confidence, improve citation (#98 )
* Round when displaying not by default
* Add LLMTrulens reranking model
* Use llmtrulensscoring in pipeline
* fix: update UI display for trulen score
---------
Co-authored-by: taprosoft <tadashi@cinnamon.is>
* feat: add question decomposition & few-shot rewrite pipeline (#89 )
* Create few-shot query-rewriting. Run and display the result in info_panel
* Fix style check
* Put the functions to separate modules
* Add zero-shot question decomposition
* Fix fewshot rewriting
* Add default few-shot examples
* Fix decompose question
* Fix importing rewriting pipelines
* fix: update decompose logic in fullQA pipeline
---------
Co-authored-by: taprosoft <tadashi@cinnamon.is>
* fix: add encoding utf-8 when save temporal markdown in vectorIndex (#101 )
* fix: improve retrieval pipeline and relevant score display (#102 )
* fix: improve retrieval pipeline by extending first round top_k with multiplier
* fix: minor fix
* feat: improve UI default settings and add quick switch option for pipeline
* fix: improve agent logics (#103 )
* fix: improve agent progres display
* fix: update retrieval logic
* fix: UI display
* fix: less verbose debug log
* feat: add warning message for low confidence
* fix: LLM scoring enabled by default
* fix: minor update logics
* fix: hotfix image citation
* feat: update docx loader for handle merged table cells + handle zip file upload (#104 )
* feat: update docx loader for handle merged table cells
* feat: handle zip file
* refactor: pre-commit
* fix: escape text in download UI
* feat: optimize vector store query db (#105 )
* feat: optimize vector store query db
* feat: add file_id to chroma metadatas
* feat: remove unnecessary logs and update migrate script
* feat: iterate through file index
* fix: remove unused code
---------
Co-authored-by: taprosoft <tadashi@cinnamon.is>
* fix: add openai embedidng exponential back-off
* fix: update import download_loader
* refactor: codespell
* fix: update some default settings
* fix: update installation instruction
* fix: default chunk length in simple QA
* feat: add share converstation feature and enable retrieval history (#108 )
* feat: add share converstation feature and enable retrieval history
* fix: update share conversation UI
---------
Co-authored-by: taprosoft <tadashi@cinnamon.is>
* fix: allow exponential backoff for failed OCR call (#109 )
* fix: update default prompt when no retrieval is used
* fix: create embedding for long image chunks
* fix: add exception handling for additional table retriever
* fix: clean conversation & file selection UI
* fix: elastic search with empty doc_ids
* feat: add thumbnail PDF reader for quick multimodal QA
* feat: add thumbnail handling logic in indexing
* fix: UI text update
* fix: PDF thumb loader page number logic
* feat: add quick indexing pipeline and update UI
* feat: add conv name suggestion
* fix: minor UI change
* feat: citation in thread
* fix: add conv name suggestion in regen
* chore: add assets for usage doc
* chore: update usage doc
* feat: pdf viewer (#110 )
* feat: update pdfviewer
* feat: update missing files
* fix: update rendering logic of infor panel
* fix: improve thumbnail retrieval logic
* fix: update PDF evidence rendering logic
* fix: remove pdfjs built dist
* fix: reduce thumbnail evidence count
* chore: update gitignore
* fix: add js event on chat msg select
* fix: update css for viewer
* fix: add env var for PDFJS prebuilt
* fix: move language setting to reasoning utils
---------
Co-authored-by: phv2312 <kat87yb@gmail.com>
Co-authored-by: trducng <trungduc1992@gmail.com>
* feat: graph rag (#116 )
* fix: reload server when add/delete index
* fix: rework indexing pipeline to be able to disable vectorstore and splitter if needed
* feat: add graphRAG index with plot view
* fix: update requirement for graphRAG and lighten unnecessary packages
* feat: add knowledge network index (#118 )
* feat: add Knowledge Network index
* fix: update reader mode setting for knet
* fix: update init knet
* fix: update collection name to index pipeline
* fix: missing req
---------
Co-authored-by: jeff52415 <jeff.yang@cinnamon.is>
* fix: update info panel return for graphrag
* fix: retriever setting graphrag
* feat: local llm settings (#122 )
* feat: expose context length as reasoning setting to better fit local models
* fix: update context length setting for agents
* fix: rework threadpool llm call
* fix: fix improve indexing logic
* fix: fix improve UI
* feat: add lancedb
* fix: improve lancedb logic
* feat: add lancedb vectorstore
* fix: lighten requirement
* fix: improve lanceDB vs
* fix: improve UI
* fix: openai retry
* fix: update reqs
* fix: update launch command
* feat: update Dockerfile
* feat: add plot history
* fix: update default config
* fix: remove verbose print
* fix: update default setting
* fix: update gradio plot return
* fix: default gradio tmp
* fix: improve lancedb docstore
* fix: fix question decompose pipeline
* feat: add multimodal reader in UI
* fix: udpate docs
* fix: update default settings & docker build
* fix: update app startup
* chore: update documentation
* chore: update README
* chore: update README
---------
Co-authored-by: trducng <trungduc1992@gmail.com>
* chore: update README
* chore: update README
---------
Co-authored-by: trducng <trungduc1992@gmail.com>
Co-authored-by: cin-ace <ace@cinnamon.is>
Co-authored-by: Linh Nguyen <70562198+linhnguyen-cinnamon@users.noreply.github.com>
Co-authored-by: linhnguyen-cinnamon <cinmc0019@CINMC0019-LinhNguyen.local>
Co-authored-by: cin-jacky <101088014+jacky0218@users.noreply.github.com>
Co-authored-by: jacky0218 <jacky0218@github.com>
Co-authored-by: kan_cin <kan@cinnamon.is>
Co-authored-by: phv2312 <kat87yb@gmail.com>
Co-authored-by: jeff52415 <jeff.yang@cinnamon.is>
2024-08-26 08:50:37 +07:00
ian
86d60e1649
Update docs ( #88 )
...
Co-authored-by: ian <ian@cinnamon.is>
2024-05-31 17:49:02 +07:00
ian_Cin
654501e01c
(bump:minor) Feat: Add mechanism for user-site update and auto creating releases ( #56 )
...
* move flowsettings.py and launch.py to root
* update docs
* sync sub package versions
* rename launch.py to app.py and make run scripts work with installation package
* add update scripts
* auto version for root package
* rename authors and update doc dir
* Update auto-bump-and-release.yaml to trigger on push to main branch
* latest as branch instead of tag
* pin deps versions
* cache the changelogs
2024-05-15 16:34:50 +07:00
ian_Cin
bbe862fe47
Update docs ( #49 )
2024-04-25 17:33:19 +07:00
ian_Cin
1130aa78d1
add demo gif ( #34 )
2024-04-15 22:57:02 +07:00
ian_Cin
8985963e1e
Setup app data dir ( #32 )
...
* setup local data dir
* update readme
* update chat panel
* update help page
2024-04-13 23:26:06 +07:00
Duc Nguyen (john)
917fb0a082
Treat index id as auto-generated field ( #27 )
...
* Treat index id as auto-generated field
* fix Can't create index: KeyError: 'embedding' #28
* udpate docs
* Update requirement
* Use lighter default local embedding model
---------
Co-authored-by: ian <ian@cinnamon.is>
2024-04-13 18:29:37 +07:00
ian_Cin
4efe9c02a8
Update documentations ( #23 )
2024-04-11 19:41:45 +07:00
ian_Cin
a3bf728400
Update various docs ( #4 )
...
* rename cli tool
* remove redundant docs
* update docs
* update macos instructions
* add badges
2024-03-29 19:47:03 +07:00
Albert (Quang)
cc87aaa783
Add one-click installers for Linux, Windows, and MacOS ( #146 )
...
* feat: Add installers for linux, windows, and macos
* docs: Update README
* pre-commit fix styles
* Update installers and README
* Remove env vars check and fix paths
* Update installers:
* Remove start.py and move install and launch part back to .sh/.bat
* Add conda deactivate
* Make messages more informative
* Improve kotaemon based on insights from projects (#147 )
- Include static files in the package.
- More reliable information panel. Faster & not breaking randomly.
- Add directory upload.
- Enable zip file to upload.
- Allow setting endpoint for the OCR reader using environment variable.
* feat: Add installers for linux, windows, and macos
* docs: Update README
* pre-commit fix styles
* Update installers and README
* Remove env vars check and fix paths
* Update installers:
* Remove start.py and move install and launch part back to .sh/.bat
* Add conda deactivate
* Make messages more informative
* Make macOS installer runable and improve Windows, Linux installers
* Minor fix macos commands
* installation should pause before exit
* Update Windows installer: add a new label to exit function with error
* put install_dir to .gitignore
* chore: Add comments to clarify the 'end' labels
---------
Co-authored-by: Duc Nguyen (john) <trungduc1992@gmail.com>
Co-authored-by: ian <ian@cinnamon.is>
2024-03-06 10:59:30 +07:00
ian_Cin
a86c727869
add albert to git-secret ( #140 )
...
* add albert to git-secret
* update readme
* Limit llama-index version
* Langchain upgrade their wikipedia tool name
---------
Co-authored-by: trducng <trungduc1992@gmail.com>
2024-02-20 17:28:06 +07:00
Duc Nguyen (john)
65852b7d71
Add docx + html reader ( #139 )
2024-01-31 19:21:30 +07:00
ian_Cin
116919b346
Update docs ( #106 )
2024-01-30 18:50:17 +07:00
ian_Cin
230328c62f
Best docs Cinnamon will probably ever have ( #105 )
2023-12-20 11:30:25 +07:00
ian_Cin
2638152054
[Feat] Add support for f-string syntax in PromptTemplate ( #38 )
...
* Add support for f-string syntax in PromptTemplate
2023-10-04 16:40:09 +07:00
ian_Cin
d83c22aa4e
[AUR-395, AUR-415] Adopt Example1 Injury pipeline; add .flow() for enabling bottom-up pipeline execution ( #32 )
...
* add example1/injury pipeline example
* add dotenv
* update various api
2023-10-02 16:24:56 +07:00
ian_Cin
1061192731
[AUR-418] Add member public keys to git-secret: John, Ian, Tadashi, Jacky
2023-09-06 17:19:22 +07:00
ian_Cin
5241edbc46
[AUR-361] Setup pre-commit, pytest, GitHub actions, ssh-secret ( #3 )
...
Co-authored-by: trducng <trungduc1992@gmail.com>
2023-08-30 07:22:01 +07:00
