QuantumGhost
349c3cf7b8
feat(api): Add image multimodal support for LLMNode ( #17372 )
...
Enhance `LLMNode` with multimodal capability, introducing support for
image outputs.
This implementation extracts base64-encoded images from LLM responses,
saves them to the storage service, and records the file metadata in the
`ToolFile` table. In conversations, these images are rendered as
markdown-based inline images.
Additionally, the images are included in the LLMNode's output as
file variables, enabling subsequent nodes in the workflow to utilize them.
To integrate file outputs into workflows, adjustments to the frontend code
are necessary.
For multimodal output functionality, updates to related model configurations
are required. Currently, this capability has been applied exclusively to
Google's Gemini models.
Close #15814 .
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
2025-04-30 17:28:02 +08:00
bravomark
9bcc8041e9
fix : #18744 The model order defined in position.yaml in the Model Plugin is not taking effect. ( #18756 )
2025-04-25 16:45:48 +08:00
ZalterCitty
a1158cc946
fix: Update prompt message content types to use Literal and add union type for content ( #17136 )
...
Co-authored-by: 朱庆超 <zhuqingchao@xiaomi.com>
Co-authored-by: crazywoola <427733928@qq.com>
2025-04-22 16:17:55 +08:00
Novice
f191d372f0
fix(promptMessage): correct field_serializer implementation for content serialization ( #18458 )
2025-04-21 15:09:49 +08:00
Novice
da9269ca97
feat: structured output ( #17877 )
2025-04-18 16:33:53 +08:00
-LAN-
8e6f6d64a4
feat: re-add prompt messages to result and chunks in llm ( #17883 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-04-11 17:04:49 +08:00
yihong
f04d52c044
fix: autocorrect everything in api ( #17859 )
...
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-04-11 15:24:39 +08:00
Yongtao Huang
963b6f628a
Chore: PromptMessage is not an abstract base class ( #15965 )
2025-03-18 10:57:52 +08:00
Yeuoly
403e2d58b9
Introduce Plugins ( #13836 )
...
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Signed-off-by: -LAN- <laipz8200@outlook.com>
Signed-off-by: xhe <xw897002528@gmail.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: takatost <takatost@gmail.com>
Co-authored-by: kurokobo <kuro664@gmail.com>
Co-authored-by: Novice Lee <novicelee@NoviPro.local>
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
Co-authored-by: AkaraChen <akarachen@outlook.com>
Co-authored-by: Yi <yxiaoisme@gmail.com>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: twwu <twwu@dify.ai>
Co-authored-by: Hiroshi Fujita <fujita-h@users.noreply.github.com>
Co-authored-by: AkaraChen <85140972+AkaraChen@users.noreply.github.com>
Co-authored-by: NFish <douxc512@gmail.com>
Co-authored-by: Wu Tianwei <30284043+WTW0313@users.noreply.github.com>
Co-authored-by: 非法操作 <hjlarry@163.com>
Co-authored-by: Novice <857526207@qq.com>
Co-authored-by: Hiroki Nagai <82458324+nagaihiroki-git@users.noreply.github.com>
Co-authored-by: Gen Sato <52241300+halogen22@users.noreply.github.com>
Co-authored-by: eux <euxuuu@gmail.com>
Co-authored-by: huangzhuo1949 <167434202+huangzhuo1949@users.noreply.github.com>
Co-authored-by: huangzhuo <huangzhuo1@xiaomi.com>
Co-authored-by: lotsik <lotsik@mail.ru>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: nite-knite <nkCoding@gmail.com>
Co-authored-by: Jyong <76649700+JohnJyong@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: gakkiyomi <gakkiyomi@aliyun.com>
Co-authored-by: CN-P5 <heibai2006@gmail.com>
Co-authored-by: CN-P5 <heibai2006@qq.com>
Co-authored-by: Chuehnone <1897025+chuehnone@users.noreply.github.com>
Co-authored-by: yihong <zouzou0208@gmail.com>
Co-authored-by: Kevin9703 <51311316+Kevin9703@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Boris Feld <lothiraldan@gmail.com>
Co-authored-by: mbo <himabo@gmail.com>
Co-authored-by: mabo <mabo@aeyes.ai>
Co-authored-by: Warren Chen <warren.chen830@gmail.com>
Co-authored-by: JzoNgKVO <27049666+JzoNgKVO@users.noreply.github.com>
Co-authored-by: jiandanfeng <chenjh3@wangsu.com>
Co-authored-by: zhu-an <70234959+xhdd123321@users.noreply.github.com>
Co-authored-by: zhaoqingyu.1075 <zhaoqingyu.1075@bytedance.com>
Co-authored-by: 海狸大師 <86974027+yenslife@users.noreply.github.com>
Co-authored-by: Xu Song <xusong.vip@gmail.com>
Co-authored-by: rayshaw001 <396301947@163.com>
Co-authored-by: Ding Jiatong <dingjiatong@gmail.com>
Co-authored-by: Bowen Liang <liangbowen@gf.com.cn>
Co-authored-by: JasonVV <jasonwangiii@outlook.com>
Co-authored-by: le0zh <newlight@qq.com>
Co-authored-by: zhuxinliang <zhuxinliang@didiglobal.com>
Co-authored-by: k-zaku <zaku99@outlook.jp>
Co-authored-by: luckylhb90 <luckylhb90@gmail.com>
Co-authored-by: hobo.l <hobo.l@binance.com>
Co-authored-by: jiangbo721 <365065261@qq.com>
Co-authored-by: 刘江波 <jiangbo721@163.com>
Co-authored-by: Shun Miyazawa <34241526+miya@users.noreply.github.com>
Co-authored-by: EricPan <30651140+Egfly@users.noreply.github.com>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: sino <sino2322@gmail.com>
Co-authored-by: Jhvcc <37662342+Jhvcc@users.noreply.github.com>
Co-authored-by: lowell <lowell.hu@zkteco.in>
Co-authored-by: Boris Polonsky <BorisPolonsky@users.noreply.github.com>
Co-authored-by: Ademílson Tonato <ademilsonft@outlook.com>
Co-authored-by: Ademílson Tonato <ademilson.tonato@refurbed.com>
Co-authored-by: IWAI, Masaharu <iwaim.sub@gmail.com>
Co-authored-by: Yueh-Po Peng (Yabi) <94939112+y10ab1@users.noreply.github.com>
Co-authored-by: Jason <ggbbddjm@gmail.com>
Co-authored-by: Xin Zhang <sjhpzx@gmail.com>
Co-authored-by: yjc980121 <3898524+yjc980121@users.noreply.github.com>
Co-authored-by: heyszt <36215648+hieheihei@users.noreply.github.com>
Co-authored-by: Abdullah AlOsaimi <osaimiacc@gmail.com>
Co-authored-by: Abdullah AlOsaimi <189027247+osaimi@users.noreply.github.com>
Co-authored-by: Yingchun Lai <laiyingchun@apache.org>
Co-authored-by: Hash Brown <hi@xzd.me>
Co-authored-by: zuodongxu <192560071+zuodongxu@users.noreply.github.com>
Co-authored-by: Masashi Tomooka <tmokmss@users.noreply.github.com>
Co-authored-by: aplio <ryo.091219@gmail.com>
Co-authored-by: Obada Khalili <54270856+obadakhalili@users.noreply.github.com>
Co-authored-by: Nam Vu <zuzoovn@gmail.com>
Co-authored-by: Kei YAMAZAKI <1715090+kei-yamazaki@users.noreply.github.com>
Co-authored-by: TechnoHouse <13776377+deephbz@users.noreply.github.com>
Co-authored-by: Riddhimaan-Senapati <114703025+Riddhimaan-Senapati@users.noreply.github.com>
Co-authored-by: MaFee921 <31881301+2284730142@users.noreply.github.com>
Co-authored-by: te-chan <t-nakanome@sakura-is.co.jp>
Co-authored-by: HQidea <HQidea@users.noreply.github.com>
Co-authored-by: Joshbly <36315710+Joshbly@users.noreply.github.com>
Co-authored-by: xhe <xw897002528@gmail.com>
Co-authored-by: weiwenyan-dev <154779315+weiwenyan-dev@users.noreply.github.com>
Co-authored-by: ex_wenyan.wei <ex_wenyan.wei@tcl.com>
Co-authored-by: engchina <12236799+engchina@users.noreply.github.com>
Co-authored-by: engchina <atjapan2015@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: 呆萌闷油瓶 <253605712@qq.com>
Co-authored-by: Kemal <kemalmeler@outlook.com>
Co-authored-by: Lazy_Frog <4590648+lazyFrogLOL@users.noreply.github.com>
Co-authored-by: Yi Xiao <54782454+YIXIAO0@users.noreply.github.com>
Co-authored-by: Steven sun <98230804+Tuyohai@users.noreply.github.com>
Co-authored-by: steven <sunzwj@digitalchina.com>
Co-authored-by: Kalo Chin <91766386+fdb02983rhy@users.noreply.github.com>
Co-authored-by: Katy Tao <34019945+KatyTao@users.noreply.github.com>
Co-authored-by: depy <42985524+h4ckdepy@users.noreply.github.com>
Co-authored-by: 胡春东 <gycm520@gmail.com>
Co-authored-by: Junjie.M <118170653@qq.com>
Co-authored-by: MuYu <mr.muzea@gmail.com>
Co-authored-by: Naoki Takashima <39912547+takatea@users.noreply.github.com>
Co-authored-by: Summer-Gu <37869445+gubinjie@users.noreply.github.com>
Co-authored-by: Fei He <droxer.he@gmail.com>
Co-authored-by: ybalbert001 <120714773+ybalbert001@users.noreply.github.com>
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
Co-authored-by: douxc <7553076+douxc@users.noreply.github.com>
Co-authored-by: liuzhenghua <1090179900@qq.com>
Co-authored-by: Wu Jiayang <62842862+Wu-Jiayang@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: kimjion <45935338+kimjion@users.noreply.github.com>
Co-authored-by: AugNSo <song.tiankai@icloud.com>
Co-authored-by: llinvokerl <38915183+llinvokerl@users.noreply.github.com>
Co-authored-by: liusurong.lsr <liusurong.lsr@alibaba-inc.com>
Co-authored-by: Vasu Negi <vasu-negi@users.noreply.github.com>
Co-authored-by: Hundredwz <1808096180@qq.com>
Co-authored-by: Xiyuan Chen <52963600+GareArc@users.noreply.github.com>
2025-02-17 17:05:13 +08:00
-LAN-
413dfd5628
feat: add completion mode and context size options for LLM configuration ( #13325 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-02-07 15:08:53 +08:00
-LAN-
f9515901cc
fix: Azure AI Foundry model cannot be used in the workflow ( #13323 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-02-07 14:52:57 +08:00
yihong
56e15d09a9
feat: mypy for all type check ( #10921 )
2024-12-24 18:38:51 +08:00
-LAN-
996a9135f6
feat(llm_node): support order in text and files ( #11837 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2024-12-20 14:12:50 +08:00
非法操作
c9b4029ce7
chore: the consistency of MultiModalPromptMessageContent ( #11721 )
2024-12-17 15:01:38 +08:00
非法操作
74fdc16bd1
feat: enhance gemini models ( #11497 )
2024-12-17 12:05:13 +08:00
Bowen Liang
e79eac688a
chore(lint): sort __all__ definitions ( #11243 )
2024-12-03 13:26:33 +08:00
Bowen Liang
6c8e208ef3
chore: bump minimum supported Python version to 3.11 ( #10386 )
2024-11-24 13:28:46 +08:00
非法操作
08ac36812b
feat: support LLM process document file ( #10966 )
...
Co-authored-by: -LAN- <laipz8200@outlook.com>
2024-11-22 19:32:44 +08:00
-LAN-
c5f7d650b5
feat: Allow using file variables directly in the LLM node and support more file types. ( #10679 )
...
Co-authored-by: Joel <iamjoel007@gmail.com>
2024-11-22 16:30:22 +08:00
非法操作
033ab5490b
feat: support LLM understand video ( #9828 )
2024-11-08 13:22:52 +08:00
guogeer
70ddc0ce43
openai compatiable api usage and id ( #9800 )
...
Co-authored-by: jinqi.guo <jinqi.guo@ubtrobot.com>
2024-10-24 21:51:36 +08:00
-LAN-
e61752bd3a
feat/enhance the multi-modal support ( #8818 )
2024-10-21 10:43:49 +08:00
Bowen Liang
a1104ab97e
chore: refurish python code by applying Pylint linter rules ( #8322 )
2024-09-13 22:42:08 +08:00
Bowen Liang
c69f5b07ba
chore: apply ruff E501 line-too-long linter rule ( #8275 )
...
Co-authored-by: -LAN- <laipz8200@outlook.com>
2024-09-12 14:00:36 +08:00
Bowen Liang
2cf1187b32
chore(api/core): apply ruff reformatting ( #7624 )
2024-09-10 17:00:20 +08:00
takatost
dabfd74622
feat: Parallel Execution of Nodes in Workflows ( #8192 )
...
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: Yi <yxiaoisme@gmail.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
2024-09-10 15:23:16 +08:00
-LAN-
4313d92e6b
feat(api/core/model_runtime/entities/defaults.py): Add TOP_K in default parameters. ( #8167 )
2024-09-10 09:11:31 +08:00
非法操作
6ff7fd80a1
feat: support OPENAI json_schema ( #7258 )
2024-08-15 11:29:19 +08:00
-LAN-
23fa3dedc4
fix(core): Fix incorrect type hints. ( #5427 )
2024-06-20 15:16:21 +08:00
-LAN-
5a99aeb864
fix(core): Reorder field_validator and classmethod to fit Pydantic V2. ( #5257 )
2024-06-17 10:04:28 +08:00
Bowen Liang
f976740b57
improve: mordernizing validation by migrating pydantic from 1.x to 2.x ( #4592 )
2024-06-14 01:05:37 +08:00
takatost
b890c11c14
feat: filter empty content messages in llm node ( #3547 )
2024-04-17 13:30:33 +08:00
orangeclk
4fb9606361
fix: max_token default help info improved ( #2951 )
2024-03-25 10:07:32 +08:00
呆萌闷油瓶
8f9125b08a
fix:typo ( #2808 )
2024-03-13 13:00:46 +08:00
Yeuoly
40c646cf7a
Feat/model as tool ( #2744 )
2024-03-08 15:22:55 +08:00
Yeuoly
3e63abd335
Feat/json mode ( #2563 )
2024-02-26 23:34:40 +08:00
Charlie.Wei
300d9892a5
tts add voice choose ( #2391 )
...
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2024-02-15 22:41:18 +08:00
Bowen Liang
063191889d
chore: apply ruff's pyupgrade linter rules to modernize Python code with targeted version ( #2419 )
2024-02-09 15:21:33 +08:00
Bowen Liang
843280f82b
enhancement: introduce Ruff for Python linter for reordering and removing unused imports with automated pre-commit and sytle check ( #2366 )
2024-02-06 13:21:13 +08:00
Yeuoly
6d5b386394
Feat/blocking function call ( #2247 )
2024-01-30 15:25:37 +08:00
Charlie.Wei
6355e61eb8
tts models support ( #2033 )
...
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: Yeuoly <45712896+Yeuoly@users.noreply.github.com>
2024-01-24 01:05:37 +08:00
takatost
b582fc13c3
fix: qwen top_p min/max wrong ( #2044 )
2024-01-16 01:12:55 +08:00
Bowen Liang
cc9e74123c
improve: introduce isort for linting Python imports ( #1983 )
2024-01-12 12:34:01 +08:00
Charlie.Wei
5b24d7129e
Azure openai init ( #1929 )
...
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2024-01-09 19:17:47 +08:00
takatost
7be77c19f5
fix: default model parameter precision ( #1864 )
2024-01-03 08:52:22 +08:00
takatost
d069c668f8
Model Runtime ( #1858 )
...
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: Garfield Dai <dai.hai@foxmail.com>
Co-authored-by: chenhe <guchenhe@gmail.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
2024-01-02 23:42:00 +08:00
