349c3cf7b8
Enhance `LLMNode` with multimodal capability, introducing support for image outputs. This implementation extracts base64-encoded images from LLM responses, saves them to the storage service, and records the file metadata in the `ToolFile` table. In conversations, these images are rendered as markdown-based inline images. Additionally, the images are included in the LLMNode's output as file variables, enabling subsequent nodes in the workflow to utilize them. To integrate file outputs into workflows, adjustments to the frontend code are necessary. For multimodal output functionality, updates to related model configurations are required. Currently, this capability has been applied exclusively to Google's Gemini models. Close #15814. Signed-off-by: -LAN- <laipz8200@outlook.com> Co-authored-by: -LAN- <laipz8200@outlook.com>
Dify Frontend
This is a Next.js project bootstrapped with create-next-app.
Getting Started
Run by source code
Before starting the web frontend service, please make sure the following environment is ready.
First, install the dependencies:
pnpm install
Then, configure the environment variables. Create a file named .env.local in the current directory and copy the contents from .env.example. Modify the values of these environment variables according to your requirements:
cp .env.example .env.local
# For production release, change this to PRODUCTION
NEXT_PUBLIC_DEPLOY_ENV=DEVELOPMENT
# The deployment edition, SELF_HOSTED
NEXT_PUBLIC_EDITION=SELF_HOSTED
# The base URL of console application, refers to the Console base URL of WEB service if console domain is
# different from api or web app domain.
# example: http://cloud.dify.ai/console/api
NEXT_PUBLIC_API_PREFIX=http://localhost:5001/console/api
# The URL for Web APP, refers to the Web App base URL of WEB service if web app domain is different from
# console or api domain.
# example: http://udify.app/api
NEXT_PUBLIC_PUBLIC_API_PREFIX=http://localhost:5001/api
# SENTRY
NEXT_PUBLIC_SENTRY_DSN=
Finally, run the development server:
pnpm run dev
Open http://localhost:3000 with your browser to see the result.
You can start editing the file under folder app. The page auto-updates as you edit the file.
Deploy
Deploy on server
First, build the app for production:
pnpm run build
Then, start the server:
pnpm run start
If you want to customize the host and port:
pnpm run start --port=3001 --host=0.0.0.0
If you want to customize the number of instances launched by PM2, you can configure PM2_INSTANCES in docker-compose.yaml or Dockerfile.
Storybook
This project uses Storybook for UI component development.
To start the storybook server, run:
pnpm storybook
Open http://localhost:6006 with your browser to see the result.
Lint Code
If your IDE is VSCode, rename web/.vscode/settings.example.json to web/.vscode/settings.json for lint code setting.
Test
We start to use Jest and React Testing Library for Unit Testing.
You can create a test file with a suffix of .spec beside the file that to be tested. For example, if you want to test a file named util.ts. The test file name should be util.spec.ts.
Run test:
pnpm run test
If you are not familiar with writing tests, here is some code to refer to:
Documentation
Visit https://docs.dify.ai/getting-started/readme to view the full documentation.
Community
The Dify community can be found on Discord community, where you can ask questions, voice ideas, and share your projects.