KAG/kag/tools/graph_api/graph_api_abc.py
zhuzhongshu123 13cea5f6fe
feat(kag): update to v0.7 (#456)
* add think cost

* update csv scanner

* add final rerank

* add reasoner

* add iterative planner

* fix dpr search

* fix dpr search

* add reference data

* move odps import

* update requirement.txt

* update 2wiki

* add missing file

* fix markdown reader

* add iterative planning

* update version

* update runner

* update 2wiki example

* update bridge

* merge solver and solver_new

* add cur day

* writer delete

* update multi process

* add missing files

* fix report

* add chunk retrieved executor

* update try in stream runner result

* add path

* add math executor

* update hotpotqa example

* remove log

* fix python coder solver

* update hotpotqa example

* fix python coder solver

* update config

* fix bad

* add log

* remove unused code

* commit with task thought

* move kag model to common

* add default chat llm

* fix

* use static planner

* support chunk graph node

* add args

* support naive rag

* llm client support tool calls

* add default async

* add openai

* fix result

* fix markdown reader

* fix thinker

* update asyncio interface

* feat(solver): add mcp support (#444)

* 上传mcp client相关代码

* 1、完成一套mcp client的调用,从pipeline到planner、executor
2、允许json中传入多个mcp_server,通过大模型进行调用并选择
3、调通baidu_map_mcp的使用

* 1、schema

* bugfix:删减冗余代码

---------

Co-authored-by: wanxingyu.wxy <wanxingyu.wxy@antgroup.com>

* fix affairqa after solver refactor

* fix affairqa after solver refactor

* fix readme

* add params

* update version

* update mcp executor

* update mcp executor

* solver add mcp executor

* add missing file

* add mpc executor

* add executor

* x

* update

* fix requirement

* fix main llm config

* fix solver

* bugfix:修复invoke函数调用逻辑

* chg eva

* update example

* add kag layer

* add step task

* support dot refresh

* support dot refresh

* support dot refresh

* support dot refresh

* add retrieved num

* add retrieved num

* add pipelineconf

* update ppr

* update musique prompts

* update

* add to_dict for BuilderComponentData

* async build

* add deduce prompt

* add deduce prompt

* add deduce prompt

* fix reader

* add deduce prompt

* add page thinker report

* modify prmpt

* add step status

* add self cognition

* add self cognition

* add memory graph storage

* add now time

* update memory config

* add now time

* chg graph loader

* 添加prqa数据集和代码

* bugfix:prqa调用逻辑修复

* optimize:优化代码逻辑,生成答案规范化

* add retry py code

* update memory graph

* update memory graph

* fix

* fix ner

* add with_out_refer generator prompt

* fix

* close ckpt

* fix query

* fix query

* update version

* add llm checker

* add llm checker

* 1、上传evalutor.py以及修改gold_answer.json格式
2、优化代码逻辑
3、修改README.md文件

* update exp

* update exp

* rerank support

* add static rewrite query

* recall more chunks

* fix graph load

* add static rewrite query

* fix bugs

* add finish check

* add finish check

* add finish check

* add finish check

* 1、上传evalutor.py的结果
2、优化代码逻辑,优化readme文件

* add lf retry

* add memory graph api

* fix reader api

* add ner

* add metrics

* fix bug

* remove ner

* add reraise fo retry

* add edge prop to memory graph

* add memory graph

* 1、评测数据集结果修正
2、优化evaluator.py代码
3、删除结果不存在而gold_answer中有答案的问题

* 删除评测结果文件

* fix knext host addr

* async eva

* add lf prompt

* add lf prompt

* add config

* add retry

* add unknown check

* add rc result

* add rc result

* add rc result

* add rc result

* 依据kag pipeline格式修改代码逻辑并通过测试

* bugfix:删除冗余代码

* fix report prompt

* bugfix:触发重试机制

* bugfix:中文符号错误

* fix rethinker prompt

* update version to 0.6.2b78

* update version

* 1、修改evaluator.py,通过大模型计算准确率,符合最新调用逻辑
2、修改prompt,让没有回答的结果重复测试

* update affairqa for evaluate

* update affairqa for evaluate

* bugfix:修正数据集

* bugfix:修正数据集

* bugfix:修正数据集

* fix name conflict

* bugfix:删除错误问题

* bugfix:文件名命名错误导致evaluator失败

* update for affairqa eval

* bugfix:修改代码保持evaluate逻辑一致

* x

* update for affairqa readme

* remove temp eval scripts

* bugfix for math deduce

* merge 0.6.2_dev

* merge 0.6.2_dev

* fix

* update client addr

* updated version

* update for affairqa eval

* evaUtils 支持中文

* fix affairqa eval:

* remove unused example

* update kag config

* fix default value

* update readme

* fix init

* 注释信息修改,并添加部分class说明

* update example config

* Tc 0.7.0 (#459)

* 提交affairQA 代码

* fix affairqa eval

---------

Co-authored-by: zhengke.gzk <zhengke.gzk@antgroup.com>

* fix all examples

* reformat

---------

Co-authored-by: peilong <peilong.zpl@antgroup.com>
Co-authored-by: 锦呈 <zhangxinhong.zxh@antgroup.com>
Co-authored-by: wanxingyu.wxy <wanxingyu.wxy@antgroup.com>
Co-authored-by: zhengke.gzk <zhengke.gzk@antgroup.com>
2025-04-17 17:23:52 +08:00

83 lines
2.8 KiB
Python

from abc import abstractmethod
from typing import Dict, List
from kag.common.registry import Registrable
from kag.interface.solver.base_model import SPOEntity, SPOBase
from kag.interface.solver.model.one_hop_graph import (
EntityData,
OneHopGraphData,
)
from kag.tools.graph_api.model.table_model import TableData
def replace_qota(s: str):
return s.replace('"', '\\"')
def generate_label(s: SPOBase, heads: List[EntityData], schema):
if heads:
return list(set([f"{h.type}" for h in heads]))
if not isinstance(s, SPOEntity):
return ["Entity"]
std_types = s.get_entity_type_set()
std_types_with_prefix = []
for std_type in std_types:
std_type_with_prefix = schema.get_label_within_prefix(std_type)
if std_types_with_prefix != std_type:
std_types_with_prefix.append(f"`{std_type_with_prefix}`")
if len(std_types_with_prefix):
return list(set(std_types_with_prefix))
return ["Entity"]
def generate_gql_id_params(ids: List[str]):
s_biz_id_set = [f'"{replace_qota(biz_id)}"' for biz_id in ids]
return f'[{",".join(s_biz_id_set)}]'
class GraphApiABC(Registrable):
def __init__(self, **kwargs):
super().__init__(**kwargs)
@abstractmethod
def get_entity_prop_by_id(self, biz_id, label) -> Dict:
pass
@abstractmethod
def get_entity(self, entity: SPOEntity) -> List[EntityData]:
pass
@abstractmethod
def get_entity_one_hop(self, entity: EntityData) -> OneHopGraphData:
pass
@abstractmethod
def convert_spo_to_one_graph(self, table: TableData) -> Dict[str, OneHopGraphData]:
pass
@abstractmethod
def execute_dsl(self, dsl: str, **kwargs) -> TableData:
pass
@abstractmethod
def calculate_pagerank_scores(
self, target_vertex_type, start_nodes: List[Dict], top_k=10
) -> Dict:
"""
Calculate and retrieve PageRank scores for the given starting nodes.
Parameters:
target_vertex_type (str): Return target vectex type ppr score
start_nodes (list): A list containing document fragment IDs to be used as starting nodes for the PageRank algorithm.
Returns:
ppr_doc_scores (dict): A dictionary containing each document fragment ID and its corresponding PageRank score.
This method uses the PageRank algorithm in the graph store to compute scores for document fragments. If `start_nodes` is empty,
it returns an empty dictionary. Otherwise, it attempts to retrieve PageRank scores from the graph store and converts the result
into a dictionary format where keys are document fragment IDs and values are their respective PageRank scores. Any exceptions,
such as failures in running `run_pagerank_igraph_chunk`, are logged.
"""