diff --git a/README.md b/README.md
index 00017df55ebc9c61d60461061a0697fb57dd439d..6d8e3eb62717b7ab62d2af40ee2391a992508d1b 100644
--- a/README.md
+++ b/README.md
@@ -22,13 +22,13 @@ pinned: false
**如果喜欢这个项目，请给它一个Star；如果您发明了好用的快捷键或函数插件，欢迎发pull requests！**
If you like this project, please give it a Star. If you've come up with more useful academic shortcuts or functional plugins, feel free to open an issue or pull request. We also have a README in [English|](docs/README_EN.md)[日本語|](docs/README_JP.md)[한국어|](https://github.com/mldljyh/ko_gpt_academic)[Русский|](docs/README_RS.md)[Français](docs/README_FR.md) translated by this project itself.
-To translate this project to arbitary language with GPT, read and run [`multi_language.py`](multi_language.py) (experimental).
+To translate this project to arbitrary language with GPT, read and run [`multi_language.py`](multi_language.py) (experimental).
> **Note**
>
-> 1.请注意只有 **高亮(如红色)** 标识的函数插件（按钮）才支持读取文件，部分插件位于插件区的**下拉菜单**中。另外我们以**最高优先级**欢迎和处理任何新插件的PR。
+> 1.请注意只有 **高亮** 标识的函数插件（按钮）才支持读取文件，部分插件位于插件区的**下拉菜单**中。另外我们以**最高优先级**欢迎和处理任何新插件的PR。
>
-> 2.本项目中每个文件的功能都在自译解[`self_analysis.md`](https://github.com/binary-husky/gpt_academic/wiki/chatgpt-academic%E9%A1%B9%E7%9B%AE%E8%87%AA%E8%AF%91%E8%A7%A3%E6%8A%A5%E5%91%8A)详细说明。随着版本的迭代，您也可以随时自行点击相关函数插件，调用GPT重新生成项目的自我解析报告。常见问题汇总在[`wiki`](https://github.com/binary-husky/gpt_academic/wiki/%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98)当中。[安装方法](#installation)。
+> 2.本项目中每个文件的功能都在[自译解报告`self_analysis.md`](https://github.com/binary-husky/gpt_academic/wiki/GPT‐Academic项目自译解报告)详细说明。随着版本的迭代，您也可以随时自行点击相关函数插件，调用GPT重新生成项目的自我解析报告。常见问题[`wiki`](https://github.com/binary-husky/gpt_academic/wiki)。[安装方法](#installation) | [配置说明](https://github.com/binary-husky/gpt_academic/wiki/%E9%A1%B9%E7%9B%AE%E9%85%8D%E7%BD%AE%E8%AF%B4%E6%98%8E)。
>
> 3.本项目兼容并鼓励尝试国产大语言模型ChatGLM和Moss等等。支持多个api-key共存，可在配置文件中填写如`API_KEY="openai-key1,openai-key2,azure-key3,api2d-key4"`。需要临时更换`API_KEY`时，在输入区输入临时的`API_KEY`然后回车键提交后即可生效。
@@ -65,7 +65,8 @@ Latex论文一键校对 | [函数插件] 仿Grammarly对Latex文章进行语法
[多LLM模型](https://www.bilibili.com/video/BV1wT411p7yf)支持 | 同时被GPT3.5、GPT4、[清华ChatGLM2](https://github.com/THUDM/ChatGLM2-6B)、[复旦MOSS](https://github.com/OpenLMLab/MOSS)同时伺候的感觉一定会很不错吧？
⭐ChatGLM2微调模型 | 支持加载ChatGLM2微调模型，提供ChatGLM2微调辅助插件
更多LLM模型接入，支持[huggingface部署](https://huggingface.co/spaces/qingxu98/gpt-academic) | 加入Newbing接口(新必应)，引入清华[Jittorllms](https://github.com/Jittor/JittorLLMs)支持[LLaMA](https://github.com/facebookresearch/llama)和[盘古α](https://openi.org.cn/pangu/)
-⭐[虚空终端](https://github.com/binary-husky/void-terminal)pip包 | 脱离GUI，在Python中直接调用本项目的函数插件（开发中）
+⭐[void-terminal](https://github.com/binary-husky/void-terminal) pip包 | 脱离GUI，在Python中直接调用本项目的所有函数插件（开发中）
+⭐虚空终端插件 | [函数插件] 用自然语言，直接调度本项目其他插件
更多新功能展示 (图像生成等) …… | 见本文档结尾处 ……
</div>
@@ -114,7 +115,7 @@ cd gpt_academic
在`config.py`中，配置API KEY等设置，[点击查看特殊网络环境设置方法](https://github.com/binary-husky/gpt_academic/issues/1) 。
-(P.S. 程序运行时会优先检查是否存在名为`config_private.py`的私密配置文件，并用其中的配置覆盖`config.py`的同名配置。因此，如果您能理解我们的配置读取逻辑，我们强烈建议您在`config.py`旁边创建一个名为`config_private.py`的新配置文件，并把`config.py`中的配置转移（复制）到`config_private.py`中。`config_private.py`不受git管控，可以让您的隐私信息更加安全。P.S.项目同样支持通过`环境变量`配置大多数选项，环境变量的书写格式参考`docker-compose`文件。读取优先级: `环境变量` > `config_private.py` > `config.py`)
+(P.S. 程序运行时会优先检查是否存在名为`config_private.py`的私密配置文件，并用其中的配置覆盖`config.py`的同名配置。因此，如果您能理解我们的配置读取逻辑，我们强烈建议您在`config.py`旁边创建一个名为`config_private.py`的新配置文件，并把`config.py`中的配置转移（复制）到`config_private.py`中（仅复制您修改过的配置条目即可）。`config_private.py`不受git管控，可以让您的隐私信息更加安全。P.S.项目同样支持通过`环境变量`配置大多数选项，环境变量的书写格式参考`docker-compose`文件。读取优先级: `环境变量` > `config_private.py` > `config.py`)
3. 安装依赖
@@ -160,11 +161,14 @@ python main.py
### 安装方法II：使用Docker
+[](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-audio-assistant.yml)
+
1. 仅ChatGPT（推荐大多数人选择，等价于docker-compose方案1）
[](https://github.com/binary-husky/gpt_academic/actions/workflows/build-without-local-llms.yml)
[](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-latex.yml)
[](https://github.com/binary-husky/gpt_academic/actions/workflows/build-with-audio-assistant.yml)
+
``` sh
git clone --depth=1 https://github.com/binary-husky/gpt_academic.git # 下载项目
cd gpt_academic # 进入路径
@@ -261,10 +265,13 @@ Tip：不指定文件直接点击 `载入对话历史存档` 可以查看历史h
<img src="https://github.com/binary-husky/gpt_academic/assets/96192199/9fdcc391-f823-464f-9322-f8719677043b" height="250" >
</div>
-3. 生成报告。大部分插件都会在执行结束后，生成工作报告
+3. 虚空终端（从自然语言输入中，理解用户意图+自动调用其他插件）
+
+- 步骤一：输入 “ 请调用插件翻译PDF论文，地址为https://storage.googleapis.com/deepmind-media/alphago/AlphaGoNaturePaper.pdf ”
+- 步骤二：点击“虚空终端”
+
<div align="center">
-<img src="https://user-images.githubusercontent.com/96192199/227503770-fe29ce2c-53fd-47b0-b0ff-93805f0c2ff4.png" height="250" >
-<img src="https://user-images.githubusercontent.com/96192199/227504617-7a497bb3-0a2a-4b50-9a8a-95ae60ea7afd.png" height="250" >
+<img src="https://github.com/binary-husky/gpt_academic/assets/96192199/66f1b044-e9ff-4eed-9126-5d4f3668f1ed" width="500" >
</div>
4. 模块化功能设计，简单的接口却能支持强大的功能
@@ -311,8 +318,10 @@ Tip：不指定文件直接点击 `载入对话历史存档` 可以查看历史h
</div>
+
### II：版本:
-- version 3.5(Todo): 使用自然语言调用本项目的所有函数插件（高优先级）
+- version 3.60（todo）: 优化虚空终端，引入code interpreter和更多插件
+- version 3.50: 使用自然语言调用本项目的所有函数插件（虚空终端），支持插件分类，改进UI，设计新主题
- version 3.49: 支持百度千帆平台和文心一言
- version 3.48: 支持阿里达摩院通义千问，上海AI-Lab书生，讯飞星火
- version 3.46: 支持完全脱手操作的实时语音对话
diff --git a/app.py b/app.py
index 3463fb6b69c90a799c779df343e709624f5c4248..2da187937dc3a59a08bf7c08dd11eeb91f20be9d 100644
--- a/app.py
+++ b/app.py
@@ -7,18 +7,18 @@ def main():
from request_llm.bridge_all import predict
from toolbox import format_io, find_free_port, on_file_uploaded, on_report_generated, get_conf, ArgsGeneralWrapper, load_chat_cookies, DummyWith
# 建议您复制一个config_private.py放自己的秘密, 如API和代理网址, 避免不小心传github被别人看到
- proxies, WEB_PORT, LLM_MODEL, CONCURRENT_COUNT, AUTHENTICATION, CHATBOT_HEIGHT, LAYOUT, AVAIL_LLM_MODELS, AUTO_CLEAR_TXT = \
- get_conf('proxies', 'WEB_PORT', 'LLM_MODEL', 'CONCURRENT_COUNT', 'AUTHENTICATION', 'CHATBOT_HEIGHT', 'LAYOUT', 'AVAIL_LLM_MODELS', 'AUTO_CLEAR_TXT')
+ proxies, WEB_PORT, LLM_MODEL, CONCURRENT_COUNT, AUTHENTICATION = get_conf('proxies', 'WEB_PORT', 'LLM_MODEL', 'CONCURRENT_COUNT', 'AUTHENTICATION')
+ CHATBOT_HEIGHT, LAYOUT, AVAIL_LLM_MODELS, AUTO_CLEAR_TXT = get_conf('CHATBOT_HEIGHT', 'LAYOUT', 'AVAIL_LLM_MODELS', 'AUTO_CLEAR_TXT')
ENABLE_AUDIO, AUTO_CLEAR_TXT = get_conf('ENABLE_AUDIO', 'AUTO_CLEAR_TXT')
+
# 如果WEB_PORT是-1, 则随机选取WEB端口
PORT = find_free_port() if WEB_PORT <= 0 else WEB_PORT
- if not AUTHENTICATION: AUTHENTICATION = None
-
from check_proxy import get_current_version
from themes.theme import adjust_theme, advanced_css, theme_declaration
initial_prompt = "Serve me as a writing and programming assistant."
title_html = f"<h1 align=\"center\">GPT 学术优化 {get_current_version()}</h1>{theme_declaration}"
- description = """代码开源和更新[地址🚀](https://github.com/binary-husky/chatgpt_academic)，感谢热情的[开发者们❤️](https://github.com/binary-husky/chatgpt_academic/graphs/contributors)"""
+ description = "代码开源和更新[地址🚀](https://github.com/binary-husky/gpt_academic)，"
+ description += "感谢热情的[开发者们❤️](https://github.com/binary-husky/gpt_academic/graphs/contributors)"
# 问询记录, python 版本建议3.9+（越新越好）
import logging, uuid
@@ -35,7 +35,10 @@ def main():
# 高级函数插件
from crazy_functional import get_crazy_functions
- crazy_fns = get_crazy_functions()
+ DEFAULT_FN_GROUPS, = get_conf('DEFAULT_FN_GROUPS')
+ plugins = get_crazy_functions()
+ all_plugin_groups = list(set([g for _, plugin in plugins.items() for g in plugin['Group'].split('|')]))
+ match_group = lambda tags, groups: any([g in groups for g in tags.split('|')])
# 处理markdown文本格式的转变
gr.Chatbot.postprocess = format_io
@@ -85,25 +88,33 @@ def main():
if ("Visible" in functional[k]) and (not functional[k]["Visible"]): continue
variant = functional[k]["Color"] if "Color" in functional[k] else "secondary"
functional[k]["Button"] = gr.Button(k, variant=variant)
+ functional[k]["Button"].style(size="sm")
with gr.Accordion("函数插件区", open=True, elem_id="plugin-panel") as area_crazy_fn:
with gr.Row():
gr.Markdown("插件可读取“输入区”文本/路径作为参数（上传文件自动修正路径）")
+ with gr.Row(elem_id="input-plugin-group"):
+ plugin_group_sel = gr.Dropdown(choices=all_plugin_groups, label='', show_label=False, value=DEFAULT_FN_GROUPS,
+ multiselect=True, interactive=True, elem_classes='normal_mut_select').style(container=False)
with gr.Row():
- for k in crazy_fns:
- if not crazy_fns[k].get("AsButton", True): continue
- variant = crazy_fns[k]["Color"] if "Color" in crazy_fns[k] else "secondary"
- crazy_fns[k]["Button"] = gr.Button(k, variant=variant)
- crazy_fns[k]["Button"].style(size="sm")
+ for k, plugin in plugins.items():
+ if not plugin.get("AsButton", True): continue
+ visible = True if match_group(plugin['Group'], DEFAULT_FN_GROUPS) else False
+ variant = plugins[k]["Color"] if "Color" in plugin else "secondary"
+ plugin['Button'] = plugins[k]['Button'] = gr.Button(k, variant=variant, visible=visible).style(size="sm")
with gr.Row():
with gr.Accordion("更多函数插件", open=True):
- dropdown_fn_list = [k for k in crazy_fns.keys() if not crazy_fns[k].get("AsButton", True)]
+ dropdown_fn_list = []
+ for k, plugin in plugins.items():
+ if not match_group(plugin['Group'], DEFAULT_FN_GROUPS): continue
+ if not plugin.get("AsButton", True): dropdown_fn_list.append(k) # 排除已经是按钮的插件
+ elif plugin.get('AdvancedArgs', False): dropdown_fn_list.append(k) # 对于需要高级参数的插件，亦在下拉菜单中显示
with gr.Row():
dropdown = gr.Dropdown(dropdown_fn_list, value=r"打开插件列表", label="", show_label=False).style(container=False)
with gr.Row():
plugin_advanced_arg = gr.Textbox(show_label=True, label="高级参数输入区", visible=False,
placeholder="这里是特殊函数插件的高级参数输入区").style(container=False)
with gr.Row():
- switchy_bt = gr.Button(r"请先从插件列表中选择", variant="secondary")
+ switchy_bt = gr.Button(r"请先从插件列表中选择", variant="secondary").style(size="sm")
with gr.Row():
with gr.Accordion("点击展开“文件上传区”。上传本地文件/压缩包供函数插件调用。", open=False) as area_file_up:
file_upload = gr.Files(label="任何文件, 但推荐上传压缩文件(zip, tar)", file_count="multiple")
@@ -114,7 +125,6 @@ def main():
max_length_sl = gr.Slider(minimum=256, maximum=8192, value=4096, step=1, interactive=True, label="Local LLM MaxLength",)
checkboxes = gr.CheckboxGroup(["基础功能区", "函数插件区", "底部输入区", "输入清除键", "插件参数区"], value=["基础功能区", "函数插件区"], label="显示/隐藏功能区")
md_dropdown = gr.Dropdown(AVAIL_LLM_MODELS, value=LLM_MODEL, label="更换LLM模型/请求源").style(container=False)
-
gr.Markdown(description)
with gr.Accordion("备选输入区", open=True, visible=False, elem_id="input-panel2") as area_input_secondary:
with gr.Row():
@@ -125,6 +135,7 @@ def main():
resetBtn2 = gr.Button("重置", variant="secondary"); resetBtn2.style(size="sm")
stopBtn2 = gr.Button("停止", variant="secondary"); stopBtn2.style(size="sm")
clearBtn2 = gr.Button("清除", variant="secondary", visible=False); clearBtn2.style(size="sm")
+
# 功能区显示开关与功能区的互动
def fn_area_visibility(a):
ret = {}
@@ -162,19 +173,19 @@ def main():
click_handle = functional[k]["Button"].click(fn=ArgsGeneralWrapper(predict), inputs=[*input_combo, gr.State(True), gr.State(k)], outputs=output_combo)
cancel_handles.append(click_handle)
# 文件上传区，接收文件后与chatbot的互动
- file_upload.upload(on_file_uploaded, [file_upload, chatbot, txt, txt2, checkboxes], [chatbot, txt, txt2])
+ file_upload.upload(on_file_uploaded, [file_upload, chatbot, txt, txt2, checkboxes, cookies], [chatbot, txt, txt2, cookies])
# 函数插件-固定按钮区
- for k in crazy_fns:
- if not crazy_fns[k].get("AsButton", True): continue
- click_handle = crazy_fns[k]["Button"].click(ArgsGeneralWrapper(crazy_fns[k]["Function"]), [*input_combo, gr.State(PORT)], output_combo)
+ for k in plugins:
+ if not plugins[k].get("AsButton", True): continue
+ click_handle = plugins[k]["Button"].click(ArgsGeneralWrapper(plugins[k]["Function"]), [*input_combo, gr.State(PORT)], output_combo)
click_handle.then(on_report_generated, [cookies, file_upload, chatbot], [cookies, file_upload, chatbot])
cancel_handles.append(click_handle)
# 函数插件-下拉菜单与随变按钮的互动
def on_dropdown_changed(k):
- variant = crazy_fns[k]["Color"] if "Color" in crazy_fns[k] else "secondary"
+ variant = plugins[k]["Color"] if "Color" in plugins[k] else "secondary"
ret = {switchy_bt: gr.update(value=k, variant=variant)}
- if crazy_fns[k].get("AdvancedArgs", False): # 是否唤起高级插件参数区
- ret.update({plugin_advanced_arg: gr.update(visible=True, label=f"插件[{k}]的高级参数说明：" + crazy_fns[k].get("ArgsReminder", [f"没有提供高级参数功能说明"]))})
+ if plugins[k].get("AdvancedArgs", False): # 是否唤起高级插件参数区
+ ret.update({plugin_advanced_arg: gr.update(visible=True, label=f"插件[{k}]的高级参数说明：" + plugins[k].get("ArgsReminder", [f"没有提供高级参数功能说明"]))})
else:
ret.update({plugin_advanced_arg: gr.update(visible=False, label=f"插件[{k}]不需要高级参数。")})
return ret
@@ -185,13 +196,26 @@ def main():
# 随变按钮的回调函数注册
def route(request: gr.Request, k, *args, **kwargs):
if k in [r"打开插件列表", r"请先从插件列表中选择"]: return
- yield from ArgsGeneralWrapper(crazy_fns[k]["Function"])(request, *args, **kwargs)
+ yield from ArgsGeneralWrapper(plugins[k]["Function"])(request, *args, **kwargs)
click_handle = switchy_bt.click(route,[switchy_bt, *input_combo, gr.State(PORT)], output_combo)
click_handle.then(on_report_generated, [cookies, file_upload, chatbot], [cookies, file_upload, chatbot])
cancel_handles.append(click_handle)
# 终止按钮的回调函数注册
stopBtn.click(fn=None, inputs=None, outputs=None, cancels=cancel_handles)
stopBtn2.click(fn=None, inputs=None, outputs=None, cancels=cancel_handles)
+ plugins_as_btn = {name:plugin for name, plugin in plugins.items() if plugin.get('Button', None)}
+ def on_group_change(group_list):
+ btn_list = []
+ fns_list = []
+ if not group_list: # 处理特殊情况：没有选择任何插件组
+ return [*[plugin['Button'].update(visible=False) for _, plugin in plugins_as_btn.items()], gr.Dropdown.update(choices=[])]
+ for k, plugin in plugins.items():
+ if plugin.get("AsButton", True):
+ btn_list.append(plugin['Button'].update(visible=match_group(plugin['Group'], group_list))) # 刷新按钮
+ if plugin.get('AdvancedArgs', False): dropdown_fn_list.append(k) # 对于需要高级参数的插件，亦在下拉菜单中显示
+ elif match_group(plugin['Group'], group_list): fns_list.append(k) # 刷新下拉列表
+ return [*btn_list, gr.Dropdown.update(choices=fns_list)]
+ plugin_group_sel.select(fn=on_group_change, inputs=[plugin_group_sel], outputs=[*[plugin['Button'] for name, plugin in plugins_as_btn.items()], dropdown])
if ENABLE_AUDIO:
from crazy_functions.live_audio.audio_io import RealtimeAudioDistribution
rad = RealtimeAudioDistribution()
diff --git a/check_proxy.py b/check_proxy.py
index 474988c129d51e3f1a4fa59634f073cbda553c09..b6fe99f878df4006cb1bc0031f86ce1ef8eba563 100644
--- a/check_proxy.py
+++ b/check_proxy.py
@@ -5,7 +5,7 @@ def check_proxy(proxies):
try:
response = requests.get("https://ipapi.co/json/", proxies=proxies, timeout=4)
data = response.json()
- print(f'查询代理的地理位置，返回的结果是{data}')
+ # print(f'查询代理的地理位置，返回的结果是{data}')
if 'country_name' in data:
country = data['country_name']
result = f"代理配置 {proxies_https}, 代理所在地：{country}"
diff --git a/config.py b/config.py
index a161bb191b91e27d1b48f25e93de67c79671b70d..4225a443b5293ac05a5c4df1cbf9f8bbec236f35 100644
--- a/config.py
+++ b/config.py
@@ -47,7 +47,11 @@ API_URL_REDIRECT = {}
DEFAULT_WORKER_NUM = 3
-# 对话窗的高度
+# 色彩主题，可选 ["Default", "Chuanhu-Small-and-Beautiful", "High-Contrast"]
+THEME = "Default"
+
+
+# 对话窗的高度 （仅在LAYOUT="TOP-DOWN"时生效）
CHATBOT_HEIGHT = 1115
@@ -75,8 +79,26 @@ MAX_RETRY = 2
LLM_MODEL = "gpt-3.5-turbo" # 可选 "chatglm"
AVAIL_LLM_MODELS = ["gpt-3.5-turbo", "gpt-4", "api2d-gpt-4", "api2d-gpt-3.5-turbo", "spark", "azure-gpt-3.5"]
-# ChatGLM(2) Finetune Model Path （如果使用ChatGLM2微调模型，需要把"chatglmft"加入AVAIL_LLM_MODELS中）
-ChatGLM_PTUNING_CHECKPOINT = "" # 例如"/home/hmp/ChatGLM2-6B/ptuning/output/6b-pt-128-1e-2/checkpoint-100"
+# 插件分类默认选项
+DEFAULT_FN_GROUPS = ['对话', '编程', '学术']
+
+
+# 模型选择是 (注意: LLM_MODEL是默认选中的模型, 它*必须*被包含在AVAIL_LLM_MODELS列表中 )
+LLM_MODEL = "gpt-3.5-turbo" # 可选 ↓↓↓
+AVAIL_LLM_MODELS = ["gpt-3.5-turbo-16k", "gpt-3.5-turbo", "azure-gpt-3.5", "api2d-gpt-3.5-turbo",
+ "gpt-4", "api2d-gpt-4", "chatglm", "moss", "newbing", "stack-claude"]
+# P.S. 其他可用的模型还包括 ["qianfan", "llama2", "qwen", "gpt-3.5-turbo-0613", "gpt-3.5-turbo-16k-0613",
+# "spark", "sparkv2", "chatglm_onnx", "claude-1-100k", "claude-2", "internlm", "jittorllms_pangualpha", "jittorllms_llama"]
+
+
+# 百度千帆（LLM_MODEL="qianfan"）
+BAIDU_CLOUD_API_KEY = ''
+BAIDU_CLOUD_SECRET_KEY = ''
+BAIDU_CLOUD_QIANFAN_MODEL = 'ERNIE-Bot' # 可选 "ERNIE-Bot"(文心一言), "ERNIE-Bot-turbo", "BLOOMZ-7B", "Llama-2-70B-Chat", "Llama-2-13B-Chat", "Llama-2-7B-Chat"
+
+
+# 如果使用ChatGLM2微调模型，请把 LLM_MODEL="chatglmft"，并在此处指定模型路径
+CHATGLM_PTUNING_CHECKPOINT = "" # 例如"/home/hmp/ChatGLM2-6B/ptuning/output/6b-pt-128-1e-2/checkpoint-100"
# 本地LLM模型如ChatGLM的执行方式 CPU/GPU
@@ -92,10 +114,6 @@ CONCURRENT_COUNT = 100
AUTO_CLEAR_TXT = False
-# 色彩主体，可选 ["Default", "Chuanhu-Small-and-Beautiful"]
-THEME = "Default"
-
-
# 加一个live2d装饰
ADD_WAIFU = False
@@ -161,10 +179,13 @@ HUGGINGFACE_ACCESS_TOKEN = "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"
# 获取方法：复制以下空间https://huggingface.co/spaces/qingxu98/grobid，设为public，然后GROBID_URL = "https://(你的hf用户名如qingxu98)-(你的填写的空间名如grobid).hf.space"
GROBID_URLS = [
"https://qingxu98-grobid.hf.space","https://qingxu98-grobid2.hf.space","https://qingxu98-grobid3.hf.space",
- "https://shaocongma-grobid.hf.space","https://FBR123-grobid.hf.space",
+ "https://shaocongma-grobid.hf.space","https://FBR123-grobid.hf.space", "https://yeku-grobid.hf.space",
]
+# 是否允许通过自然语言描述修改本页的配置，该功能具有一定的危险性，默认关闭
+ALLOW_RESET_CONFIG = False
+
"""
在线大模型配置关联关系示意图
@@ -182,7 +203,7 @@ GROBID_URLS = [
│ ├── AZURE_ENGINE
│ └── API_URL_REDIRECT
│
-├── "spark" 星火认知大模型
+├── "spark" 星火认知大模型 spark & sparkv2
│ ├── XFYUN_APPID
│ ├── XFYUN_API_SECRET
│ └── XFYUN_API_KEY
@@ -203,6 +224,18 @@ GROBID_URLS = [
├── NEWBING_STYLE
└── NEWBING_COOKIES
+
+用户图形界面布局依赖关系示意图
+│
+├── CHATBOT_HEIGHT 对话窗的高度
+├── CODE_HIGHLIGHT 代码高亮
+├── LAYOUT 窗口布局
+├── DARK_MODE 暗色模式 / 亮色模式
+├── DEFAULT_FN_GROUPS 插件分类默认选项
+├── THEME 色彩主题
+├── AUTO_CLEAR_TXT 是否在提交时自动清空输入框
+├── ADD_WAIFU 加一个live2d装饰
+├── ALLOW_RESET_CONFIG 是否允许通过自然语言描述修改本页的配置，该功能具有一定的危险性
插件在线服务配置依赖关系示意图
diff --git a/core_functional.py b/core_functional.py
index b04e1e0df07ce20a9700307bc898934c3f1d90cc..c4519ef8a73e3c01386a48ef3a26bf4560f3a2fc 100644
--- a/core_functional.py
+++ b/core_functional.py
@@ -63,6 +63,7 @@ def get_core_functions():
"英译中": {
"Prefix": r"翻译成地道的中文：" + "\n\n",
"Suffix": r"",
+ "Visible": False,
},
"找图片": {
"Prefix": r"我需要你找一张网络图片。使用Unsplash API(https://source.unsplash.com/960x640/?<英语关键词>)获取图片URL，" +
@@ -78,6 +79,7 @@ def get_core_functions():
"Prefix": r"Here are some bibliography items, please transform them into bibtex style." +
r"Note that, reference styles maybe more than one kind, you should transform each item correctly." +
r"Items need to be transformed:",
+ "Visible": False,
"Suffix": r"",
}
}
diff --git a/crazy_functional.py b/crazy_functional.py
index 119ff0cdb358631aa768929822bcea584f60d6ca..c6578554dc327a28f87960dd19ae68977f763c08 100644
--- a/crazy_functional.py
+++ b/crazy_functional.py
@@ -2,7 +2,6 @@ from toolbox import HotReload # HotReload 的意思是热更新，修改函数
def get_crazy_functions():
- ###################### 第一组插件 ###########################
from crazy_functions.读文章写摘要 import 读文章写摘要
from crazy_functions.生成函数注释 import 批量生成函数注释
from crazy_functions.解析项目源代码 import 解析项目本身
@@ -25,204 +24,258 @@ def get_crazy_functions():
from crazy_functions.对话历史存档 import 载入对话历史存档
from crazy_functions.对话历史存档 import 删除所有本地对话历史记录
from crazy_functions.辅助功能 import 清除缓存
-
from crazy_functions.批量Markdown翻译 import Markdown英译中
+ from crazy_functions.批量总结PDF文档 import 批量总结PDF文档
+ from crazy_functions.批量翻译PDF文档_多线程 import 批量翻译PDF文档
+ from crazy_functions.谷歌检索小助手 import 谷歌检索小助手
+ from crazy_functions.理解PDF文档内容 import 理解PDF文档内容标准文件输入
+ from crazy_functions.Latex全文润色 import Latex中文润色
+ from crazy_functions.Latex全文润色 import Latex英文纠错
+ from crazy_functions.Latex全文翻译 import Latex中译英
+ from crazy_functions.Latex全文翻译 import Latex英译中
+ from crazy_functions.批量Markdown翻译 import Markdown中译英
+ from crazy_functions.虚空终端 import 虚空终端
+
+
function_plugins = {
+ "虚空终端": {
+ "Group": "对话|编程|学术",
+ "Color": "stop",
+ "AsButton": True,
+ "Function": HotReload(虚空终端)
+ },
"解析整个Python项目": {
- "Color": "stop", # 按钮颜色
+ "Group": "编程",
+ "Color": "stop",
+ "AsButton": True,
+ "Info": "解析一个Python项目的所有源文件(.py) | 输入参数为路径",
"Function": HotReload(解析一个Python项目)
},
"载入对话历史存档（先上传存档或输入路径）": {
+ "Group": "对话",
"Color": "stop",
- "AsButton":False,
+ "AsButton": False,
+ "Info": "载入对话历史存档 | 输入参数为路径",
"Function": HotReload(载入对话历史存档)
},
- "删除所有本地对话历史记录（请谨慎操作）": {
- "AsButton":False,
+ "删除所有本地对话历史记录（谨慎操作）": {
+ "Group": "对话",
+ "AsButton": False,
+ "Info": "删除所有本地对话历史记录，谨慎操作 | 不需要输入参数",
"Function": HotReload(删除所有本地对话历史记录)
},
- "清除所有缓存文件（请谨慎操作）": {
+ "清除所有缓存文件（谨慎操作）": {
+ "Group": "对话",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "清除所有缓存文件，谨慎操作 | 不需要输入参数",
"Function": HotReload(清除缓存)
},
- "解析Jupyter Notebook文件": {
- "Color": "stop",
- "AsButton":False,
- "Function": HotReload(解析ipynb文件),
- "AdvancedArgs": True, # 调用时，唤起高级参数输入区（默认False）
- "ArgsReminder": "若输入0，则不解析notebook中的Markdown块", # 高级参数输入区的显示提示
- },
"批量总结Word文档": {
+ "Group": "学术",
"Color": "stop",
+ "AsButton": True,
+ "Info": "批量总结word文档 | 输入参数为路径",
"Function": HotReload(总结word文档)
},
"解析整个C++项目头文件": {
- "Color": "stop", # 按钮颜色
+ "Group": "编程",
+ "Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "解析一个C++项目的所有头文件(.h/.hpp) | 输入参数为路径",
"Function": HotReload(解析一个C项目的头文件)
},
"解析整个C++项目（.cpp/.hpp/.c/.h）": {
- "Color": "stop", # 按钮颜色
+ "Group": "编程",
+ "Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "解析一个C++项目的所有源文件（.cpp/.hpp/.c/.h）| 输入参数为路径",
"Function": HotReload(解析一个C项目)
},
"解析整个Go项目": {
- "Color": "stop", # 按钮颜色
+ "Group": "编程",
+ "Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "解析一个Go项目的所有源文件 | 输入参数为路径",
"Function": HotReload(解析一个Golang项目)
},
"解析整个Rust项目": {
- "Color": "stop", # 按钮颜色
+ "Group": "编程",
+ "Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "解析一个Rust项目的所有源文件 | 输入参数为路径",
"Function": HotReload(解析一个Rust项目)
},
"解析整个Java项目": {
- "Color": "stop", # 按钮颜色
+ "Group": "编程",
+ "Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "解析一个Java项目的所有源文件 | 输入参数为路径",
"Function": HotReload(解析一个Java项目)
},
"解析整个前端项目（js,ts,css等）": {
- "Color": "stop", # 按钮颜色
+ "Group": "编程",
+ "Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "解析一个前端项目的所有源文件（js,ts,css等） | 输入参数为路径",
"Function": HotReload(解析一个前端项目)
},
"解析整个Lua项目": {
- "Color": "stop", # 按钮颜色
+ "Group": "编程",
+ "Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "解析一个Lua项目的所有源文件 | 输入参数为路径",
"Function": HotReload(解析一个Lua项目)
},
"解析整个CSharp项目": {
- "Color": "stop", # 按钮颜色
+ "Group": "编程",
+ "Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "解析一个CSharp项目的所有源文件 | 输入参数为路径",
"Function": HotReload(解析一个CSharp项目)
},
+ "解析Jupyter Notebook文件": {
+ "Group": "编程",
+ "Color": "stop",
+ "AsButton": False,
+ "Info": "解析Jupyter Notebook文件 | 输入参数为路径",
+ "Function": HotReload(解析ipynb文件),
+ "AdvancedArgs": True, # 调用时，唤起高级参数输入区（默认False）
+ "ArgsReminder": "若输入0，则不解析notebook中的Markdown块", # 高级参数输入区的显示提示
+ },
"读Tex论文写摘要": {
- "Color": "stop", # 按钮颜色
+ "Group": "学术",
+ "Color": "stop",
+ "AsButton": False,
+ "Info": "读取Tex论文并写摘要 | 输入参数为路径",
"Function": HotReload(读文章写摘要)
},
- "Markdown/Readme英译中": {
- # HotReload 的意思是热更新，修改函数插件代码后，不需要重启程序，代码直接生效
+ "翻译README或MD": {
+ "Group": "编程",
"Color": "stop",
+ "AsButton": True,
+ "Info": "将Markdown翻译为中文 | 输入参数为路径或URL",
+ "Function": HotReload(Markdown英译中)
+ },
+ "翻译Markdown或README（支持Github链接）": {
+ "Group": "编程",
+ "Color": "stop",
+ "AsButton": False,
+ "Info": "将Markdown或README翻译为中文 | 输入参数为路径或URL",
"Function": HotReload(Markdown英译中)
},
"批量生成函数注释": {
- "Color": "stop", # 按钮颜色
+ "Group": "编程",
+ "Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "批量生成函数的注释 | 输入参数为路径",
"Function": HotReload(批量生成函数注释)
},
"保存当前的对话": {
+ "Group": "对话",
+ "AsButton": True,
+ "Info": "保存当前的对话 | 不需要输入参数",
"Function": HotReload(对话历史存档)
},
- "[多线程Demo] 解析此项目本身（源码自译解）": {
+ "[多线程Demo]解析此项目本身（源码自译解）": {
+ "Group": "对话|编程",
"AsButton": False, # 加入下拉菜单中
+ "Info": "多线程解析并翻译此项目的源码 | 不需要输入参数",
"Function": HotReload(解析项目本身)
},
- # "[老旧的Demo] 把本项目源代码切换成全英文": {
- # # HotReload 的意思是热更新，修改函数插件代码后，不需要重启程序，代码直接生效
- # "AsButton": False, # 加入下拉菜单中
- # "Function": HotReload(全项目切换英文)
- # },
- "[插件demo] 历史上的今天": {
- # HotReload 的意思是热更新，修改函数插件代码后，不需要重启程序，代码直接生效
+ "[插件demo]历史上的今天": {
+ "Group": "对话",
+ "AsButton": True,
+ "Info": "查看历史上的今天事件 | 不需要输入参数",
"Function": HotReload(高阶功能模板函数)
},
-
- }
- ###################### 第二组插件 ###########################
- # [第二组插件]: 经过充分测试
- from crazy_functions.批量总结PDF文档 import 批量总结PDF文档
- # from crazy_functions.批量总结PDF文档pdfminer import 批量总结PDF文档pdfminer
- from crazy_functions.批量翻译PDF文档_多线程 import 批量翻译PDF文档
- from crazy_functions.谷歌检索小助手 import 谷歌检索小助手
- from crazy_functions.理解PDF文档内容 import 理解PDF文档内容标准文件输入
- from crazy_functions.Latex全文润色 import Latex中文润色
- from crazy_functions.Latex全文润色 import Latex英文纠错
- from crazy_functions.Latex全文翻译 import Latex中译英
- from crazy_functions.Latex全文翻译 import Latex英译中
- from crazy_functions.批量Markdown翻译 import Markdown中译英
-
- function_plugins.update({
- "批量翻译PDF文档（多线程）": {
+ "精准翻译PDF论文": {
+ "Group": "学术",
"Color": "stop",
- "AsButton": True, # 加入下拉菜单中
+ "AsButton": True,
+ "Info": "精准翻译PDF论文为中文 | 输入参数为路径",
"Function": HotReload(批量翻译PDF文档)
},
"询问多个GPT模型": {
- "Color": "stop", # 按钮颜色
+ "Group": "对话",
+ "Color": "stop",
+ "AsButton": True,
"Function": HotReload(同时问询)
},
- "[测试功能] 批量总结PDF文档": {
+ "批量总结PDF文档": {
+ "Group": "学术",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
- # HotReload 的意思是热更新，修改函数插件代码后，不需要重启程序，代码直接生效
+ "Info": "批量总结PDF文档的内容 | 输入参数为路径",
"Function": HotReload(批量总结PDF文档)
},
- # "[测试功能] 批量总结PDF文档pdfminer": {
- # "Color": "stop",
- # "AsButton": False, # 加入下拉菜单中
- # "Function": HotReload(批量总结PDF文档pdfminer)
- # },
"谷歌学术检索助手（输入谷歌学术搜索页url）": {
+ "Group": "学术",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "使用谷歌学术检索助手搜索指定URL的结果 | 输入参数为谷歌学术搜索页的URL",
"Function": HotReload(谷歌检索小助手)
},
"理解PDF文档内容 （模仿ChatPDF）": {
- # HotReload 的意思是热更新，修改函数插件代码后，不需要重启程序，代码直接生效
+ "Group": "学术",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "理解PDF文档的内容并进行回答 | 输入参数为路径",
"Function": HotReload(理解PDF文档内容标准文件输入)
},
"英文Latex项目全文润色（输入路径或上传压缩包）": {
- # HotReload 的意思是热更新，修改函数插件代码后，不需要重启程序，代码直接生效
+ "Group": "学术",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "对英文Latex项目全文进行润色处理 | 输入参数为路径或上传压缩包",
"Function": HotReload(Latex英文润色)
},
"英文Latex项目全文纠错（输入路径或上传压缩包）": {
- # HotReload 的意思是热更新，修改函数插件代码后，不需要重启程序，代码直接生效
+ "Group": "学术",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "对英文Latex项目全文进行纠错处理 | 输入参数为路径或上传压缩包",
"Function": HotReload(Latex英文纠错)
},
"中文Latex项目全文润色（输入路径或上传压缩包）": {
- # HotReload 的意思是热更新，修改函数插件代码后，不需要重启程序，代码直接生效
+ "Group": "学术",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "对中文Latex项目全文进行润色处理 | 输入参数为路径或上传压缩包",
"Function": HotReload(Latex中文润色)
},
"Latex项目全文中译英（输入路径或上传压缩包）": {
- # HotReload 的意思是热更新，修改函数插件代码后，不需要重启程序，代码直接生效
+ "Group": "学术",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "对Latex项目全文进行中译英处理 | 输入参数为路径或上传压缩包",
"Function": HotReload(Latex中译英)
},
"Latex项目全文英译中（输入路径或上传压缩包）": {
- # HotReload 的意思是热更新，修改函数插件代码后，不需要重启程序，代码直接生效
+ "Group": "学术",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "对Latex项目全文进行英译中处理 | 输入参数为路径或上传压缩包",
"Function": HotReload(Latex英译中)
},
"批量Markdown中译英（输入路径或上传压缩包）": {
- # HotReload 的意思是热更新，修改函数插件代码后，不需要重启程序，代码直接生效
+ "Group": "编程",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "批量将Markdown文件中文翻译为英文 | 输入参数为路径或上传压缩包",
"Function": HotReload(Markdown中译英)
},
+ }
-
- })
-
- ###################### 第三组插件 ###########################
- # [第三组插件]: 尚未充分测试的函数插件
-
+ # -=--=- 尚未充分测试的实验性插件 & 需要额外依赖的插件 -=--=-
try:
from crazy_functions.下载arxiv论文翻译摘要 import 下载arxiv论文并翻译摘要
function_plugins.update({
"一键下载arxiv论文并翻译摘要（先在input输入编号，如1812.10695）": {
+ "Group": "学术",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ # "Info": "下载arxiv论文并翻译摘要 | 输入参数为arxiv编号如1812.10695",
"Function": HotReload(下载arxiv论文并翻译摘要)
}
})
@@ -233,16 +286,20 @@ def get_crazy_functions():
from crazy_functions.联网的ChatGPT import 连接网络回答问题
function_plugins.update({
"连接网络回答问题（输入问题后点击该插件，需要访问谷歌）": {
+ "Group": "对话",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ # "Info": "连接网络回答问题（需要访问谷歌）| 输入参数是一个问题",
"Function": HotReload(连接网络回答问题)
}
})
from crazy_functions.联网的ChatGPT_bing版 import 连接bing搜索回答问题
function_plugins.update({
"连接网络回答问题（中文Bing版，输入问题后点击该插件）": {
+ "Group": "对话",
"Color": "stop",
"AsButton": False, # 加入下拉菜单中
+ "Info": "连接网络回答问题（需要访问中文Bing）| 输入参数是一个问题",
"Function": HotReload(连接bing搜索回答问题)
}
})
@@ -253,10 +310,11 @@ def get_crazy_functions():
from crazy_functions.解析项目源代码 import 解析任意code项目
function_plugins.update({
"解析项目源代码（手动指定和筛选源代码文件类型）": {
+ "Group": "编程",
"Color": "stop",
"AsButton": False,
- "AdvancedArgs": True, # 调用时，唤起高级参数输入区（默认False）
- "ArgsReminder": "输入时用逗号隔开, *代表通配符, 加了^代表不匹配; 不输入代表全部匹配。例如: \"*.c, ^*.cpp, config.toml, ^*.toml\"", # 高级参数输入区的显示提示
+ "AdvancedArgs": True, # 调用时，唤起高级参数输入区（默认False）
+ "ArgsReminder": "输入时用逗号隔开, *代表通配符, 加了^代表不匹配; 不输入代表全部匹配。例如: \"*.c, ^*.cpp, config.toml, ^*.toml\"", # 高级参数输入区的显示提示
"Function": HotReload(解析任意code项目)
},
})
@@ -267,10 +325,11 @@ def get_crazy_functions():
from crazy_functions.询问多个大语言模型 import 同时问询_指定模型
function_plugins.update({
"询问多个GPT模型（手动指定询问哪些模型）": {
+ "Group": "对话",
"Color": "stop",
"AsButton": False,
- "AdvancedArgs": True, # 调用时，唤起高级参数输入区（默认False）
- "ArgsReminder": "支持任意数量的llm接口，用&符号分隔。例如chatglm&gpt-3.5-turbo&api2d-gpt-4", # 高级参数输入区的显示提示
+ "AdvancedArgs": True, # 调用时，唤起高级参数输入区（默认False）
+ "ArgsReminder": "支持任意数量的llm接口，用&符号分隔。例如chatglm&gpt-3.5-turbo&api2d-gpt-4", # 高级参数输入区的显示提示
"Function": HotReload(同时问询_指定模型)
},
})
@@ -281,10 +340,12 @@ def get_crazy_functions():
from crazy_functions.图片生成 import 图片生成
function_plugins.update({
"图片生成（先切换模型到openai或api2d）": {
+ "Group": "对话",
"Color": "stop",
"AsButton": False,
- "AdvancedArgs": True, # 调用时，唤起高级参数输入区（默认False）
- "ArgsReminder": "在这里输入分辨率, 如256x256（默认）", # 高级参数输入区的显示提示
+ "AdvancedArgs": True, # 调用时，唤起高级参数输入区（默认False）
+ "ArgsReminder": "在这里输入分辨率, 如256x256（默认）", # 高级参数输入区的显示提示
+ "Info": "图片生成 | 输入参数字符串，提供图像的内容",
"Function": HotReload(图片生成)
},
})
@@ -295,10 +356,12 @@ def get_crazy_functions():
from crazy_functions.总结音视频 import 总结音视频
function_plugins.update({
"批量总结音视频（输入路径或上传压缩包）": {
+ "Group": "对话",
"Color": "stop",
"AsButton": False,
"AdvancedArgs": True,
"ArgsReminder": "调用openai api 使用whisper-1模型, 目前支持的格式:mp4, m4a, wav, mpga, mpeg, mp3。此处可以输入解析提示，例如：解析为简体中文（默认）。",
+ "Info": "批量总结音频或视频 | 输入参数为路径",
"Function": HotReload(总结音视频)
}
})
@@ -309,8 +372,10 @@ def get_crazy_functions():
from crazy_functions.数学动画生成manim import 动画生成
function_plugins.update({
"数学动画生成（Manim）": {
+ "Group": "对话",
"Color": "stop",
"AsButton": False,
+ "Info": "按照自然语言描述生成一个动画 | 输入参数是一段话",
"Function": HotReload(动画生成)
}
})
@@ -321,6 +386,7 @@ def get_crazy_functions():
from crazy_functions.批量Markdown翻译 import Markdown翻译指定语言
function_plugins.update({
"Markdown翻译（手动指定语言）": {
+ "Group": "编程",
"Color": "stop",
"AsButton": False,
"AdvancedArgs": True,
@@ -335,6 +401,7 @@ def get_crazy_functions():
from crazy_functions.Langchain知识库 import 知识库问答
function_plugins.update({
"构建知识库（请先上传文件素材）": {
+ "Group": "对话",
"Color": "stop",
"AsButton": False,
"AdvancedArgs": True,
@@ -349,6 +416,7 @@ def get_crazy_functions():
from crazy_functions.Langchain知识库 import 读取知识库作答
function_plugins.update({
"知识库问答": {
+ "Group": "对话",
"Color": "stop",
"AsButton": False,
"AdvancedArgs": True,
@@ -358,11 +426,12 @@ def get_crazy_functions():
})
except:
print('Load function plugin failed')
-
+
try:
from crazy_functions.交互功能函数模板 import 交互功能模板函数
function_plugins.update({
"交互功能模板函数": {
+ "Group": "对话",
"Color": "stop",
"AsButton": False,
"Function": HotReload(交互功能模板函数)
@@ -371,24 +440,11 @@ def get_crazy_functions():
except:
print('Load function plugin failed')
- # try:
- # from crazy_functions.chatglm微调工具 import 微调数据集生成
- # function_plugins.update({
- # "黑盒模型学习: 微调数据集生成 (先上传数据集)": {
- # "Color": "stop",
- # "AsButton": False,
- # "AdvancedArgs": True,
- # "ArgsReminder": "针对数据集输入（如 绿帽子*深蓝色衬衫*黑色运动裤）给出指令，例如您可以将以下命令复制到下方: --llm_to_learn=azure-gpt-3.5 --prompt_prefix='根据下面的服装类型提示，想象一个穿着者，对这个人外貌、身处的环境、内心世界、过去经历进行描写。要求：100字以内，用第二人称。' --system_prompt=''",
- # "Function": HotReload(微调数据集生成)
- # }
- # })
- # except:
- # print('Load function plugin failed')
-
try:
from crazy_functions.Latex输出PDF结果 import Latex英文纠错加PDF对比
function_plugins.update({
"Latex英文纠错+高亮修正位置 [需Latex]": {
+ "Group": "学术",
"Color": "stop",
"AsButton": False,
"AdvancedArgs": True,
@@ -399,41 +455,110 @@ def get_crazy_functions():
from crazy_functions.Latex输出PDF结果 import Latex翻译中文并重新编译PDF
function_plugins.update({
"Arixv论文精细翻译（输入arxivID）[需Latex]": {
+ "Group": "学术",
"Color": "stop",
"AsButton": False,
"AdvancedArgs": True,
- "ArgsReminder":
- "如果有必要, 请在此处给出自定义翻译命令, 解决部分词汇翻译不准确的问题。 "+
- "例如当单词'agent'翻译不准确时, 请尝试把以下指令复制到高级参数区: " + 'If the term "agent" is used in this section, it should be translated to "智能体". ',
+ "ArgsReminder":
+ "如果有必要, 请在此处给出自定义翻译命令, 解决部分词汇翻译不准确的问题。 " +
+ "例如当单词'agent'翻译不准确时, 请尝试把以下指令复制到高级参数区: " +
+ 'If the term "agent" is used in this section, it should be translated to "智能体". ',
+ "Info": "Arixv论文精细翻译 | 输入参数arxiv论文的ID，比如1812.10695",
"Function": HotReload(Latex翻译中文并重新编译PDF)
}
})
function_plugins.update({
"本地Latex论文精细翻译（上传Latex项目）[需Latex]": {
+ "Group": "学术",
"Color": "stop",
"AsButton": False,
"AdvancedArgs": True,
- "ArgsReminder":
- "如果有必要, 请在此处给出自定义翻译命令, 解决部分词汇翻译不准确的问题。 "+
- "例如当单词'agent'翻译不准确时, 请尝试把以下指令复制到高级参数区: " + 'If the term "agent" is used in this section, it should be translated to "智能体". ',
+ "ArgsReminder":
+ "如果有必要, 请在此处给出自定义翻译命令, 解决部分词汇翻译不准确的问题。 " +
+ "例如当单词'agent'翻译不准确时, 请尝试把以下指令复制到高级参数区: " +
+ 'If the term "agent" is used in this section, it should be translated to "智能体". ',
+ "Info": "本地Latex论文精细翻译 | 输入参数是路径",
"Function": HotReload(Latex翻译中文并重新编译PDF)
}
})
except:
print('Load function plugin failed')
+ try:
+ from toolbox import get_conf
+ ENABLE_AUDIO, = get_conf('ENABLE_AUDIO')
+ if ENABLE_AUDIO:
+ from crazy_functions.语音助手 import 语音助手
+ function_plugins.update({
+ "实时音频采集": {
+ "Group": "对话",
+ "Color": "stop",
+ "AsButton": True,
+ "Info": "开始语言对话 | 没有输入参数",
+ "Function": HotReload(语音助手)
+ }
+ })
+ except:
+ print('Load function plugin failed')
+
+ try:
+ from crazy_functions.批量翻译PDF文档_NOUGAT import 批量翻译PDF文档
+ function_plugins.update({
+ "精准翻译PDF文档（NOUGAT）": {
+ "Group": "学术",
+ "Color": "stop",
+ "AsButton": False,
+ "Function": HotReload(批量翻译PDF文档)
+ }
+ })
+ except:
+ print('Load function plugin failed')
+
+
+ # try:
+ # from crazy_functions.CodeInterpreter import 虚空终端CodeInterpreter
+ # function_plugins.update({
+ # "CodeInterpreter（开发中，仅供测试）": {
+ # "Group": "编程|对话",
+ # "Color": "stop",
+ # "AsButton": False,
+ # "Function": HotReload(虚空终端CodeInterpreter)
+ # }
+ # })
+ # except:
+ # print('Load function plugin failed')
+
# try:
- # from crazy_functions.虚空终端 import 终端
+ # from crazy_functions.chatglm微调工具 import 微调数据集生成
# function_plugins.update({
- # "超级终端": {
+ # "黑盒模型学习: 微调数据集生成 (先上传数据集)": {
# "Color": "stop",
# "AsButton": False,
- # # "AdvancedArgs": True,
- # # "ArgsReminder": "",
- # "Function": HotReload(终端)
+ # "AdvancedArgs": True,
+ # "ArgsReminder": "针对数据集输入（如 绿帽子*深蓝色衬衫*黑色运动裤）给出指令，例如您可以将以下命令复制到下方: --llm_to_learn=azure-gpt-3.5 --prompt_prefix='根据下面的服装类型提示，想象一个穿着者，对这个人外貌、身处的环境、内心世界、过去经历进行描写。要求：100字以内，用第二人称。' --system_prompt=''",
+ # "Function": HotReload(微调数据集生成)
# }
# })
# except:
# print('Load function plugin failed')
+
+
+ """
+ 设置默认值:
+ - 默认 Group = 对话
+ - 默认 AsButton = True
+ - 默认 AdvancedArgs = False
+ - 默认 Color = secondary
+ """
+ for name, function_meta in function_plugins.items():
+ if "Group" not in function_meta:
+ function_plugins[name]["Group"] = '对话'
+ if "AsButton" not in function_meta:
+ function_plugins[name]["AsButton"] = True
+ if "AdvancedArgs" not in function_meta:
+ function_plugins[name]["AdvancedArgs"] = False
+ if "Color" not in function_meta:
+ function_plugins[name]["Color"] = 'secondary'
+
return function_plugins
diff --git a/crazy_functions/CodeInterpreter.py b/crazy_functions/CodeInterpreter.py
new file mode 100644
index 0000000000000000000000000000000000000000..3c970f3571f6b6a188c8166a9c26c7f07901dd21
--- /dev/null
+++ b/crazy_functions/CodeInterpreter.py
@@ -0,0 +1,231 @@
+from collections.abc import Callable, Iterable, Mapping
+from typing import Any
+from toolbox import CatchException, update_ui, gen_time_str, trimmed_format_exc, promote_file_to_downloadzone, clear_file_downloadzone
+from .crazy_utils import request_gpt_model_in_new_thread_with_ui_alive
+from .crazy_utils import input_clipping, try_install_deps
+from multiprocessing import Process, Pipe
+import os
+import time
+
+templete = """
+```python
+import ... # Put dependencies here, e.g. import numpy as np
+
+class TerminalFunction(object): # Do not change the name of the class, The name of the class must be `TerminalFunction`
+
+ def run(self, path): # The name of the function must be `run`, it takes only a positional argument.
+ # rewrite the function you have just written here
+ ...
+ return generated_file_path
+```
+"""
+
+def inspect_dependency(chatbot, history):
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+ return True
+
+def get_code_block(reply):
+ import re
+ pattern = r"```([\s\S]*?)```" # regex pattern to match code blocks
+ matches = re.findall(pattern, reply) # find all code blocks in text
+ if len(matches) == 1:
+ return matches[0].strip('python') # code block
+ for match in matches:
+ if 'class TerminalFunction' in match:
+ return match.strip('python') # code block
+ raise RuntimeError("GPT is not generating proper code.")
+
+def gpt_interact_multi_step(txt, file_type, llm_kwargs, chatbot, history):
+ # 输入
+ prompt_compose = [
+ f'Your job:\n'
+ f'1. write a single Python function, which takes a path of a `{file_type}` file as the only argument and returns a `string` containing the result of analysis or the path of generated files. \n',
+ f"2. You should write this function to perform following task: " + txt + "\n",
+ f"3. Wrap the output python function with markdown codeblock."
+ ]
+ i_say = "".join(prompt_compose)
+ demo = []
+
+ # 第一步
+ gpt_say = yield from request_gpt_model_in_new_thread_with_ui_alive(
+ inputs=i_say, inputs_show_user=i_say,
+ llm_kwargs=llm_kwargs, chatbot=chatbot, history=demo,
+ sys_prompt= r"You are a programmer."
+ )
+ history.extend([i_say, gpt_say])
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 界面更新
+
+ # 第二步
+ prompt_compose = [
+ "If previous stage is successful, rewrite the function you have just written to satisfy following templete: \n",
+ templete
+ ]
+ i_say = "".join(prompt_compose); inputs_show_user = "If previous stage is successful, rewrite the function you have just written to satisfy executable templete. "
+ gpt_say = yield from request_gpt_model_in_new_thread_with_ui_alive(
+ inputs=i_say, inputs_show_user=inputs_show_user,
+ llm_kwargs=llm_kwargs, chatbot=chatbot, history=history,
+ sys_prompt= r"You are a programmer."
+ )
+ code_to_return = gpt_say
+ history.extend([i_say, gpt_say])
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 界面更新
+
+ # # 第三步
+ # i_say = "Please list to packages to install to run the code above. Then show me how to use `try_install_deps` function to install them."
+ # i_say += 'For instance. `try_install_deps(["opencv-python", "scipy", "numpy"])`'
+ # installation_advance = yield from request_gpt_model_in_new_thread_with_ui_alive(
+ # inputs=i_say, inputs_show_user=inputs_show_user,
+ # llm_kwargs=llm_kwargs, chatbot=chatbot, history=history,
+ # sys_prompt= r"You are a programmer."
+ # )
+ # # # 第三步
+ # i_say = "Show me how to use `pip` to install packages to run the code above. "
+ # i_say += 'For instance. `pip install -r opencv-python scipy numpy`'
+ # installation_advance = yield from request_gpt_model_in_new_thread_with_ui_alive(
+ # inputs=i_say, inputs_show_user=i_say,
+ # llm_kwargs=llm_kwargs, chatbot=chatbot, history=history,
+ # sys_prompt= r"You are a programmer."
+ # )
+ installation_advance = ""
+
+ return code_to_return, installation_advance, txt, file_type, llm_kwargs, chatbot, history
+
+def make_module(code):
+ module_file = 'gpt_fn_' + gen_time_str().replace('-','_')
+ with open(f'gpt_log/{module_file}.py', 'w', encoding='utf8') as f:
+ f.write(code)
+
+ def get_class_name(class_string):
+ import re
+ # Use regex to extract the class name
+ class_name = re.search(r'class (\w+)\(', class_string).group(1)
+ return class_name
+
+ class_name = get_class_name(code)
+ return f"gpt_log.{module_file}->{class_name}"
+
+def init_module_instance(module):
+ import importlib
+ module_, class_ = module.split('->')
+ init_f = getattr(importlib.import_module(module_), class_)
+ return init_f()
+
+def for_immediate_show_off_when_possible(file_type, fp, chatbot):
+ if file_type in ['png', 'jpg']:
+ image_path = os.path.abspath(fp)
+ chatbot.append(['这是一张图片, 展示如下:',
+ f'本地文件地址: <br/>`{image_path}`<br/>'+
+ f'本地文件预览: <br/><div align="center"><img src="file={image_path}"></div>'
+ ])
+ return chatbot
+
+def subprocess_worker(instance, file_path, return_dict):
+ return_dict['result'] = instance.run(file_path)
+
+def have_any_recent_upload_files(chatbot):
+ _5min = 5 * 60
+ if not chatbot: return False # chatbot is None
+ most_recent_uploaded = chatbot._cookies.get("most_recent_uploaded", None)
+ if not most_recent_uploaded: return False # most_recent_uploaded is None
+ if time.time() - most_recent_uploaded["time"] < _5min: return True # most_recent_uploaded is new
+ else: return False # most_recent_uploaded is too old
+
+def get_recent_file_prompt_support(chatbot):
+ most_recent_uploaded = chatbot._cookies.get("most_recent_uploaded", None)
+ path = most_recent_uploaded['path']
+ return path
+
+@CatchException
+def 虚空终端CodeInterpreter(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
+ """
+ txt 输入栏用户输入的文本，例如需要翻译的一段话，再例如一个包含了待处理文件的路径
+ llm_kwargs gpt模型参数，如温度和top_p等，一般原样传递下去就行
+ plugin_kwargs 插件模型的参数，暂时没有用武之地
+ chatbot 聊天显示框的句柄，用于显示给用户
+ history 聊天历史，前情提要
+ system_prompt 给gpt的静默提醒
+ web_port 当前软件运行的端口号
+ """
+ raise NotImplementedError
+
+ # 清空历史，以免输入溢出
+ history = []; clear_file_downloadzone(chatbot)
+
+ # 基本信息：功能、贡献者
+ chatbot.append([
+ "函数插件功能？",
+ "CodeInterpreter开源版, 此插件处于开发阶段, 建议暂时不要使用, 插件初始化中 ..."
+ ])
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+
+ if have_any_recent_upload_files(chatbot):
+ file_path = get_recent_file_prompt_support(chatbot)
+ else:
+ chatbot.append(["文件检索", "没有发现任何近期上传的文件。"])
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+
+ # 读取文件
+ if ("recently_uploaded_files" in plugin_kwargs) and (plugin_kwargs["recently_uploaded_files"] == ""): plugin_kwargs.pop("recently_uploaded_files")
+ recently_uploaded_files = plugin_kwargs.get("recently_uploaded_files", None)
+ file_path = recently_uploaded_files[-1]
+ file_type = file_path.split('.')[-1]
+
+ # 粗心检查
+ if 'private_upload' in txt:
+ chatbot.append([
+ "...",
+ f"请在输入框内填写需求，然后再次点击该插件（文件路径 {file_path} 已经被记忆）"
+ ])
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+ return
+
+ # 开始干正事
+ for j in range(5): # 最多重试5次
+ try:
+ code, installation_advance, txt, file_type, llm_kwargs, chatbot, history = \
+ yield from gpt_interact_multi_step(txt, file_type, llm_kwargs, chatbot, history)
+ code = get_code_block(code)
+ res = make_module(code)
+ instance = init_module_instance(res)
+ break
+ except Exception as e:
+ chatbot.append([f"第{j}次代码生成尝试，失败了", f"错误追踪\n```\n{trimmed_format_exc()}\n```\n"])
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+
+ # 代码生成结束, 开始执行
+ try:
+ import multiprocessing
+ manager = multiprocessing.Manager()
+ return_dict = manager.dict()
+
+ p = multiprocessing.Process(target=subprocess_worker, args=(instance, file_path, return_dict))
+ # only has 10 seconds to run
+ p.start(); p.join(timeout=10)
+ if p.is_alive(): p.terminate(); p.join()
+ p.close()
+ res = return_dict['result']
+ # res = instance.run(file_path)
+ except Exception as e:
+ chatbot.append(["执行失败了", f"错误追踪\n```\n{trimmed_format_exc()}\n```\n"])
+ # chatbot.append(["如果是缺乏依赖，请参考以下建议", installation_advance])
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+ return
+
+ # 顺利完成，收尾
+ res = str(res)
+ if os.path.exists(res):
+ chatbot.append(["执行成功了，结果是一个有效文件", "结果：" + res])
+ new_file_path = promote_file_to_downloadzone(res, chatbot=chatbot)
+ chatbot = for_immediate_show_off_when_possible(file_type, new_file_path, chatbot)
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 界面更新
+ else:
+ chatbot.append(["执行成功了，结果是一个字符串", "结果：" + res])
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 界面更新
+
+"""
+测试：
+ 裁剪图像，保留下半部分
+ 交换图像的蓝色通道和红色通道
+ 将图像转为灰度图像
+ 将csv文件转excel表格
+"""
\ No newline at end of file
diff --git "a/crazy_functions/Latex\350\276\223\345\207\272PDF\347\273\223\346\236\234.py" "b/crazy_functions/Latex\350\276\223\345\207\272PDF\347\273\223\346\236\234.py"
index e79cf8223826e74c31b4cfb0220a395f38b335b2..8686f7e9c6b1a83101cc5d8ac994cf28b8853000 100644
--- "a/crazy_functions/Latex\350\276\223\345\207\272PDF\347\273\223\346\236\234.py"
+++ "b/crazy_functions/Latex\350\276\223\345\207\272PDF\347\273\223\346\236\234.py"
@@ -6,7 +6,7 @@ pj = os.path.join
ARXIV_CACHE_DIR = os.path.expanduser(f"~/arxiv_cache/")
# =================================== 工具函数 ===============================================
-专业词汇声明 = 'If the term "agent" is used in this section, it should be translated to "智能体". '
+# 专业词汇声明 = 'If the term "agent" is used in this section, it should be translated to "智能体". '
def switch_prompt(pfg, mode, more_requirement):
"""
Generate prompts and system prompts based on the mode for proofreading or translating.
@@ -109,7 +109,7 @@ def arxiv_download(chatbot, history, txt):
url_ = txt # https://arxiv.org/abs/1707.06690
if not txt.startswith('https://arxiv.org/abs/'):
- msg = f"解析arxiv网址失败, 期望格式例如: https://arxiv.org/abs/1707.06690。实际得到格式: {url_}"
+ msg = f"解析arxiv网址失败, 期望格式例如: https://arxiv.org/abs/1707.06690。实际得到格式: {url_}。"
yield from update_ui_lastest_msg(msg, chatbot=chatbot, history=history) # 刷新界面
return msg, None
# <-------------- set format ------------->
@@ -255,7 +255,7 @@ def Latex翻译中文并重新编译PDF(txt, llm_kwargs, plugin_kwargs, chatbot,
project_folder = txt
else:
if txt == "": txt = '空空如也的输入栏'
- report_execption(chatbot, history, a = f"解析项目: {txt}", b = f"找不到本地项目或无权访问: {txt}")
+ report_execption(chatbot, history, a = f"解析项目: {txt}", b = f"找不到本地项目或无法处理: {txt}")
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
return
@@ -291,7 +291,7 @@ def Latex翻译中文并重新编译PDF(txt, llm_kwargs, plugin_kwargs, chatbot,
yield from update_ui(chatbot=chatbot, history=history); time.sleep(1) # 刷新界面
promote_file_to_downloadzone(file=zip_res, chatbot=chatbot)
else:
- chatbot.append((f"失败了", '虽然PDF生成失败了, 但请查收结果（压缩包）, 内含已经翻译的Tex文档, 也是可读的, 您可以到Github Issue区, 用该压缩包+对话历史存档进行反馈 ...'))
+ chatbot.append((f"失败了", '虽然PDF生成失败了, 但请查收结果（压缩包）, 内含已经翻译的Tex文档, 您可以到Github Issue区, 用该压缩包进行反馈。如系统是Linux，请检查系统字体（见Github wiki） ...'))
yield from update_ui(chatbot=chatbot, history=history); time.sleep(1) # 刷新界面
promote_file_to_downloadzone(file=zip_res, chatbot=chatbot)
diff --git a/crazy_functions/crazy_utils.py b/crazy_functions/crazy_utils.py
index ffe95e2be56e969c787f5d2895fa502540501660..5a314b37971927793b2627d502693e62d36f11fa 100644
--- a/crazy_functions/crazy_utils.py
+++ b/crazy_functions/crazy_utils.py
@@ -591,11 +591,16 @@ def get_files_from_everything(txt, type): # type='.md'
# 网络的远程文件
import requests
from toolbox import get_conf
+ from toolbox import get_log_folder, gen_time_str
proxies, = get_conf('proxies')
- r = requests.get(txt, proxies=proxies)
- with open('./gpt_log/temp'+type, 'wb+') as f: f.write(r.content)
- project_folder = './gpt_log/'
- file_manifest = ['./gpt_log/temp'+type]
+ try:
+ r = requests.get(txt, proxies=proxies)
+ except:
+ raise ConnectionRefusedError(f"无法下载资源{txt}，请检查。")
+ path = os.path.join(get_log_folder(plugin_name='web_download'), gen_time_str()+type)
+ with open(path, 'wb+') as f: f.write(r.content)
+ project_folder = get_log_folder(plugin_name='web_download')
+ file_manifest = [path]
elif txt.endswith(type):
# 直接给定文件
file_manifest = [txt]
diff --git a/crazy_functions/json_fns/pydantic_io.py b/crazy_functions/json_fns/pydantic_io.py
new file mode 100644
index 0000000000000000000000000000000000000000..4e300d65dd918f890d64e68e0cc5a37f36366585
--- /dev/null
+++ b/crazy_functions/json_fns/pydantic_io.py
@@ -0,0 +1,111 @@
+"""
+https://github.com/langchain-ai/langchain/blob/master/docs/extras/modules/model_io/output_parsers/pydantic.ipynb
+
+Example 1.
+
+# Define your desired data structure.
+class Joke(BaseModel):
+ setup: str = Field(description="question to set up a joke")
+ punchline: str = Field(description="answer to resolve the joke")
+
+ # You can add custom validation logic easily with Pydantic.
+ @validator("setup")
+ def question_ends_with_question_mark(cls, field):
+ if field[-1] != "?":
+ raise ValueError("Badly formed question!")
+ return field
+
+
+Example 2.
+
+# Here's another example, but with a compound typed field.
+class Actor(BaseModel):
+ name: str = Field(description="name of an actor")
+ film_names: List[str] = Field(description="list of names of films they starred in")
+"""
+
+import json, re, logging
+
+
+PYDANTIC_FORMAT_INSTRUCTIONS = """The output should be formatted as a JSON instance that conforms to the JSON schema below.
+
+As an example, for the schema {{"properties": {{"foo": {{"title": "Foo", "description": "a list of strings", "type": "array", "items": {{"type": "string"}}}}}}, "required": ["foo"]}}
+the object {{"foo": ["bar", "baz"]}} is a well-formatted instance of the schema. The object {{"properties": {{"foo": ["bar", "baz"]}}}} is not well-formatted.
+
+Here is the output schema:
+```
+{schema}
+```"""
+
+
+PYDANTIC_FORMAT_INSTRUCTIONS_SIMPLE = """The output should be formatted as a JSON instance that conforms to the JSON schema below.
+```
+{schema}
+```"""
+
+class JsonStringError(Exception): ...
+
+class GptJsonIO():
+
+ def __init__(self, schema, example_instruction=True):
+ self.pydantic_object = schema
+ self.example_instruction = example_instruction
+ self.format_instructions = self.generate_format_instructions()
+
+ def generate_format_instructions(self):
+ schema = self.pydantic_object.schema()
+
+ # Remove extraneous fields.
+ reduced_schema = schema
+ if "title" in reduced_schema:
+ del reduced_schema["title"]
+ if "type" in reduced_schema:
+ del reduced_schema["type"]
+ # Ensure json in context is well-formed with double quotes.
+ if self.example_instruction:
+ schema_str = json.dumps(reduced_schema)
+ return PYDANTIC_FORMAT_INSTRUCTIONS.format(schema=schema_str)
+ else:
+ return PYDANTIC_FORMAT_INSTRUCTIONS_SIMPLE.format(schema=schema_str)
+
+ def generate_output(self, text):
+ # Greedy search for 1st json candidate.
+ match = re.search(
+ r"\{.*\}", text.strip(), re.MULTILINE | re.IGNORECASE | re.DOTALL
+ )
+ json_str = ""
+ if match: json_str = match.group()
+ json_object = json.loads(json_str, strict=False)
+ final_object = self.pydantic_object.parse_obj(json_object)
+ return final_object
+
+ def generate_repair_prompt(self, broken_json, error):
+ prompt = "Fix a broken json string.\n\n" + \
+ "(1) The broken json string need to fix is: \n\n" + \
+ "```" + "\n" + \
+ broken_json + "\n" + \
+ "```" + "\n\n" + \
+ "(2) The error message is: \n\n" + \
+ error + "\n\n" + \
+ "Now, fix this json string. \n\n"
+ return prompt
+
+ def generate_output_auto_repair(self, response, gpt_gen_fn):
+ """
+ response: string containing canidate json
+ gpt_gen_fn: gpt_gen_fn(inputs, sys_prompt)
+ """
+ try:
+ result = self.generate_output(response)
+ except Exception as e:
+ try:
+ logging.info(f'Repairing json：{response}')
+ repair_prompt = self.generate_repair_prompt(broken_json = response, error=repr(e))
+ result = self.generate_output(gpt_gen_fn(repair_prompt, self.format_instructions))
+ logging.info('Repaire json success.')
+ except Exception as e:
+ # 没辙了，放弃治疗
+ logging.info('Repaire json fail.')
+ raise JsonStringError('Cannot repair json.', str(e))
+ return result
+
diff --git a/crazy_functions/live_audio/aliyunASR.py b/crazy_functions/live_audio/aliyunASR.py
index 964105779f15c92e8cb87dbd5ad7c883ed32538d..ed67fcd3fb391409d7e6aced033d46585e62a858 100644
--- a/crazy_functions/live_audio/aliyunASR.py
+++ b/crazy_functions/live_audio/aliyunASR.py
@@ -1,4 +1,4 @@
-import time, threading, json
+import time, logging, json
class AliyunASR():
@@ -12,14 +12,14 @@ class AliyunASR():
message = json.loads(message)
self.parsed_sentence = message['payload']['result']
self.event_on_entence_end.set()
- print(self.parsed_sentence)
+ # print(self.parsed_sentence)
def test_on_start(self, message, *args):
# print("test_on_start:{}".format(message))
pass
def test_on_error(self, message, *args):
- print("on_error args=>{}".format(args))
+ logging.error("on_error args=>{}".format(args))
pass
def test_on_close(self, *args):
@@ -36,7 +36,6 @@ class AliyunASR():
# print("on_completed:args=>{} message=>{}".format(args, message))
pass
-
def audio_convertion_thread(self, uuid):
# 在一个异步线程中采集音频
import nls # pip install git+https://github.com/aliyun/alibabacloud-nls-python-sdk.git
diff --git a/crazy_functions/pdf_fns/parse_pdf.py b/crazy_functions/pdf_fns/parse_pdf.py
index 00016be594334e841f95c291e635dd7c30de6da4..8a7117adb61a1fb67c911e04d8968ac803885dd1 100644
--- a/crazy_functions/pdf_fns/parse_pdf.py
+++ b/crazy_functions/pdf_fns/parse_pdf.py
@@ -20,6 +20,11 @@ def get_avail_grobid_url():
def parse_pdf(pdf_path, grobid_url):
import scipdf # pip install scipdf_parser
if grobid_url.endswith('/'): grobid_url = grobid_url.rstrip('/')
- article_dict = scipdf.parse_pdf_to_dict(pdf_path, grobid_url=grobid_url)
+ try:
+ article_dict = scipdf.parse_pdf_to_dict(pdf_path, grobid_url=grobid_url)
+ except GROBID_OFFLINE_EXCEPTION:
+ raise GROBID_OFFLINE_EXCEPTION("GROBID服务不可用，请修改config中的GROBID_URL，可修改成本地GROBID服务。")
+ except:
+ raise RuntimeError("解析PDF失败，请检查PDF是否损坏。")
return article_dict
diff --git a/crazy_functions/vt_fns/vt_call_plugin.py b/crazy_functions/vt_fns/vt_call_plugin.py
new file mode 100644
index 0000000000000000000000000000000000000000..455ac88bc0d7861441301c37490181181243a59a
--- /dev/null
+++ b/crazy_functions/vt_fns/vt_call_plugin.py
@@ -0,0 +1,114 @@
+from pydantic import BaseModel, Field
+from typing import List
+from toolbox import update_ui_lastest_msg, disable_auto_promotion
+from request_llm.bridge_all import predict_no_ui_long_connection
+from crazy_functions.json_fns.pydantic_io import GptJsonIO, JsonStringError
+import copy, json, pickle, os, sys, time
+
+
+def read_avail_plugin_enum():
+ from crazy_functional import get_crazy_functions
+ plugin_arr = get_crazy_functions()
+ # remove plugins with out explaination
+ plugin_arr = {k:v for k, v in plugin_arr.items() if 'Info' in v}
+ plugin_arr_info = {"F_{:04d}".format(i):v["Info"] for i, v in enumerate(plugin_arr.values(), start=1)}
+ plugin_arr_dict = {"F_{:04d}".format(i):v for i, v in enumerate(plugin_arr.values(), start=1)}
+ plugin_arr_dict_parse = {"F_{:04d}".format(i):v for i, v in enumerate(plugin_arr.values(), start=1)}
+ plugin_arr_dict_parse.update({f"F_{i}":v for i, v in enumerate(plugin_arr.values(), start=1)})
+ prompt = json.dumps(plugin_arr_info, ensure_ascii=False, indent=2)
+ prompt = "\n\nThe defination of PluginEnum:\nPluginEnum=" + prompt
+ return prompt, plugin_arr_dict, plugin_arr_dict_parse
+
+def wrap_code(txt):
+ txt = txt.replace('```','')
+ return f"\n```\n{txt}\n```\n"
+
+def have_any_recent_upload_files(chatbot):
+ _5min = 5 * 60
+ if not chatbot: return False # chatbot is None
+ most_recent_uploaded = chatbot._cookies.get("most_recent_uploaded", None)
+ if not most_recent_uploaded: return False # most_recent_uploaded is None
+ if time.time() - most_recent_uploaded["time"] < _5min: return True # most_recent_uploaded is new
+ else: return False # most_recent_uploaded is too old
+
+def get_recent_file_prompt_support(chatbot):
+ most_recent_uploaded = chatbot._cookies.get("most_recent_uploaded", None)
+ path = most_recent_uploaded['path']
+ prompt = "\nAdditional Information:\n"
+ prompt = "In case that this plugin requires a path or a file as argument,"
+ prompt += f"it is important for you to know that the user has recently uploaded a file, located at: `{path}`"
+ prompt += f"Only use it when necessary, otherwise, you can ignore this file."
+ return prompt
+
+def get_inputs_show_user(inputs, plugin_arr_enum_prompt):
+ # remove plugin_arr_enum_prompt from inputs string
+ inputs_show_user = inputs.replace(plugin_arr_enum_prompt, "")
+ inputs_show_user += plugin_arr_enum_prompt[:200] + '...'
+ inputs_show_user += '\n...\n'
+ inputs_show_user += '...\n'
+ inputs_show_user += '...}'
+ return inputs_show_user
+
+def execute_plugin(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_intention):
+ plugin_arr_enum_prompt, plugin_arr_dict, plugin_arr_dict_parse = read_avail_plugin_enum()
+ class Plugin(BaseModel):
+ plugin_selection: str = Field(description="The most related plugin from one of the PluginEnum.", default="F_0000")
+ reason_of_selection: str = Field(description="The reason why you should select this plugin.", default="This plugin satisfy user requirement most")
+ # ⭐ ⭐ ⭐ 选择插件
+ yield from update_ui_lastest_msg(lastmsg=f"正在执行任务: {txt}\n\n查找可用插件中...", chatbot=chatbot, history=history, delay=0)
+ gpt_json_io = GptJsonIO(Plugin)
+ gpt_json_io.format_instructions = "The format of your output should be a json that can be parsed by json.loads.\n"
+ gpt_json_io.format_instructions += """Output example: {"plugin_selection":"F_1234", "reason_of_selection":"F_1234 plugin satisfy user requirement most"}\n"""
+ gpt_json_io.format_instructions += "The plugins you are authorized to use are listed below:\n"
+ gpt_json_io.format_instructions += plugin_arr_enum_prompt
+ inputs = "Choose the correct plugin according to user requirements, the user requirement is: \n\n" + \
+ ">> " + txt.rstrip('\n').replace('\n','\n>> ') + '\n\n' + gpt_json_io.format_instructions
+
+ run_gpt_fn = lambda inputs, sys_prompt: predict_no_ui_long_connection(
+ inputs=inputs, llm_kwargs=llm_kwargs, history=[], sys_prompt=sys_prompt, observe_window=[])
+ try:
+ gpt_reply = run_gpt_fn(inputs, "")
+ plugin_sel = gpt_json_io.generate_output_auto_repair(gpt_reply, run_gpt_fn)
+ except JsonStringError:
+ msg = f"抱歉, {llm_kwargs['llm_model']}无法理解您的需求。"
+ msg += "请求的Prompt为：\n" + wrap_code(get_inputs_show_user(inputs, plugin_arr_enum_prompt))
+ msg += "语言模型回复为：\n" + wrap_code(gpt_reply)
+ msg += "\n但您可以尝试再试一次\n"
+ yield from update_ui_lastest_msg(lastmsg=msg, chatbot=chatbot, history=history, delay=2)
+ return
+ if plugin_sel.plugin_selection not in plugin_arr_dict_parse:
+ msg = f"抱歉, 找不到合适插件执行该任务, 或者{llm_kwargs['llm_model']}无法理解您的需求。"
+ msg += f"语言模型{llm_kwargs['llm_model']}选择了不存在的插件：\n" + wrap_code(gpt_reply)
+ msg += "\n但您可以尝试再试一次\n"
+ yield from update_ui_lastest_msg(lastmsg=msg, chatbot=chatbot, history=history, delay=2)
+ return
+
+ # ⭐ ⭐ ⭐ 确认插件参数
+ if not have_any_recent_upload_files(chatbot):
+ appendix_info = ""
+ else:
+ appendix_info = get_recent_file_prompt_support(chatbot)
+
+ plugin = plugin_arr_dict_parse[plugin_sel.plugin_selection]
+ yield from update_ui_lastest_msg(lastmsg=f"正在执行任务: {txt}\n\n提取插件参数...", chatbot=chatbot, history=history, delay=0)
+ class PluginExplicit(BaseModel):
+ plugin_selection: str = plugin_sel.plugin_selection
+ plugin_arg: str = Field(description="The argument of the plugin.", default="")
+ gpt_json_io = GptJsonIO(PluginExplicit)
+ gpt_json_io.format_instructions += "The information about this plugin is:" + plugin["Info"]
+ inputs = f"A plugin named {plugin_sel.plugin_selection} is selected, " + \
+ "you should extract plugin_arg from the user requirement, the user requirement is: \n\n" + \
+ ">> " + (txt + appendix_info).rstrip('\n').replace('\n','\n>> ') + '\n\n' + \
+ gpt_json_io.format_instructions
+ run_gpt_fn = lambda inputs, sys_prompt: predict_no_ui_long_connection(
+ inputs=inputs, llm_kwargs=llm_kwargs, history=[], sys_prompt=sys_prompt, observe_window=[])
+ plugin_sel = gpt_json_io.generate_output_auto_repair(run_gpt_fn(inputs, ""), run_gpt_fn)
+
+
+ # ⭐ ⭐ ⭐ 执行插件
+ fn = plugin['Function']
+ fn_name = fn.__name__
+ msg = f'{llm_kwargs["llm_model"]}为您选择了插件: `{fn_name}`\n\n插件说明：{plugin["Info"]}\n\n插件参数：{plugin_sel.plugin_arg}\n\n假如偏离了您的要求，按停止键终止。'
+ yield from update_ui_lastest_msg(lastmsg=msg, chatbot=chatbot, history=history, delay=2)
+ yield from fn(plugin_sel.plugin_arg, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, -1)
+ return
\ No newline at end of file
diff --git a/crazy_functions/vt_fns/vt_modify_config.py b/crazy_functions/vt_fns/vt_modify_config.py
new file mode 100644
index 0000000000000000000000000000000000000000..e7fd745c3dc2ee1cf260ac2ac97a053b2985d4c8
--- /dev/null
+++ b/crazy_functions/vt_fns/vt_modify_config.py
@@ -0,0 +1,81 @@
+from pydantic import BaseModel, Field
+from typing import List
+from toolbox import update_ui_lastest_msg, get_conf
+from request_llm.bridge_all import predict_no_ui_long_connection
+from crazy_functions.json_fns.pydantic_io import GptJsonIO
+import copy, json, pickle, os, sys
+
+
+def modify_configuration_hot(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_intention):
+ ALLOW_RESET_CONFIG, = get_conf('ALLOW_RESET_CONFIG')
+ if not ALLOW_RESET_CONFIG:
+ yield from update_ui_lastest_msg(
+ lastmsg=f"当前配置不允许被修改！如需激活本功能，请在config.py中设置ALLOW_RESET_CONFIG=True后重启软件。",
+ chatbot=chatbot, history=history, delay=2
+ )
+ return
+
+ # ⭐ ⭐ ⭐ 读取可配置项目条目
+ names = {}
+ from enum import Enum
+ import config
+ for k, v in config.__dict__.items():
+ if k.startswith('__'): continue
+ names.update({k:k})
+ # if len(names) > 20: break # 限制最多前10个配置项，如果太多了会导致gpt无法理解
+
+ ConfigOptions = Enum('ConfigOptions', names)
+ class ModifyConfigurationIntention(BaseModel):
+ which_config_to_modify: ConfigOptions = Field(description="the name of the configuration to modify, you must choose from one of the ConfigOptions enum.", default=None)
+ new_option_value: str = Field(description="the new value of the option", default=None)
+
+ # ⭐ ⭐ ⭐ 分析用户意图
+ yield from update_ui_lastest_msg(lastmsg=f"正在执行任务: {txt}\n\n读取新配置中", chatbot=chatbot, history=history, delay=0)
+ gpt_json_io = GptJsonIO(ModifyConfigurationIntention)
+ inputs = "Analyze how to change configuration according to following user input, answer me with json: \n\n" + \
+ ">> " + txt.rstrip('\n').replace('\n','\n>> ') + '\n\n' + \
+ gpt_json_io.format_instructions
+
+ run_gpt_fn = lambda inputs, sys_prompt: predict_no_ui_long_connection(
+ inputs=inputs, llm_kwargs=llm_kwargs, history=[], sys_prompt=sys_prompt, observe_window=[])
+ user_intention = gpt_json_io.generate_output_auto_repair(run_gpt_fn(inputs, ""), run_gpt_fn)
+
+ explicit_conf = user_intention.which_config_to_modify.value
+
+ ok = (explicit_conf in txt)
+ if ok:
+ yield from update_ui_lastest_msg(
+ lastmsg=f"正在执行任务: {txt}\n\n新配置{explicit_conf}={user_intention.new_option_value}",
+ chatbot=chatbot, history=history, delay=1
+ )
+ yield from update_ui_lastest_msg(
+ lastmsg=f"正在执行任务: {txt}\n\n新配置{explicit_conf}={user_intention.new_option_value}\n\n正在修改配置中",
+ chatbot=chatbot, history=history, delay=2
+ )
+
+ # ⭐ ⭐ ⭐ 立即应用配置
+ from toolbox import set_conf
+ set_conf(explicit_conf, user_intention.new_option_value)
+
+ yield from update_ui_lastest_msg(
+ lastmsg=f"正在执行任务: {txt}\n\n配置修改完成，重新页面即可生效。", chatbot=chatbot, history=history, delay=1
+ )
+ else:
+ yield from update_ui_lastest_msg(
+ lastmsg=f"失败，如果需要配置{explicit_conf}，您需要明确说明并在指令中提到它。", chatbot=chatbot, history=history, delay=5
+ )
+
+def modify_configuration_reboot(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_intention):
+ ALLOW_RESET_CONFIG, = get_conf('ALLOW_RESET_CONFIG')
+ if not ALLOW_RESET_CONFIG:
+ yield from update_ui_lastest_msg(
+ lastmsg=f"当前配置不允许被修改！如需激活本功能，请在config.py中设置ALLOW_RESET_CONFIG=True后重启软件。",
+ chatbot=chatbot, history=history, delay=2
+ )
+ return
+
+ yield from modify_configuration_hot(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_intention)
+ yield from update_ui_lastest_msg(
+ lastmsg=f"正在执行任务: {txt}\n\n配置修改完成，五秒后即将重启！若出现报错请无视即可。", chatbot=chatbot, history=history, delay=5
+ )
+ os.execl(sys.executable, sys.executable, *sys.argv)
diff --git a/crazy_functions/vt_fns/vt_state.py b/crazy_functions/vt_fns/vt_state.py
new file mode 100644
index 0000000000000000000000000000000000000000..18187286383ce2f3e881510852cf3aba7e6c43d1
--- /dev/null
+++ b/crazy_functions/vt_fns/vt_state.py
@@ -0,0 +1,28 @@
+import pickle
+
+class VoidTerminalState():
+ def __init__(self):
+ self.reset_state()
+
+ def reset_state(self):
+ self.has_provided_explaination = False
+
+ def lock_plugin(self, chatbot):
+ chatbot._cookies['lock_plugin'] = 'crazy_functions.虚空终端->虚空终端'
+ chatbot._cookies['plugin_state'] = pickle.dumps(self)
+
+ def unlock_plugin(self, chatbot):
+ self.reset_state()
+ chatbot._cookies['lock_plugin'] = None
+ chatbot._cookies['plugin_state'] = pickle.dumps(self)
+
+ def set_state(self, chatbot, key, value):
+ setattr(self, key, value)
+ chatbot._cookies['plugin_state'] = pickle.dumps(self)
+
+ def get_state(chatbot):
+ state = chatbot._cookies.get('plugin_state', None)
+ if state is not None: state = pickle.loads(state)
+ else: state = VoidTerminalState()
+ state.chatbot = chatbot
+ return state
\ No newline at end of file
diff --git "a/crazy_functions/\346\211\271\351\207\217Markdown\347\277\273\350\257\221.py" "b/crazy_functions/\346\211\271\351\207\217Markdown\347\277\273\350\257\221.py"
index ea8dee960b25e37ac061871cfca3592510157ac9..8af98640bf51a4edf19254c05d68fad282d02776 100644
--- "a/crazy_functions/\346\211\271\351\207\217Markdown\347\277\273\350\257\221.py"
+++ "b/crazy_functions/\346\211\271\351\207\217Markdown\347\277\273\350\257\221.py"
@@ -145,6 +145,8 @@ def get_files_from_everything(txt, preference=''):
project_folder = txt
file_manifest = [f for f in glob.glob(f'{project_folder}/**/*.md', recursive=True)]
else:
+ project_folder = None
+ file_manifest = []
success = False
return success, file_manifest, project_folder
diff --git "a/crazy_functions/\346\211\271\351\207\217\347\277\273\350\257\221PDF\346\226\207\346\241\243_NOUGAT.py" "b/crazy_functions/\346\211\271\351\207\217\347\277\273\350\257\221PDF\346\226\207\346\241\243_NOUGAT.py"
new file mode 100644
index 0000000000000000000000000000000000000000..ed15121159837d0f543063708a2cbff50b9e5491
--- /dev/null
+++ "b/crazy_functions/\346\211\271\351\207\217\347\277\273\350\257\221PDF\346\226\207\346\241\243_NOUGAT.py"
@@ -0,0 +1,271 @@
+from toolbox import CatchException, report_execption, gen_time_str
+from toolbox import update_ui, promote_file_to_downloadzone, update_ui_lastest_msg, disable_auto_promotion
+from toolbox import write_history_to_file, get_log_folder
+from .crazy_utils import request_gpt_model_in_new_thread_with_ui_alive
+from .crazy_utils import request_gpt_model_multi_threads_with_very_awesome_ui_and_high_efficiency
+from .crazy_utils import read_and_clean_pdf_text
+from .pdf_fns.parse_pdf import parse_pdf, get_avail_grobid_url
+from colorful import *
+import os
+import math
+import logging
+
+def markdown_to_dict(article_content):
+ import markdown
+ from bs4 import BeautifulSoup
+ cur_t = ""
+ cur_c = ""
+ results = {}
+ for line in article_content:
+ if line.startswith('#'):
+ if cur_t!="":
+ if cur_t not in results:
+ results.update({cur_t:cur_c.lstrip('\n')})
+ else:
+ # 处理重名的章节
+ results.update({cur_t + " " + gen_time_str():cur_c.lstrip('\n')})
+ cur_t = line.rstrip('\n')
+ cur_c = ""
+ else:
+ cur_c += line
+ results_final = {}
+ for k in list(results.keys()):
+ if k.startswith('# '):
+ results_final['title'] = k.split('# ')[-1]
+ results_final['authors'] = results.pop(k).lstrip('\n')
+ if k.startswith('###### Abstract'):
+ results_final['abstract'] = results.pop(k).lstrip('\n')
+
+ results_final_sections = []
+ for k,v in results.items():
+ results_final_sections.append({
+ 'heading':k.lstrip("# "),
+ 'text':v if len(v) > 0 else f"The beginning of {k.lstrip('# ')} section."
+ })
+ results_final['sections'] = results_final_sections
+ return results_final
+
+
+@CatchException
+def 批量翻译PDF文档(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
+
+ disable_auto_promotion(chatbot)
+ # 基本信息：功能、贡献者
+ chatbot.append([
+ "函数插件功能？",
+ "批量翻译PDF文档。函数插件贡献者: Binary-Husky"])
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+
+ # 尝试导入依赖，如果缺少依赖，则给出安装建议
+ try:
+ import nougat
+ import tiktoken
+ except:
+ report_execption(chatbot, history,
+ a=f"解析项目: {txt}",
+ b=f"导入软件依赖失败。使用该模块需要额外依赖，安装方法```pip install --upgrade nougat-ocr tiktoken```。")
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+ return
+
+ # 清空历史，以免输入溢出
+ history = []
+
+ from .crazy_utils import get_files_from_everything
+ success, file_manifest, project_folder = get_files_from_everything(txt, type='.pdf')
+ # 检测输入参数，如没有给定输入参数，直接退出
+ if not success:
+ if txt == "": txt = '空空如也的输入栏'
+
+ # 如果没找到任何文件
+ if len(file_manifest) == 0:
+ report_execption(chatbot, history,
+ a=f"解析项目: {txt}", b=f"找不到任何.tex或.pdf文件: {txt}")
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+ return
+
+ # 开始正式执行任务
+ yield from 解析PDF_基于NOUGAT(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt)
+
+
+def nougat_with_timeout(command, cwd, timeout=3600):
+ import subprocess
+ process = subprocess.Popen(command, shell=True, cwd=cwd)
+ try:
+ stdout, stderr = process.communicate(timeout=timeout)
+ except subprocess.TimeoutExpired:
+ process.kill()
+ stdout, stderr = process.communicate()
+ print("Process timed out!")
+ return False
+ return True
+
+
+def NOUGAT_parse_pdf(fp):
+ import glob
+ from toolbox import get_log_folder, gen_time_str
+ dst = os.path.join(get_log_folder(plugin_name='nougat'), gen_time_str())
+ os.makedirs(dst)
+ nougat_with_timeout(f'nougat --out "{os.path.abspath(dst)}" "{os.path.abspath(fp)}"', os.getcwd())
+ res = glob.glob(os.path.join(dst,'*.mmd'))
+ if len(res) == 0:
+ raise RuntimeError("Nougat解析论文失败。")
+ return res[0]
+
+
+def 解析PDF_基于NOUGAT(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt):
+ import copy
+ import tiktoken
+ TOKEN_LIMIT_PER_FRAGMENT = 1280
+ generated_conclusion_files = []
+ generated_html_files = []
+ DST_LANG = "中文"
+ for index, fp in enumerate(file_manifest):
+ chatbot.append(["当前进度：", f"正在解析论文，请稍候。（第一次运行时，需要花费较长时间下载NOUGAT参数）"]); yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+ fpp = NOUGAT_parse_pdf(fp)
+
+ with open(fpp, 'r', encoding='utf8') as f:
+ article_content = f.readlines()
+ article_dict = markdown_to_dict(article_content)
+ logging.info(article_dict)
+
+ prompt = "以下是一篇学术论文的基本信息:\n"
+ # title
+ title = article_dict.get('title', '无法获取 title'); prompt += f'title:{title}\n\n'
+ # authors
+ authors = article_dict.get('authors', '无法获取 authors'); prompt += f'authors:{authors}\n\n'
+ # abstract
+ abstract = article_dict.get('abstract', '无法获取 abstract'); prompt += f'abstract:{abstract}\n\n'
+ # command
+ prompt += f"请将题目和摘要翻译为{DST_LANG}。"
+ meta = [f'# Title:\n\n', title, f'# Abstract:\n\n', abstract ]
+
+ # 单线，获取文章meta信息
+ paper_meta_info = yield from request_gpt_model_in_new_thread_with_ui_alive(
+ inputs=prompt,
+ inputs_show_user=prompt,
+ llm_kwargs=llm_kwargs,
+ chatbot=chatbot, history=[],
+ sys_prompt="You are an academic paper reader。",
+ )
+
+ # 多线，翻译
+ inputs_array = []
+ inputs_show_user_array = []
+
+ # get_token_num
+ from request_llm.bridge_all import model_info
+ enc = model_info[llm_kwargs['llm_model']]['tokenizer']
+ def get_token_num(txt): return len(enc.encode(txt, disallowed_special=()))
+ from .crazy_utils import breakdown_txt_to_satisfy_token_limit_for_pdf
+
+ def break_down(txt):
+ raw_token_num = get_token_num(txt)
+ if raw_token_num <= TOKEN_LIMIT_PER_FRAGMENT:
+ return [txt]
+ else:
+ # raw_token_num > TOKEN_LIMIT_PER_FRAGMENT
+ # find a smooth token limit to achieve even seperation
+ count = int(math.ceil(raw_token_num / TOKEN_LIMIT_PER_FRAGMENT))
+ token_limit_smooth = raw_token_num // count + count
+ return breakdown_txt_to_satisfy_token_limit_for_pdf(txt, get_token_fn=get_token_num, limit=token_limit_smooth)
+
+ for section in article_dict.get('sections'):
+ if len(section['text']) == 0: continue
+ section_frags = break_down(section['text'])
+ for i, fragment in enumerate(section_frags):
+ heading = section['heading']
+ if len(section_frags) > 1: heading += f' Part-{i+1}'
+ inputs_array.append(
+ f"你需要翻译{heading}章节，内容如下: \n\n{fragment}"
+ )
+ inputs_show_user_array.append(
+ f"# {heading}\n\n{fragment}"
+ )
+
+ gpt_response_collection = yield from request_gpt_model_multi_threads_with_very_awesome_ui_and_high_efficiency(
+ inputs_array=inputs_array,
+ inputs_show_user_array=inputs_show_user_array,
+ llm_kwargs=llm_kwargs,
+ chatbot=chatbot,
+ history_array=[meta for _ in inputs_array],
+ sys_prompt_array=[
+ "请你作为一个学术翻译，负责把学术论文准确翻译成中文。注意文章中的每一句话都要翻译。" for _ in inputs_array],
+ )
+ res_path = write_history_to_file(meta + ["# Meta Translation" , paper_meta_info] + gpt_response_collection, file_basename=None, file_fullname=None)
+ promote_file_to_downloadzone(res_path, rename_file=os.path.basename(fp)+'.md', chatbot=chatbot)
+ generated_conclusion_files.append(res_path)
+
+ ch = construct_html()
+ orig = ""
+ trans = ""
+ gpt_response_collection_html = copy.deepcopy(gpt_response_collection)
+ for i,k in enumerate(gpt_response_collection_html):
+ if i%2==0:
+ gpt_response_collection_html[i] = inputs_show_user_array[i//2]
+ else:
+ gpt_response_collection_html[i] = gpt_response_collection_html[i]
+
+ final = ["", "", "一、论文概况", "", "Abstract", paper_meta_info, "二、论文翻译", ""]
+ final.extend(gpt_response_collection_html)
+ for i, k in enumerate(final):
+ if i%2==0:
+ orig = k
+ if i%2==1:
+ trans = k
+ ch.add_row(a=orig, b=trans)
+ create_report_file_name = f"{os.path.basename(fp)}.trans.html"
+ html_file = ch.save_file(create_report_file_name)
+ generated_html_files.append(html_file)
+ promote_file_to_downloadzone(html_file, rename_file=os.path.basename(html_file), chatbot=chatbot)
+
+ chatbot.append(("给出输出文件清单", str(generated_conclusion_files + generated_html_files)))
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+
+
+
+class construct_html():
+ def __init__(self) -> None:
+ self.css = """
+.row {
+ display: flex;
+ flex-wrap: wrap;
+}
+
+.column {
+ flex: 1;
+ padding: 10px;
+}
+
+.table-header {
+ font-weight: bold;
+ border-bottom: 1px solid black;
+}
+
+.table-row {
+ border-bottom: 1px solid lightgray;
+}
+
+.table-cell {
+ padding: 5px;
+}
+ """
+ self.html_string = f'<!DOCTYPE html><head><meta charset="utf-8"><title>翻译结果</title><style>{self.css}</style></head>'
+
+
+ def add_row(self, a, b):
+ tmp = """
+<div class="row table-row">
+ <div class="column table-cell">REPLACE_A</div>
+ <div class="column table-cell">REPLACE_B</div>
+</div>
+ """
+ from toolbox import markdown_convertion
+ tmp = tmp.replace('REPLACE_A', markdown_convertion(a))
+ tmp = tmp.replace('REPLACE_B', markdown_convertion(b))
+ self.html_string += tmp
+
+
+ def save_file(self, file_name):
+ with open(os.path.join(get_log_folder(), file_name), 'w', encoding='utf8') as f:
+ f.write(self.html_string.encode('utf-8', 'ignore').decode())
+ return os.path.join(get_log_folder(), file_name)
diff --git "a/crazy_functions/\346\211\271\351\207\217\347\277\273\350\257\221PDF\346\226\207\346\241\243_\345\244\232\347\272\277\347\250\213.py" "b/crazy_functions/\346\211\271\351\207\217\347\277\273\350\257\221PDF\346\226\207\346\241\243_\345\244\232\347\272\277\347\250\213.py"
index e0558e905c7748e2a067642db79dd29b06fe45a9..6e9fe6a84a85f6fc13d4181bea0fbd86037d9743 100644
--- "a/crazy_functions/\346\211\271\351\207\217\347\277\273\350\257\221PDF\346\226\207\346\241\243_\345\244\232\347\272\277\347\250\213.py"
+++ "b/crazy_functions/\346\211\271\351\207\217\347\277\273\350\257\221PDF\346\226\207\346\241\243_\345\244\232\347\272\277\347\250\213.py"
@@ -24,10 +24,11 @@ def 批量翻译PDF文档(txt, llm_kwargs, plugin_kwargs, chatbot, history, syst
try:
import fitz
import tiktoken
+ import scipdf
except:
report_execption(chatbot, history,
a=f"解析项目: {txt}",
- b=f"导入软件依赖失败。使用该模块需要额外依赖，安装方法```pip install --upgrade pymupdf tiktoken```。")
+ b=f"导入软件依赖失败。使用该模块需要额外依赖，安装方法```pip install --upgrade pymupdf tiktoken scipdf_parser```。")
yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
return
@@ -58,7 +59,6 @@ def 批量翻译PDF文档(txt, llm_kwargs, plugin_kwargs, chatbot, history, syst
def 解析PDF_基于GROBID(file_manifest, project_folder, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, grobid_url):
import copy
- import tiktoken
TOKEN_LIMIT_PER_FRAGMENT = 1280
generated_conclusion_files = []
generated_html_files = []
@@ -66,7 +66,7 @@ def 解析PDF_基于GROBID(file_manifest, project_folder, llm_kwargs, plugin_kwa
for index, fp in enumerate(file_manifest):
chatbot.append(["当前进度：", f"正在连接GROBID服务，请稍候: {grobid_url}\n如果等待时间过长，请修改config中的GROBID_URL，可修改成本地GROBID服务。"]); yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
article_dict = parse_pdf(fp, grobid_url)
- print(article_dict)
+ if article_dict is None: raise RuntimeError("解析PDF失败，请检查PDF是否损坏。")
prompt = "以下是一篇学术论文的基本信息:\n"
# title
title = article_dict.get('title', '无法获取 title'); prompt += f'title:{title}\n\n'
@@ -113,7 +113,7 @@ def 解析PDF_基于GROBID(file_manifest, project_folder, llm_kwargs, plugin_kwa
section_frags = break_down(section['text'])
for i, fragment in enumerate(section_frags):
heading = section['heading']
- if len(section_frags) > 1: heading += f'Part-{i+1}'
+ if len(section_frags) > 1: heading += f' Part-{i+1}'
inputs_array.append(
f"你需要翻译{heading}章节，内容如下: \n\n{fragment}"
)
diff --git "a/crazy_functions/\350\201\224\347\275\221\347\232\204ChatGPT.py" "b/crazy_functions/\350\201\224\347\275\221\347\232\204ChatGPT.py"
index 6a7d118b4439605db6e10b9a416a2e725b99a672..4ed9aebf97834418d145ab1dd5b22ca7f4f9b214 100644
--- "a/crazy_functions/\350\201\224\347\275\221\347\232\204ChatGPT.py"
+++ "b/crazy_functions/\350\201\224\347\275\221\347\232\204ChatGPT.py"
@@ -75,7 +75,11 @@ def 连接网络回答问题(txt, llm_kwargs, plugin_kwargs, chatbot, history, s
proxies, = get_conf('proxies')
urls = google(txt, proxies)
history = []
-
+ if len(urls) == 0:
+ chatbot.append((f"结论：{txt}",
+ "[Local Message] 受到google限制，无法从google获取信息！"))
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 由于请求gpt需要一段时间，我们先及时地做一次界面更新
+ return
# ------------- < 第2步：依次访问网页 > -------------
max_search_result = 5 # 最多收纳多少个网页的结果
for index, url in enumerate(urls[:max_search_result]):
diff --git "a/crazy_functions/\350\201\224\347\275\221\347\232\204ChatGPT_bing\347\211\210.py" "b/crazy_functions/\350\201\224\347\275\221\347\232\204ChatGPT_bing\347\211\210.py"
index 93a84a0c5b47d44ee10e2a8a732c68d693388694..db5adb7992f765db3e5b0e7ecea7e71e44dbe855 100644
--- "a/crazy_functions/\350\201\224\347\275\221\347\232\204ChatGPT_bing\347\211\210.py"
+++ "b/crazy_functions/\350\201\224\347\275\221\347\232\204ChatGPT_bing\347\211\210.py"
@@ -75,7 +75,11 @@ def 连接bing搜索回答问题(txt, llm_kwargs, plugin_kwargs, chatbot, histor
proxies, = get_conf('proxies')
urls = bing_search(txt, proxies)
history = []
-
+ if len(urls) == 0:
+ chatbot.append((f"结论：{txt}",
+ "[Local Message] 受到bing限制，无法从bing获取信息！"))
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面 # 由于请求gpt需要一段时间，我们先及时地做一次界面更新
+ return
# ------------- < 第2步：依次访问网页 > -------------
max_search_result = 8 # 最多收纳多少个网页的结果
for index, url in enumerate(urls[:max_search_result]):
diff --git "a/crazy_functions/\350\231\232\347\251\272\347\273\210\347\253\257.py" "b/crazy_functions/\350\231\232\347\251\272\347\273\210\347\253\257.py"
index 36667e90d67730d75d0ad17039920b7d535103ff..2e1b5236bdc4cd4523743c265a98d7279c9cbf93 100644
--- "a/crazy_functions/\350\231\232\347\251\272\347\273\210\347\253\257.py"
+++ "b/crazy_functions/\350\231\232\347\251\272\347\273\210\347\253\257.py"
@@ -1,119 +1,179 @@
+"""
+Explanation of the Void Terminal Plugin:
+
+Please describe in natural language what you want to do.
+
+1. You can open the plugin's dropdown menu to explore various capabilities of this project, and then describe your needs in natural language, for example:
+- "Please call the plugin to translate a PDF paper for me. I just uploaded the paper to the upload area."
+- "Please use the plugin to translate a PDF paper, with the address being https://www.nature.com/articles/s41586-019-1724-z.pdf."
+- "Generate an image with blooming flowers and lush green grass using the plugin."
+- "Translate the README using the plugin. The GitHub URL is https://github.com/facebookresearch/co-tracker."
+- "Translate an Arxiv paper for me. The Arxiv ID is 1812.10695. Remember to use the plugin and don't do it manually!"
+- "I don't like the current interface color. Modify the configuration and change the theme to THEME="High-Contrast"."
+- "Could you please explain the structure of the Transformer network?"
+
+2. If you use keywords like "call the plugin xxx", "modify the configuration xxx", "please", etc., your intention can be recognized more accurately.
+
+3. Your intention can be recognized more accurately when using powerful models like GPT4. This plugin is relatively new, so please feel free to provide feedback on GitHub.
+
+4. Now, if you need to process a file, please upload the file (drag the file to the file upload area) or describe the path to the file.
+
+5. If you don't need to upload a file, you can simply repeat your command again.
+"""
+explain_msg = """
+## 虚空终端插件说明:
+
+1. 请用**自然语言**描述您需要做什么。例如：
+ - 「请调用插件，为我翻译PDF论文，论文我刚刚放到上传区了」
+ - 「请调用插件翻译PDF论文，地址为https://aaa/bbb/ccc.pdf」
+ - 「把Arxiv论文翻译成中文PDF，arxiv论文的ID是1812.10695，记得用插件！」
+ - 「生成一张图片，图中鲜花怒放，绿草如茵，用插件实现」
+ - 「用插件翻译README，Github网址是https://github.com/facebookresearch/co-tracker」
+ - 「我不喜欢当前的界面颜色，修改配置，把主题THEME更换为THEME="High-Contrast"」
+ - 「请问Transformer网络的结构是怎样的？」
+
+2. 您可以打开插件下拉菜单以了解本项目的各种能力。
+
+3. 如果您使用「调用插件xxx」、「修改配置xxx」、「请问」等关键词，您的意图可以被识别的更准确。
+
+4. 建议使用 GPT3.5 或更强的模型，弱模型可能无法理解您的想法。该插件诞生时间不长，欢迎您前往Github反馈问题。
+
+5. 现在，如果需要处理文件，请您上传文件（将文件拖动到文件上传区），或者描述文件所在的路径。
+
+6. 如果不需要上传文件，现在您只需要再次重复一次您的指令即可。
+"""
+
+from pydantic import BaseModel, Field
+from typing import List
from toolbox import CatchException, update_ui, gen_time_str
-from .crazy_utils import request_gpt_model_in_new_thread_with_ui_alive
-from .crazy_utils import input_clipping
-import copy, json
-
-def get_fn_lib():
- return {
- "BatchTranslatePDFDocuments_MultiThreaded": {
- "module": "crazy_functions.批量翻译PDF文档_多线程",
- "function": "批量翻译PDF文档",
- "description": "Translate PDF Documents",
- "arg_1_description": "A path containing pdf files.",
- },
- "SummarizingWordDocuments": {
- "module": "crazy_functions.总结word文档",
- "function": "总结word文档",
- "description": "Summarize Word Documents",
- "arg_1_description": "A path containing Word files.",
- },
- "ImageGeneration": {
- "module": "crazy_functions.图片生成",
- "function": "图片生成",
- "description": "Generate a image that satisfies some description.",
- "arg_1_description": "Descriptions about the image to be generated.",
- },
- "TranslateMarkdownFromEnglishToChinese": {
- "module": "crazy_functions.批量Markdown翻译",
- "function": "Markdown中译英",
- "description": "Translate Markdown Documents from English to Chinese.",
- "arg_1_description": "A path containing Markdown files.",
- },
- "SummaryAudioVideo": {
- "module": "crazy_functions.总结音视频",
- "function": "总结音视频",
- "description": "Get text from a piece of audio and summarize this audio.",
- "arg_1_description": "A path containing audio files.",
- },
- }
-
-functions = [
- {
- "name": k,
- "description": v['description'],
- "parameters": {
- "type": "object",
- "properties": {
- "plugin_arg_1": {
- "type": "string",
- "description": v['arg_1_description'],
- },
- },
- "required": ["plugin_arg_1"],
- },
- } for k, v in get_fn_lib().items()
-]
-
-def inspect_dependency(chatbot, history):
- return True
-
-def eval_code(code, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
- import importlib
- try:
- tmp = get_fn_lib()[code['name']]
- fp, fn = tmp['module'], tmp['function']
- fn_plugin = getattr(importlib.import_module(fp, fn), fn)
- arg = json.loads(code['arguments'])['plugin_arg_1']
- yield from fn_plugin(arg, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port)
- except:
- from toolbox import trimmed_format_exc
- chatbot.append(["执行错误", f"\n```\n{trimmed_format_exc()}\n```\n"])
- yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
-
-def get_code_block(reply):
- import re
- pattern = r"```([\s\S]*?)```" # regex pattern to match code blocks
- matches = re.findall(pattern, reply) # find all code blocks in text
- if len(matches) != 1:
- raise RuntimeError("GPT is not generating proper code.")
- return matches[0].strip('python') # code block
+from toolbox import update_ui_lastest_msg, disable_auto_promotion
+from request_llm.bridge_all import predict_no_ui_long_connection
+from crazy_functions.crazy_utils import request_gpt_model_in_new_thread_with_ui_alive
+from crazy_functions.crazy_utils import input_clipping
+from crazy_functions.json_fns.pydantic_io import GptJsonIO, JsonStringError
+from crazy_functions.vt_fns.vt_state import VoidTerminalState
+from crazy_functions.vt_fns.vt_modify_config import modify_configuration_hot
+from crazy_functions.vt_fns.vt_modify_config import modify_configuration_reboot
+from crazy_functions.vt_fns.vt_call_plugin import execute_plugin
-@CatchException
-def 终端(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
- """
- txt 输入栏用户输入的文本, 例如需要翻译的一段话, 再例如一个包含了待处理文件的路径
- llm_kwargs gpt模型参数, 如温度和top_p等, 一般原样传递下去就行
- plugin_kwargs 插件模型的参数, 暂时没有用武之地
- chatbot 聊天显示框的句柄, 用于显示给用户
- history 聊天历史, 前情提要
- system_prompt 给gpt的静默提醒
- web_port 当前软件运行的端口号
- """
- # 清空历史, 以免输入溢出
- history = []
-
- # 基本信息：功能、贡献者
- chatbot.append(["虚空终端插件的功能？", "根据自然语言的描述, 执行任意插件的命令."])
- yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
-
- # 输入
- i_say = txt
- # 开始
- llm_kwargs_function_call = copy.deepcopy(llm_kwargs)
- llm_kwargs_function_call['llm_model'] = 'gpt-call-fn' # 修改调用函数
+class UserIntention(BaseModel):
+ user_prompt: str = Field(description="the content of user input", default="")
+ intention_type: str = Field(description="the type of user intention, choose from ['ModifyConfiguration', 'ExecutePlugin', 'Chat']", default="ExecutePlugin")
+ user_provide_file: bool = Field(description="whether the user provides a path to a file", default=False)
+ user_provide_url: bool = Field(description="whether the user provides a url", default=False)
+
+
+def chat(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_intention):
gpt_say = yield from request_gpt_model_in_new_thread_with_ui_alive(
- inputs=i_say, inputs_show_user=txt,
- llm_kwargs=llm_kwargs_function_call, chatbot=chatbot, history=[],
- sys_prompt=functions
+ inputs=txt, inputs_show_user=txt,
+ llm_kwargs=llm_kwargs, chatbot=chatbot, history=[],
+ sys_prompt=system_prompt
)
+ chatbot[-1] = [txt, gpt_say]
+ history.extend([txt, gpt_say])
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+ pass
+
+
+explain_intention_to_user = {
+ 'Chat': "聊天对话",
+ 'ExecutePlugin': "调用插件",
+ 'ModifyConfiguration': "修改配置",
+}
- # 将代码转为动画
- res = json.loads(gpt_say)['choices'][0]
- if res['finish_reason'] == 'function_call':
- code = json.loads(gpt_say)['choices'][0]
- yield from eval_code(code['message']['function_call'], llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port)
+
+def analyze_intention_with_simple_rules(txt):
+ user_intention = UserIntention()
+ user_intention.user_prompt = txt
+ is_certain = False
+
+ if '请问' in txt:
+ is_certain = True
+ user_intention.intention_type = 'Chat'
+
+ if '用插件' in txt:
+ is_certain = True
+ user_intention.intention_type = 'ExecutePlugin'
+
+ if '修改配置' in txt:
+ is_certain = True
+ user_intention.intention_type = 'ModifyConfiguration'
+
+ return is_certain, user_intention
+
+
+@CatchException
+def 虚空终端(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
+ disable_auto_promotion(chatbot=chatbot)
+ # 获取当前虚空终端状态
+ state = VoidTerminalState.get_state(chatbot)
+ appendix_msg = ""
+
+ # 用简单的关键词检测用户意图
+ is_certain, _ = analyze_intention_with_simple_rules(txt)
+ if txt.startswith('private_upload/') and len(txt) == 34:
+ state.set_state(chatbot=chatbot, key='has_provided_explaination', value=False)
+ appendix_msg = "\n\n**很好，您已经上传了文件**，现在请您描述您的需求。"
+
+ if is_certain or (state.has_provided_explaination):
+ # 如果意图明确，跳过提示环节
+ state.set_state(chatbot=chatbot, key='has_provided_explaination', value=True)
+ state.unlock_plugin(chatbot=chatbot)
+ yield from update_ui(chatbot=chatbot, history=history)
+ yield from 虚空终端主路由(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port)
+ return
else:
- chatbot.append(["无法调用相关功能", res])
- yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+ # 如果意图模糊，提示
+ state.set_state(chatbot=chatbot, key='has_provided_explaination', value=True)
+ state.lock_plugin(chatbot=chatbot)
+ chatbot.append(("虚空终端状态:", explain_msg+appendix_msg))
+ yield from update_ui(chatbot=chatbot, history=history)
+ return
+
+
+
+def 虚空终端主路由(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
+ history = []
+ chatbot.append(("虚空终端状态: ", f"正在执行任务: {txt}"))
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
+
+ # ⭐ ⭐ ⭐ 分析用户意图
+ is_certain, user_intention = analyze_intention_with_simple_rules(txt)
+ if not is_certain:
+ yield from update_ui_lastest_msg(
+ lastmsg=f"正在执行任务: {txt}\n\n分析用户意图中", chatbot=chatbot, history=history, delay=0)
+ gpt_json_io = GptJsonIO(UserIntention)
+ rf_req = "\nchoose from ['ModifyConfiguration', 'ExecutePlugin', 'Chat']"
+ inputs = "Analyze the intention of the user according to following user input: \n\n" + \
+ ">> " + (txt+rf_req).rstrip('\n').replace('\n','\n>> ') + '\n\n' + gpt_json_io.format_instructions
+ run_gpt_fn = lambda inputs, sys_prompt: predict_no_ui_long_connection(
+ inputs=inputs, llm_kwargs=llm_kwargs, history=[], sys_prompt=sys_prompt, observe_window=[])
+ analyze_res = run_gpt_fn(inputs, "")
+ try:
+ user_intention = gpt_json_io.generate_output_auto_repair(analyze_res, run_gpt_fn)
+ lastmsg=f"正在执行任务: {txt}\n\n用户意图理解: 意图={explain_intention_to_user[user_intention.intention_type]}",
+ except JsonStringError as e:
+ yield from update_ui_lastest_msg(
+ lastmsg=f"正在执行任务: {txt}\n\n用户意图理解: 失败 当前语言模型（{llm_kwargs['llm_model']}）不能理解您的意图", chatbot=chatbot, history=history, delay=0)
+ return
+ else:
+ pass
+
+ yield from update_ui_lastest_msg(
+ lastmsg=f"正在执行任务: {txt}\n\n用户意图理解: 意图={explain_intention_to_user[user_intention.intention_type]}",
+ chatbot=chatbot, history=history, delay=0)
+
+ # 用户意图: 修改本项目的配置
+ if user_intention.intention_type == 'ModifyConfiguration':
+ yield from modify_configuration_reboot(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_intention)
+
+ # 用户意图: 调度插件
+ if user_intention.intention_type == 'ExecutePlugin':
+ yield from execute_plugin(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_intention)
+
+ # 用户意图: 聊天
+ if user_intention.intention_type == 'Chat':
+ yield from chat(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, user_intention)
+ return
diff --git "a/crazy_functions/\350\257\255\351\237\263\345\212\251\346\211\213.py" "b/crazy_functions/\350\257\255\351\237\263\345\212\251\346\211\213.py"
index 6fe1afc0b61a0a09282f8314c5eceeeba5bf328e..b1c8c41e63f3cedc97affb66d5ea124f9c5d4837 100644
--- "a/crazy_functions/\350\257\255\351\237\263\345\212\251\346\211\213.py"
+++ "b/crazy_functions/\350\257\255\351\237\263\345\212\251\346\211\213.py"
@@ -80,9 +80,9 @@ class InterviewAssistant(AliyunASR):
def __init__(self):
self.capture_interval = 0.5 # second
self.stop = False
- self.parsed_text = ""
- self.parsed_sentence = ""
- self.buffered_sentence = ""
+ self.parsed_text = "" # 下个句子中已经说完的部分, 由 test_on_result_chg() 写入
+ self.parsed_sentence = "" # 某段话的整个句子,由 test_on_sentence_end() 写入
+ self.buffered_sentence = "" #
self.event_on_result_chg = threading.Event()
self.event_on_entence_end = threading.Event()
self.event_on_commit_question = threading.Event()
@@ -132,7 +132,7 @@ class InterviewAssistant(AliyunASR):
self.plugin_wd.feed()
if self.event_on_result_chg.is_set():
- # update audio decode result
+ # called when some words have finished
self.event_on_result_chg.clear()
chatbot[-1] = list(chatbot[-1])
chatbot[-1][0] = self.buffered_sentence + self.parsed_text
@@ -144,7 +144,11 @@ class InterviewAssistant(AliyunASR):
# called when a sentence has ended
self.event_on_entence_end.clear()
self.parsed_text = self.parsed_sentence
- self.buffered_sentence += self.parsed_sentence
+ self.buffered_sentence += self.parsed_text
+ chatbot[-1] = list(chatbot[-1])
+ chatbot[-1][0] = self.buffered_sentence
+ history = chatbot2history(chatbot)
+ yield from update_ui(chatbot=chatbot, history=history) # 刷新界面
if self.event_on_commit_question.is_set():
# called when a question should be commited
diff --git "a/crazy_functions/\350\260\267\346\255\214\346\243\200\347\264\242\345\260\217\345\212\251\346\211\213.py" "b/crazy_functions/\350\260\267\346\255\214\346\243\200\347\264\242\345\260\217\345\212\251\346\211\213.py"
index 46c10de4f7148e2fa5de13b93183ab119f1af165..05e80d2c8432cd3db46b8ca5a30b045ca9f5c7ca 100644
--- "a/crazy_functions/\350\260\267\346\255\214\346\243\200\347\264\242\345\260\217\345\212\251\346\211\213.py"
+++ "b/crazy_functions/\350\260\267\346\255\214\346\243\200\347\264\242\345\260\217\345\212\251\346\211\213.py"
@@ -1,26 +1,75 @@
from .crazy_utils import request_gpt_model_in_new_thread_with_ui_alive
-from toolbox import CatchException, report_execption, write_results_to_file
-from toolbox import update_ui
+from toolbox import CatchException, report_execption, promote_file_to_downloadzone
+from toolbox import update_ui, update_ui_lastest_msg, disable_auto_promotion, write_history_to_file
+import logging
+import requests
+import time
+import random
+
+ENABLE_ALL_VERSION_SEARCH = True
def get_meta_information(url, chatbot, history):
- import requests
import arxiv
import difflib
+ import re
from bs4 import BeautifulSoup
from toolbox import get_conf
+ from urllib.parse import urlparse
+ session = requests.session()
+
proxies, = get_conf('proxies')
headers = {
- 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36',
+ 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36',
+ 'Accept-Encoding': 'gzip, deflate, br',
+ 'Accept-Language': 'en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7',
+ 'Cache-Control':'max-age=0',
+ 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7',
+ 'Connection': 'keep-alive'
}
- # 发送 GET 请求
- response = requests.get(url, proxies=proxies, headers=headers)
+ session.proxies.update(proxies)
+ session.headers.update(headers)
+ response = session.get(url)
# 解析网页内容
soup = BeautifulSoup(response.text, "html.parser")
def string_similar(s1, s2):
return difflib.SequenceMatcher(None, s1, s2).quick_ratio()
+ if ENABLE_ALL_VERSION_SEARCH:
+ def search_all_version(url):
+ time.sleep(random.randint(1,5)) # 睡一会防止触发google反爬虫
+ response = session.get(url)
+ soup = BeautifulSoup(response.text, "html.parser")
+
+ for result in soup.select(".gs_ri"):
+ try:
+ url = result.select_one(".gs_rt").a['href']
+ except:
+ continue
+ arxiv_id = extract_arxiv_id(url)
+ if not arxiv_id:
+ continue
+ search = arxiv.Search(
+ id_list=[arxiv_id],
+ max_results=1,
+ sort_by=arxiv.SortCriterion.Relevance,
+ )
+ try: paper = next(search.results())
+ except: paper = None
+ return paper
+
+ return None
+
+ def extract_arxiv_id(url):
+ # 返回给定的url解析出的arxiv_id，如url未成功匹配返回None
+ pattern = r'arxiv.org/abs/([^/]+)'
+ match = re.search(pattern, url)
+ if match:
+ return match.group(1)
+ else:
+ return None
+
profile = []
# 获取所有文章的标题和作者
for result in soup.select(".gs_ri"):
@@ -31,32 +80,45 @@ def get_meta_information(url, chatbot, history):
except:
citation = 'cited by 0'
abstract = result.select_one(".gs_rs").text.strip() # 摘要在 .gs_rs 中的文本，需要清除首尾空格
+
+ # 首先在arxiv上搜索，获取文章摘要
search = arxiv.Search(
query = title,
max_results = 1,
sort_by = arxiv.SortCriterion.Relevance,
)
- try:
- paper = next(search.results())
- if string_similar(title, paper.title) > 0.90: # same paper
- abstract = paper.summary.replace('\n', ' ')
- is_paper_in_arxiv = True
- else: # different paper
- abstract = abstract
- is_paper_in_arxiv = False
- paper = next(search.results())
- except:
+ try: paper = next(search.results())
+ except: paper = None
+
+ is_match = paper is not None and string_similar(title, paper.title) > 0.90
+
+ # 如果在Arxiv上匹配失败，检索文章的历史版本的题目
+ if not is_match and ENABLE_ALL_VERSION_SEARCH:
+ other_versions_page_url = [tag['href'] for tag in result.select_one('.gs_flb').select('.gs_nph') if 'cluster' in tag['href']]
+ if len(other_versions_page_url) > 0:
+ other_versions_page_url = other_versions_page_url[0]
+ paper = search_all_version('http://' + urlparse(url).netloc + other_versions_page_url)
+ is_match = paper is not None and string_similar(title, paper.title) > 0.90
+
+ if is_match:
+ # same paper
+ abstract = paper.summary.replace('\n', ' ')
+ is_paper_in_arxiv = True
+ else:
+ # different paper
abstract = abstract
is_paper_in_arxiv = False
- print(title)
- print(author)
- print(citation)
+
+ logging.info('[title]:' + title)
+ logging.info('[author]:' + author)
+ logging.info('[citation]:' + citation)
+
profile.append({
- 'title':title,
- 'author':author,
- 'citation':citation,
- 'abstract':abstract,
- 'is_paper_in_arxiv':is_paper_in_arxiv,
+ 'title': title,
+ 'author': author,
+ 'citation': citation,
+ 'abstract': abstract,
+ 'is_paper_in_arxiv': is_paper_in_arxiv,
})
chatbot[-1] = [chatbot[-1][0], title + f'\n\n是否在arxiv中（不在arxiv中无法获取完整摘要）:{is_paper_in_arxiv}\n\n' + abstract]
@@ -65,6 +127,7 @@ def get_meta_information(url, chatbot, history):
@CatchException
def 谷歌检索小助手(txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port):
+ disable_auto_promotion(chatbot=chatbot)
# 基本信息：功能、贡献者
chatbot.append([
"函数插件功能？",
@@ -86,6 +149,9 @@ def 谷歌检索小助手(txt, llm_kwargs, plugin_kwargs, chatbot, history, syst
# 清空历史，以免输入溢出
history = []
meta_paper_info_list = yield from get_meta_information(txt, chatbot, history)
+ if len(meta_paper_info_list) == 0:
+ yield from update_ui_lastest_msg(lastmsg='获取文献失败，可能触发了google反爬虫机制。',chatbot=chatbot, history=history, delay=0)
+ return
batchsize = 5
for batch in range(math.ceil(len(meta_paper_info_list)/batchsize)):
if len(meta_paper_info_list[:batchsize]) > 0:
@@ -107,6 +173,7 @@ def 谷歌检索小助手(txt, llm_kwargs, plugin_kwargs, chatbot, history, syst
"已经全部完成，您可以试试让AI写一个Related Works，例如您可以继续输入Write a \"Related Works\" section about \"你搜索的研究领域\" for me."])
msg = '正常'
yield from update_ui(chatbot=chatbot, history=history, msg=msg) # 刷新界面
- res = write_results_to_file(history)
- chatbot.append(("完成了吗？", res));
+ path = write_history_to_file(history)
+ promote_file_to_downloadzone(path, chatbot=chatbot)
+ chatbot.append(("完成了吗？", path));
yield from update_ui(chatbot=chatbot, history=history, msg=msg) # 刷新界面
diff --git a/docker-compose.yml b/docker-compose.yml
index cf753b56ca7673242f7dc7327d952944faadfd0e..2387527fce707643a03c763aa177826108d5cfbc 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -1,7 +1,7 @@
#【请修改完参数后，删除此行】请在以下方案中选择一种，然后删除其他的方案，最后docker-compose up运行 | Please choose from one of these options below, delete other options as well as This Line
## ===================================================
-## 【方案一】 如果不需要运行本地模型（仅chatgpt,newbing类远程服务）
+## 【方案一】 如果不需要运行本地模型（仅 chatgpt, azure, 星火, 千帆, claude 等在线大模型服务）
## ===================================================
version: '3'
services:
@@ -13,7 +13,7 @@ services:
USE_PROXY: ' True '
proxies: ' { "http": "socks5h://localhost:10880", "https": "socks5h://localhost:10880", } '
LLM_MODEL: ' gpt-3.5-turbo '
- AVAIL_LLM_MODELS: ' ["gpt-3.5-turbo", "api2d-gpt-3.5-turbo", "gpt-4", "api2d-gpt-4", "newbing"] '
+ AVAIL_LLM_MODELS: ' ["gpt-3.5-turbo", "api2d-gpt-3.5-turbo", "gpt-4", "api2d-gpt-4", "sparkv2", "qianfan"] '
WEB_PORT: ' 22303 '
ADD_WAIFU: ' True '
# THEME: ' Chuanhu-Small-and-Beautiful '
diff --git a/docs/Dockerfile+ChatGLM b/docs/Dockerfile+ChatGLM
index 75c1fa7479de516655f0e1166642f0eb37e1b1a9..f0d7c7586f14f3a77557684d62609dba8d5a83a5 100644
--- a/docs/Dockerfile+ChatGLM
+++ b/docs/Dockerfile+ChatGLM
@@ -1,62 +1,2 @@
-# How to build | 如何构建: docker build -t gpt-academic --network=host -f Dockerfile+ChatGLM .
-# How to run | (1) 我想直接一键运行（选择0号GPU）: docker run --rm -it --net=host --gpus \"device=0\" gpt-academic
-# How to run | (2) 我想运行之前进容器做一些调整（选择1号GPU）: docker run --rm -it --net=host --gpus \"device=1\" gpt-academic bash
-
-# 从NVIDIA源，从而支持显卡运损（检查宿主的nvidia-smi中的cuda版本必须>=11.3）
-FROM nvidia/cuda:11.3.1-runtime-ubuntu20.04
-ARG useProxyNetwork=''
-RUN apt-get update
-RUN apt-get install -y curl proxychains curl
-RUN apt-get install -y git python python3 python-dev python3-dev --fix-missing
+# 此Dockerfile不再维护，请前往docs/GithubAction+ChatGLM+Moss
-# 配置代理网络（构建Docker镜像时使用）
-# # comment out below if you do not need proxy network | 如果不需要翻墙 - 从此行向下删除
-RUN $useProxyNetwork curl cip.cc
-RUN sed -i '$ d' /etc/proxychains.conf
-RUN sed -i '$ d' /etc/proxychains.conf
-# 在这里填写主机的代理协议（用于从github拉取代码）
-RUN echo "socks5 127.0.0.1 10880" >> /etc/proxychains.conf
-ARG useProxyNetwork=proxychains
-# # comment out above if you do not need proxy network | 如果不需要翻墙 - 从此行向上删除
-
-
-# use python3 as the system default python
-RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.8
-# 下载pytorch
-RUN $useProxyNetwork python3 -m pip install torch --extra-index-url https://download.pytorch.org/whl/cu113
-# 下载分支
-WORKDIR /gpt
-RUN $useProxyNetwork git clone https://github.com/binary-husky/gpt_academic.git
-WORKDIR /gpt/gpt_academic
-RUN $useProxyNetwork python3 -m pip install -r requirements.txt
-RUN $useProxyNetwork python3 -m pip install -r request_llm/requirements_chatglm.txt
-RUN $useProxyNetwork python3 -m pip install -r request_llm/requirements_newbing.txt
-
-# 预热CHATGLM参数（非必要 可选步骤）
-RUN echo ' \n\
-from transformers import AutoModel, AutoTokenizer \n\
-chatglm_tokenizer = AutoTokenizer.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True) \n\
-chatglm_model = AutoModel.from_pretrained("THUDM/chatglm-6b", trust_remote_code=True).float() ' >> warm_up_chatglm.py
-RUN python3 -u warm_up_chatglm.py
-
-# 禁用缓存，确保更新代码
-ADD "https://www.random.org/cgi-bin/randbyte?nbytes=10&format=h" skipcache
-RUN $useProxyNetwork git pull
-
-# 预热Tiktoken模块
-RUN python3 -c 'from check_proxy import warm_up_modules; warm_up_modules()'
-
-# 为chatgpt-academic配置代理和API-KEY （非必要 可选步骤）
-# 可同时填写多个API-KEY，支持openai的key和api2d的key共存，用英文逗号分割，例如API_KEY = "sk-openaikey1,fkxxxx-api2dkey2,........"
-# LLM_MODEL 是选择初始的模型
-# LOCAL_MODEL_DEVICE 是选择chatglm等本地模型运行的设备，可选 cpu 和 cuda
-# [说明: 以下内容与`config.py`一一对应，请查阅config.py来完成一下配置的填写]
-RUN echo ' \n\
-API_KEY = "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,fkxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \n\
-USE_PROXY = True \n\
-LLM_MODEL = "chatglm" \n\
-LOCAL_MODEL_DEVICE = "cuda" \n\
-proxies = { "http": "socks5h://localhost:10880", "https": "socks5h://localhost:10880", } ' >> config_private.py
-
-# 启动
-CMD ["python3", "-u", "main.py"]
diff --git a/docs/Dockerfile+JittorLLM b/docs/Dockerfile+JittorLLM
index ea53402926b8a9529f3224d8e9b65c0a9476b5a5..2bd1237b9386545ed681a9540af968872bddd140 100644
--- a/docs/Dockerfile+JittorLLM
+++ b/docs/Dockerfile+JittorLLM
@@ -1,59 +1 @@
-# How to build | 如何构建: docker build -t gpt-academic-jittor --network=host -f Dockerfile+ChatGLM .
-# How to run | (1) 我想直接一键运行（选择0号GPU）: docker run --rm -it --net=host --gpus \"device=0\" gpt-academic-jittor bash
-# How to run | (2) 我想运行之前进容器做一些调整（选择1号GPU）: docker run --rm -it --net=host --gpus \"device=1\" gpt-academic-jittor bash
-
-# 从NVIDIA源，从而支持显卡运损（检查宿主的nvidia-smi中的cuda版本必须>=11.3）
-FROM nvidia/cuda:11.3.1-runtime-ubuntu20.04
-ARG useProxyNetwork=''
-RUN apt-get update
-RUN apt-get install -y curl proxychains curl g++
-RUN apt-get install -y git python python3 python-dev python3-dev --fix-missing
-
-# 配置代理网络（构建Docker镜像时使用）
-# # comment out below if you do not need proxy network | 如果不需要翻墙 - 从此行向下删除
-RUN $useProxyNetwork curl cip.cc
-RUN sed -i '$ d' /etc/proxychains.conf
-RUN sed -i '$ d' /etc/proxychains.conf
-# 在这里填写主机的代理协议（用于从github拉取代码）
-RUN echo "socks5 127.0.0.1 10880" >> /etc/proxychains.conf
-ARG useProxyNetwork=proxychains
-# # comment out above if you do not need proxy network | 如果不需要翻墙 - 从此行向上删除
-
-
-# use python3 as the system default python
-RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.8
-# 下载pytorch
-RUN $useProxyNetwork python3 -m pip install torch --extra-index-url https://download.pytorch.org/whl/cu113
-# 下载分支
-WORKDIR /gpt
-RUN $useProxyNetwork git clone https://github.com/binary-husky/gpt_academic.git
-WORKDIR /gpt/gpt_academic
-RUN $useProxyNetwork python3 -m pip install -r requirements.txt
-RUN $useProxyNetwork python3 -m pip install -r request_llm/requirements_chatglm.txt
-RUN $useProxyNetwork python3 -m pip install -r request_llm/requirements_newbing.txt
-RUN $useProxyNetwork python3 -m pip install -r request_llm/requirements_jittorllms.txt -i https://pypi.jittor.org/simple -I
-
-# 下载JittorLLMs
-RUN $useProxyNetwork git clone https://github.com/binary-husky/JittorLLMs.git --depth 1 request_llm/jittorllms
-
-# 禁用缓存，确保更新代码
-ADD "https://www.random.org/cgi-bin/randbyte?nbytes=10&format=h" skipcache
-RUN $useProxyNetwork git pull
-
-# 预热Tiktoken模块
-RUN python3 -c 'from check_proxy import warm_up_modules; warm_up_modules()'
-
-# 为chatgpt-academic配置代理和API-KEY （非必要 可选步骤）
-# 可同时填写多个API-KEY，支持openai的key和api2d的key共存，用英文逗号分割，例如API_KEY = "sk-openaikey1,fkxxxx-api2dkey2,........"
-# LLM_MODEL 是选择初始的模型
-# LOCAL_MODEL_DEVICE 是选择chatglm等本地模型运行的设备，可选 cpu 和 cuda
-# [说明: 以下内容与`config.py`一一对应，请查阅config.py来完成一下配置的填写]
-RUN echo ' \n\
-API_KEY = "sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,fkxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \n\
-USE_PROXY = True \n\
-LLM_MODEL = "chatglm" \n\
-LOCAL_MODEL_DEVICE = "cuda" \n\
-proxies = { "http": "socks5h://localhost:10880", "https": "socks5h://localhost:10880", } ' >> config_private.py
-
-# 启动
-CMD ["python3", "-u", "main.py"]
+# 此Dockerfile不再维护，请前往docs/GithubAction+JittorLLMs
\ No newline at end of file
diff --git a/docs/Dockerfile+NoLocal+Latex b/docs/Dockerfile+NoLocal+Latex
index 0f9ac8a156b76c72c58764dd3e33f46715eeb9df..a0f162af784263c47e0df85b4ba1ef49e031bf8c 100644
--- a/docs/Dockerfile+NoLocal+Latex
+++ b/docs/Dockerfile+NoLocal+Latex
@@ -1,27 +1 @@
-# 此Dockerfile适用于“无本地模型”的环境构建，如果需要使用chatglm等本地模型，请参考 docs/Dockerfile+ChatGLM
-# - 1 修改 `config.py`
-# - 2 构建 docker build -t gpt-academic-nolocal-latex -f docs/Dockerfile+NoLocal+Latex .
-# - 3 运行 docker run -v /home/fuqingxu/arxiv_cache:/root/arxiv_cache --rm -it --net=host gpt-academic-nolocal-latex
-
-FROM fuqingxu/python311_texlive_ctex:latest
-
-# 指定路径
-WORKDIR /gpt
-
-ARG useProxyNetwork=''
-
-RUN $useProxyNetwork pip3 install gradio openai numpy arxiv rich -i https://pypi.douban.com/simple/
-RUN $useProxyNetwork pip3 install colorama Markdown pygments pymupdf -i https://pypi.douban.com/simple/
-
-# 装载项目文件
-COPY . .
-
-
-# 安装依赖
-RUN $useProxyNetwork pip3 install -r requirements.txt -i https://pypi.douban.com/simple/
-
-# 可选步骤，用于预热模块
-RUN python3 -c 'from check_proxy import warm_up_modules; warm_up_modules()'
-
-# 启动
-CMD ["python3", "-u", "main.py"]
+# 此Dockerfile不再维护，请前往docs/GithubAction+NoLocal+Latex
diff --git a/docs/GithubAction+AllCapacity b/docs/GithubAction+AllCapacity
new file mode 100644
index 0000000000000000000000000000000000000000..f6f05daa8f5c93dcc15b4fc9098059e6dd35115b
--- /dev/null
+++ b/docs/GithubAction+AllCapacity
@@ -0,0 +1,37 @@
+# docker build -t gpt-academic-all-capacity -f docs/GithubAction+AllCapacity --network=host --build-arg http_proxy=http://localhost:10881 --build-arg https_proxy=http://localhost:10881 .
+
+# 从NVIDIA源，从而支持显卡（检查宿主的nvidia-smi中的cuda版本必须>=11.3）
+FROM fuqingxu/11.3.1-runtime-ubuntu20.04-with-texlive:latest
+
+# use python3 as the system default python
+WORKDIR /gpt
+RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.8
+# 下载pytorch
+RUN python3 -m pip install torch --extra-index-url https://download.pytorch.org/whl/cu113
+# 准备pip依赖
+RUN python3 -m pip install openai numpy arxiv rich
+RUN python3 -m pip install colorama Markdown pygments pymupdf
+RUN python3 -m pip install python-docx moviepy pdfminer
+RUN python3 -m pip install zh_langchain==0.2.1
+RUN python3 -m pip install nougat-ocr
+RUN python3 -m pip install rarfile py7zr
+RUN python3 -m pip install aliyun-python-sdk-core==2.13.3 pyOpenSSL scipy git+https://github.com/aliyun/alibabacloud-nls-python-sdk.git
+# 下载分支
+WORKDIR /gpt
+RUN git clone --depth=1 https://github.com/binary-husky/gpt_academic.git
+WORKDIR /gpt/gpt_academic
+RUN git clone https://github.com/OpenLMLab/MOSS.git request_llm/moss
+
+RUN python3 -m pip install -r requirements.txt
+RUN python3 -m pip install -r request_llm/requirements_moss.txt
+RUN python3 -m pip install -r request_llm/requirements_qwen.txt
+RUN python3 -m pip install -r request_llm/requirements_chatglm.txt
+RUN python3 -m pip install -r request_llm/requirements_newbing.txt
+
+
+
+# 预热Tiktoken模块
+RUN python3 -c 'from check_proxy import warm_up_modules; warm_up_modules()'
+
+# 启动
+CMD ["python3", "-u", "main.py"]
diff --git a/docs/GithubAction+ChatGLM+Moss b/docs/GithubAction+ChatGLM+Moss
index 7bb11a2c636c2519f50c3e004a9c6397b83bdfa5..3087d5513c646759c33f08d29d62ed68602eaa93 100644
--- a/docs/GithubAction+ChatGLM+Moss
+++ b/docs/GithubAction+ChatGLM+Moss
@@ -1,7 +1,6 @@
# 从NVIDIA源，从而支持显卡运损（检查宿主的nvidia-smi中的cuda版本必须>=11.3）
FROM nvidia/cuda:11.3.1-runtime-ubuntu20.04
-ARG useProxyNetwork=''
RUN apt-get update
RUN apt-get install -y curl proxychains curl gcc
RUN apt-get install -y git python python3 python-dev python3-dev --fix-missing
diff --git a/docs/GithubAction+NoLocal+Latex b/docs/GithubAction+NoLocal+Latex
index 5ff9bb828ccb02e4ef528c0177b3d8dbef42b523..2f2608ccb77eb01e162b0176baf8f39dc3391019 100644
--- a/docs/GithubAction+NoLocal+Latex
+++ b/docs/GithubAction+NoLocal+Latex
@@ -1,6 +1,6 @@
# 此Dockerfile适用于“无本地模型”的环境构建，如果需要使用chatglm等本地模型，请参考 docs/Dockerfile+ChatGLM
# - 1 修改 `config.py`
-# - 2 构建 docker build -t gpt-academic-nolocal-latex -f docs/Dockerfile+NoLocal+Latex .
+# - 2 构建 docker build -t gpt-academic-nolocal-latex -f docs/GithubAction+NoLocal+Latex .
# - 3 运行 docker run -v /home/fuqingxu/arxiv_cache:/root/arxiv_cache --rm -it --net=host gpt-academic-nolocal-latex
FROM fuqingxu/python311_texlive_ctex:latest
@@ -10,6 +10,10 @@ WORKDIR /gpt
RUN pip3 install gradio openai numpy arxiv rich
RUN pip3 install colorama Markdown pygments pymupdf
+RUN pip3 install python-docx moviepy pdfminer
+RUN pip3 install zh_langchain==0.2.1
+RUN pip3 install nougat-ocr
+RUN pip3 install aliyun-python-sdk-core==2.13.3 pyOpenSSL scipy git+https://github.com/aliyun/alibabacloud-nls-python-sdk.git
# 装载项目文件
COPY . .
diff --git a/docs/translate_english.json b/docs/translate_english.json
index 9721dcfa1572120ccc157d3445bb4dddc03d4790..667956c82ac9f3b10413619e4fa6f134e56639db 100644
--- a/docs/translate_english.json
+++ b/docs/translate_english.json
@@ -2161,5 +2161,292 @@
"在运行过程中动态地修改配置": "Dynamically modify configurations during runtime",
"请先把模型切换至gpt-*或者api2d-*": "Please switch the model to gpt-* or api2d-* first",
"获取简单聊天的句柄": "Get handle of simple chat",
- "获取插件的默认参数": "Get default parameters of plugin"
+ "获取插件的默认参数": "Get default parameters of plugin",
+ "GROBID服务不可用": "GROBID service is unavailable",
+ "请问": "May I ask",
+ "如果等待时间过长": "If the waiting time is too long",
+ "编程": "programming",
+ "5. 现在": "5. Now",
+ "您不必读这个else分支": "You don't have to read this else branch",
+ "用插件实现": "Implement with plugins",
+ "插件分类默认选项": "Default options for plugin classification",
+ "填写多个可以均衡负载": "Filling in multiple can balance the load",
+ "色彩主题": "Color theme",
+ "可能附带额外依赖 -=-=-=-=-=-=-": "May come with additional dependencies -=-=-=-=-=-=-",
+ "讯飞星火认知大模型": "Xunfei Xinghuo cognitive model",
+ "ParsingLuaProject的所有源文件 | 输入参数为路径": "All source files of ParsingLuaProject | Input parameter is path",
+ "复制以下空间https": "Copy the following space https",
+ "如果意图明确": "If the intention is clear",
+ "如系统是Linux": "If the system is Linux",
+ "├── 语音功能": "├── Voice function",
+ "见Github wiki": "See Github wiki",
+ "⭐ ⭐ ⭐ 立即应用配置": "⭐ ⭐ ⭐ Apply configuration immediately",
+ "现在您只需要再次重复一次您的指令即可": "Now you just need to repeat your command again",
+ "没辙了": "No way",
+ "解析Jupyter Notebook文件 | 输入参数为路径": "Parse Jupyter Notebook file | Input parameter is path",
+ "⭐ ⭐ ⭐ 确认插件参数": "⭐ ⭐ ⭐ Confirm plugin parameters",
+ "找不到合适插件执行该任务": "Cannot find a suitable plugin to perform this task",
+ "接驳VoidTerminal": "Connect to VoidTerminal",
+ "**很好": "**Very good",
+ "对话|编程": "Conversation|Programming",
+ "对话|编程|学术": "Conversation|Programming|Academic",
+ "4. 建议使用 GPT3.5 或更强的模型": "4. It is recommended to use GPT3.5 or a stronger model",
+ "「请调用插件翻译PDF论文": "Please call the plugin to translate the PDF paper",
+ "3. 如果您使用「调用插件xxx」、「修改配置xxx」、「请问」等关键词": "3. If you use keywords such as 'call plugin xxx', 'modify configuration xxx', 'please', etc.",
+ "以下是一篇学术论文的基本信息": "The following is the basic information of an academic paper",
+ "GROBID服务器地址": "GROBID server address",
+ "修改配置": "Modify configuration",
+ "理解PDF文档的内容并进行回答 | 输入参数为路径": "Understand the content of the PDF document and answer | Input parameter is path",
+ "对于需要高级参数的插件": "For plugins that require advanced parameters",
+ "🏃♂️🏃♂️🏃♂️ 主进程执行": "Main process execution 🏃♂️🏃♂️🏃♂️",
+ "没有填写 HUGGINGFACE_ACCESS_TOKEN": "HUGGINGFACE_ACCESS_TOKEN not filled in",
+ "调度插件": "Scheduling plugin",
+ "语言模型": "Language model",
+ "├── ADD_WAIFU 加一个live2d装饰": "├── ADD_WAIFU Add a live2d decoration",
+ "初始化": "Initialization",
+ "选择了不存在的插件": "Selected a non-existent plugin",
+ "修改本项目的配置": "Modify the configuration of this project",
+ "如果输入的文件路径是正确的": "If the input file path is correct",
+ "2. 您可以打开插件下拉菜单以了解本项目的各种能力": "2. You can open the plugin dropdown menu to learn about various capabilities of this project",
+ "VoidTerminal插件说明": "VoidTerminal plugin description",
+ "无法理解您的需求": "Unable to understand your requirements",
+ "默认 AdvancedArgs = False": "Default AdvancedArgs = False",
+ "「请问Transformer网络的结构是怎样的": "What is the structure of the Transformer network?",
+ "比如1812.10695": "For example, 1812.10695",
+ "翻译README或MD": "Translate README or MD",
+ "读取新配置中": "Reading new configuration",
+ "假如偏离了您的要求": "If it deviates from your requirements",
+ "├── THEME 色彩主题": "├── THEME color theme",
+ "如果还找不到": "If still not found",
+ "问": "Ask",
+ "请检查系统字体": "Please check system fonts",
+ "如果错误": "If there is an error",
+ "作为替代": "As an alternative",
+ "ParseJavaProject的所有源文件 | 输入参数为路径": "All source files of ParseJavaProject | Input parameter is path",
+ "比对相同参数时生成的url与自己代码生成的url是否一致": "Check if the generated URL matches the one generated by your code when comparing the same parameters",
+ "清除本地缓存数据": "Clear local cache data",
+ "使用谷歌学术检索助手搜索指定URL的结果 | 输入参数为谷歌学术搜索页的URL": "Use Google Scholar search assistant to search for results of a specific URL | Input parameter is the URL of Google Scholar search page",
+ "运行方法": "Running method",
+ "您已经上传了文件**": "You have uploaded the file **",
+ "「给爷翻译Arxiv论文": "Translate Arxiv papers for me",
+ "请修改config中的GROBID_URL": "Please modify GROBID_URL in the config",
+ "处理特殊情况": "Handling special cases",
+ "不要自己瞎搞！」": "Don't mess around by yourself!",
+ "LoadConversationHistoryArchive | 输入参数为路径": "LoadConversationHistoryArchive | Input parameter is a path",
+ "| 输入参数是一个问题": "| Input parameter is a question",
+ "├── CHATBOT_HEIGHT 对话窗的高度": "├── CHATBOT_HEIGHT Height of the chat window",
+ "对C": "To C",
+ "默认关闭": "Default closed",
+ "当前进度": "Current progress",
+ "HUGGINGFACE的TOKEN": "HUGGINGFACE's TOKEN",
+ "查找可用插件中": "Searching for available plugins",
+ "下载LLAMA时起作用 https": "Works when downloading LLAMA https",
+ "使用 AK": "Using AK",
+ "正在执行任务": "Executing task",
+ "保存当前的对话 | 不需要输入参数": "Save current conversation | No input parameters required",
+ "对话": "Conversation",
+ "图中鲜花怒放": "Flowers blooming in the picture",
+ "批量将Markdown文件中文翻译为英文 | 输入参数为路径或上传压缩包": "Batch translate Chinese to English in Markdown files | Input parameter is a path or upload a compressed package",
+ "ParsingCSharpProject的所有源文件 | 输入参数为路径": "ParsingCSharpProject's all source files | Input parameter is a path",
+ "为我翻译PDF论文": "Translate PDF papers for me",
+ "聊天对话": "Chat conversation",
+ "拼接鉴权参数": "Concatenate authentication parameters",
+ "请检查config中的GROBID_URL": "Please check the GROBID_URL in the config",
+ "拼接字符串": "Concatenate strings",
+ "您的意图可以被识别的更准确": "Your intent can be recognized more accurately",
+ "该模型有七个 bin 文件": "The model has seven bin files",
+ "但思路相同": "But the idea is the same",
+ "你需要翻译": "You need to translate",
+ "或者描述文件所在的路径": "Or the path of the description file",
+ "请您上传文件": "Please upload the file",
+ "不常用": "Not commonly used",
+ "尚未充分测试的实验性插件 & 需要额外依赖的插件 -=--=-": "Experimental plugins that have not been fully tested & plugins that require additional dependencies -=--=-",
+ "⭐ ⭐ ⭐ 选择插件": "⭐ ⭐ ⭐ Select plugin",
+ "当前配置不允许被修改！如需激活本功能": "The current configuration does not allow modification! To activate this feature",
+ "正在连接GROBID服务": "Connecting to GROBID service",
+ "用户图形界面布局依赖关系示意图": "Diagram of user interface layout dependencies",
+ "是否允许通过自然语言描述修改本页的配置": "Allow modifying the configuration of this page through natural language description",
+ "self.chatbot被序列化": "self.chatbot is serialized",
+ "本地Latex论文精细翻译 | 输入参数是路径": "Locally translate Latex papers with fine-grained translation | Input parameter is the path",
+ "抱歉": "Sorry",
+ "以下这部分是最早加入的最稳定的模型 -=-=-=-=-=-=-": "The following section is the earliest and most stable model added",
+ "「用插件翻译README": "Translate README with plugins",
+ "如果不正确": "If incorrect",
+ "⭐ ⭐ ⭐ 读取可配置项目条目": "⭐ ⭐ ⭐ Read configurable project entries",
+ "开始语言对话 | 没有输入参数": "Start language conversation | No input parameters",
+ "谨慎操作 | 不需要输入参数": "Handle with caution | No input parameters required",
+ "对英文Latex项目全文进行纠错处理 | 输入参数为路径或上传压缩包": "Correct the entire English Latex project | Input parameter is the path or upload compressed package",
+ "如果需要处理文件": "If file processing is required",
+ "提供图像的内容": "Provide the content of the image",
+ "查看历史上的今天事件 | 不需要输入参数": "View historical events of today | No input parameters required",
+ "这个稍微啰嗦一点": "This is a bit verbose",
+ "多线程解析并翻译此项目的源码 | 不需要输入参数": "Parse and translate the source code of this project in multi-threading | No input parameters required",
+ "此处打印出建立连接时候的url": "Print the URL when establishing the connection here",
+ "精准翻译PDF论文为中文 | 输入参数为路径": "Translate PDF papers accurately into Chinese | Input parameter is the path",
+ "检测到操作错误！当您上传文档之后": "Operation error detected! After you upload the document",
+ "在线大模型配置关联关系示意图": "Online large model configuration relationship diagram",
+ "你的填写的空间名如grobid": "Your filled space name such as grobid",
+ "获取方法": "Get method",
+ "| 输入参数为路径": "| Input parameter is the path",
+ "⭐ ⭐ ⭐ 执行插件": "⭐ ⭐ ⭐ Execute plugin",
+ "├── ALLOW_RESET_CONFIG 是否允许通过自然语言描述修改本页的配置": "├── ALLOW_RESET_CONFIG Whether to allow modifying the configuration of this page through natural language description",
+ "重新页面即可生效": "Refresh the page to take effect",
+ "设为public": "Set as public",
+ "并在此处指定模型路径": "And specify the model path here",
+ "分析用户意图中": "Analyzing user intent",
+ "刷新下拉列表": "Refresh the drop-down list",
+ "失败 当前语言模型": "Failed current language model",
+ "1. 请用**自然语言**描述您需要做什么": "1. Please describe what you need to do in **natural language**",
+ "对Latex项目全文进行中译英处理 | 输入参数为路径或上传压缩包": "Translate the full text of Latex projects from Chinese to English | Input parameter is the path or upload a compressed package",
+ "没有配置BAIDU_CLOUD_API_KEY": "No configuration for BAIDU_CLOUD_API_KEY",
+ "设置默认值": "Set default value",
+ "如果太多了会导致gpt无法理解": "If there are too many, it will cause GPT to be unable to understand",
+ "绿草如茵": "Green grass",
+ "├── LAYOUT 窗口布局": "├── LAYOUT window layout",
+ "用户意图理解": "User intent understanding",
+ "生成RFC1123格式的时间戳": "Generate RFC1123 formatted timestamp",
+ "欢迎您前往Github反馈问题": "Welcome to go to Github to provide feedback",
+ "排除已经是按钮的插件": "Exclude plugins that are already buttons",
+ "亦在下拉菜单中显示": "Also displayed in the dropdown menu",
+ "导致无法反序列化": "Causing deserialization failure",
+ "意图=": "Intent =",
+ "章节": "Chapter",
+ "调用插件": "Invoke plugin",
+ "ParseRustProject的所有源文件 | 输入参数为路径": "All source files of ParseRustProject | Input parameter is path",
+ "需要点击“函数插件区”按钮进行处理": "Need to click the 'Function Plugin Area' button for processing",
+ "默认 AsButton = True": "Default AsButton = True",
+ "收到websocket错误的处理": "Handling websocket errors",
+ "用插件": "Use Plugin",
+ "没有选择任何插件组": "No plugin group selected",
+ "答": "Answer",
+ "可修改成本地GROBID服务": "Can modify to local GROBID service",
+ "用户意图": "User intent",
+ "对英文Latex项目全文进行润色处理 | 输入参数为路径或上传压缩包": "Polish the full text of English Latex projects | Input parameters are paths or uploaded compressed packages",
+ "「我不喜欢当前的界面颜色": "I don't like the current interface color",
+ "「请调用插件": "Please call the plugin",
+ "VoidTerminal状态": "VoidTerminal status",
+ "新配置": "New configuration",
+ "支持Github链接": "Support Github links",
+ "没有配置BAIDU_CLOUD_SECRET_KEY": "No BAIDU_CLOUD_SECRET_KEY configured",
+ "获取当前VoidTerminal状态": "Get the current VoidTerminal status",
+ "刷新按钮": "Refresh button",
+ "为了防止pickle.dumps": "To prevent pickle.dumps",
+ "放弃治疗": "Give up treatment",
+ "可指定不同的生成长度、top_p等相关超参": "Can specify different generation lengths, top_p and other related hyperparameters",
+ "请将题目和摘要翻译为": "Translate the title and abstract",
+ "通过appid和用户的提问来生成请参数": "Generate request parameters through appid and user's question",
+ "ImageGeneration | 输入参数字符串": "ImageGeneration | Input parameter string",
+ "将文件拖动到文件上传区": "Drag and drop the file to the file upload area",
+ "如果意图模糊": "If the intent is ambiguous",
+ "星火认知大模型": "Spark Cognitive Big Model",
+ "执行中. 删除 gpt_log & private_upload": "Executing. Delete gpt_log & private_upload",
+ "默认 Color = secondary": "Default Color = secondary",
+ "此处也不需要修改": "No modification is needed here",
+ "⭐ ⭐ ⭐ 分析用户意图": "⭐ ⭐ ⭐ Analyze user intent",
+ "再试一次": "Try again",
+ "请写bash命令实现以下功能": "Please write a bash command to implement the following function",
+ "批量SummarizingWordDocuments | 输入参数为路径": "Batch SummarizingWordDocuments | Input parameter is the path",
+ "/Users/fuqingxu/Desktop/旧文件/gpt/chatgpt_academic/crazy_functions/latex_fns中的python文件进行解析": "Parse the python file in /Users/fuqingxu/Desktop/旧文件/gpt/chatgpt_academic/crazy_functions/latex_fns",
+ "当我要求你写bash命令时": "When I ask you to write a bash command",
+ "├── AUTO_CLEAR_TXT 是否在提交时自动清空输入框": "├── AUTO_CLEAR_TXT Whether to automatically clear the input box when submitting",
+ "按停止键终止": "Press the stop key to terminate",
+ "文心一言": "Original text",
+ "不能理解您的意图": "Cannot understand your intention",
+ "用简单的关键词检测用户意图": "Detect user intention with simple keywords",
+ "中文": "Chinese",
+ "解析一个C++项目的所有源文件": "Parse all source files of a C++ project",
+ "请求的Prompt为": "Requested prompt is",
+ "参考本demo的时候可取消上方打印的注释": "You can remove the comments above when referring to this demo",
+ "开始接收回复": "Start receiving replies",
+ "接入讯飞星火大模型 https": "Access to Xunfei Xinghuo large model https",
+ "用该压缩包进行反馈": "Use this compressed package for feedback",
+ "翻译Markdown或README": "Translate Markdown or README",
+ "SK 生成鉴权签名": "SK generates authentication signature",
+ "插件参数": "Plugin parameters",
+ "需要访问中文Bing": "Need to access Chinese Bing",
+ "ParseFrontendProject的所有源文件": "Parse all source files of ParseFrontendProject",
+ "现在将执行效果稍差的旧版代码": "Now execute the older version code with slightly worse performance",
+ "您需要明确说明并在指令中提到它": "You need to specify and mention it in the command",
+ "请在config.py中设置ALLOW_RESET_CONFIG=True后重启软件": "Please set ALLOW_RESET_CONFIG=True in config.py and restart the software",
+ "按照自然语言描述生成一个动画 | 输入参数是一段话": "Generate an animation based on natural language description | Input parameter is a sentence",
+ "你的hf用户名如qingxu98": "Your hf username is qingxu98",
+ "Arixv论文精细翻译 | 输入参数arxiv论文的ID": "Fine translation of Arixv paper | Input parameter is the ID of arxiv paper",
+ "无法获取 abstract": "Unable to retrieve abstract",
+ "尽可能地仅用一行命令解决我的要求": "Try to solve my request using only one command",
+ "提取插件参数": "Extract plugin parameters",
+ "配置修改完成": "Configuration modification completed",
+ "正在修改配置中": "Modifying configuration",
+ "ParsePythonProject的所有源文件": "All source files of ParsePythonProject",
+ "请求错误": "Request error",
+ "精准翻译PDF论文": "Accurate translation of PDF paper",
+ "无法获取 authors": "Unable to retrieve authors",
+ "该插件诞生时间不长": "This plugin has not been around for long",
+ "返回项目根路径": "Return project root path",
+ "BatchSummarizePDFDocuments的内容 | 输入参数为路径": "Content of BatchSummarizePDFDocuments | Input parameter is a path",
+ "百度千帆": "Baidu Qianfan",
+ "解析一个C++项目的所有头文件": "Parse all header files of a C++ project",
+ "现在请您描述您的需求": "Now please describe your requirements",
+ "该功能具有一定的危险性": "This feature has a certain level of danger",
+ "收到websocket关闭的处理": "Processing when receiving websocket closure",
+ "读取Tex论文并写摘要 | 输入参数为路径": "Read Tex paper and write abstract | Input parameter is the path",
+ "地址为https": "The address is https",
+ "限制最多前10个配置项": "Limit up to 10 configuration items",
+ "6. 如果不需要上传文件": "6. If file upload is not needed",
+ "默认 Group = 对话": "Default Group = Conversation",
+ "五秒后即将重启！若出现报错请无视即可": "Restarting in five seconds! Please ignore if there is an error",
+ "收到websocket连接建立的处理": "Processing when receiving websocket connection establishment",
+ "批量生成函数的注释 | 输入参数为路径": "Batch generate function comments | Input parameter is the path",
+ "聊天": "Chat",
+ "但您可以尝试再试一次": "But you can try again",
+ "千帆大模型平台": "Qianfan Big Model Platform",
+ "直接运行 python tests/test_plugins.py": "Run python tests/test_plugins.py directly",
+ "或是None": "Or None",
+ "进行hmac-sha256进行加密": "Perform encryption using hmac-sha256",
+ "批量总结音频或视频 | 输入参数为路径": "Batch summarize audio or video | Input parameter is path",
+ "插件在线服务配置依赖关系示意图": "Plugin online service configuration dependency diagram",
+ "开始初始化模型": "Start initializing model",
+ "弱模型可能无法理解您的想法": "Weak model may not understand your ideas",
+ "解除大小写限制": "Remove case sensitivity restriction",
+ "跳过提示环节": "Skip prompt section",
+ "接入一些逆向工程https": "Access some reverse engineering https",
+ "执行完成": "Execution completed",
+ "如果需要配置": "If configuration is needed",
+ "此处不修改；如果使用本地或无地域限制的大模型时": "Do not modify here; if using local or region-unrestricted large models",
+ "你是一个Linux大师级用户": "You are a Linux master-level user",
+ "arxiv论文的ID是1812.10695": "The ID of the arxiv paper is 1812.10695",
+ "而不是点击“提交”按钮": "Instead of clicking the 'Submit' button",
+ "解析一个Go项目的所有源文件 | 输入参数为路径": "Parse all source files of a Go project | Input parameter is path",
+ "对中文Latex项目全文进行润色处理 | 输入参数为路径或上传压缩包": "Polish the entire text of a Chinese Latex project | Input parameter is path or upload compressed package",
+ "「生成一张图片": "Generate an image",
+ "将Markdown或README翻译为中文 | 输入参数为路径或URL": "Translate Markdown or README to Chinese | Input parameters are path or URL",
+ "训练时间": "Training time",
+ "将请求的鉴权参数组合为字典": "Combine the requested authentication parameters into a dictionary",
+ "对Latex项目全文进行英译中处理 | 输入参数为路径或上传压缩包": "Translate the entire text of Latex project from English to Chinese | Input parameters are path or uploaded compressed package",
+ "内容如下": "The content is as follows",
+ "用于高质量地读取PDF文档": "Used for high-quality reading of PDF documents",
+ "上下文太长导致 token 溢出": "The context is too long, causing token overflow",
+ "├── DARK_MODE 暗色模式 / 亮色模式": "├── DARK_MODE Dark mode / Light mode",
+ "语言模型回复为": "The language model replies as",
+ "from crazy_functions.chatglm微调工具 import 微调数据集生成": "from crazy_functions.chatglm fine-tuning tool import fine-tuning dataset generation",
+ "为您选择了插件": "Selected plugin for you",
+ "无法获取 title": "Unable to get title",
+ "收到websocket消息的处理": "Processing of received websocket messages",
+ "2023年": "2023",
+ "清除所有缓存文件": "Clear all cache files",
+ "├── PDF文档精准解析": "├── Accurate parsing of PDF documents",
+ "论文我刚刚放到上传区了": "I just put the paper in the upload area",
+ "生成url": "Generate URL",
+ "以下部分是新加入的模型": "The following section is the newly added model",
+ "学术": "Academic",
+ "├── DEFAULT_FN_GROUPS 插件分类默认选项": "├── DEFAULT_FN_GROUPS Plugin classification default options",
+ "不推荐使用": "Not recommended for use",
+ "正在同时咨询": "Consulting simultaneously",
+ "将Markdown翻译为中文 | 输入参数为路径或URL": "Translate Markdown to Chinese | Input parameters are path or URL",
+ "Github网址是https": "The Github URL is https",
+ "试着加上.tex后缀试试": "Try adding the .tex suffix",
+ "对项目中的各个插件进行测试": "Test each plugin in the project",
+ "插件说明": "Plugin description",
+ "├── CODE_HIGHLIGHT 代码高亮": "├── CODE_HIGHLIGHT Code highlighting",
+ "记得用插件": "Remember to use the plugin",
+ "谨慎操作": "Handle with caution"
}
\ No newline at end of file
diff --git a/docs/translate_std.json b/docs/translate_std.json
index 8c47de9075d1e7f8f6fe550e4898ea1dc7e097f9..63b6baa4182c764cf76a8f486deef621f86fa5c8 100644
--- a/docs/translate_std.json
+++ b/docs/translate_std.json
@@ -83,5 +83,10 @@
"图片生成": "ImageGeneration",
"动画生成": "AnimationGeneration",
"语音助手": "VoiceAssistant",
- "启动微调": "StartFineTuning"
+ "启动微调": "StartFineTuning",
+ "清除缓存": "ClearCache",
+ "辅助功能": "Accessibility",
+ "虚空终端": "VoidTerminal",
+ "解析PDF_基于GROBID": "ParsePDF_BasedOnGROBID",
+ "虚空终端主路由": "VoidTerminalMainRoute"
}
\ No newline at end of file
diff --git a/multi_language.py b/multi_language.py
index 7786b8652486a60ca33ea793dbef921f1793b35e..c4ed36eb72f8c5f9a87accbea4a35c633e26854c 100644
--- a/multi_language.py
+++ b/multi_language.py
@@ -478,6 +478,8 @@ def step_2_core_key_translate():
up = trans_json(need_translate, language=LANG, special=False)
map_to_json(up, language=LANG)
cached_translation = read_map_from_json(language=LANG)
+ LANG_STD = 'std'
+ cached_translation.update(read_map_from_json(language=LANG_STD))
cached_translation = dict(sorted(cached_translation.items(), key=lambda x: -len(x[0])))
# ===============================================
diff --git a/request_llm/bridge_all.py b/request_llm/bridge_all.py
index e167825ae906dd977c65786ada25d60b05f5e6d5..bb325e460742cececeaf1683d331c593bcba2915 100644
--- a/request_llm/bridge_all.py
+++ b/request_llm/bridge_all.py
@@ -398,6 +398,22 @@ if "spark" in AVAIL_LLM_MODELS: # 讯飞星火认知大模型
})
except:
print(trimmed_format_exc())
+if "sparkv2" in AVAIL_LLM_MODELS: # 讯飞星火认知大模型
+ try:
+ from .bridge_spark import predict_no_ui_long_connection as spark_noui
+ from .bridge_spark import predict as spark_ui
+ model_info.update({
+ "sparkv2": {
+ "fn_with_ui": spark_ui,
+ "fn_without_ui": spark_noui,
+ "endpoint": None,
+ "max_token": 4096,
+ "tokenizer": tokenizer_gpt35,
+ "token_cnt": get_token_num_gpt35,
+ }
+ })
+ except:
+ print(trimmed_format_exc())
if "llama2" in AVAIL_LLM_MODELS: # llama2
try:
from .bridge_llama2 import predict_no_ui_long_connection as llama2_noui
diff --git a/request_llm/bridge_chatglmft.py b/request_llm/bridge_chatglmft.py
index 4e21c989b75a463dc7ecd695d23d2c24202788e3..71af94213e1824f8c664d5759812b6057d3b29a2 100644
--- a/request_llm/bridge_chatglmft.py
+++ b/request_llm/bridge_chatglmft.py
@@ -63,9 +63,9 @@ class GetGLMFTHandle(Process):
# if not os.path.exists(conf): raise RuntimeError('找不到微调模型信息')
# with open(conf, 'r', encoding='utf8') as f:
# model_args = json.loads(f.read())
- ChatGLM_PTUNING_CHECKPOINT, = get_conf('ChatGLM_PTUNING_CHECKPOINT')
- assert os.path.exists(ChatGLM_PTUNING_CHECKPOINT), "找不到微调模型检查点"
- conf = os.path.join(ChatGLM_PTUNING_CHECKPOINT, "config.json")
+ CHATGLM_PTUNING_CHECKPOINT, = get_conf('CHATGLM_PTUNING_CHECKPOINT')
+ assert os.path.exists(CHATGLM_PTUNING_CHECKPOINT), "找不到微调模型检查点"
+ conf = os.path.join(CHATGLM_PTUNING_CHECKPOINT, "config.json")
with open(conf, 'r', encoding='utf8') as f:
model_args = json.loads(f.read())
if 'model_name_or_path' not in model_args:
@@ -78,9 +78,9 @@ class GetGLMFTHandle(Process):
config.pre_seq_len = model_args['pre_seq_len']
config.prefix_projection = model_args['prefix_projection']
- print(f"Loading prefix_encoder weight from {ChatGLM_PTUNING_CHECKPOINT}")
+ print(f"Loading prefix_encoder weight from {CHATGLM_PTUNING_CHECKPOINT}")
model = AutoModel.from_pretrained(model_args['model_name_or_path'], config=config, trust_remote_code=True)
- prefix_state_dict = torch.load(os.path.join(ChatGLM_PTUNING_CHECKPOINT, "pytorch_model.bin"))
+ prefix_state_dict = torch.load(os.path.join(CHATGLM_PTUNING_CHECKPOINT, "pytorch_model.bin"))
new_prefix_state_dict = {}
for k, v in prefix_state_dict.items():
if k.startswith("transformer.prefix_encoder."):
diff --git a/request_llm/bridge_chatgpt.py b/request_llm/bridge_chatgpt.py
index 5a7a274e1d42336ed4ac6d7b008db2a78bd2b4e5..929a7546c56cff1a305ced54df819bb992f6b8a5 100644
--- a/request_llm/bridge_chatgpt.py
+++ b/request_llm/bridge_chatgpt.py
@@ -137,6 +137,12 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp
chatbot.append((inputs, ""))
yield from update_ui(chatbot=chatbot, history=history, msg="等待响应") # 刷新界面
+ # check mis-behavior
+ if raw_input.startswith('private_upload/') and len(raw_input) == 34:
+ chatbot[-1] = (inputs, f"[Local Message] 检测到操作错误！当您上传文档之后，需要点击“函数插件区”按钮进行处理，而不是点击“提交”按钮。")
+ yield from update_ui(chatbot=chatbot, history=history, msg="正常") # 刷新界面
+ time.sleep(2)
+
try:
headers, payload = generate_payload(inputs, llm_kwargs, history, system_prompt, stream)
except RuntimeError as e:
@@ -178,7 +184,7 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp
return
chunk_decoded = chunk.decode()
- if is_head_of_the_stream and (r'"object":"error"' not in chunk_decoded) and (r"choices" not in chunk_decoded):
+ if is_head_of_the_stream and (r'"object":"error"' not in chunk_decoded) and (r"content" not in chunk_decoded):
# 数据流的第一帧不携带content
is_head_of_the_stream = False; continue
diff --git a/request_llm/bridge_qianfan.py b/request_llm/bridge_qianfan.py
index e2cdb0eef7433a0d4f701858206f68415f969f47..be7397607a71681edc240be48d1889ce51008af5 100644
--- a/request_llm/bridge_qianfan.py
+++ b/request_llm/bridge_qianfan.py
@@ -49,16 +49,17 @@ def get_access_token():
def generate_message_payload(inputs, llm_kwargs, history, system_prompt):
conversation_cnt = len(history) // 2
+ if system_prompt == "": system_prompt = "Hello"
messages = [{"role": "user", "content": system_prompt}]
messages.append({"role": "assistant", "content": 'Certainly!'})
if conversation_cnt:
for index in range(0, 2*conversation_cnt, 2):
what_i_have_asked = {}
what_i_have_asked["role"] = "user"
- what_i_have_asked["content"] = history[index]
+ what_i_have_asked["content"] = history[index] if history[index]!="" else "Hello"
what_gpt_answer = {}
what_gpt_answer["role"] = "assistant"
- what_gpt_answer["content"] = history[index+1]
+ what_gpt_answer["content"] = history[index+1] if history[index]!="" else "Hello"
if what_i_have_asked["content"] != "":
if what_gpt_answer["content"] == "": continue
if what_gpt_answer["content"] == timeout_bot_msg: continue
diff --git a/request_llm/bridge_spark.py b/request_llm/bridge_spark.py
index 551b6f305daf2f904056e6090261b7609dae0512..0fe925f7a0354fe6361e9d11ae074dd287813e9f 100644
--- a/request_llm/bridge_spark.py
+++ b/request_llm/bridge_spark.py
@@ -2,11 +2,17 @@
import time
import threading
import importlib
-from toolbox import update_ui, get_conf
+from toolbox import update_ui, get_conf, update_ui_lastest_msg
from multiprocessing import Process, Pipe
model_name = '星火认知大模型'
+def validate_key():
+ XFYUN_APPID, = get_conf('XFYUN_APPID', )
+ if XFYUN_APPID == '00000000' or XFYUN_APPID == '':
+ return False
+ return True
+
def predict_no_ui_long_connection(inputs, llm_kwargs, history=[], sys_prompt="", observe_window=[], console_slience=False):
"""
⭐多线程方法
@@ -15,6 +21,9 @@ def predict_no_ui_long_connection(inputs, llm_kwargs, history=[], sys_prompt="",
watch_dog_patience = 5
response = ""
+ if validate_key() is False:
+ raise RuntimeError('请配置讯飞星火大模型的XFYUN_APPID, XFYUN_API_KEY, XFYUN_API_SECRET')
+
from .com_sparkapi import SparkRequestInstance
sri = SparkRequestInstance()
for response in sri.generate(inputs, llm_kwargs, history, sys_prompt):
@@ -30,6 +39,11 @@ def predict(inputs, llm_kwargs, plugin_kwargs, chatbot, history=[], system_promp
函数的说明请见 request_llm/bridge_all.py
"""
chatbot.append((inputs, ""))
+ yield from update_ui(chatbot=chatbot, history=history)
+
+ if validate_key() is False:
+ yield from update_ui_lastest_msg(lastmsg="[Local Message]: 请配置讯飞星火大模型的XFYUN_APPID, XFYUN_API_KEY, XFYUN_API_SECRET", chatbot=chatbot, history=history, delay=0)
+ return
if additional_fn is not None:
from core_functional import handle_core_functionality
diff --git a/request_llm/com_sparkapi.py b/request_llm/com_sparkapi.py
index c83710b22457ce5d496e1790d754f6d5d488823a..0b8d655dffd41c18f5533e71a7709ebd626158a7 100644
--- a/request_llm/com_sparkapi.py
+++ b/request_llm/com_sparkapi.py
@@ -58,11 +58,13 @@ class Ws_Param(object):
class SparkRequestInstance():
def __init__(self):
XFYUN_APPID, XFYUN_API_SECRET, XFYUN_API_KEY = get_conf('XFYUN_APPID', 'XFYUN_API_SECRET', 'XFYUN_API_KEY')
-
+ if XFYUN_APPID == '00000000' or XFYUN_APPID == '': raise RuntimeError('请配置讯飞星火大模型的XFYUN_APPID, XFYUN_API_KEY, XFYUN_API_SECRET')
self.appid = XFYUN_APPID
self.api_secret = XFYUN_API_SECRET
self.api_key = XFYUN_API_KEY
self.gpt_url = "ws://spark-api.xf-yun.com/v1.1/chat"
+ self.gpt_url_v2 = "ws://spark-api.xf-yun.com/v2.1/chat"
+
self.time_to_yield_event = threading.Event()
self.time_to_exit_event = threading.Event()
@@ -83,7 +85,12 @@ class SparkRequestInstance():
def create_blocking_request(self, inputs, llm_kwargs, history, system_prompt):
- wsParam = Ws_Param(self.appid, self.api_key, self.api_secret, self.gpt_url)
+ if llm_kwargs['llm_model'] == 'sparkv2':
+ gpt_url = self.gpt_url_v2
+ else:
+ gpt_url = self.gpt_url
+
+ wsParam = Ws_Param(self.appid, self.api_key, self.api_secret, gpt_url)
websocket.enableTrace(False)
wsUrl = wsParam.create_url()
@@ -167,7 +174,7 @@ def gen_params(appid, inputs, llm_kwargs, history, system_prompt):
},
"parameter": {
"chat": {
- "domain": "general",
+ "domain": "generalv2" if llm_kwargs['llm_model'] == 'sparkv2' else "general",
"temperature": llm_kwargs["temperature"],
"random_threshold": 0.5,
"max_tokens": 4096,
diff --git a/requirements.txt b/requirements.txt
index ea93933971f9b85b075373994542e8c4764b27d4..0a9a4c83f90cd9796ff9e5c52b3512eeb4c19e68 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -19,4 +19,4 @@ arxiv
rich
pypdf2==2.12.1
websocket-client
-scipdf_parser==0.3
+scipdf_parser>=0.3
diff --git a/tests/test_plugins.py b/tests/test_plugins.py
index 4913a59bfcec5d803d980dcfcbe9a8bdf0b12600..ec28af1e671282f4e3b7f8ef14fb6ac7bdb36e65 100644
--- a/tests/test_plugins.py
+++ b/tests/test_plugins.py
@@ -9,6 +9,11 @@ validate_path() # 返回项目根路径
from tests.test_utils import plugin_test
if __name__ == "__main__":
+ # plugin_test(plugin='crazy_functions.虚空终端->虚空终端', main_input='修改api-key为sk-jhoejriotherjep')
+ plugin_test(plugin='crazy_functions.批量翻译PDF文档_NOUGAT->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
+
+ # plugin_test(plugin='crazy_functions.虚空终端->虚空终端', main_input='调用插件，对C:/Users/fuqingxu/Desktop/旧文件/gpt/chatgpt_academic/crazy_functions/latex_fns中的python文件进行解析')
+
# plugin_test(plugin='crazy_functions.命令行助手->命令行助手', main_input='查看当前的docker容器列表')
# plugin_test(plugin='crazy_functions.解析项目源代码->解析一个Python项目', main_input="crazy_functions/test_project/python/dqn")
@@ -19,7 +24,7 @@ if __name__ == "__main__":
# plugin_test(plugin='crazy_functions.批量Markdown翻译->Markdown中译英', main_input="README.md")
- plugin_test(plugin='crazy_functions.批量翻译PDF文档_多线程->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
+ # plugin_test(plugin='crazy_functions.批量翻译PDF文档_多线程->批量翻译PDF文档', main_input='crazy_functions/test_project/pdf_and_word/aaai.pdf')
# plugin_test(plugin='crazy_functions.谷歌检索小助手->谷歌检索小助手', main_input="https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=auto+reinforcement+learning&btnG=")
diff --git a/themes/common.css b/themes/common.css
new file mode 100644
index 0000000000000000000000000000000000000000..5880d00e56ad7a1872911b7652ff6c1cd1886cb5
--- /dev/null
+++ b/themes/common.css
@@ -0,0 +1,21 @@
+/* hide remove all button */
+.remove-all.svelte-aqlk7e.svelte-aqlk7e.svelte-aqlk7e {
+ visibility: hidden;
+}
+
+/* hide selector border */
+#input-plugin-group .wrap.svelte-aqlk7e.svelte-aqlk7e.svelte-aqlk7e {
+ border: 0px;
+ box-shadow: none;
+}
+
+/* hide selector label */
+#input-plugin-group .svelte-1gfkn6j {
+ visibility: hidden;
+}
+
+
+/* height of the upload box */
+.wrap.svelte-xwlu1w {
+ min-height: var(--size-32);
+}
diff --git a/themes/common.js b/themes/common.js
index dcdeb50e2bbf508abb2d38624a5d60b057a27425..7733c7b1e424ea486a6fd52071a04fca8feaf106 100644
--- a/themes/common.js
+++ b/themes/common.js
@@ -1,6 +1,6 @@
function ChatBotHeight() {
function update_height(){
- var { panel_height_target, chatbot_height, chatbot } = get_elements();
+ var { panel_height_target, chatbot_height, chatbot } = get_elements(true);
if (panel_height_target!=chatbot_height)
{
var pixelString = panel_height_target.toString() + 'px';
@@ -28,18 +28,24 @@ function ChatBotHeight() {
}, 50); // 每100毫秒执行一次
}
-function get_elements() {
+function get_elements(consider_state_panel=false) {
var chatbot = document.querySelector('#gpt-chatbot > div.wrap.svelte-18telvq');
if (!chatbot) {
chatbot = document.querySelector('#gpt-chatbot');
}
- const panel1 = document.querySelector('#input-panel');
- const panel2 = document.querySelector('#basic-panel');
- const panel3 = document.querySelector('#plugin-panel');
- const panel4 = document.querySelector('#interact-panel');
- const panel5 = document.querySelector('#input-panel2');
- const panel_active = document.querySelector('#state-panel');
- var panel_height_target = (20-panel_active.offsetHeight) + panel1.offsetHeight + panel2.offsetHeight + panel3.offsetHeight + panel4.offsetHeight + panel5.offsetHeight + 21;
+ const panel1 = document.querySelector('#input-panel').getBoundingClientRect();
+ const panel2 = document.querySelector('#basic-panel').getBoundingClientRect()
+ const panel3 = document.querySelector('#plugin-panel').getBoundingClientRect();
+ const panel4 = document.querySelector('#interact-panel').getBoundingClientRect();
+ const panel5 = document.querySelector('#input-panel2').getBoundingClientRect();
+ const panel_active = document.querySelector('#state-panel').getBoundingClientRect();
+ if (consider_state_panel || panel_active.height < 25){
+ document.state_panel_height = panel_active.height;
+ }
+ // 25 是chatbot的label高度, 16 是右侧的gap
+ var panel_height_target = panel1.height + panel2.height + panel3.height + panel4.height + panel5.height - 25 + 16*3;
+ // 禁止动态的state-panel高度影响
+ panel_height_target = panel_height_target + (document.state_panel_height-panel_active.height)
var panel_height_target = parseInt(panel_height_target);
var chatbot_height = chatbot.style.height;
var chatbot_height = parseInt(chatbot_height);
diff --git a/themes/contrast.css b/themes/contrast.css
new file mode 100644
index 0000000000000000000000000000000000000000..54a1b2b4480bde6ac17938fd73691b13f43d84ef
--- /dev/null
+++ b/themes/contrast.css
@@ -0,0 +1,482 @@
+:root {
+ --body-text-color: #FFFFFF;
+ --link-text-color: #FFFFFF;
+ --link-text-color-active: #FFFFFF;
+ --link-text-color-hover: #FFFFFF;
+ --link-text-color-visited: #FFFFFF;
+ --body-text-color-subdued: #FFFFFF;
+ --block-info-text-color: #FFFFFF;
+ --block-label-text-color: #FFFFFF;
+ --block-title-text-color: #FFFFFF;
+ --checkbox-label-text-color: #FFFFFF;
+ --checkbox-label-text-color-selected: #FFFFFF;
+ --error-text-color: #FFFFFF;
+ --button-cancel-text-color: #FFFFFF;
+ --button-cancel-text-color-hover: #FFFFFF;
+ --button-primary-text-color: #FFFFFF;
+ --button-primary-text-color-hover: #FFFFFF;
+ --button-secondary-text-color: #FFFFFF;
+ --button-secondary-text-color-hover: #FFFFFF;
+
+
+ --border-bottom-right-radius: 0px;
+ --border-bottom-left-radius: 0px;
+ --border-top-right-radius: 0px;
+ --border-top-left-radius: 0px;
+ --block-radius: 0px;
+ --button-large-radius: 0px;
+ --button-small-radius: 0px;
+ --block-background-fill: #000000;
+
+ --border-color-accent: #3cff00;
+ --border-color-primary: #3cff00;
+ --block-border-color: #3cff00;
+ --block-label-border-color: #3cff00;
+ --block-title-border-color: #3cff00;
+ --panel-border-color: #3cff00;
+ --checkbox-border-color: #3cff00;
+ --checkbox-border-color-focus: #3cff00;
+ --checkbox-border-color-hover: #3cff00;
+ --checkbox-border-color-selected: #3cff00;
+ --checkbox-label-border-color: #3cff00;
+ --checkbox-label-border-color-hover: #3cff00;
+ --error-border-color: #3cff00;
+ --input-border-color: #3cff00;
+ --input-border-color-focus: #3cff00;
+ --input-border-color-hover: #3cff00;
+ --table-border-color: #3cff00;
+ --button-cancel-border-color: #3cff00;
+ --button-cancel-border-color-hover: #3cff00;
+ --button-primary-border-color: #3cff00;
+ --button-primary-border-color-hover: #3cff00;
+ --button-secondary-border-color: #3cff00;
+ --button-secondary-border-color-hover: #3cff00;
+
+
+ --body-background-fill: #000000;
+ --background-fill-primary: #000000;
+ --background-fill-secondary: #000000;
+ --block-background-fill: #000000;
+ --block-label-background-fill: #000000;
+ --block-title-background-fill: #000000;
+ --panel-background-fill: #000000;
+ --chatbot-code-background-color: #000000;
+ --checkbox-background-color: #000000;
+ --checkbox-background-color-focus: #000000;
+ --checkbox-background-color-hover: #000000;
+ --checkbox-background-color-selected: #000000;
+ --checkbox-label-background-fill: #000000;
+ --checkbox-label-background-fill-hover: #000000;
+ --checkbox-label-background-fill-selected: #000000;
+ --error-background-fill: #000000;
+ --input-background-fill: #000000;
+ --input-background-fill-focus: #000000;
+ --input-background-fill-hover: #000000;
+ --stat-background-fill: #000000;
+ --table-even-background-fill: #000000;
+ --table-odd-background-fill: #000000;
+ --button-cancel-background-fill: #000000;
+ --button-cancel-background-fill-hover: #000000;
+ --button-primary-background-fill: #000000;
+ --button-primary-background-fill-hover: #000000;
+ --button-secondary-background-fill: #000000;
+ --button-secondary-background-fill-hover: #000000;
+ --color-accent-soft: #000000;
+}
+
+.dark {
+ --body-text-color: #FFFFFF;
+ --link-text-color: #FFFFFF;
+ --link-text-color-active: #FFFFFF;
+ --link-text-color-hover: #FFFFFF;
+ --link-text-color-visited: #FFFFFF;
+ --body-text-color-subdued: #FFFFFF;
+ --block-info-text-color: #FFFFFF;
+ --block-label-text-color: #FFFFFF;
+ --block-title-text-color: #FFFFFF;
+ --checkbox-label-text-color: #FFFFFF;
+ --checkbox-label-text-color-selected: #FFFFFF;
+ --error-text-color: #FFFFFF;
+ --button-cancel-text-color: #FFFFFF;
+ --button-cancel-text-color-hover: #FFFFFF;
+ --button-primary-text-color: #FFFFFF;
+ --button-primary-text-color-hover: #FFFFFF;
+ --button-secondary-text-color: #FFFFFF;
+ --button-secondary-text-color-hover: #FFFFFF;
+
+
+
+ --border-bottom-right-radius: 0px;
+ --border-bottom-left-radius: 0px;
+ --border-top-right-radius: 0px;
+ --border-top-left-radius: 0px;
+ --block-radius: 0px;
+ --button-large-radius: 0px;
+ --button-small-radius: 0px;
+ --block-background-fill: #000000;
+
+ --border-color-accent: #3cff00;
+ --border-color-primary: #3cff00;
+ --block-border-color: #3cff00;
+ --block-label-border-color: #3cff00;
+ --block-title-border-color: #3cff00;
+ --panel-border-color: #3cff00;
+ --checkbox-border-color: #3cff00;
+ --checkbox-border-color-focus: #3cff00;
+ --checkbox-border-color-hover: #3cff00;
+ --checkbox-border-color-selected: #3cff00;
+ --checkbox-label-border-color: #3cff00;
+ --checkbox-label-border-color-hover: #3cff00;
+ --error-border-color: #3cff00;
+ --input-border-color: #3cff00;
+ --input-border-color-focus: #3cff00;
+ --input-border-color-hover: #3cff00;
+ --table-border-color: #3cff00;
+ --button-cancel-border-color: #3cff00;
+ --button-cancel-border-color-hover: #3cff00;
+ --button-primary-border-color: #3cff00;
+ --button-primary-border-color-hover: #3cff00;
+ --button-secondary-border-color: #3cff00;
+ --button-secondary-border-color-hover: #3cff00;
+
+
+ --body-background-fill: #000000;
+ --background-fill-primary: #000000;
+ --background-fill-secondary: #000000;
+ --block-background-fill: #000000;
+ --block-label-background-fill: #000000;
+ --block-title-background-fill: #000000;
+ --panel-background-fill: #000000;
+ --chatbot-code-background-color: #000000;
+ --checkbox-background-color: #000000;
+ --checkbox-background-color-focus: #000000;
+ --checkbox-background-color-hover: #000000;
+ --checkbox-background-color-selected: #000000;
+ --checkbox-label-background-fill: #000000;
+ --checkbox-label-background-fill-hover: #000000;
+ --checkbox-label-background-fill-selected: #000000;
+ --error-background-fill: #000000;
+ --input-background-fill: #000000;
+ --input-background-fill-focus: #000000;
+ --input-background-fill-hover: #000000;
+ --stat-background-fill: #000000;
+ --table-even-background-fill: #000000;
+ --table-odd-background-fill: #000000;
+ --button-cancel-background-fill: #000000;
+ --button-cancel-background-fill-hover: #000000;
+ --button-primary-background-fill: #000000;
+ --button-primary-background-fill-hover: #000000;
+ --button-secondary-background-fill: #000000;
+ --button-secondary-background-fill-hover: #000000;
+ --color-accent-soft: #000000;
+}
+
+
+
+.block.svelte-mppz8v {
+ border-color: #3cff00;
+}
+
+/* 插件下拉菜单 */
+#plugin-panel .wrap.svelte-aqlk7e.svelte-aqlk7e.svelte-aqlk7e {
+ box-shadow: var(--input-shadow);
+ border: var(--input-border-width) dashed var(--border-color-primary);
+ border-radius: 4px;
+}
+
+#plugin-panel .dropdown-arrow.svelte-p5edak {
+ width: 50px;
+}
+#plugin-panel input.svelte-aqlk7e.svelte-aqlk7e.svelte-aqlk7e {
+ padding-left: 5px;
+}
+.root{
+ border-bottom-right-radius: 0px;
+ border-bottom-left-radius: 0px;
+ border-top-right-radius: 0px;
+ border-top-left-radius: 0px;
+}
+
+/* 小按钮 */
+.sm.svelte-1ipelgc {
+ font-family: "Microsoft YaHei UI", "Helvetica", "Microsoft YaHei", "ui-sans-serif", "sans-serif", "system-ui";
+ --button-small-text-weight: 600;
+ --button-small-text-size: 16px;
+ border-bottom-right-radius: 0px;
+ border-bottom-left-radius: 0px;
+ border-top-right-radius: 0px;
+ border-top-left-radius: 0px;
+}
+
+#plugin-panel .sm.svelte-1ipelgc {
+ font-family: "Microsoft YaHei UI", "Helvetica", "Microsoft YaHei", "ui-sans-serif", "sans-serif", "system-ui";
+ --button-small-text-weight: 400;
+ --button-small-text-size: 14px;
+ border-bottom-right-radius: 0px;
+ border-bottom-left-radius: 0px;
+ border-top-right-radius: 0px;
+ border-top-left-radius: 0px;
+}
+
+.wrap-inner.svelte-aqlk7e.svelte-aqlk7e.svelte-aqlk7e {
+ padding: 0%;
+}
+
+.markdown-body table {
+ margin: 1em 0;
+ border-collapse: collapse;
+ empty-cells: show;
+}
+
+.markdown-body th, .markdown-body td {
+ border: 1.2px solid var(--border-color-primary);
+ padding: 5px;
+}
+
+.markdown-body thead {
+ background-color: rgb(0, 0, 0);
+}
+
+.markdown-body thead th {
+ padding: .5em .2em;
+}
+
+.normal_mut_select .svelte-1gfkn6j {
+ float: left;
+ width: auto;
+ line-height: 260% !important;
+}
+
+.markdown-body ol, .markdown-body ul {
+ padding-inline-start: 2em !important;
+}
+
+/* chat box. */
+[class *= "message"] {
+ border-radius: var(--radius-xl) !important;
+ /* padding: var(--spacing-xl) !important; */
+ /* font-size: var(--text-md) !important; */
+ /* line-height: var(--line-md) !important; */
+ /* min-height: calc(var(--text-md)*var(--line-md) + 2*var(--spacing-xl)); */
+ /* min-width: calc(var(--text-md)*var(--line-md) + 2*var(--spacing-xl)); */
+}
+[data-testid = "bot"] {
+ max-width: 95%;
+ /* width: auto !important; */
+ border-bottom-left-radius: 0 !important;
+}
+[data-testid = "user"] {
+ max-width: 100%;
+ /* width: auto !important; */
+ border-bottom-right-radius: 0 !important;
+}
+
+/* linein code block. */
+.markdown-body code {
+ display: inline;
+ white-space: break-spaces;
+ border-radius: 6px;
+ margin: 0 2px 0 2px;
+ padding: .2em .4em .1em .4em;
+ background-color: rgba(0, 0, 0, 0.95);
+ color: #c9d1d9;
+}
+
+.dark .markdown-body code {
+ display: inline;
+ white-space: break-spaces;
+ border-radius: 6px;
+ margin: 0 2px 0 2px;
+ padding: .2em .4em .1em .4em;
+ background-color: rgba(0,0,0,0.2);
+}
+
+/* code block css */
+.markdown-body pre code {
+ display: block;
+ overflow: auto;
+ white-space: pre;
+ background-color: rgba(0, 0, 0, 0.95);
+ border-radius: 10px;
+ padding: 1em;
+ margin: 1em 2em 1em 0.5em;
+}
+
+.dark .markdown-body pre code {
+ display: block;
+ overflow: auto;
+ white-space: pre;
+ background-color: rgba(0,0,0,0.2);
+ border-radius: 10px;
+ padding: 1em;
+ margin: 1em 2em 1em 0.5em;
+}
+
+/* .mic-wrap.svelte-1thnwz {
+
+} */
+.block.svelte-mppz8v > .mic-wrap.svelte-1thnwz{
+ justify-content: center;
+ display: flex;
+ padding: 0;
+
+}
+
+.codehilite .hll { background-color: #6e7681 }
+.codehilite .c { color: #8b949e; font-style: italic } /* Comment */
+.codehilite .err { color: #f85149 } /* Error */
+.codehilite .esc { color: #c9d1d9 } /* Escape */
+.codehilite .g { color: #c9d1d9 } /* Generic */
+.codehilite .k { color: #ff7b72 } /* Keyword */
+.codehilite .l { color: #a5d6ff } /* Literal */
+.codehilite .n { color: #c9d1d9 } /* Name */
+.codehilite .o { color: #ff7b72; font-weight: bold } /* Operator */
+.codehilite .x { color: #c9d1d9 } /* Other */
+.codehilite .p { color: #c9d1d9 } /* Punctuation */
+.codehilite .ch { color: #8b949e; font-style: italic } /* Comment.Hashbang */
+.codehilite .cm { color: #8b949e; font-style: italic } /* Comment.Multiline */
+.codehilite .cp { color: #8b949e; font-weight: bold; font-style: italic } /* Comment.Preproc */
+.codehilite .cpf { color: #8b949e; font-style: italic } /* Comment.PreprocFile */
+.codehilite .c1 { color: #8b949e; font-style: italic } /* Comment.Single */
+.codehilite .cs { color: #8b949e; font-weight: bold; font-style: italic } /* Comment.Special */
+.codehilite .gd { color: #ffa198; background-color: #490202 } /* Generic.Deleted */
+.codehilite .ge { color: #c9d1d9; font-style: italic } /* Generic.Emph */
+.codehilite .gr { color: #ffa198 } /* Generic.Error */
+.codehilite .gh { color: #79c0ff; font-weight: bold } /* Generic.Heading */
+.codehilite .gi { color: #56d364; background-color: #0f5323 } /* Generic.Inserted */
+.codehilite .go { color: #8b949e } /* Generic.Output */
+.codehilite .gp { color: #8b949e } /* Generic.Prompt */
+.codehilite .gs { color: #c9d1d9; font-weight: bold } /* Generic.Strong */
+.codehilite .gu { color: #79c0ff } /* Generic.Subheading */
+.codehilite .gt { color: #ff7b72 } /* Generic.Traceback */
+.codehilite .g-Underline { color: #c9d1d9; text-decoration: underline } /* Generic.Underline */
+.codehilite .kc { color: #79c0ff } /* Keyword.Constant */
+.codehilite .kd { color: #ff7b72 } /* Keyword.Declaration */
+.codehilite .kn { color: #ff7b72 } /* Keyword.Namespace */
+.codehilite .kp { color: #79c0ff } /* Keyword.Pseudo */
+.codehilite .kr { color: #ff7b72 } /* Keyword.Reserved */
+.codehilite .kt { color: #ff7b72 } /* Keyword.Type */
+.codehilite .ld { color: #79c0ff } /* Literal.Date */
+.codehilite .m { color: #a5d6ff } /* Literal.Number */
+.codehilite .s { color: #a5d6ff } /* Literal.String */
+.codehilite .na { color: #c9d1d9 } /* Name.Attribute */
+.codehilite .nb { color: #c9d1d9 } /* Name.Builtin */
+.codehilite .nc { color: #f0883e; font-weight: bold } /* Name.Class */
+.codehilite .no { color: #79c0ff; font-weight: bold } /* Name.Constant */
+.codehilite .nd { color: #d2a8ff; font-weight: bold } /* Name.Decorator */
+.codehilite .ni { color: #ffa657 } /* Name.Entity */
+.codehilite .ne { color: #f0883e; font-weight: bold } /* Name.Exception */
+.codehilite .nf { color: #d2a8ff; font-weight: bold } /* Name.Function */
+.codehilite .nl { color: #79c0ff; font-weight: bold } /* Name.Label */
+.codehilite .nn { color: #ff7b72 } /* Name.Namespace */
+.codehilite .nx { color: #c9d1d9 } /* Name.Other */
+.codehilite .py { color: #79c0ff } /* Name.Property */
+.codehilite .nt { color: #7ee787 } /* Name.Tag */
+.codehilite .nv { color: #79c0ff } /* Name.Variable */
+.codehilite .ow { color: #ff7b72; font-weight: bold } /* Operator.Word */
+.codehilite .pm { color: #c9d1d9 } /* Punctuation.Marker */
+.codehilite .w { color: #6e7681 } /* Text.Whitespace */
+.codehilite .mb { color: #a5d6ff } /* Literal.Number.Bin */
+.codehilite .mf { color: #a5d6ff } /* Literal.Number.Float */
+.codehilite .mh { color: #a5d6ff } /* Literal.Number.Hex */
+.codehilite .mi { color: #a5d6ff } /* Literal.Number.Integer */
+.codehilite .mo { color: #a5d6ff } /* Literal.Number.Oct */
+.codehilite .sa { color: #79c0ff } /* Literal.String.Affix */
+.codehilite .sb { color: #a5d6ff } /* Literal.String.Backtick */
+.codehilite .sc { color: #a5d6ff } /* Literal.String.Char */
+.codehilite .dl { color: #79c0ff } /* Literal.String.Delimiter */
+.codehilite .sd { color: #a5d6ff } /* Literal.String.Doc */
+.codehilite .s2 { color: #a5d6ff } /* Literal.String.Double */
+.codehilite .se { color: #79c0ff } /* Literal.String.Escape */
+.codehilite .sh { color: #79c0ff } /* Literal.String.Heredoc */
+.codehilite .si { color: #a5d6ff } /* Literal.String.Interpol */
+.codehilite .sx { color: #a5d6ff } /* Literal.String.Other */
+.codehilite .sr { color: #79c0ff } /* Literal.String.Regex */
+.codehilite .s1 { color: #a5d6ff } /* Literal.String.Single */
+.codehilite .ss { color: #a5d6ff } /* Literal.String.Symbol */
+.codehilite .bp { color: #c9d1d9 } /* Name.Builtin.Pseudo */
+.codehilite .fm { color: #d2a8ff; font-weight: bold } /* Name.Function.Magic */
+.codehilite .vc { color: #79c0ff } /* Name.Variable.Class */
+.codehilite .vg { color: #79c0ff } /* Name.Variable.Global */
+.codehilite .vi { color: #79c0ff } /* Name.Variable.Instance */
+.codehilite .vm { color: #79c0ff } /* Name.Variable.Magic */
+.codehilite .il { color: #a5d6ff } /* Literal.Number.Integer.Long */
+
+.dark .codehilite .hll { background-color: #2C3B41 }
+.dark .codehilite .c { color: #79d618; font-style: italic } /* Comment */
+.dark .codehilite .err { color: #FF5370 } /* Error */
+.dark .codehilite .esc { color: #89DDFF } /* Escape */
+.dark .codehilite .g { color: #EEFFFF } /* Generic */
+.dark .codehilite .k { color: #BB80B3 } /* Keyword */
+.dark .codehilite .l { color: #C3E88D } /* Literal */
+.dark .codehilite .n { color: #EEFFFF } /* Name */
+.dark .codehilite .o { color: #89DDFF } /* Operator */
+.dark .codehilite .p { color: #89DDFF } /* Punctuation */
+.dark .codehilite .ch { color: #79d618; font-style: italic } /* Comment.Hashbang */
+.dark .codehilite .cm { color: #79d618; font-style: italic } /* Comment.Multiline */
+.dark .codehilite .cp { color: #79d618; font-style: italic } /* Comment.Preproc */
+.dark .codehilite .cpf { color: #79d618; font-style: italic } /* Comment.PreprocFile */
+.dark .codehilite .c1 { color: #79d618; font-style: italic } /* Comment.Single */
+.dark .codehilite .cs { color: #79d618; font-style: italic } /* Comment.Special */
+.dark .codehilite .gd { color: #FF5370 } /* Generic.Deleted */
+.dark .codehilite .ge { color: #89DDFF } /* Generic.Emph */
+.dark .codehilite .gr { color: #FF5370 } /* Generic.Error */
+.dark .codehilite .gh { color: #C3E88D } /* Generic.Heading */
+.dark .codehilite .gi { color: #C3E88D } /* Generic.Inserted */
+.dark .codehilite .go { color: #79d618 } /* Generic.Output */
+.dark .codehilite .gp { color: #FFCB6B } /* Generic.Prompt */
+.dark .codehilite .gs { color: #FF5370 } /* Generic.Strong */
+.dark .codehilite .gu { color: #89DDFF } /* Generic.Subheading */
+.dark .codehilite .gt { color: #FF5370 } /* Generic.Traceback */
+.dark .codehilite .kc { color: #89DDFF } /* Keyword.Constant */
+.dark .codehilite .kd { color: #BB80B3 } /* Keyword.Declaration */
+.dark .codehilite .kn { color: #89DDFF; font-style: italic } /* Keyword.Namespace */
+.dark .codehilite .kp { color: #89DDFF } /* Keyword.Pseudo */
+.dark .codehilite .kr { color: #BB80B3 } /* Keyword.Reserved */
+.dark .codehilite .kt { color: #BB80B3 } /* Keyword.Type */
+.dark .codehilite .ld { color: #C3E88D } /* Literal.Date */
+.dark .codehilite .m { color: #F78C6C } /* Literal.Number */
+.dark .codehilite .s { color: #C3E88D } /* Literal.String */
+.dark .codehilite .na { color: #BB80B3 } /* Name.Attribute */
+.dark .codehilite .nb { color: #82AAFF } /* Name.Builtin */
+.dark .codehilite .nc { color: #FFCB6B } /* Name.Class */
+.dark .codehilite .no { color: #EEFFFF } /* Name.Constant */
+.dark .codehilite .nd { color: #82AAFF } /* Name.Decorator */
+.dark .codehilite .ni { color: #89DDFF } /* Name.Entity */
+.dark .codehilite .ne { color: #FFCB6B } /* Name.Exception */
+.dark .codehilite .nf { color: #82AAFF } /* Name.Function */
+.dark .codehilite .nl { color: #82AAFF } /* Name.Label */
+.dark .codehilite .nn { color: #FFCB6B } /* Name.Namespace */
+.dark .codehilite .nx { color: #EEFFFF } /* Name.Other */
+.dark .codehilite .py { color: #FFCB6B } /* Name.Property */
+.dark .codehilite .nt { color: #FF5370 } /* Name.Tag */
+.dark .codehilite .nv { color: #89DDFF } /* Name.Variable */
+.dark .codehilite .ow { color: #89DDFF; font-style: italic } /* Operator.Word */
+.dark .codehilite .pm { color: #89DDFF } /* Punctuation.Marker */
+.dark .codehilite .w { color: #EEFFFF } /* Text.Whitespace */
+.dark .codehilite .mb { color: #F78C6C } /* Literal.Number.Bin */
+.dark .codehilite .mf { color: #F78C6C } /* Literal.Number.Float */
+.dark .codehilite .mh { color: #F78C6C } /* Literal.Number.Hex */
+.dark .codehilite .mi { color: #F78C6C } /* Literal.Number.Integer */
+.dark .codehilite .mo { color: #F78C6C } /* Literal.Number.Oct */
+.dark .codehilite .sa { color: #BB80B3 } /* Literal.String.Affix */
+.dark .codehilite .sb { color: #C3E88D } /* Literal.String.Backtick */
+.dark .codehilite .sc { color: #C3E88D } /* Literal.String.Char */
+.dark .codehilite .dl { color: #EEFFFF } /* Literal.String.Delimiter */
+.dark .codehilite .sd { color: #79d618; font-style: italic } /* Literal.String.Doc */
+.dark .codehilite .s2 { color: #C3E88D } /* Literal.String.Double */
+.dark .codehilite .se { color: #EEFFFF } /* Literal.String.Escape */
+.dark .codehilite .sh { color: #C3E88D } /* Literal.String.Heredoc */
+.dark .codehilite .si { color: #89DDFF } /* Literal.String.Interpol */
+.dark .codehilite .sx { color: #C3E88D } /* Literal.String.Other */
+.dark .codehilite .sr { color: #89DDFF } /* Literal.String.Regex */
+.dark .codehilite .s1 { color: #C3E88D } /* Literal.String.Single */
+.dark .codehilite .ss { color: #89DDFF } /* Literal.String.Symbol */
+.dark .codehilite .bp { color: #89DDFF } /* Name.Builtin.Pseudo */
+.dark .codehilite .fm { color: #82AAFF } /* Name.Function.Magic */
+.dark .codehilite .vc { color: #89DDFF } /* Name.Variable.Class */
+.dark .codehilite .vg { color: #89DDFF } /* Name.Variable.Global */
+.dark .codehilite .vi { color: #89DDFF } /* Name.Variable.Instance */
+.dark .codehilite .vm { color: #82AAFF } /* Name.Variable.Magic */
+.dark .codehilite .il { color: #F78C6C } /* Literal.Number.Integer.Long */
+
diff --git a/themes/contrast.py b/themes/contrast.py
new file mode 100644
index 0000000000000000000000000000000000000000..fd4ef0465bd97db3a4cb5c20ac749e364a640677
--- /dev/null
+++ b/themes/contrast.py
@@ -0,0 +1,88 @@
+import gradio as gr
+from toolbox import get_conf
+CODE_HIGHLIGHT, ADD_WAIFU, LAYOUT = get_conf('CODE_HIGHLIGHT', 'ADD_WAIFU', 'LAYOUT')
+
+def adjust_theme():
+
+ try:
+ color_er = gr.themes.utils.colors.fuchsia
+ set_theme = gr.themes.Default(
+ primary_hue=gr.themes.utils.colors.orange,
+ neutral_hue=gr.themes.utils.colors.gray,
+ font=["Helvetica", "Microsoft YaHei", "ui-sans-serif", "sans-serif", "system-ui"],
+ font_mono=["ui-monospace", "Consolas", "monospace"])
+ set_theme.set(
+ # Colors
+ input_background_fill_dark="*neutral_800",
+ # Transition
+ button_transition="none",
+ # Shadows
+ button_shadow="*shadow_drop",
+ button_shadow_hover="*shadow_drop_lg",
+ button_shadow_active="*shadow_inset",
+ input_shadow="0 0 0 *shadow_spread transparent, *shadow_inset",
+ input_shadow_focus="0 0 0 *shadow_spread *secondary_50, *shadow_inset",
+ input_shadow_focus_dark="0 0 0 *shadow_spread *neutral_700, *shadow_inset",
+ checkbox_label_shadow="*shadow_drop",
+ block_shadow="*shadow_drop",
+ form_gap_width="1px",
+ # Button borders
+ input_border_width="1px",
+ input_background_fill="white",
+ # Gradients
+ stat_background_fill="linear-gradient(to right, *primary_400, *primary_200)",
+ stat_background_fill_dark="linear-gradient(to right, *primary_400, *primary_600)",
+ error_background_fill=f"linear-gradient(to right, {color_er.c100}, *background_fill_secondary)",
+ error_background_fill_dark="*background_fill_primary",
+ checkbox_label_background_fill="linear-gradient(to top, *neutral_50, white)",
+ checkbox_label_background_fill_dark="linear-gradient(to top, *neutral_900, *neutral_800)",
+ checkbox_label_background_fill_hover="linear-gradient(to top, *neutral_100, white)",
+ checkbox_label_background_fill_hover_dark="linear-gradient(to top, *neutral_900, *neutral_800)",
+ button_primary_background_fill="linear-gradient(to bottom right, *primary_100, *primary_300)",
+ button_primary_background_fill_dark="linear-gradient(to bottom right, *primary_500, *primary_600)",
+ button_primary_background_fill_hover="linear-gradient(to bottom right, *primary_100, *primary_200)",
+ button_primary_background_fill_hover_dark="linear-gradient(to bottom right, *primary_500, *primary_500)",
+ button_primary_border_color_dark="*primary_500",
+ button_secondary_background_fill="linear-gradient(to bottom right, *neutral_100, *neutral_200)",
+ button_secondary_background_fill_dark="linear-gradient(to bottom right, *neutral_600, *neutral_700)",
+ button_secondary_background_fill_hover="linear-gradient(to bottom right, *neutral_100, *neutral_100)",
+ button_secondary_background_fill_hover_dark="linear-gradient(to bottom right, *neutral_600, *neutral_600)",
+ button_cancel_background_fill=f"linear-gradient(to bottom right, {color_er.c100}, {color_er.c200})",
+ button_cancel_background_fill_dark=f"linear-gradient(to bottom right, {color_er.c600}, {color_er.c700})",
+ button_cancel_background_fill_hover=f"linear-gradient(to bottom right, {color_er.c100}, {color_er.c100})",
+ button_cancel_background_fill_hover_dark=f"linear-gradient(to bottom right, {color_er.c600}, {color_er.c600})",
+ button_cancel_border_color=color_er.c200,
+ button_cancel_border_color_dark=color_er.c600,
+ button_cancel_text_color=color_er.c600,
+ button_cancel_text_color_dark="white",
+ )
+
+ if LAYOUT=="TOP-DOWN":
+ js = ""
+ else:
+ with open('themes/common.js', 'r', encoding='utf8') as f:
+ js = f"<script>{f.read()}</script>"
+
+ # 添加一个萌萌的看板娘
+ if ADD_WAIFU:
+ js += """
+ <script src="file=docs/waifu_plugin/jquery.min.js"></script>
+ <script src="file=docs/waifu_plugin/jquery-ui.min.js"></script>
+ <script src="file=docs/waifu_plugin/autoload.js"></script>
+ """
+ gradio_original_template_fn = gr.routes.templates.TemplateResponse
+ def gradio_new_template_fn(*args, **kwargs):
+ res = gradio_original_template_fn(*args, **kwargs)
+ res.body = res.body.replace(b'</html>', f'{js}</html>'.encode("utf8"))
+ res.init_headers()
+ return res
+ gr.routes.templates.TemplateResponse = gradio_new_template_fn # override gradio template
+ except:
+ set_theme = None
+ print('gradio版本较旧, 不能自定义字体和颜色')
+ return set_theme
+
+with open("themes/contrast.css", "r", encoding="utf-8") as f:
+ advanced_css = f.read()
+with open("themes/common.css", "r", encoding="utf-8") as f:
+ advanced_css += f.read()
diff --git a/themes/default.css b/themes/default.css
index 07b5383c2b62b4150416b51a1b49b8ada70f1475..a35cd1d420576e36e82aff8d3e1147629bac2546 100644
--- a/themes/default.css
+++ b/themes/default.css
@@ -1,3 +1,46 @@
+.dark {
+ --background-fill-primary: #050810;
+ --body-background-fill: var(--background-fill-primary);
+}
+/* 插件下拉菜单 */
+#plugin-panel .wrap.svelte-aqlk7e.svelte-aqlk7e.svelte-aqlk7e {
+ box-shadow: var(--input-shadow);
+ border: var(--input-border-width) dashed var(--border-color-primary);
+ border-radius: 4px;
+}
+
+#plugin-panel .dropdown-arrow.svelte-p5edak {
+ width: 50px;
+}
+#plugin-panel input.svelte-aqlk7e.svelte-aqlk7e.svelte-aqlk7e {
+ padding-left: 5px;
+}
+
+/* 小按钮 */
+.sm.svelte-1ipelgc {
+ font-family: "Microsoft YaHei UI", "Helvetica", "Microsoft YaHei", "ui-sans-serif", "sans-serif", "system-ui";
+ --button-small-text-weight: 600;
+ --button-small-text-size: 16px;
+ border-bottom-right-radius: 6px;
+ border-bottom-left-radius: 6px;
+ border-top-right-radius: 6px;
+ border-top-left-radius: 6px;
+}
+
+#plugin-panel .sm.svelte-1ipelgc {
+ font-family: "Microsoft YaHei UI", "Helvetica", "Microsoft YaHei", "ui-sans-serif", "sans-serif", "system-ui";
+ --button-small-text-weight: 400;
+ --button-small-text-size: 14px;
+ border-bottom-right-radius: 6px;
+ border-bottom-left-radius: 6px;
+ border-top-right-radius: 6px;
+ border-top-left-radius: 6px;
+}
+
+.wrap-inner.svelte-aqlk7e.svelte-aqlk7e.svelte-aqlk7e {
+ padding: 0%;
+}
+
.markdown-body table {
margin: 1em 0;
border-collapse: collapse;
@@ -17,6 +60,12 @@
padding: .5em .2em;
}
+.normal_mut_select .svelte-1gfkn6j {
+ float: left;
+ width: auto;
+ line-height: 260% !important;
+}
+
.markdown-body ol, .markdown-body ul {
padding-inline-start: 2em !important;
}
diff --git a/themes/default.py b/themes/default.py
index 4efde5bd6f9334e3814743c35cb3bceb019bbea0..2611e7aab7e7b6a41a3ef4c21a4f47b22409bebe 100644
--- a/themes/default.py
+++ b/themes/default.py
@@ -9,7 +9,7 @@ def adjust_theme():
set_theme = gr.themes.Default(
primary_hue=gr.themes.utils.colors.orange,
neutral_hue=gr.themes.utils.colors.gray,
- font=["sans-serif", "Microsoft YaHei", "ui-sans-serif", "system-ui"],
+ font=["Helvetica", "Microsoft YaHei", "ui-sans-serif", "sans-serif", "system-ui"],
font_mono=["ui-monospace", "Consolas", "monospace"])
set_theme.set(
# Colors
@@ -83,4 +83,6 @@ def adjust_theme():
return set_theme
with open("themes/default.css", "r", encoding="utf-8") as f:
- advanced_css = f.read()
\ No newline at end of file
+ advanced_css = f.read()
+with open("themes/common.css", "r", encoding="utf-8") as f:
+ advanced_css += f.read()
diff --git a/themes/green.css b/themes/green.css
index 950ac2e1feda07424a164c6b52facb6d740a2884..dd109d53fda81949834f74d767c77940709d557c 100644
--- a/themes/green.css
+++ b/themes/green.css
@@ -24,7 +24,11 @@ mspace {
border-color: yellow;
}
}
-
+.normal_mut_select .svelte-1gfkn6j {
+ float: left;
+ width: auto;
+ line-height: 260% !important;
+}
#highlight_update {
animation-name: highlight;
animation-duration: 0.75s;
diff --git a/themes/green.py b/themes/green.py
index e14f4b67706d194c1d0b087e4c7848af35116e58..5aa9e8b0f7673a9cf83d9916bcb249158f8b95d0 100644
--- a/themes/green.py
+++ b/themes/green.py
@@ -106,3 +106,5 @@ def adjust_theme():
with open("themes/green.css", "r", encoding="utf-8") as f:
advanced_css = f.read()
+with open("themes/common.css", "r", encoding="utf-8") as f:
+ advanced_css += f.read()
diff --git a/themes/theme.py b/themes/theme.py
index c08d9bb0f7b30cdb1c7679c2b8979ec3ab7e8977..5cba541cc4c597299e9a908edbb82598e9b55c2f 100644
--- a/themes/theme.py
+++ b/themes/theme.py
@@ -5,6 +5,9 @@ THEME, = get_conf('THEME')
if THEME == 'Chuanhu-Small-and-Beautiful':
from .green import adjust_theme, advanced_css
theme_declaration = "<h2 align=\"center\" class=\"small\">[Chuanhu-Small-and-Beautiful主题]</h2>"
+elif THEME == 'High-Contrast':
+ from .contrast import adjust_theme, advanced_css
+ theme_declaration = ""
else:
from .default import adjust_theme, advanced_css
theme_declaration = ""
diff --git a/toolbox.py b/toolbox.py
index 73e3e8d664d38c5abdbd3ce327dd822d46a0d72a..ab8d046a7d6d1ef1b631ee1eb05d5ad5c8beecff 100644
--- a/toolbox.py
+++ b/toolbox.py
@@ -24,6 +24,19 @@ pj = os.path.join
class ChatBotWithCookies(list):
def __init__(self, cookie):
+ """
+ cookies = {
+ 'top_p': top_p,
+ 'temperature': temperature,
+ 'lock_plugin': bool,
+ "files_to_promote": ["file1", "file2"],
+ "most_recent_uploaded": {
+ "path": "uploaded_path",
+ "time": time.time(),
+ "time_str": "timestr",
+ }
+ }
+ """
self._cookies = cookie
def write_list(self, list):
@@ -47,6 +60,8 @@ def ArgsGeneralWrapper(f):
# 引入一个有cookie的chatbot
cookies.update({
'top_p':top_p,
+ 'api_key': cookies['api_key'],
+ 'llm_model': llm_model,
'temperature':temperature,
})
llm_kwargs = {
@@ -69,7 +84,7 @@ def ArgsGeneralWrapper(f):
# 处理个别特殊插件的锁定状态
module, fn_name = cookies['lock_plugin'].split('->')
f_hot_reload = getattr(importlib.import_module(module, fn_name), fn_name)
- yield from f_hot_reload(txt_passon, llm_kwargs, plugin_kwargs, chatbot_with_cookie, history, system_prompt, *args)
+ yield from f_hot_reload(txt_passon, llm_kwargs, plugin_kwargs, chatbot_with_cookie, history, system_prompt, request)
return decorated
@@ -266,8 +281,7 @@ def report_execption(chatbot, history, a, b):
向chatbot中添加错误信息
"""
chatbot.append((a, b))
- history.append(a)
- history.append(b)
+ history.extend([a, b])
def text_divide_paragraph(text):
@@ -290,6 +304,7 @@ def text_divide_paragraph(text):
text = "</br>".join(lines)
return pre + text + suf
+
@lru_cache(maxsize=128) # 使用 lru缓存 加快转换速度
def markdown_convertion(txt):
"""
@@ -344,19 +359,41 @@ def markdown_convertion(txt):
content = content.replace('</script>\n</script>', '</script>')
return content
- def no_code(txt):
- if '```' not in txt:
- return True
- else:
- if '```reference' in txt: return True # newbing
- else: return False
-
- if ('$' in txt) and no_code(txt): # 有$标识的公式符号，且没有代码段```的标识
+ def is_equation(txt):
+ """
+ 判定是否为公式 | 测试1 写出洛伦兹定律，使用tex格式公式 测试2 给出柯西不等式，使用latex格式 测试3 写出麦克斯韦方程组
+ """
+ if '```' in txt and '```reference' not in txt: return False
+ if '$' not in txt and '\\[' not in txt: return False
+ mathpatterns = {
+ r'(?<!\\|\$)(\$)([^\$]+)(\$)': {'allow_multi_lines': False}, # $...$
+ r'(?<!\\)(\$\$)([^\$]+)(\$\$)': {'allow_multi_lines': True}, # $$...$$
+ r'(?<!\\)(\\\[)(.+?)(\\\])': {'allow_multi_lines': False}, # \[...\]
+ # r'(?<!\\)(\\\()(.+?)(\\\))': {'allow_multi_lines': False}, # \(...\)
+ # r'(?<!\\)(\\begin{([a-z]+?\*?)})(.+?)(\\end{\2})': {'allow_multi_lines': True}, # \begin...\end
+ # r'(?<!\\)(\$`)([^`]+)(`\$)': {'allow_multi_lines': False}, # $`...`$
+ }
+ matches = []
+ for pattern, property in mathpatterns.items():
+ flags = re.ASCII|re.DOTALL if property['allow_multi_lines'] else re.ASCII
+ matches.extend(re.findall(pattern, txt, flags))
+ if len(matches) == 0: return False
+ contain_any_eq = False
+ illegal_pattern = re.compile(r'[^\x00-\x7F]|echo')
+ for match in matches:
+ if len(match) != 3: return False
+ eq_canidate = match[1]
+ if illegal_pattern.search(eq_canidate):
+ return False
+ else:
+ contain_any_eq = True
+ return contain_any_eq
+
+ if is_equation(txt): # 有$标识的公式符号，且没有代码段```的标识
# convert everything to html format
split = markdown.markdown(text='---')
- convert_stage_1 = markdown.markdown(text=txt, extensions=['mdx_math', 'fenced_code', 'tables', 'sane_lists'], extension_configs=markdown_extension_configs)
+ convert_stage_1 = markdown.markdown(text=txt, extensions=['sane_lists', 'tables', 'mdx_math', 'fenced_code'], extension_configs=markdown_extension_configs)
convert_stage_1 = markdown_bug_hunt(convert_stage_1)
- # re.DOTALL: Make the '.' special character match any character at all, including a newline; without this flag, '.' will match anything except a newline. Corresponds to the inline flag (?s).
# 1. convert to easy-to-copy tex (do not render math)
convert_stage_2_1, n = re.subn(find_equation_pattern, replace_math_no_render, convert_stage_1, flags=re.DOTALL)
# 2. convert to rendered equation
@@ -364,7 +401,7 @@ def markdown_convertion(txt):
# cat them together
return pre + convert_stage_2_1 + f'{split}' + convert_stage_2_2 + suf
else:
- return pre + markdown.markdown(txt, extensions=['fenced_code', 'codehilite', 'tables', 'sane_lists']) + suf
+ return pre + markdown.markdown(txt, extensions=['sane_lists', 'tables', 'fenced_code', 'codehilite']) + suf
def close_up_code_segment_during_stream(gpt_reply):
@@ -479,7 +516,8 @@ def find_recent_files(directory):
current_time = time.time()
one_minute_ago = current_time - 60
recent_files = []
-
+ if not os.path.exists(directory):
+ os.makedirs(directory, exist_ok=True)
for filename in os.listdir(directory):
file_path = os.path.join(directory, filename)
if file_path.endswith('.log'):
@@ -503,15 +541,15 @@ def promote_file_to_downloadzone(file, rename_file=None, chatbot=None):
if not os.path.exists(new_path): shutil.copyfile(file, new_path)
# 将文件添加到chatbot cookie中，避免多用户干扰
if chatbot:
- if 'file_to_promote' in chatbot._cookies: current = chatbot._cookies['file_to_promote']
+ if 'files_to_promote' in chatbot._cookies: current = chatbot._cookies['files_to_promote']
else: current = []
- chatbot._cookies.update({'file_to_promote': [new_path] + current})
+ chatbot._cookies.update({'files_to_promote': [new_path] + current})
def disable_auto_promotion(chatbot):
- chatbot._cookies.update({'file_to_promote': []})
+ chatbot._cookies.update({'files_to_promote': []})
return
-def on_file_uploaded(files, chatbot, txt, txt2, checkboxes):
+def on_file_uploaded(files, chatbot, txt, txt2, checkboxes, cookies):
"""
当文件被上传时的回调函数
"""
@@ -545,15 +583,21 @@ def on_file_uploaded(files, chatbot, txt, txt2, checkboxes):
chatbot.append(['我上传了文件，请查收',
f'[Local Message] 收到以下文件: \n\n{moved_files_str}' +
f'\n\n调用路径参数已自动修正到: \n\n{txt}' +
- f'\n\n现在您点击任意“红颜色”标识的函数插件时，以上文件将被作为输入参数'+err_msg])
- return chatbot, txt, txt2
+ f'\n\n现在您点击任意函数插件时，以上文件将被作为输入参数'+err_msg])
+ cookies.update({
+ 'most_recent_uploaded': {
+ 'path': f'private_upload/{time_tag}',
+ 'time': time.time(),
+ 'time_str': time_tag
+ }})
+ return chatbot, txt, txt2, cookies
def on_report_generated(cookies, files, chatbot):
from toolbox import find_recent_files
- if 'file_to_promote' in cookies:
- report_files = cookies['file_to_promote']
- cookies.pop('file_to_promote')
+ if 'files_to_promote' in cookies:
+ report_files = cookies['files_to_promote']
+ cookies.pop('files_to_promote')
else:
report_files = find_recent_files('gpt_log')
if len(report_files) == 0:
@@ -1001,7 +1045,7 @@ def get_plugin_default_kwargs():
chatbot = ChatBotWithCookies(llm_kwargs)
# txt, llm_kwargs, plugin_kwargs, chatbot, history, system_prompt, web_port
- default_plugin_kwargs = {
+ DEFAULT_FN_GROUPS_kwargs = {
"main_input": "./README.md",
"llm_kwargs": llm_kwargs,
"plugin_kwargs": {},
@@ -1010,7 +1054,7 @@ def get_plugin_default_kwargs():
"system_prompt": "You are a good AI.",
"web_port": WEB_PORT
}
- return default_plugin_kwargs
+ return DEFAULT_FN_GROUPS_kwargs
def get_chat_default_kwargs():
"""
diff --git a/version b/version
index 303a44b06dd7e103ff37ff6944828e8f7f8639ac..369a80084f5f009fdc9f7011a90bfab840bf6d1e 100644
--- a/version
+++ b/version
@@ -1,5 +1,5 @@
{
- "version": 3.49,
+ "version": 3.50,
"show_feature": true,
- "new_feature": "支持借助GROBID实现PDF高精度翻译 <-> 接入百度千帆平台和文心一言 <-> 接入阿里通义千问、讯飞星火、上海AI-Lab书生 <-> 优化一键升级 <-> 提高arxiv翻译速度和成功率 <-> 支持自定义APIKEY格式 <-> 临时修复theme的文件丢失问题 <-> 新增实时语音对话插件（自动断句，脱手对话） <-> 支持加载自定义的ChatGLM2微调模型 <-> 动态ChatBot窗口高度 <-> 修复Azure接口的BUG <-> 完善多语言模块"
+ "new_feature": "支持插件分类！ <-> 支持用户使用自然语言调度各个插件(虚空终端) ！ <-> 改进UI，设计新主题 <-> 支持借助GROBID实现PDF高精度翻译 <-> 接入百度千帆平台和文心一言 <-> 接入阿里通义千问、讯飞星火、上海AI-Lab书生 <-> 优化一键升级 <-> 提高arxiv翻译速度和成功率"
}