-
Notifications
You must be signed in to change notification settings - Fork 334
Open
Description
想请问下对于一段长的内容,要如何设计提示词进行总结比较好,用的是glm4-chat-1m int4量化模型,没按照我的模板进行总结,prompt.txt是具体事情内容。
我的代码如下
prompt = "/home/lili/project/chatglm.cpp/prompt.txt"
with open(prompt, "r", encoding="utf-8") as f:
content = f.read()
system = "你是一个总结助手,帮我根据内容按如下格式总结,内容包括:\
一、主题 \
二、目的 \
三、主要内容 \
四、结论 \"
pipeline = chatglm_cpp.Pipeline(DEFAULT_MODEL_PATH, max_length=20000)
generation_kwargs = dict(
max_length=20000,
max_new_tokens=2000,
max_context_length=1024,
do_sample=0.95,
top_k=0,
top_p=0.8, #0.7
temperature=0.6, #0.95
repetition_penalty=1.2, #1.0
stream=True,
)
system_messages: List[chatglm_cpp.ChatMessage] = []
if system is not None:
system_messages.append(chatglm_cpp.ChatMessage(role="system", content=system))
messages = system_messages.copy()
#print("system: ", messages)
messages.append(chatglm_cpp.ChatMessage(role="user", content=content[0:10000]))
for chunk in pipeline.chat(messages, **generation_kwargs):
print(chunk.content, sep="", end="", flush=True)
Metadata
Metadata
Assignees
Labels
No labels