EricLu/SCP-116K
Viewer
•
Updated
•
117k
•
47
•
19
Mert
Sengil
AI & ML interests
LLM's
Recent Activity
liked
a dataset
1 day ago
EricLu/SCP-116K
liked
a model
2 days ago
emredeveloper/DeepSeek-R1-Medical-COT
updated
a model
12 days ago
Sengil/ModernBERT-NewsClassifier-EN-small
Organizations
None yet
Sengil's activity
liked
a
dataset
1 day ago
liked
a
model
2 days ago
updated
a
model
12 days ago
published
a
model
12 days ago
liked
a
model
15 days ago
liked
a
dataset
29 days ago
liked
a
model
30 days ago
ValueError: The checkpoint you are trying to load has model type `modernbert`
2
#37 opened 30 days ago
by
Sengil
liked
a
model
30 days ago
liked
3
datasets
about 1 month ago
liked
a
model
about 1 month ago
updated
a
model
about 1 month ago
updated
a
model
about 2 months ago
updated
a
dataset
about 2 months ago
liked
a
model
about 2 months ago
reacted to
gabrielchua's
post with 👀
2 months ago
Post
1348
Sharing my first paper!
==
Large Language Models (LLMs) are powerful, but they're prone to off-topic misuse, where users push them beyond their intended scope. Think harmful prompts, jailbreaks, and misuse. So how do we build better guardrails?
Traditional guardrails rely on curated examples or classifiers. The problem?
⚠️ High false-positive rates
⚠️ Poor adaptability to new misuse types
⚠️ Require real-world data, which is often unavailable during pre-production
Our method skips the need for real-world misuse examples. Instead, we:
1️⃣ Define the problem space qualitatively
2️⃣ Use an LLM to generate synthetic misuse prompts
3️⃣ Train and test guardrails on this dataset
We apply this to the off-topic prompt detection problem, and fine-tune simple bi- and cross-encoder classifiers that outperform heuristics based on cosine similarity or prompt engineering.
Additionally, framing the problem as prompt relevance allows these fine-tuned classifiers to generalise to other risk categories (e.g., jailbreak, toxic prompts).
Through this work, we also open-source our dataset (2M examples, ~50M+ tokens) and models.
paper: A Flexible Large Language Models Guardrail Development Methodology Applied to Off-Topic Prompt Detection (2411.12946)
artifacts: govtech/off-topic-guardrail-673838a62e4c661f248e81a4
==
Large Language Models (LLMs) are powerful, but they're prone to off-topic misuse, where users push them beyond their intended scope. Think harmful prompts, jailbreaks, and misuse. So how do we build better guardrails?
Traditional guardrails rely on curated examples or classifiers. The problem?
⚠️ High false-positive rates
⚠️ Poor adaptability to new misuse types
⚠️ Require real-world data, which is often unavailable during pre-production
Our method skips the need for real-world misuse examples. Instead, we:
1️⃣ Define the problem space qualitatively
2️⃣ Use an LLM to generate synthetic misuse prompts
3️⃣ Train and test guardrails on this dataset
We apply this to the off-topic prompt detection problem, and fine-tune simple bi- and cross-encoder classifiers that outperform heuristics based on cosine similarity or prompt engineering.
Additionally, framing the problem as prompt relevance allows these fine-tuned classifiers to generalise to other risk categories (e.g., jailbreak, toxic prompts).
Through this work, we also open-source our dataset (2M examples, ~50M+ tokens) and models.
paper: A Flexible Large Language Models Guardrail Development Methodology Applied to Off-Topic Prompt Detection (2411.12946)
artifacts: govtech/off-topic-guardrail-673838a62e4c661f248e81a4
reacted to
maxiw's
post with 👍
2 months ago
Post
2128
You can now try out computer use models from the hub to automate your local machine with https://github.com/askui/vision-agent. 💻
Currently these models are integrated with Gradio Spaces API. Also planning to add local inference soon!
Currently supported:
- Qwen/Qwen2-VL-7B-Instruct
- Qwen/Qwen2-VL-2B-Instruct
- AskUI/PTA-1
- OS-Copilot/OS-Atlas-Base-7B
import time
from askui import VisionAgent
with VisionAgent() as agent:
agent.tools.webbrowser.open_new("http://www.google.com")
time.sleep(0.5)
agent.click("search field in the center of the screen", model_name="Qwen/Qwen2-VL-7B-Instruct")
agent.type("cats")
agent.keyboard("enter")
time.sleep(0.5)
agent.click("text 'Images'", model_name="AskUI/PTA-1")
time.sleep(0.5)
agent.click("second cat image", model_name="OS-Copilot/OS-Atlas-Base-7B")Currently these models are integrated with Gradio Spaces API. Also planning to add local inference soon!
Currently supported:
- Qwen/Qwen2-VL-7B-Instruct
- Qwen/Qwen2-VL-2B-Instruct
- AskUI/PTA-1
- OS-Copilot/OS-Atlas-Base-7B
- 3 replies
liked
a
Space
2 months ago