Eguneratuta, bukatutzat eman genezake, emaitzak falta dira soilik
Browse files
README.md
CHANGED
|
@@ -45,9 +45,11 @@ tags:
|
|
| 45 |
|
| 46 |
This model achieves state-of-the-art performance on zero-shot Named Entity Recognition (NER) by first training on `GuideX`, a large-scale synthetic dataset with executable guidelines, and then fine-tuning on a collection of gold-standard IE datasets.
|
| 47 |
|
| 48 |
-
-
|
| 49 |
-
-
|
| 50 |
-
-
|
|
|
|
|
|
|
| 51 |
|
| 52 |
## Model Description
|
| 53 |
|
|
@@ -58,3 +60,91 @@ This model achieves state-of-the-art performance on zero-shot Named Entity Recog
|
|
| 58 |
- **License:** Llama 2 Community License
|
| 59 |
- **Finetuned from model:** `meta-llama/Llama-3.1-8B`
|
| 60 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
|
| 46 |
This model achieves state-of-the-art performance on zero-shot Named Entity Recognition (NER) by first training on `GuideX`, a large-scale synthetic dataset with executable guidelines, and then fine-tuning on a collection of gold-standard IE datasets.
|
| 47 |
|
| 48 |
+
- 💻 **Project Page:** [https://neilus03.github.io/guidex.com/](https://neilus03.github.io/guidex.com/)
|
| 49 |
+
- 📒 **Code:** [GuideX codebase](https://github.com/Neilus03/GUIDEX)
|
| 50 |
+
- 📖 **Paper:** [GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction](https://arxiv.org/abs/2506.00649)
|
| 51 |
+
- 🐕 **GuideX Colection in the 🤗HuggingFace Hub:** [GuideX Collection](https://huggingface.co/collections/neildlf/guidex-6842ef478e8d9bb0a00c844d)
|
| 52 |
+
- 🚀 **Example Jupyter Notebooks:** [GuideX Notebooks](https://github.com/Neilus03/GUIDEX/tree/dev-neil/notebooks)
|
| 53 |
|
| 54 |
## Model Description
|
| 55 |
|
|
|
|
| 60 |
- **License:** Llama 2 Community License
|
| 61 |
- **Finetuned from model:** `meta-llama/Llama-3.1-8B`
|
| 62 |
|
| 63 |
+
## Schema definition and inference example
|
| 64 |
+
|
| 65 |
+
The labels are represented as Python classes, and the guidelines or instructions are introduced as docstrings. The model start generating after the `result = [` line.
|
| 66 |
+
```Python
|
| 67 |
+
# Entity definitions
|
| 68 |
+
@dataclass
|
| 69 |
+
class Launcher(Template):
|
| 70 |
+
"""Refers to a vehicle designed primarily to transport payloads from the Earth's
|
| 71 |
+
surface to space. Launchers can carry various payloads, including satellites,
|
| 72 |
+
crewed spacecraft, and cargo, into various orbits or even beyond Earth's orbit.
|
| 73 |
+
They are usually multi-stage vehicles that use rocket engines for propulsion."""
|
| 74 |
+
|
| 75 |
+
mention: str
|
| 76 |
+
"""
|
| 77 |
+
The name of the launcher vehicle.
|
| 78 |
+
Such as: "Sturn V", "Atlas V", "Soyuz", "Ariane 5"
|
| 79 |
+
"""
|
| 80 |
+
space_company: str # The company that operates the launcher. Such as: "Blue origin", "ESA", "Boeing", "ISRO", "Northrop Grumman", "Arianespace"
|
| 81 |
+
crew: List[str] # Names of the crew members boarding the Launcher. Such as: "Neil Armstrong", "Michael Collins", "Buzz Aldrin"
|
| 82 |
+
|
| 83 |
+
|
| 84 |
+
@dataclass
|
| 85 |
+
class Mission(Template):
|
| 86 |
+
"""Any planned or accomplished journey beyond Earth's atmosphere with specific objectives,
|
| 87 |
+
either crewed or uncrewed. It includes missions to satellites, the International
|
| 88 |
+
Space Station (ISS), other celestial bodies, and deep space."""
|
| 89 |
+
|
| 90 |
+
mention: str
|
| 91 |
+
"""
|
| 92 |
+
The name of the mission.
|
| 93 |
+
Such as: "Apollo 11", "Artemis", "Mercury"
|
| 94 |
+
"""
|
| 95 |
+
date: str # The start date of the mission
|
| 96 |
+
departure: str # The place from which the vehicle will be launched. Such as: "Florida", "Houston", "French Guiana"
|
| 97 |
+
destination: str # The place or planet to which the launcher will be sent. Such as "Moon", "low-orbit", "Saturn"
|
| 98 |
+
|
| 99 |
+
# This is the text to analyze
|
| 100 |
+
text = (
|
| 101 |
+
"The Ares 3 mission to Mars is scheduled for 2032. The Starship rocket build by SpaceX will take off from Boca Chica,"
|
| 102 |
+
"carrying the astronauts Max Rutherford, Elena Soto, and Jake Martinez."
|
| 103 |
+
)
|
| 104 |
+
|
| 105 |
+
# The annotation instances that take place in the text above are listed here
|
| 106 |
+
result = [
|
| 107 |
+
Mission(mention='Ares 3', date='2032', departure='Boca Chica', destination='Mars'),
|
| 108 |
+
Launcher(mention='Starship', space_company='SpaceX', crew=['Max Rutherford', 'Elena Soto', 'Jake Martinez'])
|
| 109 |
+
]
|
| 110 |
+
```
|
| 111 |
+
|
| 112 |
+
## How to Get Started with the Model
|
| 113 |
+
|
| 114 |
+
Please read our [🚀 Example Jupyter Notebooks](https://github.com/Neilus03/GUIDEX/tree/dev-neil/notebooks) to get started with GuideX.
|
| 115 |
+
|
| 116 |
+
The best way to load the model is using our custom `load_model` fuction. However, you can also load them using the AutoModelForCausalLM class.
|
| 117 |
+
|
| 118 |
+
**Important**: Our flash attention implementation has small numerical differences compared to the attention implementation in Huggingface.
|
| 119 |
+
You must use the flag `trust_remote_code=True` or you will get inferior results. Flash attention requires an available CUDA GPU. Running GuideX
|
| 120 |
+
pre-trained models on a CPU is not supported. We plan to address this in future releases. First, install flash attention 2:
|
| 121 |
+
```bash
|
| 122 |
+
pip install flash-attn --no-build-isolation
|
| 123 |
+
pip install git+https://github.com/HazyResearch/flash-attention.git#subdirectory=csrc/rotary
|
| 124 |
+
```
|
| 125 |
+
|
| 126 |
+
Then you can load the model using
|
| 127 |
+
|
| 128 |
+
```python
|
| 129 |
+
import torch
|
| 130 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 131 |
+
|
| 132 |
+
tokenizer = AutoTokenizer.from_pretrained("HiTZ/Llama-3.1-GuideX-8B")
|
| 133 |
+
model = AutoModelForCausalLM.from_pretrained("HiTZ/Llama-3.1-GuideX-8B", trust_remote_code=True, torch_dtype=torch.bfloat16)
|
| 134 |
+
model.to("cuda")
|
| 135 |
+
```
|
| 136 |
+
|
| 137 |
+
Read our [🚀 Example Jupyter Notebooks](https://github.com/hitz-zentroa/GoLLIE/tree/main/notebooks) to learn how to easily define guidelines, generate model inputs and parse the output!
|
| 138 |
+
|
| 139 |
+
```
|
| 140 |
+
## Citation
|
| 141 |
+
@misc{delafuente2025guidexguidedsyntheticdata,
|
| 142 |
+
title={GuideX: Guided Synthetic Data Generation for Zero-Shot Information Extraction},
|
| 143 |
+
author={Neil De La Fuente and Oscar Sainz and Iker García-Ferrero and Eneko Agirre},
|
| 144 |
+
year={2025},
|
| 145 |
+
eprint={2506.00649},
|
| 146 |
+
archivePrefix={arXiv},
|
| 147 |
+
primaryClass={cs.CL},
|
| 148 |
+
url={https://arxiv.org/abs/2506.00649},
|
| 149 |
+
}
|
| 150 |
+
```
|