README.md

---
license: openrail
datasets:
- universal-dependencies/universal_dependencies
language:
- bg
- ca
- zh
- hr
- cs
- da
- nl
- en
- fi
- fr
- de
- hu
- id
- it
- ja
- ko
- 'no'
- pl
- pt
- ro
- ru
- sl
- es
- sv
- uk
base_model:
- meta-llama/Llama-2-13b-hf
---

# Introduction
The paper explores the capabilities of Large Language Models (LLMs) like LLaMA in syntactic parsing tasks. We introduce U-DepPLLaMA, a novel architecture that treats Dependency Parsing as a sequence-to-sequence problem, achieving state-of-the-art results in 26 languages from the Universal Dependency Treebank. Our approach demonstrates that LLMs can handle dependency parsing without the need for specialized architectures, showing robust performance even with complex sentence structures. The paper is available [here](https://journals.openedition.org/ijcol/1352).

For more details, please consult the associated [Github repository](https://github.com/crux82/u-deppllama).

# How to use it

```Python
model = AutoModelForCausalLM.from_pretrained(
            "meta-llama/Llama-2-13b-hf",
            load_in_4bit=True,
            quantization_config=quant_config,
            torch_dtype=torch.float16,
            trust_remote_code=True,
            device_map={"": 0},
        )
model = PeftModel.from_pretrained(
            model,
            "sag-uniroma2/u-depp-llama-2-13b"
        )

input_string = "He was most widely recognized for some of his books."
prompt = f"""
### Input:
{input_string}
### Answer:"""

inputs = tokenizer(prompt, return_tensors="pt", padding=do_padding, truncation=True, max_length=CUTOFF_LEN)
input_ids = inputs["input_ids"].to(model.device)

with torch.no_grad():
    gen_outputs = model.generate(
        input_ids=input_ids,
        generation_config=generation_config,
        return_dict_in_generate=True,
        output_scores=True,
        max_new_tokens=MAX_NEW_TOKENS,
        use_cache=True,
    )
s = gen_outputs.sequences[0]
output = tokenizer.decode(s, skip_special_tokens=True)

if "### Answer:" in output:
    response = output.split("### Answer:")[1].rstrip().lstrip()
else:
    response = "UNK"
    print("WARNING")
    print(row)
    print(output)
    print("\n--------------------\n")
```

# Citation
```
@article{hromei2024udeppllama,
  author  = "Hromei, Claudiu Daniel and Croce, Danilo and Basili, Roberto",
  title   = "U-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language Models",
  journal = "IJCoL",
  year    = 2024,
  volume  = "10",
  number  = "1",
  pages   = "21--38"
}
```