|
--- |
|
base_model: |
|
- facebook/musicgen-small |
|
library_name: transformers |
|
license: cc-by-4.0 |
|
pipeline_tag: text-to-audio |
|
tags: |
|
- music |
|
- art |
|
--- |
|
|
|
# tasty-musicgen-small |
|
|
|
[](https://creativecommons.org/licenses/by/4.0/) |
|
[](https://arxiv.org/abs/2503.02823) |
|
|
|
tasty-musicgen-small is a [musicgen-small](https://huggingface.co/facebook/musicgen-small) fine-tuned on a [patched version](https://github.com/matteospanio/taste-music-dataset) of the [Taste & Affect Music Database](https://osf.io/2cqa5/). |
|
It generates music that's supposed to induce gustatory synesthesia perceptions based on multimodal research. It generates mono audio in 32khz. |
|
|
|
## Code and Dataset |
|
|
|
Code and the dataset used to train this model are available at: https://osf.io/xs5jy/. |
|
|
|
## How to use |
|
|
|
Here is a showcase on how to use the model with the transformer library, it is also possible to make inference with the audiocraft library, for a detailed explanation we suggest to read the [official MusicGEN guide](https://huggingface.co/docs/transformers/main/model_doc/musicgen) by Hugging Face |
|
|
|
```python |
|
from transformers import pipeline |
|
import scipy |
|
|
|
synthesiser = pipeline("text-to-audio", "csc-unipd/tasty-musicgen-small") |
|
|
|
music = synthesiser("sweet music for fine restaurents", forward_params={"do_sample": True}) |
|
|
|
scipy.io.wavfile.write("musicgen_out.wav", rate=music["sampling_rate"], data=music["audio"]) |
|
``` |
|
|
|
## Citation |
|
|
|
If you use this model, code or the data in your research, please cite the following article: |
|
|
|
``` |
|
@misc{spanio2025multimodalsymphonyintegratingtaste, |
|
title={A Multimodal Symphony: Integrating Taste and Sound through Generative AI}, |
|
author={Matteo Spanio and Massimiliano Zampini and Antonio Rodà and Franco Pierucci}, |
|
year={2025}, |
|
eprint={2503.02823}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.SD}, |
|
url={https://arxiv.org/abs/2503.02823}, |
|
} |
|
``` |
