|
|
--- |
|
|
license: apache-2.0 |
|
|
pipeline_tag: automatic-speech-recognition |
|
|
base_model: |
|
|
- openai/whisper-large-v3 |
|
|
tags: |
|
|
- inference_endpoints |
|
|
- audio |
|
|
- transcription |
|
|
--- |
|
|
|
|
|
# Inference Endpoint - Multilingual Audio Transcription with Whisper models |
|
|
|
|
|
**Deploy OpenAI's Whisper Inference Endpoint to transcribe audio files to text in many languages** |
|
|
|
|
|
Resulting deployment exposes an [OpenAI Platform Transcription](https://platform.openai.com/docs/api-reference/audio/createTranscription) compatible HTTP endpoint |
|
|
which you can query using the `OpenAi` Libraries or directly through `cURL` for instance. |
|
|
|
|
|
## Available Routes |
|
|
| path | description | |
|
|
|:-----------------------------|:--------------------------------------------------| |
|
|
| /api/v1/audio/transcriptions | Transcription endpoint to interact with the model | |
|
|
| /docs | Visual documentation | |
|
|
|
|
|
## Getting started |
|
|
|
|
|
- **Getting text output from audio file** |
|
|
|
|
|
```bash |
|
|
curl http://localhost:8000/api/v1/audio/transcriptions \ |
|
|
--request POST \ |
|
|
--header 'Content-Type: multipart/form-data' \ |
|
|
-F file=@</path/to/audio/file> \ |
|
|
-F "response_format": "text" |
|
|
``` |
|
|
|
|
|
- **Getting JSON output from audio file** |
|
|
|
|
|
```bash |
|
|
curl http://localhost:8000/api/v1/audio/transcriptions \ |
|
|
--request POST \ |
|
|
--header 'Content-Type: multipart/form-data' \ |
|
|
-F file=@</path/to/audio/file> \ |
|
|
-F "response_format": "json" |
|
|
``` |
|
|
|
|
|
- **Getting segmented JSON output from audio file** |
|
|
|
|
|
```bash |
|
|
curl http://localhost:8000/api/v1/audio/transcriptions \ |
|
|
--request POST \ |
|
|
--header 'Content-Type: multipart/form-data' \ |
|
|
-F file=@</path/to/audio/file> \ |
|
|
-F "response_format": "verbose_json" |
|
|
``` |
|
|
|
|
|
|
|
|
## Specifications |
|
|
|
|
|
| spec | value | description | |
|
|
|:------------------ |:---------------------:|:-----------------------------------------------------------------------------------------------------------| |
|
|
| Engine | vLLM (v0.8.3) | Underlying inference engine leverages [vLLM](https://docs.vllm.ai/en/latest/) | |
|
|
| Hardware | GPU (Ada Lovelace) | Requires the target endpoint to run over NVIDIA GPUs with at least compute capabilities 8.9 (Ada Lovelace) | |
|
|
| Compute data type | `bfloat16` | Computations (matmuls, norms, etc.) are done using `bfloat16` precision | |
|
|
| KV cache data type | `float8` (e4m3) | Key-Value cache is stored on the GPU using `float8` (`float8_e4m3`) precision to save space | |
|
|
| PyTorch Compile | ✅ | Enable the use of `torch.compile` to further optimize model's execution with more optimizations | |
|
|
| CUDA Graphs | ✅ | Enable the use of so called "[CUDA Graphs](https://developer.nvidia.com/blog/cuda-graphs/)" to reduce overhead executing GPU computations | |
