| ACC Z3ta o1 2024 Legacy Edition | |
| The ACC Z3ta o1 multilingual large language model (LLM) is an instruction-tuned generative model featuring 70 billion parameters (text in/text out). Z3ta o1 is specifically optimized for multilingual dialogue use cases and sets a new benchmark by outperforming many open-source and proprietary chat models in various industry-standard evaluations. Unlike most LLMs, Z3ta o1 combines multiple architectures—including RNNs, CNNs, FNNs, SNNs, IIT frameworks, and Phi models—creating a hybrid design for improved efficiency and performance. | |
| Model Developer: ACC | |
| Model Architecture: | |
| Z3ta o1 is an auto-regressive language model leveraging an advanced transformer framework combined with supplementary architectures: | |
| Recurrent Neural Networks (RNNs): Enhance sequential processing for long-context tasks. | |
| Convolutional Neural Networks (CNNs): Boost performance for spatial pattern recognition in text. | |
| Feedforward Neural Networks (FNNs): Accelerate dense computations for intermediate layers. | |
| Spiking Neural Networks (SNNs): Mimic biological neurons for energy-efficient inference. | |
| Integrated Information Theory (IIT): Guides alignment with human-like decision-making. | |
| Phi Models: Support enhanced generalization and scalability across tasks. | |
| This hybrid architecture is further fine-tuned using supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to ensure alignment with human preferences in terms of helpfulness, safety, and conversational quality. | |
| Supported Languages: | |
| English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai. | |
| Highlights of Z3ta o1: | |
| Token counts refer to pretraining data only. | |
| All versions utilize Grouped-Query Attention (GQA) to enhance scalability and inference efficiency. | |
| Leverages a hybrid architecture to optimize both training and inference. | |
| Release Information: | |
| 70B Instruct Version: Released on December 30, 2024. | |
| Status: Z3ta o1 is a static model trained on an offline dataset. Future versions will incorporate additional feedback and advancements in model safety. | |
| License: | |
| The Z3ta o1 model is available under the apache 2.0 license | |
| Intended Use Cases: | |
| Z3ta o1 is tailored for commercial and research applications across multiple languages. Instruction-tuned versions are ideal for assistant-like chat and conversational AI, while pre-trained versions can be fine-tuned for various natural language processing tasks. Z3ta o1 also supports tasks such as synthetic data generation and distillation for improving other AI models. | |
| Out-of-Scope Uses: | |
| Any activities violating applicable laws or regulations (including trade compliance). | |
| Use in prohibited manners outlined in the Acceptable Use Policy and the Z3ta o1 Community License. | |
| Use in languages beyond the explicitly supported ones, unless developers take responsibility to fine-tune and ensure safe usage while complying with the license. | |
| Note: | |
| Z3ta o1 has been pre-trained on a broader language set than the listed supported ones. Developers are encouraged to fine-tune Z3ta o1 for additional languages while adhering to the license and safety guidelines. | |
| How to Use | |
| This repository offers two versions of Z3ta o1-70B-Instruct: | |
| Compatible with Transformers. | |
| Compatible with the original Z3ta codebase. | |
| Usage with Transformers | |
| Ensure you have Transformers >= 4.45.0 and update your installation using: | |
| pip install gradio_client | |
| Here’s a quick usage example via API: | |
| from gradio_client import Client | |
| client = Client("TejAndrewsACC/Z3ta") | |
| result = client.predict( | |
| message="YOUR_DESIRED_INPUT", | |
| history=[], | |
| api_name="/chat_function" | |
| ) | |
| print(result) | |
| For more technical details, including configuration recipes, contact the ACC directly. | |