Update README.md with new model card content
Browse files
README.md
CHANGED
|
@@ -13,6 +13,22 @@ pipeline_tag: text-generation
|
|
| 13 |
|
| 14 |
Falcon-RW-1B is a 1B parameters causal decoder-only model built by [TII](https://www.tii.ae/) and trained on 350B tokens of [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb). The architecture of the model is adopted from the GPT-3 paper ([Brown et al., 2020](https://arxiv.org/abs/2005.14165)) but it uses ALiBi.
|
| 15 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
## Use
|
| 17 |
|
| 18 |
### Direct Use
|
|
@@ -82,3 +98,53 @@ The architecture is adapted from the GPT-3 paper ([Brown et al., 2020](https://a
|
|
| 82 |
}
|
| 83 |
```
|
| 84 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
Falcon-RW-1B is a 1B parameters causal decoder-only model built by [TII](https://www.tii.ae/) and trained on 350B tokens of [RefinedWeb](https://huggingface.co/datasets/tiiuae/falcon-refinedweb). The architecture of the model is adopted from the GPT-3 paper ([Brown et al., 2020](https://arxiv.org/abs/2005.14165)) but it uses ALiBi.
|
| 15 |
|
| 16 |
+
## Links
|
| 17 |
+
|
| 18 |
+
* [Falcon Quickstart Notebook](https://www.kaggle.com/code/laxmareddypatlolla/falcon-quickstart-notebook)
|
| 19 |
+
* [Falcon API Documentation](https://keras.io/keras_hub/api/models/falcon/)
|
| 20 |
+
* [Falcon Model Card](https://huggingface.co/docs/transformers/en/model_doc/falcon)
|
| 21 |
+
* [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
|
| 22 |
+
* [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
|
| 23 |
+
|
| 24 |
+
## Presets
|
| 25 |
+
|
| 26 |
+
The following model checkpoints are provided by the Keras team. Full code examples for each are available below.
|
| 27 |
+
| Preset name | Parameters | Description |
|
| 28 |
+
|----------------|------------|--------------------------------------------------|
|
| 29 |
+
| falcon_refinedweb_1b_en | 1.31B | 24-layer Falcon model (Falcon with 1B parameters), trained on 350B tokens of RefinedWeb dataset.|
|
| 30 |
+
|
| 31 |
+
|
| 32 |
## Use
|
| 33 |
|
| 34 |
### Direct Use
|
|
|
|
| 98 |
}
|
| 99 |
```
|
| 100 |
|
| 101 |
+
## Example Usage
|
| 102 |
+
```Python
|
| 103 |
+
|
| 104 |
+
import os
|
| 105 |
+
|
| 106 |
+
os.environ["KERAS_BACKEND"] = "jax"
|
| 107 |
+
|
| 108 |
+
import keras
|
| 109 |
+
import keras_hub
|
| 110 |
+
|
| 111 |
+
# When running only inference, bfloat16 saves memory usage significantly.
|
| 112 |
+
keras.config.set_floatx("bfloat16")
|
| 113 |
+
|
| 114 |
+
causal_lm = keras_hub.models.FalconCausalLM.from_preset(
|
| 115 |
+
"falcon_refinedweb_1b_en"
|
| 116 |
+
)
|
| 117 |
+
causal_lm.summary()
|
| 118 |
+
|
| 119 |
+
outputs = causal_lm.generate([
|
| 120 |
+
"What is Jax?",
|
| 121 |
+
"Give me your best brownie recipe.",
|
| 122 |
+
], max_length=512)
|
| 123 |
+
|
| 124 |
+
```
|
| 125 |
+
|
| 126 |
+
## Example Usage with Hugging Face URI
|
| 127 |
+
|
| 128 |
+
```Python
|
| 129 |
+
|
| 130 |
+
import os
|
| 131 |
+
|
| 132 |
+
os.environ["KERAS_BACKEND"] = "jax"
|
| 133 |
+
|
| 134 |
+
import keras
|
| 135 |
+
import keras_hub
|
| 136 |
+
|
| 137 |
+
# When running only inference, bfloat16 saves memory usage significantly.
|
| 138 |
+
keras.config.set_floatx("bfloat16")
|
| 139 |
+
|
| 140 |
+
causal_lm = keras_hub.models.FalconCausalLM.from_preset(
|
| 141 |
+
"hf://keras/falcon_refinedweb_1b_en"
|
| 142 |
+
)
|
| 143 |
+
causal_lm.summary()
|
| 144 |
+
|
| 145 |
+
outputs = causal_lm.generate([
|
| 146 |
+
"What is Jax?",
|
| 147 |
+
"Give me your best brownie recipe.",
|
| 148 |
+
], max_length=512)
|
| 149 |
+
|
| 150 |
+
```
|