Update README.md
Browse files
README.md
CHANGED
|
@@ -111,6 +111,8 @@ The llama-2 models have been modified from a standard transformer in the followi
|
|
| 111 |
| tokens | 2.0T |
|
| 112 |
| vocab size | 32000 |
|
| 113 |
| sequence length | 4096 |
|
|
|
|
|
|
|
| 114 |
|
| 115 |
## Finetuning Description
|
| 116 |
|
|
|
|
| 111 |
| tokens | 2.0T |
|
| 112 |
| vocab size | 32000 |
|
| 113 |
| sequence length | 4096 |
|
| 114 |
+
| grouped-query attention | ✔️ |
|
| 115 |
+
|
| 116 |
|
| 117 |
## Finetuning Description
|
| 118 |
|