text: layers of the model are offloaded to your Metal GPU, in the most case, set it to 1 is enough for Metaln_batch - how many tokens are processed in parallel, default is 8, set to bigger number.f16_kv - for some reason, Metal only support True, otherwise you will get error such as Asserting on type 0