Support inference API for 7b model?
#14
by
wmedrano
- opened
According to the Starcoder2 paper(Table 16 page 30), the 15b model seems to have degraded fim support due to a configuration bug. Leaving the starcoder2-3b as the best fim model with inference support. Is it possible to support starcoder2-7b with the inference API?
I've been using it for my autocomplete so would appreciate it if I can used an improved model.