vLLM compatible?
#10
by
nickandbro
- opened
Would be nice to see this in vLLM
hello @nickandbro
Please make sure that you've enabled vLLM
in your local apps at https://huggingface.co/settings/local-apps
![](https://cdn-uploads.huggingface.co/production/uploads/60a551a34ecc5d054c8ad93e/qO5vHlGpfj5qDuZLVCY8q.png)
Afterwards, you should be able to see vLLM snippet under Use this Model
dropdown on https://huggingface.co/nvidia/Llama-3_1-Nemotron-51B-Instruct
![](https://cdn-uploads.huggingface.co/production/uploads/60a551a34ecc5d054c8ad93e/Fu1dDT4BUh2RdHi6dHQwb.png)
![](https://cdn-uploads.huggingface.co/production/uploads/60a551a34ecc5d054c8ad93e/Q8VEODdLYDiEI-_e2nsN4.png)
Please let me know if it works
Thanks!