Running AI Models Locally
It is also possible to run AI models locally, using ollama or similar solution. In this case, you download the model and all your requests and code are sent to the locally running model and thus are never uploaded to any third party AI provider.
Note: Although this is one way to use AI in Wing, be warned that locally run models tend
to produce results that are considerably lower in quality and usability than those you
can obtain from other AI providers. This may change over time, as more powerful models can
be run locally, but for now we recommend against trying to do serious work with locally
run models.
Ollama
To use ollama to run models locally, all you need to do is:
- Download and install ollama
- Run ollama pull <model> with <model> replaced with the model you want to try.
- Run ollama run <model>
- Configure an AI provider in Wing, setting API Host to Use Base URL with url http://localhost:11434/v1/. Set Key to Use Key and enter unused (the key is required but ignored), enter the Model identifier you selected, and set costs to 0.0.
- Select this new provider configuration as the current provider.
- In AI Chat select New Conversation from its Options menu
- Try a request like "Can you write a Python hello world?"
