Tool support

July 25, 2024 Ollama now supports tool calling with popular models such as Llama 3.1. This enables a model to answer a given prompt using tool(s) it knows about, making it possible for models to perform more complex tasks or interact with the outside world. Example tools include: Functions and APIs Web browsing Code interpreter much more! Tool calling To enable tool calling, provide a list of available tools via the tools field in Ollama’s API. import ollama response = ollama.chat( model='llama3.1', messages=[{'role': 'user', 'content': 'What is the weather in Toronto?'}], # provide a weather checking tool to the model tools=[{ 'type': 'function', 'function': { 'name': 'get_current_weather', 'description': 'Get the current weather for a city', 'parameters': { 'type': 'object', 'properties': { 'city': { 'type': 'string', 'description': 'The name of the city', }, }, 'required': ['city'], }, }, }, ], ) print(response['message']['tool_calls']) Supported models will now answer with a tool_calls response. Tool responses can be provided via messages with the tool role. See API documentation for more information. Supported models A list of supported models can be found under the Tools category on the models page: Llama 3.1 Mistral Nemo Firefunction v2 Command-R + Note: please check if you have the latest model by running ollama pull OpenAI compatibility Ollama’s OpenAI compatible endpoint also now supports tools, making it possible to switch to using Llama 3.1 and other models. import openai openai.base_url = "http://localhost:11434/v1" openai.api_key = 'ollama' response = openai.chat.completions.create( model="llama3.1", messages=messages, tools=tools, ) Examples JavaScript Future improvements Streaming tool calls: stream tool calls back to begin taking action faster when multiple tools are returned Tool choice: force a model to use a tool Let’s build together We are so excited to bring you tool support, and see what you build with it! If you have any feedback, please do not hesitate to tell us either in our Discord or via hello@ollama.com.