Ollama Function Calling #51
Conversation
andthattoo
commented
May 18, 2024
andthattoo
commented
- Resolves OpenAI compatible function calling #50
- More in-depth testing incoming
…s function calling standards.
implemented base traits removed unnecessary code & import
Added NousHermes prompts.rs
function calling structs now return ``ChatMessageResponse`if it's a tool error. That being: - Error parsing tool call - No such tool - Validation error for call format (parameter errors etc.)
|
We should also add docs about this |
added docs, cleaned comments, slight formatting
- RequestParserBase trait now has `format_query` and `format_response` for greater flexibility - Updated the prompt of NousHermes - Tool errors are now ChatMessageResponses: They are not Ollama errors and might be passed to LLMs as messages for further processing. Error handler classes added. - Removed ChatMessage::Tool, Ollama doesn't accept it yet, maybe added in future. - send_function_call_with_history now uses Ollama's set_system_response - Fixed a few bugs, added JSON parsing guardails
|
Did some manual testing with different models also added a few more tests to An example run with finance tool + nous hermes 2: > What are the current risk factors to $APPL?
[2024-05-20T00:01:32Z INFO ollama_rs::generation::functions] Function call: To provide you with an analysis of the current risk factors for Apple Inc. (AAPL), I will use the "Stock Scraper" tool from the provided functions. This tool allows me to scrape stock information from Google Finance, including potential risk factors. Please bear with me as I gather this data.
<tool_call>
{"arguments": {"exchange": "NASDAQ", "ticker": "AAPL"}, "name": "Stock Scraper"}
</tool_call>
[2024-05-20T00:01:33Z INFO ollama_rs::generation::functions] Parsing done
<tool_response>
{"Market cap":"2.91T USD","Avg Volume":"63.51M","Dividend yield":"0.53%","CEO":"Tim Cook","Primary exchange":"NASDAQ","Founded":"Apr 1, 1976","Year range":"$164.08 - $199.62","Previous close":"$189.84","Website":"apple.com","P/E ratio":"29.53","Headquarters":"Cupertino, CaliforniaUnited States","Employees":"161,000","Day range":"$189.18 - $190.81"}
</tool_response>I tried to stick to the original design of ollama-rs as much as possible, and keep the recursive-looping agentic functionality excluded from this repo. Both Function calling is crucial for agentic frameworks such as ReAct (Reason + Action), but it would introduce unnecessary overhead to ollama-rs. I would probably implement it in langchain-rust or a crewAI-like Rust doppelganger. One can use responses from function calls to "Reason" over the observations done through "tools" implemented within the library. |
|
@pepperoni21 my bad, added |
|
Looks very good! I'll run the tests when I'm home 👍 |
…ad of "tool" and "tool_inputs". Testing showed it worked better with different models.
|
@pepperoni21 any estimates on when we can merge this? |
