Ollama Client

8.17.0.3

Ollama Client🔗ℹ

Bogdan Popa <bogdan@defn.io>

This package provides a client for Ollama.

This package should be considered experimental. All bindings documented here are subject to change.

1 Reference🔗ℹ

(require ollama)

package: ollama-lib

1.1 Client🔗ℹ

procedure
(ollama-client? v) → boolean?
v : any/c

Returns #t when v is an Ollama client.

procedure
(make-ollama-client [base #:auth auth]) → ollama-client?
  base : string? = "http://127.0.0.1:11434"
   auth : auth-procedure/c =
(λ (url headers params)
  (values headers params))

Returns an Ollama client that communicates with Ollama via base. When the #:auth argument is provided, it is used to authenticate requests.

procedure
(ollama-start-chat
c
model
str-or-messages
[ #:options options
#:format output-format
#:tools tools
#:response->history-entry response-converter])
→
chat-response/c chat-continuation/c
  c : ollama-client?
  model : string?
  str-or-messages : (or/c string? message? (listof (or/c string? message?)))
  options : jsexpr? = (hasheq)
  output-format : (or/c #f string?) = #f
  tools : (or/c #f (hash/c symbol? tool-info?)) = #f
   response-converter : (-> jsexpr? (or/c #f message?))
=
(λ (data)
  (make-message
   #:role 'assistant
   (hash-ref (hash-ref data 'message) 'content)))

Starts a chat with model via c by sending it str-or-messages. Returns a pair of procedures:

A procedure to return the next part of the model’s response. Returns eof when the end of response has been reached.
A procedure to continue the conversation by sending the LLM subsequent messages. Returns a new pair of chat response and chat continuation procedures. Continuing a chat preserves the chat history and passes it along to the LLM on every call.

The #:options argument can be used to pass arbitrary chat options to Ollama.

The #:format argument can be used to control the LLM’s output format. When output-format is 'json, the LLM is instructed to format its response as JSON (but you should also encourage it to do so in your message).

The #:tools argument can be used to supply the LLM with tools that it can call to perform actions.

The #:response->history-entry procedure can be used to alter LLM responses before they’re committed to history. For example, you can use this hook to filter out reasoning from an LLM response. When the result of applying this procedure is #f, no entry is stored for that response.

contract
chat-response/c : (-> (or/c jsexpr? eof-object?))

The contract for chat responses.

contract
chat-continuation/c
:
(->* [(or/c string? message? (listof (or/c string? message?)))]
     [#:format (or/c #f 'json jsexpr?)
      #:tools (or/c #f (hash/c symbol? tool-info?))]
     (values
      chat-response/c
      (recursive-contract chat-continuation/c)))

The contract for chat continuations. The #:format and #:tools arguments override the values of the same arguments passed to any previous chat continuation procedures or to the original ollama-start-chat call.

1.2 Message🔗ℹ

procedure
(message? v) → boolean?
v : any/c

Returns #t when v is a message.

procedure
(make-message content
[ #:role role
#:images images]) → message?
  content : string?
  role : (or/c 'assistant 'system 'user 'tool) = 'user
  images : (listof bytes?) = null

Creates a new message with the given content and role. The #:images argument can be used to attach base64-encoded images to the message for multimodal LLMs.

1.3 Tool🔗ℹ

Tools are Racket functions that an LLM can call.

procedure
(tool-info? v) → boolean?
v : any/c

Returns #t when v is a tool.

syntax
(define-tool-definer definer-id
#:getter getter-id
#:caller caller-id)

Binds definer-id as a definition form that declares tools.

Binds getter-id to a procedure that returns a hash of all the tools that have been defined so far via definer-id.

Binds caller-id to a procedure that, given LLM tool call data, calls one of the defined procedures and returns response that can be sent to the LLM.

The syntax of define-id is

syntax
:
syntax
(define-id (tool-name [arg-id : type-expr] ...)
  maybe-description
  maybe-example
  e ...+)

maybe-description =
| #:description description-expr

maybe-example =
| #:example example-expr maybe-example

A define-id form binds tool-name as a Racket procedure with the given set of arg-id positional arguments. The procedure and its argument metadata is recorded in an internal registry so that all tools defined by define-id can be retrieved at runtime and their metadata provided to an LLM.

procedure
(Array elem) → type
  elem : type
value
Boolean : type
procedure
(Enum option ...+) → type
  option : string?
value
Null : type
value
Number : type
syntax
(Object [property : type] ...+)
procedure
(Optional t) → type
  t : type
procedure
(Or t ...+) → type
  t : type
value
String : type

Tool arguments must declare one of these types.

Examples:

> (require ollama)
> (define-tool-definer define-tool
    #:getter get-tools
    #:caller call-tool)
> (define-tool (get-temperature [location : String])
    "451 Fahrenheit")
> (get-tools)
'#hasheq((get-temperature . #<tool-info>))
> (call-tool
   (hasheq
    'function
    (hasheq
     'name "get-temperature"
     'arguments (hasheq 'location "Los Angeles"))))
"{\"function\":{\"arguments\":{\"location\":\"Los Angeles\"},\"name\":\"get-temperature\"},\"result\":\"451 Fahrenheit\"}"

The caller-id procedure raises an exception when a requested tool cannot be found or when it is called incorrectly.

procedure
(exn:fail:tool? v) → boolean?
  v : any/c
procedure
(exn:fail:tool:not-found? v) → boolean?
  v : any/c
procedure
(exn:fail:tool:call? v) → boolean?
  v : any/c

Predicates for errors raised by tools.

A tool error can be converted to JSON by applying ->jsexpr to it. The conversion adds an 'error key to the original call data with the exception message.

procedure
(raise-tool-error format-string
format-arg ...
[ #:hints hints]) → void?
  format-string : string?
  format-arg : any/c
  hints : (listof string?) = null

Raise a tool error with the given format and hints. The #:hints argument can be used to provide a list of strings to instruct the LLM on how to recover.

1.4 JSON🔗ℹ

interface
gen:to-jsexpr

An interface for values that can be converted to JSON.

procedure
(to-jsexpr? v) → boolean?
v : any/c

Returns #t when v implements gen:to-jsexpr.

procedure
(->jsexpr v) → jsexpr?
v : to-jsexpr?

Converts v to a jsexpr?, assuming v implements gen:to-jsexpr.