glean/provider

Types

GenerateRequest

Everything the provider needs to make a generation call.

pub type GenerateRequest {
  GenerateRequest(
    messages: List(message.Message),
    settings: ModelSettings,
    tools: List(ToolSpec),
    tool_choice: ToolChoice,
    response_format: ResponseFormat,
  )
}

Constructors

GenerateRequest(
  messages: List(message.Message),
  settings: ModelSettings,
  tools: List(ToolSpec),
  tool_choice: ToolChoice,
  response_format: ResponseFormat,
)

GenerateResult

</>

The result of a generation call.

pub type GenerateResult {
  GenerateResult(
    content: List(GeneratedContent),
    finish_reason: error.FinishReason,
    usage: stream.Usage,
    model_id: String,
  )
}

Constructors

GenerateResult(
  content: List(GeneratedContent),
  finish_reason: error.FinishReason,
  usage: stream.Usage,
  model_id: String,
)

GeneratedContent

</>

Content generated by the model.

pub type GeneratedContent {
  GeneratedText(text: String)
  GeneratedReasoning(
    text: String,
    signature: option.Option(String),
  )
  GeneratedRedactedReasoning(data: String)
  GeneratedToolCall(
    tool_call_id: String,
    tool_name: String,
    arguments_json: String,
  )
}

Constructors

```
GeneratedText(text: String)
```
```
GeneratedReasoning(
  text: String,
  signature: option.Option(String),
)
```
Reasoning text. signature is set when the provider signs its reasoning (Claude) and must be sent back with it.
```
GeneratedRedactedReasoning(data: String)
```
Encrypted reasoning (Claude’s redacted thinking), to send back verbatim.

GeneratedToolCall(
  tool_call_id: String,
  tool_name: String,
  arguments_json: String,
)

ModelSettings

</>

Model settings for controlling generation behavior.

pub type ModelSettings {
  ModelSettings(
    max_output_tokens: option.Option(Int),
    temperature: option.Option(Float),
    top_p: option.Option(Float),
    top_k: option.Option(Int),
    stop_sequences: option.Option(List(String)),
    seed: option.Option(Int),
    presence_penalty: option.Option(Float),
    frequency_penalty: option.Option(Float),
    reasoning_effort: option.Option(ReasoningEffort),
  )
}

Constructors

ModelSettings(
  max_output_tokens: option.Option(Int),
  temperature: option.Option(Float),
  top_p: option.Option(Float),
  top_k: option.Option(Int),
  stop_sequences: option.Option(List(String)),
  seed: option.Option(Int),
  presence_penalty: option.Option(Float),
  frequency_penalty: option.Option(Float),
  reasoning_effort: option.Option(ReasoningEffort),
)

Arguments

reasoning_effort: How much a reasoning model should think before answering. Only sent by providers whose API has the parameter.

Provider

</>

The provider abstraction. Contains function fields for generation.

pub type Provider {
  Provider(
    name: String,
    model_id: String,
    generate: fn(GenerateRequest) -> Result(
      GenerateResult,
      error.GleanError,
    ),
    stream: fn(GenerateRequest, fn(stream.StreamEvent) -> Nil) -> Result(
      GenerateResult,
      error.GleanError,
    ),
  )
}

Constructors

Provider(
  name: String,
  model_id: String,
  generate: fn(GenerateRequest) -> Result(
    GenerateResult,
    error.GleanError,
  ),
  stream: fn(GenerateRequest, fn(stream.StreamEvent) -> Nil) -> Result(
    GenerateResult,
    error.GleanError,
  ),
)

Arguments

name: Human-readable provider name (e.g., “openai”, “anthropic”)
model_id: The model ID (e.g., “gpt-4o”, “claude-sonnet-4-20250514”)
generate: Non-streaming generation
stream: Streaming generation: calls callback for each event, returns final result

ReasoningEffort

</>

Reasoning effort for models that think before answering. Providers map these onto their own scale (e.g. DeepSeek folds Medium into high).

pub type ReasoningEffort {
  LowEffort
  MediumEffort
  HighEffort
}

Constructors

```
LowEffort
```
```
MediumEffort
```
```
HighEffort
```

ResponseFormat

</>

The format the model should respond in.

pub type ResponseFormat {
  TextFormat
  JsonFormat(
    schema: option.Option(schema.Schema),
    name: option.Option(String),
  )
}

Constructors

```
TextFormat
```

JsonFormat(
  schema: option.Option(schema.Schema),
  name: option.Option(String),
)

ToolChoice

</>

How the model should choose tools.

pub type ToolChoice {
  Auto
  Required
  NoTools
  SpecificTool(name: String)
}

Constructors

```
Auto
```
Model decides whether to call tools
```
Required
```
Model must call at least one tool
```
NoTools
```
Model must not call any tools
```
SpecificTool(name: String)
```
Model must call this specific tool

ToolSpec

</>

A tool specification as sent to the provider API.

pub type ToolSpec {
  ToolSpec(
    name: String,
    description: String,
    input_schema: schema.Schema,
  )
}

Constructors

ToolSpec(
  name: String,
  description: String,
  input_schema: schema.Schema,
)

Values

default_settings

</>

pub fn default_settings() -> ModelSettings

Default model settings (all None — provider uses its defaults).

reasoning_effort_to_string

</>

pub fn reasoning_effort_to_string(
  effort: ReasoningEffort,
) -> String

The wire value most OpenAI-compatible APIs use for reasoning_effort.

result_text

</>

pub fn result_text(result: GenerateResult) -> String

Extract all text content from a generate result.

result_tool_calls

</>

pub fn result_tool_calls(
  result: GenerateResult,
) -> List(#(String, String, String))

Extract tool calls from generated content.