CoreModel

Properties

topK

number

topP

number

temperature

number

repeatPenality

number

flashAttention

boolean

contextSize

number

grammar extension web

any

---

Methods

prompt

async (input, options) => string

See promptStreaming

promptStreaming

(input, options) => ReadableStream

This prompts the core model with the input provided. No templating or extra control characters are added, the raw input is passed to the model.

Prompt
A string with the whole prompt that's passed to the model

Options (optional)
signal optional AbortSignal

Returns a readable stream that updates each time new tokens are available from the language model

countPromptTokens

async (input, options) => number

Input
A string with the whole prompt that's passed to the model

Options (optional)
signal optional AbortSignal