Skip to content

xnano.ai.models.text

xnano.ai.models.text

xnano.ai.models.text

Prose from a language model, written whole or streamed into a placement as it arrives.

Classes:

  • TextModel –

    Write prose for the canvas: a description, a narration, a line of dialogue.

Functions:

  • stream_into_async –

    Show text in a placement as its pieces arrive, and return the whole text.

  • stream_into –

    Show text in a placement as its pieces arrive, for code that does not await.

TextModel

TextModel(model: Model | KnownModelName | str | None = None, *, style: str | None = None, max_sentences: int | None = None, instructions: str | None = None, model_settings: ModelSettings | None = None, capabilities: Sequence[AbstractCapability[Any]] = (), retries: int = 3, usage_limits: UsageLimits | None = None, thinking: ThinkingLevel | None = None, cache: bool = True)

Bases: TaskModel

Write prose for the canvas: a description, a narration, a line of dialogue.

generate returns the whole text; stream gives it piece by piece as the model writes it, which stream_into shows in a placement as it arrives.

Parameters:

  • model (Model | KnownModelName | str | None, default: None ) –

    The language model to ask.

  • style (str | None, default: None ) –

    How the text should read, such as 'wry, second person'.

  • max_sentences (int | None, default: None ) –

    The most sentences an answer keeps; a longer answer is cut after them.

  • instructions (str | None, default: None ) –

    What the model is told before every request.

  • model_settings (ModelSettings | None, default: None ) –

    pydantic-ai settings for every request.

  • capabilities (Sequence[AbstractCapability[Any]], default: () ) –

    pydantic-ai capabilities for the agent.

  • retries (int, default: 3 ) –

    How many times an empty answer is asked again.

  • usage_limits (UsageLimits | None, default: None ) –

    Limits on the requests and tokens of one call.

  • thinking (ThinkingLevel | None, default: None ) –

    Whether the model reasons before answering. None, the default, leaves that to the model; False turns it off where the model allows it, and some models cannot turn it off.

  • cache (bool, default: True ) –

    Whether the unchanging start of every prompt is cached by the provider.

Examples:

from xnano.ai.models.text import TextModel
from xnano.ai.testing import scripted_model

writer = TextModel(scripted_model('A door. A key. A choice.').model, max_sentences=2)
assert writer.generate('Describe the room.') == 'A door. A key.'

Create the text model; nothing is sent until the first call.

Raises:

  • ValueError –

    max_sentences is not a whole number of 1 or more.

Methods:

  • generate_async –

    Write the text asked for, with await.

  • generate –

    Write the text asked for, waiting for the whole answer.

  • stream_async –

    Write the text asked for, giving each new piece as the model writes it, with

  • stream –

    Write the text asked for, giving each piece as it is written, for code that does not

  • override –

    Ask another model inside a with block: a test double, or a cheaper draft model.

  • details –

    What the backend reported about its last answer, as pydantic-ai's provider_details.

  • probabilities –

    The distribution a decision model gave a pick-one or rubric field in its last answer.

  • confidence –

    The confidence a decision model reported for a field of its last answer, or None.

Attributes:

Source code in xnano/ai/models/text.py
def __init__(
    self,
    model: Model | KnownModelName | str | None = None,
    *,
    style: str | None = None,
    max_sentences: int | None = None,
    instructions: str | None = None,
    model_settings: ModelSettings | None = None,
    capabilities: Sequence[AbstractCapability[Any]] = (),
    retries: int = 3,
    usage_limits: UsageLimits | None = None,
    thinking: ThinkingLevel | None = None,
    cache: bool = True,
) -> None:
    """
    Create the text model; nothing is sent until the first call.

    Raises:
        ValueError: ``max_sentences`` is not a whole number of 1 or more.
    """
    if max_sentences is not None and (
        isinstance(max_sentences, bool)
        or not isinstance(max_sentences, int)
        or max_sentences < 1
    ):
        raise ValueError(
            f'max_sentences must be a whole number of 1 or more, or None, but was '
            f'{max_sentences!r}.'
        )
    parts = [instructions or '']
    if style:
        parts.append(f'Write in this style: {style}.')
    if max_sentences is not None:
        parts.append(f'Write at most {max_sentences} sentences.')
    super().__init__(
        model,
        instructions=' '.join(part for part in parts if part) or None,
        model_settings=model_settings,
        capabilities=capabilities,
        retries=retries,
        usage_limits=usage_limits,
        thinking=thinking,
        cache=cache,
    )
    self.style = style
    self.max_sentences = max_sentences
    self._retries = retries

is_decision_model property

is_decision_model: bool

Whether model is a decision model, which answers with probabilities.

generate_async

generate_async(request: str, evidence: object = None) -> Coroutine[Any, Any, str]

Write the text asked for, with await.

The evidence is read when this is called; await the result for the text.

Parameters:

  • request (str) –

    What to write, such as 'Describe the room the hero just entered.'.

  • evidence (object, default: None ) –

    What the text is about, such as the hook's context; see Evidence. None sends the request alone.

Returns:

  • Coroutine[Any, Any, str] –

    A coroutine that gives the text, cut after max_sentences sentences.

Raises:

  • TerminalError –

    The model could not be reached (ai.model), or wrote nothing, even when asked again (ai.generation), when awaited.

Source code in xnano/ai/models/text.py
def generate_async(
    self, request: str, evidence: object = None
) -> Coroutine[Any, Any, str]:
    """
    Write the text asked for, with ``await``.

    The evidence is read when this is called; await the result for the text.

    Args:
        request: What to write, such as ``'Describe the room the hero just entered.'``.
        evidence: What the text is about, such as the hook's context; see
            [`Evidence`][xnano.ai.Evidence]. ``None`` sends the request alone.

    Returns:
        A coroutine that gives the text, cut after ``max_sentences`` sentences.

    Raises:
        TerminalError: The model could not be reached (``ai.model``), or wrote nothing,
            even when asked again (``ai.generation``), when awaited.
    """
    return self._generate(self._prompt(request, snapshot(evidence)))

generate

generate(request: str, evidence: object = None) -> str

Write the text asked for, waiting for the whole answer.

The same as generate_async, for code that does not await.

Source code in xnano/ai/models/text.py
def generate(self, request: str, evidence: object = None) -> str:
    """
    Write the text asked for, waiting for the whole answer.

    The same as [`generate_async`][xnano.ai.models.text.TextModel.generate_async], for code that does
    not await.
    """
    return _run_blocking(self.generate_async(request, evidence), 'generate')

stream_async

stream_async(request: str, evidence: object = None) -> AsyncIterator[str]

Write the text asked for, giving each new piece as the model writes it, with async for.

The evidence is read when this is called.

Parameters:

  • request (str) –

    What to write.

  • evidence (object, default: None ) –

    What the text is about; see Evidence.

Yields:

  • AsyncIterator[str] –

    The pieces of the text, in order: joined, they are the whole text.

Raises:

Examples:

import asyncio

from xnano.ai.models.text import TextModel
from xnano.ai.testing import scripted_model

writer = TextModel(scripted_model('Once upon a time.').model)


async def main() -> str:
    return ''.join([piece async for piece in writer.stream_async('Begin.')])


assert asyncio.run(main()) == 'Once upon a time.'
Source code in xnano/ai/models/text.py
def stream_async(self, request: str, evidence: object = None) -> AsyncIterator[str]:
    """
    Write the text asked for, giving each new piece as the model writes it, with
    ``async for``.

    The evidence is read when this is called.

    Args:
        request: What to write.
        evidence: What the text is about; see [`Evidence`][xnano.ai.Evidence].

    Yields:
        The pieces of the text, in order: joined, they are the whole text.

    Raises:
        TerminalError: The model could not be reached (``ai.model``).

    Examples:
        ```python
        import asyncio

        from xnano.ai.models.text import TextModel
        from xnano.ai.testing import scripted_model

        writer = TextModel(scripted_model('Once upon a time.').model)


        async def main() -> str:
            return ''.join([piece async for piece in writer.stream_async('Begin.')])


        assert asyncio.run(main()) == 'Once upon a time.'
        ```
    """
    self._require_model('stream')
    return self._stream(self._prompt(request, snapshot(evidence)))

stream

stream(request: str, evidence: object = None) -> Iterator[str]

Write the text asked for, giving each piece as it is written, for code that does not await.

The same as stream_async. Stopping the loop early cancels the request.

Source code in xnano/ai/models/text.py
def stream(self, request: str, evidence: object = None) -> Iterator[str]:
    """
    Write the text asked for, giving each piece as it is written, for code that does not
    await.

    The same as [`stream_async`][xnano.ai.models.text.TextModel.stream_async]. Stopping the loop
    early cancels the request.
    """
    return _blocking_pieces(self.stream_async(request, evidence), 'stream')

override

override(*, model: Model | KnownModelName | str) -> Generator[None, None, None]

Ask another model inside a with block: a test double, or a cheaper draft model.

Parameters:

  • model (Model | KnownModelName | str) –

    The model to ask instead.

Examples:

from xnano.ai.models.decisions import ChoiceModel
from xnano.ai.testing import ScriptedDecisionModel

pick = ChoiceModel('typesafe:jev-1.14.0')
with pick.override(model=ScriptedDecisionModel({'response': 'left'})):
    assert pick.choose('the hero', ['left', 'right']).choice == 'left'
Source code in xnano/ai/models/__init__.py
@contextlib.contextmanager
def override(
    self, *, model: Model | KnownModelName | str
) -> Generator[None, None, None]:
    """
    Ask another model inside a ``with`` block: a test double, or a cheaper draft model.

    Args:
        model: The model to ask instead.

    Examples:
        ```python
        from xnano.ai.models.decisions import ChoiceModel
        from xnano.ai.testing import ScriptedDecisionModel

        pick = ChoiceModel('typesafe:jev-1.14.0')
        with pick.override(model=ScriptedDecisionModel({'response': 'left'})):
            assert pick.choose('the hero', ['left', 'right']).choice == 'left'
        ```
    """
    previous = self.model
    self.model = model
    try:
        with self.agent.override(model=model):
            yield
    finally:
        self.model = previous

details

details() -> dict[str, Any]

What the backend reported about its last answer, as pydantic-ai's provider_details.

Empty before the first call, and for a backend that reports nothing.

Source code in xnano/ai/models/__init__.py
def details(self) -> dict[str, Any]:
    """
    What the backend reported about its last answer, as pydantic-ai's ``provider_details``.

    Empty before the first call, and for a backend that reports nothing.
    """
    response = self._last_response()
    if response is None or response.provider_details is None:
        return {}
    return dict(response.provider_details)

probabilities

probabilities(field: str) -> dict[str, float] | None

The distribution a decision model gave a pick-one or rubric field in its last answer.

Parameters:

  • field (str) –

    The name of the field, such as 'response' for a single pick-one.

Returns:

  • dict[str, float] | None –

    The probability of each option, or None on a language model or for a field it

  • dict[str, float] | None –

    did not report.

Source code in xnano/ai/models/__init__.py
def probabilities(self, field: str) -> dict[str, float] | None:
    """
    The distribution a decision model gave a pick-one or rubric field in its last answer.

    Args:
        field: The name of the field, such as ``'response'`` for a single pick-one.

    Returns:
        The probability of each option, or ``None`` on a language model or for a field it
        did not report.
    """
    distribution = self.details().get('probabilities', {}).get(field)
    if distribution is None:
        return None
    return {str(option): float(value) for option, value in distribution.items()}

confidence

confidence(field: str) -> float | None

The confidence a decision model reported for a field of its last answer, or None.

The confidence is the backend's own figure; it is not how often the answer is right.

Source code in xnano/ai/models/__init__.py
def confidence(self, field: str) -> float | None:
    """
    The confidence a decision model reported for a field of its last answer, or ``None``.

    The confidence is the backend's own figure; it is not how often the answer is right.
    """
    value = self.details().get('confidence', {}).get(field)
    return None if value is None else float(value)

stream_into_async async

stream_into_async(placement: Placement[Any, Any], pieces: AsyncIterable[str], *, debounce_seconds: float = 1 / 30) -> str

Show text in a placement as its pieces arrive, and return the whole text.

The placement's text is replaced at most every debounce_seconds with everything that arrived so far, so a fast stream costs a few redraws, not one per piece. When the stream ends or is cancelled, the placement keeps all the text that arrived.

Parameters:

  • placement (Placement[Any, Any]) –

    Where to show the text; it grows with it, unless it was given a size.

  • pieces (AsyncIterable[str]) –

    The pieces, such as TextModel.stream_async(...).

  • debounce_seconds (float, default: 1 / 30 ) –

    The least time between two redraws.

Returns:

  • str –

    The whole text.

Examples:

import asyncio

import xnano
from xnano.ai.models.text import TextModel, stream_into_async
from xnano.ai.testing import scripted_model

writer = TextModel(scripted_model('The rain stops.').model)


async def main() -> str:
    async with xnano.new(width=20, height=1) as canvas:
        label = canvas.screen.write('')
        await stream_into_async(label, writer.stream_async('Narrate.'))
        return canvas.screen.render()


assert asyncio.run(main()) == 'The rain stops.'
Source code in xnano/ai/models/text.py
async def stream_into_async(
    placement: Placement[Any, Any],
    pieces: AsyncIterable[str],
    *,
    debounce_seconds: float = 1 / 30,
) -> str:
    """
    Show text in a placement as its pieces arrive, and return the whole text.

    The placement's text is replaced at most every ``debounce_seconds`` with everything that
    arrived so far, so a fast stream costs a few redraws, not one per piece. When the stream ends
    or is cancelled, the placement keeps all the text that arrived.

    Args:
        placement: Where to show the text; it grows with it, unless it was given a size.
        pieces: The pieces, such as ``TextModel.stream_async(...)``.
        debounce_seconds: The least time between two redraws.

    Returns:
        The whole text.

    Examples:
        ```python
        import asyncio

        import xnano
        from xnano.ai.models.text import TextModel, stream_into_async
        from xnano.ai.testing import scripted_model

        writer = TextModel(scripted_model('The rain stops.').model)


        async def main() -> str:
            async with xnano.new(width=20, height=1) as canvas:
                label = canvas.screen.write('')
                await stream_into_async(label, writer.stream_async('Narrate.'))
                return canvas.screen.render()


        assert asyncio.run(main()) == 'The rain stops.'
        ```
    """
    _check_debounce(debounce_seconds)
    text: list[str] = []
    shown = ''
    last = 0.0
    try:
        async for piece in pieces:
            text.append(piece)
            now = time.monotonic()
            if now - last >= debounce_seconds:
                shown = ''.join(text)
                placement.text = shown
                last = now
    finally:
        whole = ''.join(text)
        if whole != shown and not placement._removed:
            placement.text = whole
    return whole

stream_into

stream_into(placement: Placement[Any, Any], pieces: Iterable[str], *, debounce_seconds: float = 1 / 30) -> str

Show text in a placement as its pieces arrive, for code that does not await.

The same as stream_into_async, with pieces such as TextModel.stream(...).

Examples:

import xnano
from xnano.ai.models.text import TextModel, stream_into
from xnano.ai.testing import scripted_model

writer = TextModel(scripted_model('Thunder rolls.').model)
with xnano.new(width=20, height=1) as canvas:
    label = canvas.screen.write('')
    assert stream_into(label, writer.stream('Narrate.')) == 'Thunder rolls.'
    assert canvas.screen.render() == 'Thunder rolls.'
Source code in xnano/ai/models/text.py
def stream_into(
    placement: Placement[Any, Any],
    pieces: Iterable[str],
    *,
    debounce_seconds: float = 1 / 30,
) -> str:
    """
    Show text in a placement as its pieces arrive, for code that does not await.

    The same as [`stream_into_async`][xnano.ai.models.text.stream_into_async], with pieces such as
    ``TextModel.stream(...)``.

    Examples:
        ```python
        import xnano
        from xnano.ai.models.text import TextModel, stream_into
        from xnano.ai.testing import scripted_model

        writer = TextModel(scripted_model('Thunder rolls.').model)
        with xnano.new(width=20, height=1) as canvas:
            label = canvas.screen.write('')
            assert stream_into(label, writer.stream('Narrate.')) == 'Thunder rolls.'
            assert canvas.screen.render() == 'Thunder rolls.'
        ```
    """
    _check_debounce(debounce_seconds)
    text: list[str] = []
    shown = ''
    last = 0.0
    try:
        for piece in pieces:
            text.append(piece)
            now = time.monotonic()
            if now - last >= debounce_seconds:
                shown = ''.join(text)
                placement.text = shown
                last = now
    finally:
        whole = ''.join(text)
        if whole != shown and not placement._removed:
            placement.text = whole
    return whole