xnano.ai.models.text
xnano.ai.models.text
¶
xnano.ai.models.text
Prose from a language model, written whole or streamed into a placement as it arrives.
Classes:
-
TextModel–Write prose for the canvas: a description, a narration, a line of dialogue.
Functions:
-
stream_into_async–Show text in a placement as its pieces arrive, and return the whole text.
-
stream_into–Show text in a placement as its pieces arrive, for code that does not await.
TextModel
¶
TextModel(model: Model | KnownModelName | str | None = None, *, style: str | None = None, max_sentences: int | None = None, instructions: str | None = None, model_settings: ModelSettings | None = None, capabilities: Sequence[AbstractCapability[Any]] = (), retries: int = 3, usage_limits: UsageLimits | None = None, thinking: ThinkingLevel | None = None, cache: bool = True)
Bases: TaskModel
Write prose for the canvas: a description, a narration, a line of dialogue.
generate returns the whole text; stream gives it piece by piece as the model writes
it, which stream_into shows in a placement as it arrives.
Parameters:
-
model(Model | KnownModelName | str | None, default:None) –The language model to ask.
-
style(str | None, default:None) –How the text should read, such as
'wry, second person'. -
max_sentences(int | None, default:None) –The most sentences an answer keeps; a longer answer is cut after them.
-
instructions(str | None, default:None) –What the model is told before every request.
-
model_settings(ModelSettings | None, default:None) –pydantic-ai settings for every request.
-
capabilities(Sequence[AbstractCapability[Any]], default:()) –pydantic-ai capabilities for the agent.
-
retries(int, default:3) –How many times an empty answer is asked again.
-
usage_limits(UsageLimits | None, default:None) –Limits on the requests and tokens of one call.
-
thinking(ThinkingLevel | None, default:None) –Whether the model reasons before answering.
None, the default, leaves that to the model;Falseturns it off where the model allows it, and some models cannot turn it off. -
cache(bool, default:True) –Whether the unchanging start of every prompt is cached by the provider.
Examples:
from xnano.ai.models.text import TextModel
from xnano.ai.testing import scripted_model
writer = TextModel(scripted_model('A door. A key. A choice.').model, max_sentences=2)
assert writer.generate('Describe the room.') == 'A door. A key.'
Create the text model; nothing is sent until the first call.
Raises:
-
ValueError–max_sentencesis not a whole number of 1 or more.
Methods:
-
generate_async–Write the text asked for, with
await. -
generate–Write the text asked for, waiting for the whole answer.
-
stream_async–Write the text asked for, giving each new piece as the model writes it, with
-
stream–Write the text asked for, giving each piece as it is written, for code that does not
-
override–Ask another model inside a
withblock: a test double, or a cheaper draft model. -
details–What the backend reported about its last answer, as pydantic-ai's
provider_details. -
probabilities–The distribution a decision model gave a pick-one or rubric field in its last answer.
-
confidence–The confidence a decision model reported for a field of its last answer, or
None.
Attributes:
-
is_decision_model(bool) –Whether
modelis a decision model, which answers with probabilities.
Source code in xnano/ai/models/text.py
def __init__(
self,
model: Model | KnownModelName | str | None = None,
*,
style: str | None = None,
max_sentences: int | None = None,
instructions: str | None = None,
model_settings: ModelSettings | None = None,
capabilities: Sequence[AbstractCapability[Any]] = (),
retries: int = 3,
usage_limits: UsageLimits | None = None,
thinking: ThinkingLevel | None = None,
cache: bool = True,
) -> None:
"""
Create the text model; nothing is sent until the first call.
Raises:
ValueError: ``max_sentences`` is not a whole number of 1 or more.
"""
if max_sentences is not None and (
isinstance(max_sentences, bool)
or not isinstance(max_sentences, int)
or max_sentences < 1
):
raise ValueError(
f'max_sentences must be a whole number of 1 or more, or None, but was '
f'{max_sentences!r}.'
)
parts = [instructions or '']
if style:
parts.append(f'Write in this style: {style}.')
if max_sentences is not None:
parts.append(f'Write at most {max_sentences} sentences.')
super().__init__(
model,
instructions=' '.join(part for part in parts if part) or None,
model_settings=model_settings,
capabilities=capabilities,
retries=retries,
usage_limits=usage_limits,
thinking=thinking,
cache=cache,
)
self.style = style
self.max_sentences = max_sentences
self._retries = retries
is_decision_model
property
¶
is_decision_model: bool
Whether model is a decision model, which answers with probabilities.
generate_async
¶
Write the text asked for, with await.
The evidence is read when this is called; await the result for the text.
Parameters:
-
request(str) –What to write, such as
'Describe the room the hero just entered.'. -
evidence(object, default:None) –What the text is about, such as the hook's context; see
Evidence.Nonesends the request alone.
Returns:
Raises:
-
TerminalError–The model could not be reached (
ai.model), or wrote nothing, even when asked again (ai.generation), when awaited.
Source code in xnano/ai/models/text.py
def generate_async(
self, request: str, evidence: object = None
) -> Coroutine[Any, Any, str]:
"""
Write the text asked for, with ``await``.
The evidence is read when this is called; await the result for the text.
Args:
request: What to write, such as ``'Describe the room the hero just entered.'``.
evidence: What the text is about, such as the hook's context; see
[`Evidence`][xnano.ai.Evidence]. ``None`` sends the request alone.
Returns:
A coroutine that gives the text, cut after ``max_sentences`` sentences.
Raises:
TerminalError: The model could not be reached (``ai.model``), or wrote nothing,
even when asked again (``ai.generation``), when awaited.
"""
return self._generate(self._prompt(request, snapshot(evidence)))
generate
¶
Write the text asked for, waiting for the whole answer.
The same as generate_async, for code that does
not await.
Source code in xnano/ai/models/text.py
def generate(self, request: str, evidence: object = None) -> str:
"""
Write the text asked for, waiting for the whole answer.
The same as [`generate_async`][xnano.ai.models.text.TextModel.generate_async], for code that does
not await.
"""
return _run_blocking(self.generate_async(request, evidence), 'generate')
stream_async
¶
stream_async(request: str, evidence: object = None) -> AsyncIterator[str]
Write the text asked for, giving each new piece as the model writes it, with
async for.
The evidence is read when this is called.
Parameters:
-
request(str) –What to write.
-
evidence(object, default:None) –What the text is about; see
Evidence.
Yields:
-
AsyncIterator[str]–The pieces of the text, in order: joined, they are the whole text.
Raises:
-
TerminalError–The model could not be reached (
ai.model).
Examples:
import asyncio
from xnano.ai.models.text import TextModel
from xnano.ai.testing import scripted_model
writer = TextModel(scripted_model('Once upon a time.').model)
async def main() -> str:
return ''.join([piece async for piece in writer.stream_async('Begin.')])
assert asyncio.run(main()) == 'Once upon a time.'
Source code in xnano/ai/models/text.py
def stream_async(self, request: str, evidence: object = None) -> AsyncIterator[str]:
"""
Write the text asked for, giving each new piece as the model writes it, with
``async for``.
The evidence is read when this is called.
Args:
request: What to write.
evidence: What the text is about; see [`Evidence`][xnano.ai.Evidence].
Yields:
The pieces of the text, in order: joined, they are the whole text.
Raises:
TerminalError: The model could not be reached (``ai.model``).
Examples:
```python
import asyncio
from xnano.ai.models.text import TextModel
from xnano.ai.testing import scripted_model
writer = TextModel(scripted_model('Once upon a time.').model)
async def main() -> str:
return ''.join([piece async for piece in writer.stream_async('Begin.')])
assert asyncio.run(main()) == 'Once upon a time.'
```
"""
self._require_model('stream')
return self._stream(self._prompt(request, snapshot(evidence)))
stream
¶
Write the text asked for, giving each piece as it is written, for code that does not await.
The same as stream_async. Stopping the loop
early cancels the request.
Source code in xnano/ai/models/text.py
def stream(self, request: str, evidence: object = None) -> Iterator[str]:
"""
Write the text asked for, giving each piece as it is written, for code that does not
await.
The same as [`stream_async`][xnano.ai.models.text.TextModel.stream_async]. Stopping the loop
early cancels the request.
"""
return _blocking_pieces(self.stream_async(request, evidence), 'stream')
override
¶
Ask another model inside a with block: a test double, or a cheaper draft model.
Parameters:
-
model(Model | KnownModelName | str) –The model to ask instead.
Examples:
from xnano.ai.models.decisions import ChoiceModel
from xnano.ai.testing import ScriptedDecisionModel
pick = ChoiceModel('typesafe:jev-1.14.0')
with pick.override(model=ScriptedDecisionModel({'response': 'left'})):
assert pick.choose('the hero', ['left', 'right']).choice == 'left'
Source code in xnano/ai/models/__init__.py
@contextlib.contextmanager
def override(
self, *, model: Model | KnownModelName | str
) -> Generator[None, None, None]:
"""
Ask another model inside a ``with`` block: a test double, or a cheaper draft model.
Args:
model: The model to ask instead.
Examples:
```python
from xnano.ai.models.decisions import ChoiceModel
from xnano.ai.testing import ScriptedDecisionModel
pick = ChoiceModel('typesafe:jev-1.14.0')
with pick.override(model=ScriptedDecisionModel({'response': 'left'})):
assert pick.choose('the hero', ['left', 'right']).choice == 'left'
```
"""
previous = self.model
self.model = model
try:
with self.agent.override(model=model):
yield
finally:
self.model = previous
details
¶
What the backend reported about its last answer, as pydantic-ai's provider_details.
Empty before the first call, and for a backend that reports nothing.
Source code in xnano/ai/models/__init__.py
def details(self) -> dict[str, Any]:
"""
What the backend reported about its last answer, as pydantic-ai's ``provider_details``.
Empty before the first call, and for a backend that reports nothing.
"""
response = self._last_response()
if response is None or response.provider_details is None:
return {}
return dict(response.provider_details)
probabilities
¶
The distribution a decision model gave a pick-one or rubric field in its last answer.
Parameters:
-
field(str) –The name of the field, such as
'response'for a single pick-one.
Returns:
-
dict[str, float] | None–The probability of each option, or
Noneon a language model or for a field it -
dict[str, float] | None–did not report.
Source code in xnano/ai/models/__init__.py
def probabilities(self, field: str) -> dict[str, float] | None:
"""
The distribution a decision model gave a pick-one or rubric field in its last answer.
Args:
field: The name of the field, such as ``'response'`` for a single pick-one.
Returns:
The probability of each option, or ``None`` on a language model or for a field it
did not report.
"""
distribution = self.details().get('probabilities', {}).get(field)
if distribution is None:
return None
return {str(option): float(value) for option, value in distribution.items()}
confidence
¶
The confidence a decision model reported for a field of its last answer, or None.
The confidence is the backend's own figure; it is not how often the answer is right.
Source code in xnano/ai/models/__init__.py
def confidence(self, field: str) -> float | None:
"""
The confidence a decision model reported for a field of its last answer, or ``None``.
The confidence is the backend's own figure; it is not how often the answer is right.
"""
value = self.details().get('confidence', {}).get(field)
return None if value is None else float(value)
stream_into_async
async
¶
stream_into_async(placement: Placement[Any, Any], pieces: AsyncIterable[str], *, debounce_seconds: float = 1 / 30) -> str
Show text in a placement as its pieces arrive, and return the whole text.
The placement's text is replaced at most every debounce_seconds with everything that
arrived so far, so a fast stream costs a few redraws, not one per piece. When the stream ends
or is cancelled, the placement keeps all the text that arrived.
Parameters:
-
placement(Placement[Any, Any]) –Where to show the text; it grows with it, unless it was given a size.
-
pieces(AsyncIterable[str]) –The pieces, such as
TextModel.stream_async(...). -
debounce_seconds(float, default:1 / 30) –The least time between two redraws.
Returns:
-
str–The whole text.
Examples:
import asyncio
import xnano
from xnano.ai.models.text import TextModel, stream_into_async
from xnano.ai.testing import scripted_model
writer = TextModel(scripted_model('The rain stops.').model)
async def main() -> str:
async with xnano.new(width=20, height=1) as canvas:
label = canvas.screen.write('')
await stream_into_async(label, writer.stream_async('Narrate.'))
return canvas.screen.render()
assert asyncio.run(main()) == 'The rain stops.'
Source code in xnano/ai/models/text.py
async def stream_into_async(
placement: Placement[Any, Any],
pieces: AsyncIterable[str],
*,
debounce_seconds: float = 1 / 30,
) -> str:
"""
Show text in a placement as its pieces arrive, and return the whole text.
The placement's text is replaced at most every ``debounce_seconds`` with everything that
arrived so far, so a fast stream costs a few redraws, not one per piece. When the stream ends
or is cancelled, the placement keeps all the text that arrived.
Args:
placement: Where to show the text; it grows with it, unless it was given a size.
pieces: The pieces, such as ``TextModel.stream_async(...)``.
debounce_seconds: The least time between two redraws.
Returns:
The whole text.
Examples:
```python
import asyncio
import xnano
from xnano.ai.models.text import TextModel, stream_into_async
from xnano.ai.testing import scripted_model
writer = TextModel(scripted_model('The rain stops.').model)
async def main() -> str:
async with xnano.new(width=20, height=1) as canvas:
label = canvas.screen.write('')
await stream_into_async(label, writer.stream_async('Narrate.'))
return canvas.screen.render()
assert asyncio.run(main()) == 'The rain stops.'
```
"""
_check_debounce(debounce_seconds)
text: list[str] = []
shown = ''
last = 0.0
try:
async for piece in pieces:
text.append(piece)
now = time.monotonic()
if now - last >= debounce_seconds:
shown = ''.join(text)
placement.text = shown
last = now
finally:
whole = ''.join(text)
if whole != shown and not placement._removed:
placement.text = whole
return whole
stream_into
¶
stream_into(placement: Placement[Any, Any], pieces: Iterable[str], *, debounce_seconds: float = 1 / 30) -> str
Show text in a placement as its pieces arrive, for code that does not await.
The same as stream_into_async, with pieces such as
TextModel.stream(...).
Examples:
import xnano
from xnano.ai.models.text import TextModel, stream_into
from xnano.ai.testing import scripted_model
writer = TextModel(scripted_model('Thunder rolls.').model)
with xnano.new(width=20, height=1) as canvas:
label = canvas.screen.write('')
assert stream_into(label, writer.stream('Narrate.')) == 'Thunder rolls.'
assert canvas.screen.render() == 'Thunder rolls.'
Source code in xnano/ai/models/text.py
def stream_into(
placement: Placement[Any, Any],
pieces: Iterable[str],
*,
debounce_seconds: float = 1 / 30,
) -> str:
"""
Show text in a placement as its pieces arrive, for code that does not await.
The same as [`stream_into_async`][xnano.ai.models.text.stream_into_async], with pieces such as
``TextModel.stream(...)``.
Examples:
```python
import xnano
from xnano.ai.models.text import TextModel, stream_into
from xnano.ai.testing import scripted_model
writer = TextModel(scripted_model('Thunder rolls.').model)
with xnano.new(width=20, height=1) as canvas:
label = canvas.screen.write('')
assert stream_into(label, writer.stream('Narrate.')) == 'Thunder rolls.'
assert canvas.screen.render() == 'Thunder rolls.'
```
"""
_check_debounce(debounce_seconds)
text: list[str] = []
shown = ''
last = 0.0
try:
for piece in pieces:
text.append(piece)
now = time.monotonic()
if now - last >= debounce_seconds:
shown = ''.join(text)
placement.text = shown
last = now
finally:
whole = ''.join(text)
if whole != shown and not placement._removed:
placement.text = whole
return whole