Skip to content

xnano.ai.models.decisions

xnano.ai.models.decisions

xnano.ai.models.decisions

Picking one of a few options, and answering yes/no questions and rubrics, about evidence.

Classes:

Attributes:

  • Options –

    The options of a decision: labels, a mapping from label to what it means, or a mapping from

NO_CHANGE module-attribute

NO_CHANGE: Final = 'no_change'

The label of the option that keeps things as they are.

MOST_OPTIONS module-attribute

MOST_OPTIONS: Final = 255

The most options one decision can offer, which is what decision backends accept.

Options module-attribute

Options = Sequence[str] | Mapping[str, 'str | Choice[Any]']

The options of a decision: labels, a mapping from label to what it means, or a mapping from label to a pydantic-ai Choice whose value is what the label stands for. A callable value is called when its option is picked, and the decision's value is what it returned.

Decision dataclass

Decision(choice: str, value: object, probabilities: dict[str, float] | None = None, confidence: float | None = None)

The option a ChoiceModel picked, and how sure it was.

Attributes:

  • choice (str) –

    The label that was picked, such as 'left', or 'no_change'.

  • value (object) –

    What the label stands for: the Choice value, what a callable value returned, or the label itself.

  • probabilities (dict[str, float] | None) –

    The probability of every option on a decision model, in the order the options were offered, or None on a language model.

  • confidence (float | None) –

    The backend's own confidence in the pick, or None. It is not how often the pick is right; threshold on margin instead.

Examples:

import random

from xnano.ai.models.decisions import Decision

decision = Decision(
    choice='left', value='left', probabilities={'left': 0.7, 'right': 0.3}
)
assert round(decision.margin, 2) == 0.4
assert decision.sample(random.Random(3)) in ('left', 'right')

Methods:

  • sample –

    Draw an option from the distribution instead of always taking the likeliest.

margin property

margin: float | None

How far the pick is ahead: its probability minus the next one's, or None.

A margin near 0 means two options were nearly tied. Below the margin an action is worth taking for, keep things as they are.

sample

sample(source: Random | None = None, *, temperature: float = 1.0, floor: float = 0.0) -> str

Draw an option from the distribution instead of always taking the likeliest.

Use it for variety that still follows what the model thought likely: a character that mostly, but not always, does the expected thing.

Parameters:

  • source (Random | None, default: None ) –

    Where the random numbers come from, such as random.Random(7) for the same draws on every run; Python's shared generator when None.

  • temperature (float, default: 1.0 ) –

    Above 1 the options are drawn more evenly, below 1 the likeliest is drawn more often; it must be above 0.

  • floor (float, default: 0.0 ) –

    Options less likely than this are never drawn.

Returns:

  • str –

    The label drawn, or choice on a language model, which gives no distribution.

Raises:

Source code in xnano/ai/models/decisions.py
def sample(
    self,
    source: random.Random | None = None,
    *,
    temperature: float = 1.0,
    floor: float = 0.0,
) -> str:
    """
    Draw an option from the distribution instead of always taking the likeliest.

    Use it for variety that still follows what the model thought likely: a character that
    mostly, but not always, does the expected thing.

    Args:
        source: Where the random numbers come from, such as ``random.Random(7)`` for the
            same draws on every run; Python's shared generator when ``None``.
        temperature: Above ``1`` the options are drawn more evenly, below ``1`` the
            likeliest is drawn more often; it must be above ``0``.
        floor: Options less likely than this are never drawn.

    Returns:
        The label drawn, or ``choice`` on a language model, which gives no distribution.

    Raises:
        ValueError: ``temperature`` is not above ``0``.
    """
    if not temperature > 0:
        raise ValueError(
            f'temperature must be above 0, but was {temperature!r}. Use 1.0 to draw as '
            f'the model gave the odds, more to draw more evenly, less to favour the '
            f'likeliest option.'
        )
    probabilities = self.probabilities
    if not probabilities:
        return self.choice
    kept = {
        label: probability ** (1.0 / temperature)
        for label, probability in probabilities.items()
        if probability >= floor and probability > 0
    }
    if not kept:
        return self.choice
    draw = (source or random).random() * sum(kept.values())
    for label, weight in kept.items():
        draw -= weight
        if draw <= 0:
            return label
    return next(reversed(kept))

Verdict dataclass

Verdict(question: str, answer: bool, probability: float | None = None, threshold: float = 0.5)

The answer of a JudgeModel to a yes/no question.

Attributes:

  • question (str) –

    The question that was asked.

  • answer (bool) –

    Whether the answer is yes: on a decision model, whether probability is at least threshold.

  • probability (float | None) –

    How likely a yes is on a decision model, or None on a language model.

  • threshold (float) –

    The probability a yes needed.

Score dataclass

Score(level: int, label: str, probabilities: dict[str, float] | None = None, expected: float | None = None)

The level of a rubric a JudgeModel placed the evidence at.

Attributes:

  • level (int) –

    The position of the level in the rubric, from 0.

  • label (str) –

    The level as it was written in the rubric.

  • probabilities (dict[str, float] | None) –

    The probability of each level, by its label, on a decision model, or None on a language model.

  • expected (float | None) –

    The probability-weighted mean level on a decision model, between two levels when the model hesitated, or None.

ChoiceModel

ChoiceModel(*arguments: Any, **keywords: Any)

Bases: _AskingModel

Pick one of a few options for what is on the canvas.

On a decision model ('typesafe:jev-1.14.0') the answer comes with the probability of every option, so a program can act only when the pick is clear. A named no_change option is always offered first, because "keep things as they are" is a better answer than a forced guess. Identical evidence and options are answered from the saved answer, with no request.

Examples:

from xnano.ai.models.decisions import ChoiceModel
from xnano.ai.testing import ScriptedDecisionModel

pick = ChoiceModel(ScriptedDecisionModel({'response': {'left': 0.8, 'right': 0.2}}))
decision = pick.choose(
    {'hero': 'at the right wall'},
    {'left': 'the hero should step left', 'right': 'the hero should step right'},
    question='Where should the hero go?',
)
assert decision.choice == 'left' and decision.margin > 0.5

Methods:

  • choose_async –

    Pick one of the options for the evidence, with await.

  • choose –

    Pick one of the options for the evidence, waiting for the answer.

  • override –

    Ask another model inside a with block: a test double, or a cheaper draft model.

  • details –

    What the backend reported about its last answer, as pydantic-ai's provider_details.

  • probabilities –

    The distribution a decision model gave a pick-one or rubric field in its last answer.

  • confidence –

    The confidence a decision model reported for a field of its last answer, or None.

Attributes:

Source code in xnano/ai/models/decisions.py
def __init__(self, *arguments: Any, **keywords: Any) -> None:
    super().__init__(*arguments, **keywords)
    self._is_reusing_answers = keywords.get('cache', True)
    self._saved = _SavedAnswers()

is_decision_model property

is_decision_model: bool

Whether model is a decision model, which answers with probabilities.

choose_async

choose_async(evidence: object, options: Options, *, question: str | None = None, no_change: str | None = 'nothing needs to change now') -> Coroutine[Any, Any, Decision]

Pick one of the options for the evidence, with await.

The evidence is read when this is called, so the model judges the canvas as it was then, and the options are checked then too; await the result for the decision.

Parameters:

  • evidence (object) –

    What to decide about: a context, canvas or placement, or a mapping of them with the facts your program knows; see Evidence.

  • options (Options) –

    The labels to pick from, with what each means; see Options.

  • question (str | None, default: None ) –

    What is being decided, such as 'What should the hero do now?'.

  • no_change (str | None, default: 'nothing needs to change now' ) –

    What the no_change option means, or None to offer only your options.

Returns:

Raises:

  • ValueError –

    No option was given besides no_change, or more than 255.

  • TerminalError –

    The model could not be reached (ai.model), when awaited.

Source code in xnano/ai/models/decisions.py
def choose_async(
    self,
    evidence: object,
    options: Options,
    *,
    question: str | None = None,
    no_change: str | None = 'nothing needs to change now',
) -> Coroutine[Any, Any, Decision]:
    """
    Pick one of the options for the evidence, with ``await``.

    The evidence is read when this is called, so the model judges the canvas as it was
    then, and the options are checked then too; await the result for the decision.

    Args:
        evidence: What to decide about: a context, canvas or placement, or a mapping of
            them with the facts your program knows; see [`Evidence`][xnano.ai.Evidence].
        options: The labels to pick from, with what each means; see
            [`Options`][xnano.ai.models.decisions.Options].
        question: What is being decided, such as ``'What should the hero do now?'``.
        no_change: What the ``no_change`` option means, or ``None`` to offer only your
            options.

    Returns:
        A coroutine that gives the [`Decision`][xnano.ai.models.decisions.Decision].

    Raises:
        ValueError: No option was given besides ``no_change``, or more than 255.
        TerminalError: The model could not be reached (``ai.model``), when awaited.
    """
    offered = _offered(options, no_change)
    return self._choose(snapshot(evidence), offered, question)

choose

choose(evidence: object, options: Options, *, question: str | None = None, no_change: str | None = 'nothing needs to change now') -> Decision

Pick one of the options for the evidence, waiting for the answer.

The same as choose_async, for code that does not await. A callable value of the option picked is called on the calling thread, so it may change the canvas.

Source code in xnano/ai/models/decisions.py
def choose(
    self,
    evidence: object,
    options: Options,
    *,
    question: str | None = None,
    no_change: str | None = 'nothing needs to change now',
) -> Decision:
    """
    Pick one of the options for the evidence, waiting for the answer.

    The same as [`choose_async`][xnano.ai.models.decisions.ChoiceModel.choose_async], for code that does not
    await. A callable value of the option picked is called on the calling thread, so it
    may change the canvas.
    """
    offered = _offered(options, no_change)
    label, probabilities, confidence = _run_blocking(
        self._pick(snapshot(evidence), offered, question), 'choose'
    )
    value = _resolve_blocking(offered[label][1], label, 'choose')
    return Decision(
        choice=label, value=value, probabilities=probabilities, confidence=confidence
    )

override

override(*, model: Model | KnownModelName | str) -> Generator[None, None, None]

Ask another model inside a with block: a test double, or a cheaper draft model.

Parameters:

  • model (Model | KnownModelName | str) –

    The model to ask instead.

Examples:

from xnano.ai.models.decisions import ChoiceModel
from xnano.ai.testing import ScriptedDecisionModel

pick = ChoiceModel('typesafe:jev-1.14.0')
with pick.override(model=ScriptedDecisionModel({'response': 'left'})):
    assert pick.choose('the hero', ['left', 'right']).choice == 'left'
Source code in xnano/ai/models/__init__.py
@contextlib.contextmanager
def override(
    self, *, model: Model | KnownModelName | str
) -> Generator[None, None, None]:
    """
    Ask another model inside a ``with`` block: a test double, or a cheaper draft model.

    Args:
        model: The model to ask instead.

    Examples:
        ```python
        from xnano.ai.models.decisions import ChoiceModel
        from xnano.ai.testing import ScriptedDecisionModel

        pick = ChoiceModel('typesafe:jev-1.14.0')
        with pick.override(model=ScriptedDecisionModel({'response': 'left'})):
            assert pick.choose('the hero', ['left', 'right']).choice == 'left'
        ```
    """
    previous = self.model
    self.model = model
    try:
        with self.agent.override(model=model):
            yield
    finally:
        self.model = previous

details

details() -> dict[str, Any]

What the backend reported about its last answer, as pydantic-ai's provider_details.

Empty before the first call, and for a backend that reports nothing.

Source code in xnano/ai/models/__init__.py
def details(self) -> dict[str, Any]:
    """
    What the backend reported about its last answer, as pydantic-ai's ``provider_details``.

    Empty before the first call, and for a backend that reports nothing.
    """
    response = self._last_response()
    if response is None or response.provider_details is None:
        return {}
    return dict(response.provider_details)

probabilities

probabilities(field: str) -> dict[str, float] | None

The distribution a decision model gave a pick-one or rubric field in its last answer.

Parameters:

  • field (str) –

    The name of the field, such as 'response' for a single pick-one.

Returns:

  • dict[str, float] | None –

    The probability of each option, or None on a language model or for a field it

  • dict[str, float] | None –

    did not report.

Source code in xnano/ai/models/__init__.py
def probabilities(self, field: str) -> dict[str, float] | None:
    """
    The distribution a decision model gave a pick-one or rubric field in its last answer.

    Args:
        field: The name of the field, such as ``'response'`` for a single pick-one.

    Returns:
        The probability of each option, or ``None`` on a language model or for a field it
        did not report.
    """
    distribution = self.details().get('probabilities', {}).get(field)
    if distribution is None:
        return None
    return {str(option): float(value) for option, value in distribution.items()}

confidence

confidence(field: str) -> float | None

The confidence a decision model reported for a field of its last answer, or None.

The confidence is the backend's own figure; it is not how often the answer is right.

Source code in xnano/ai/models/__init__.py
def confidence(self, field: str) -> float | None:
    """
    The confidence a decision model reported for a field of its last answer, or ``None``.

    The confidence is the backend's own figure; it is not how often the answer is right.
    """
    value = self.details().get('confidence', {}).get(field)
    return None if value is None else float(value)

JudgeModel

JudgeModel(*arguments: Any, **keywords: Any)

Bases: _AskingModel

Answer yes/no questions, and place evidence on a rubric.

On a decision model each answer comes with the probability of a yes, and threshold sets how sure a yes has to be. Ask one question per thing you want to know, with "yes" meaning that it is true; several questions about the same evidence go in one request with ask_many.

Examples:

from xnano.ai.models.decisions import JudgeModel
from xnano.ai.testing import ScriptedDecisionModel

judge = JudgeModel(ScriptedDecisionModel({'answer': 0.9}))
verdict = judge.ask('Ada typed: "Ada"', 'Has the user finished typing their name?')
assert verdict.answer and verdict.probability > 0.8

Methods:

  • ask_async –

    Answer a yes/no question about the evidence, with await.

  • ask –

    Answer a yes/no question about the evidence, waiting for the answer.

  • ask_many_async –

    Answer several yes/no questions about the same evidence in one request, with await.

  • ask_many –

    Answer several yes/no questions about the same evidence in one request, waiting.

  • score_async –

    Place the evidence on a rubric of levels, with await.

  • score –

    Place the evidence on a rubric of levels, waiting for the answer.

  • override –

    Ask another model inside a with block: a test double, or a cheaper draft model.

  • details –

    What the backend reported about its last answer, as pydantic-ai's provider_details.

  • probabilities –

    The distribution a decision model gave a pick-one or rubric field in its last answer.

  • confidence –

    The confidence a decision model reported for a field of its last answer, or None.

Attributes:

Source code in xnano/ai/models/decisions.py
def __init__(self, *arguments: Any, **keywords: Any) -> None:
    super().__init__(*arguments, **keywords)
    self._is_reusing_answers = keywords.get('cache', True)
    self._saved = _SavedAnswers()

is_decision_model property

is_decision_model: bool

Whether model is a decision model, which answers with probabilities.

ask_async

ask_async(evidence: object, question: str, *, threshold: float = 0.5, yes: str | None = None, no: str | None = None) -> Coroutine[Any, Any, Verdict]

Answer a yes/no question about the evidence, with await.

The evidence is read when this is called; await the result for the verdict.

Parameters:

  • evidence (object) –

    What the question is about; see Evidence.

  • question (str) –

    The question, such as 'Is the board full?'.

  • threshold (float, default: 0.5 ) –

    How likely a yes has to be to count, on a decision model.

  • yes (str | None, default: None ) –

    What a yes means, when the question alone leaves it open.

  • no (str | None, default: None ) –

    What a no means.

Returns:

Raises:

  • ValueError –

    threshold is not from 0 to 1.

  • TerminalError –

    The model could not be reached (ai.model), when awaited.

Source code in xnano/ai/models/decisions.py
def ask_async(
    self,
    evidence: object,
    question: str,
    *,
    threshold: float = 0.5,
    yes: str | None = None,
    no: str | None = None,
) -> Coroutine[Any, Any, Verdict]:
    """
    Answer a yes/no question about the evidence, with ``await``.

    The evidence is read when this is called; await the result for the verdict.

    Args:
        evidence: What the question is about; see [`Evidence`][xnano.ai.Evidence].
        question: The question, such as ``'Is the board full?'``.
        threshold: How likely a yes has to be to count, on a decision model.
        yes: What a yes means, when the question alone leaves it open.
        no: What a no means.

    Returns:
        A coroutine that gives the [`Verdict`][xnano.ai.models.decisions.Verdict].

    Raises:
        ValueError: ``threshold`` is not from 0 to 1.
        TerminalError: The model could not be reached (``ai.model``), when awaited.
    """
    _check_threshold(threshold)
    return self._ask_one(snapshot(evidence), (question, yes, no), threshold)

ask

ask(evidence: object, question: str, *, threshold: float = 0.5, yes: str | None = None, no: str | None = None) -> Verdict

Answer a yes/no question about the evidence, waiting for the answer.

The same as ask_async, for code that does not await.

Source code in xnano/ai/models/decisions.py
def ask(
    self,
    evidence: object,
    question: str,
    *,
    threshold: float = 0.5,
    yes: str | None = None,
    no: str | None = None,
) -> Verdict:
    """
    Answer a yes/no question about the evidence, waiting for the answer.

    The same as [`ask_async`][xnano.ai.models.decisions.JudgeModel.ask_async], for code that does not await.
    """
    return _run_blocking(
        self.ask_async(evidence, question, threshold=threshold, yes=yes, no=no), 'ask'
    )

ask_many_async

ask_many_async(evidence: object, questions: Sequence[str] | Mapping[str, str], *, threshold: float = 0.5) -> Coroutine[Any, Any, dict[str, Verdict]]

Answer several yes/no questions about the same evidence in one request, with await.

The evidence is read when this is called; await the result for the verdicts.

Parameters:

  • evidence (object) –

    What the questions are about; see Evidence.

  • questions (Sequence[str] | Mapping[str, str]) –

    The questions, or a mapping from a name of your own to each question.

  • threshold (float, default: 0.5 ) –

    How likely a yes has to be to count, on a decision model.

Returns:

Raises:

  • ValueError –

    No question was given, a name is not a Python identifier, or threshold is not from 0 to 1.

  • TerminalError –

    The model could not be reached (ai.model).

Source code in xnano/ai/models/decisions.py
def ask_many_async(
    self,
    evidence: object,
    questions: Sequence[str] | Mapping[str, str],
    *,
    threshold: float = 0.5,
) -> Coroutine[Any, Any, dict[str, Verdict]]:
    """
    Answer several yes/no questions about the same evidence in one request, with ``await``.

    The evidence is read when this is called; await the result for the verdicts.

    Args:
        evidence: What the questions are about; see [`Evidence`][xnano.ai.Evidence].
        questions: The questions, or a mapping from a name of your own to each question.
        threshold: How likely a yes has to be to count, on a decision model.

    Returns:
        A coroutine that gives each [`Verdict`][xnano.ai.models.decisions.Verdict], by its question, or by
        its name when a mapping was given.

    Raises:
        ValueError: No question was given, a name is not a Python identifier, or
            ``threshold`` is not from 0 to 1.
        TerminalError: The model could not be reached (``ai.model``).
    """
    _check_threshold(threshold)
    if isinstance(questions, str):
        raise TypeError(
            f'ask_many() takes a list of questions, but got the single string '
            f'{questions!r}. Use ask() for one question.'
        )
    if isinstance(questions, Mapping):
        named = {str(name): question for name, question in questions.items()}
        for name in named:
            if not name.isidentifier():
                raise ValueError(
                    f'The name {name!r} of a question must be a Python identifier, such '
                    f"as 'is_full', because it names the answer."
                )
    else:
        named = {
            f'question_{index}': question
            for index, question in enumerate(questions, 1)
        }
    if not named:
        raise ValueError('ask_many() needs at least one question.')
    return self._ask_several(
        snapshot(evidence), named, threshold, is_named=isinstance(questions, Mapping)
    )

ask_many

ask_many(evidence: object, questions: Sequence[str] | Mapping[str, str], *, threshold: float = 0.5) -> dict[str, Verdict]

Answer several yes/no questions about the same evidence in one request, waiting.

The same as ask_many_async, for code that does not await.

Source code in xnano/ai/models/decisions.py
def ask_many(
    self,
    evidence: object,
    questions: Sequence[str] | Mapping[str, str],
    *,
    threshold: float = 0.5,
) -> dict[str, Verdict]:
    """
    Answer several yes/no questions about the same evidence in one request, waiting.

    The same as [`ask_many_async`][xnano.ai.models.decisions.JudgeModel.ask_many_async], for code that does
    not await.
    """
    return _run_blocking(
        self.ask_many_async(evidence, questions, threshold=threshold),
        'ask_many',
    )

score_async

score_async(evidence: object, levels: Sequence[str], *, question: str | None = None) -> Coroutine[Any, Any, Score]

Place the evidence on a rubric of levels, with await.

The evidence is read when this is called; await the result for the score.

Describe each level as a situation ('two or three items on the screen'), not as a degree ('somewhat busy'). A rubric is the least reliable kind of answer on current decision models: for anything that matters, ask a yes/no or pick one option instead.

Parameters:

  • evidence (object) –

    What to score; see Evidence.

  • levels (Sequence[str]) –

    The levels in order, from lowest to highest; at least two.

  • question (str | None, default: None ) –

    What the rubric measures, such as 'How busy is the screen?'.

Returns:

Raises:

  • ValueError –

    Fewer than two levels were given.

  • TerminalError –

    The model could not be reached (ai.model), when awaited.

Source code in xnano/ai/models/decisions.py
def score_async(
    self, evidence: object, levels: Sequence[str], *, question: str | None = None
) -> Coroutine[Any, Any, Score]:
    """
    Place the evidence on a rubric of levels, with ``await``.

    The evidence is read when this is called; await the result for the score.

    Describe each level as a situation (``'two or three items on the screen'``), not as a
    degree (``'somewhat busy'``). A rubric is the least reliable kind of answer on current
    decision models: for anything that matters, ask a yes/no or pick one option instead.

    Args:
        evidence: What to score; see [`Evidence`][xnano.ai.Evidence].
        levels: The levels in order, from lowest to highest; at least two.
        question: What the rubric measures, such as ``'How busy is the screen?'``.

    Returns:
        A coroutine that gives the [`Score`][xnano.ai.models.decisions.Score].

    Raises:
        ValueError: Fewer than two levels were given.
        TerminalError: The model could not be reached (``ai.model``), when awaited.
    """
    if isinstance(levels, str) or len(levels) < 2:
        raise ValueError(
            f'score() takes at least two levels, from lowest to highest, such as '
            f"['empty', 'sparse', 'busy'], but got {levels!r}."
        )
    labels = [str(level) for level in levels]
    return self._score(
        snapshot(evidence), labels, question or 'Which level describes the evidence?'
    )

score

score(evidence: object, levels: Sequence[str], *, question: str | None = None) -> Score

Place the evidence on a rubric of levels, waiting for the answer.

The same as score_async, for code that does not await.

Source code in xnano/ai/models/decisions.py
def score(
    self, evidence: object, levels: Sequence[str], *, question: str | None = None
) -> Score:
    """
    Place the evidence on a rubric of levels, waiting for the answer.

    The same as [`score_async`][xnano.ai.models.decisions.JudgeModel.score_async], for code that does not
    await.
    """
    return _run_blocking(
        self.score_async(evidence, levels, question=question), 'score'
    )

override

override(*, model: Model | KnownModelName | str) -> Generator[None, None, None]

Ask another model inside a with block: a test double, or a cheaper draft model.

Parameters:

  • model (Model | KnownModelName | str) –

    The model to ask instead.

Examples:

from xnano.ai.models.decisions import ChoiceModel
from xnano.ai.testing import ScriptedDecisionModel

pick = ChoiceModel('typesafe:jev-1.14.0')
with pick.override(model=ScriptedDecisionModel({'response': 'left'})):
    assert pick.choose('the hero', ['left', 'right']).choice == 'left'
Source code in xnano/ai/models/__init__.py
@contextlib.contextmanager
def override(
    self, *, model: Model | KnownModelName | str
) -> Generator[None, None, None]:
    """
    Ask another model inside a ``with`` block: a test double, or a cheaper draft model.

    Args:
        model: The model to ask instead.

    Examples:
        ```python
        from xnano.ai.models.decisions import ChoiceModel
        from xnano.ai.testing import ScriptedDecisionModel

        pick = ChoiceModel('typesafe:jev-1.14.0')
        with pick.override(model=ScriptedDecisionModel({'response': 'left'})):
            assert pick.choose('the hero', ['left', 'right']).choice == 'left'
        ```
    """
    previous = self.model
    self.model = model
    try:
        with self.agent.override(model=model):
            yield
    finally:
        self.model = previous

details

details() -> dict[str, Any]

What the backend reported about its last answer, as pydantic-ai's provider_details.

Empty before the first call, and for a backend that reports nothing.

Source code in xnano/ai/models/__init__.py
def details(self) -> dict[str, Any]:
    """
    What the backend reported about its last answer, as pydantic-ai's ``provider_details``.

    Empty before the first call, and for a backend that reports nothing.
    """
    response = self._last_response()
    if response is None or response.provider_details is None:
        return {}
    return dict(response.provider_details)

probabilities

probabilities(field: str) -> dict[str, float] | None

The distribution a decision model gave a pick-one or rubric field in its last answer.

Parameters:

  • field (str) –

    The name of the field, such as 'response' for a single pick-one.

Returns:

  • dict[str, float] | None –

    The probability of each option, or None on a language model or for a field it

  • dict[str, float] | None –

    did not report.

Source code in xnano/ai/models/__init__.py
def probabilities(self, field: str) -> dict[str, float] | None:
    """
    The distribution a decision model gave a pick-one or rubric field in its last answer.

    Args:
        field: The name of the field, such as ``'response'`` for a single pick-one.

    Returns:
        The probability of each option, or ``None`` on a language model or for a field it
        did not report.
    """
    distribution = self.details().get('probabilities', {}).get(field)
    if distribution is None:
        return None
    return {str(option): float(value) for option, value in distribution.items()}

confidence

confidence(field: str) -> float | None

The confidence a decision model reported for a field of its last answer, or None.

The confidence is the backend's own figure; it is not how often the answer is right.

Source code in xnano/ai/models/__init__.py
def confidence(self, field: str) -> float | None:
    """
    The confidence a decision model reported for a field of its last answer, or ``None``.

    The confidence is the backend's own figure; it is not how often the answer is right.
    """
    value = self.details().get('confidence', {}).get(field)
    return None if value is None else float(value)