trulens.feedback.generated

trulens.feedback.generated

Utilities for dealing with LLM-generated text.

Attributes

PATTERN_0_10 module-attribute

PATTERN_0_10: Pattern = compile('([0-9]+)(?=\\D*$)')

Regex that matches the last integer.

PATTERN_NUMBER module-attribute

PATTERN_NUMBER: Pattern = compile(
    "([+-]?[0-9]+\\.[0-9]*|[1-9][0-9]*|0)"
)

Regex that matches floating point and integer numbers.

PATTERN_INTEGER module-attribute

PATTERN_INTEGER: Pattern = compile('([+-]?[1-9][0-9]*|0)')

Regex that matches integers.

Classes

ParseError

Bases: Exception

Error parsing LLM-generated text.

Functions

re_configured_rating

re_configured_rating(
    s: str,
    min_score_val: int = 0,
    max_score_val: int = 3,
    allow_decimal: bool = False,
) -> int

Extract a {min_score_val}-{max_score_val} rating from a string. Configurable to the ranges like 4-point Likert scale or binary (0 or 1).

If the string does not match an integer/a float or matches an integer/a float outside the {min_score_val} - {max_score_val} range, raises an error instead. If multiple numbers are found within the expected 0-10 range, the smallest is returned.

PARAMETER	DESCRIPTION
s	String to extract rating from. TYPE: str
min_score_val	Minimum value of the rating scale. TYPE: int DEFAULT: 0
max_score_val	Maximum value of the rating scale. TYPE: int DEFAULT: 3
allow_decimal	Whether to allow and capture decimal numbers (floats). TYPE: bool DEFAULT: False

RETURNS	DESCRIPTION
int	Extracted rating. TYPE: int

RAISES	DESCRIPTION
ParseError	If no integers/floats between 0 and 10 are found in the string.

re_0_10_rating

re_0_10_rating(s: str) -> int

Extract a 0-10 rating from a string.

If the string does not match an integer/a float or matches an integer/a float outside the 0-10 range, raises an error instead. If multiple numbers are found within the expected 0-10 range, the smallest is returned.

PARAMETER	DESCRIPTION
s	String to extract rating from. TYPE: str

RETURNS	DESCRIPTION
int	Extracted rating. TYPE: int

RAISES	DESCRIPTION
ParseError	If no integers/floats between 0 and 10 are found in the string.