trulens.core.instruments¶
trulens.core.instruments
¶
Instrumentation
This module contains the core of the app instrumentation scheme employed by trulens to track and record apps. These details should not be relevant for typical use cases.
Attributes¶
Classes¶
WithInstrumentCallbacks
¶
Abstract definition of callbacks invoked by Instrument during instrumentation or when instrumented methods are called.
Needs to be mixed into App.
Attributes¶
_context_contexts
class-attribute
instance-attribute
¶
_context_contexts = ContextVar(
"context_contexts", default=set()
)
ContextVars for storing collections of RecordingContext .
_stack_contexts
class-attribute
instance-attribute
¶
_stack_contexts = ContextVar('stack_contexts', default={})
ContextVars for storing call stacks.
Functions¶
on_method_instrumented
¶
Callback to be called by instrumentation system for every function requested to be instrumented.
Given are the object of the class in which func
belongs
(i.e. the "self" for that function), the func
itself, and the path
of the owner object in the app hierarchy.
PARAMETER | DESCRIPTION |
---|---|
obj
|
The object of the class in which
TYPE:
|
func
|
The function that was instrumented. Expects the unbound version (self not yet bound).
TYPE:
|
path
|
The path of the owner object in the app hierarchy.
TYPE:
|
get_method_path
¶
Get the path of the instrumented function func
, a member of the class
of obj
relative to this app.
PARAMETER | DESCRIPTION |
---|---|
obj
|
The object of the class in which
TYPE:
|
func
|
The function that was instrumented. Expects the unbound version (self not yet bound).
TYPE:
|
wrap_lazy_values
¶
wrap_lazy_values(
rets: Any,
wrap: Callable[[T], T],
on_done: Callable[[T], T],
context_vars: Optional[ContextVarsOrValues],
) -> Any
Wrap any lazy values in the return value of a method call to invoke handle_done when the value is ready.
This is used to handle library-specific lazy values that are hidden in containers not visible otherwise. Visible lazy values like iterators, generators, awaitables, and async generators are handled elsewhere.
PARAMETER | DESCRIPTION |
---|---|
rets
|
The return value of the method call.
TYPE:
|
wrap
|
A callback to be called when the lazy value is ready. Should return the input value or a wrapped version of it.
TYPE:
|
on_done
|
Called when the lazy values is done and is no longer lazy. This as opposed to a lazy value that evaluates to another lazy values. Should return the value or wrapper.
TYPE:
|
context_vars
|
The contextvars to be captured by the lazy value. If not given, all contexts are captured.
TYPE:
|
RETURNS | DESCRIPTION |
---|---|
Any
|
The return value with lazy values wrapped. |
get_methods_for_func
¶
Get the methods (rather the inner functions) matching the given func
and the path of each.
PARAMETER | DESCRIPTION |
---|---|
func
|
The function to match.
TYPE:
|
on_new_record
¶
on_new_record(func: Callable)
Called by instrumented methods in cases where they cannot find a record call list in the stack. If we are inside a context manager, return a new call list.
on_add_record
¶
on_add_record(
ctx: _RecordingContext,
func: Callable,
sig: Signature,
bindings: BoundArguments,
ret: Any,
error: Any,
perf: Perf,
cost: Cost,
existing_record: Optional[Record] = None,
final: bool = True,
)
Called by instrumented methods if they are root calls (first instrumented methods in a call stack).
PARAMETER | DESCRIPTION |
---|---|
ctx
|
The context of the recording.
TYPE:
|
func
|
The function that was called.
TYPE:
|
sig
|
The signature of the function.
TYPE:
|
bindings
|
The bound arguments of the function.
TYPE:
|
ret
|
The return value of the function.
TYPE:
|
error
|
The error raised by the function if any.
TYPE:
|
perf
|
The performance of the function.
TYPE:
|
cost
|
The cost of the function.
TYPE:
|
existing_record
|
If the record has already been produced (i.e. because it was an awaitable), it can be passed here to avoid re-creating it. |
final
|
Whether this is record is final in that it is ready for feedback evaluation.
TYPE:
|
_RecordingContext
¶
Manager of the creation of records from record calls.
An instance of this class is produced when using an App as a context manager, i.e.:
Example
app = ... # your app
truapp: TruChain = TruChain(app, ...) # recorder for LangChain apps
with truapp as recorder:
app.invoke(...) # use your app
recorder: RecordingContext
Each instance of this class produces a record for every "root" instrumented method called. Root method here means the first instrumented method in a call stack. Note that there may be more than one of these contexts in play at the same time due to:
- More than one wrapper of the same app.
- More than one context manager ("with" statement) surrounding calls to the same app.
- Calls to "with_record" on methods that themselves contain recording.
- Calls to apps that use trulens internally to track records in any of the supported ways.
- Combinations of the above.
Attributes¶
calls
instance-attribute
¶
calls: Dict[CallID, RecordAppCall] = {}
A record (in terms of its RecordAppCall) in process of being created.
Storing as a map as we want to override calls with the same id which may happen due to methods producing awaitables or generators. These result in calls before the awaitables are awaited and then get updated after the result is ready.
lock
instance-attribute
¶
Lock blocking access to calls
and records
when adding calls or
finishing a record.
app
instance-attribute
¶
app: ProxyType[WithInstrumentCallbacks] = proxy(app)
App for which we are recording.
record_metadata
instance-attribute
¶
record_metadata = record_metadata
Metadata to attach to all records produced in this context.
Functions¶
get
¶
get() -> Record
Get the single record only if there was exactly one. Otherwise throw an error.
finish_record
¶
finish_record(
calls_to_record: Callable[
[List[RecordAppCall], Metadata, Optional[Record]],
Record,
],
existing_record: Optional[Record] = None,
)
Run the given function to build a record from the tracked calls and any pre-specified metadata.
If existing_record is provided, updates that record with new data.
Instrument
¶
Instrumentation tools.
Attributes¶
INSTRUMENT
class-attribute
instance-attribute
¶
INSTRUMENT = '__tru_instrumented'
Attribute name to be used to flag instrumented objects/methods/others.
APPS
class-attribute
instance-attribute
¶
APPS = '__tru_apps'
Attribute name for storing apps that expect to be notified of calls.
Classes¶
Default
¶
Default instrumentation configuration.
Additional components are included in subclasses of Instrument.
Functions¶
print_instrumentation
¶
print_instrumentation() -> None
Print out description of the modules, classes, methods this class will instrument.
to_instrument_object
¶
Determine whether the given object should be instrumented.
to_instrument_class
¶
Determine whether the given class should be instrumented.
to_instrument_module
¶
Determine whether a module with the given (full) name should be instrumented.
_have_context
staticmethod
¶
_have_context() -> bool
Determine whether context vars we need for recording are available.
tracked_method_wrapper
¶
tracked_method_wrapper(
query: Lens,
func: Callable,
method_name: str,
cls: type,
obj: object,
span_type: Optional[SpanType] = None,
attributes: Optional[Attributes] = None,
must_be_first_wrapper: bool = False,
)
Wrap a method to capture its inputs/outputs/errors.
instrument_class
¶
instrument_class(cls)
Instrument the given class cls
's new method.
This is done so we can be aware when new instances are created and is needed for wrapped methods that dynamically create instances of classes we wish to instrument. As they will not be visible at the time we wrap the app, we need to pay attention to new to make a note of them when they are created and the creator's path. This path will be used to place these new instances in the app json structure.
AddInstruments
¶
instrument
¶
Bases: AddInstruments
Decorator for marking methods to be instrumented in custom classes that are wrapped by App.
Functions¶
class_filter_disjunction
¶
class_filter_disjunction(
f1: ClassFilter, f2: ClassFilter
) -> ClassFilter
Create a disjunction of two class filters.
PARAMETER | DESCRIPTION |
---|---|
f1
|
The first filter.
TYPE:
|
f2
|
The second filter.
TYPE:
|
class_filter_matches
¶
Check whether given object matches a class-based filter.
A class-based filter here means either a type to match against object (isinstance if object is not a type or issubclass if object is a type), or a tuple of types to match against interpreted disjunctively.
PARAMETER | DESCRIPTION |
---|---|
f
|
The filter to match against.
TYPE:
|
obj
|
The object to match against. If type, uses |