Hierarchy

  • EvaluatorType

Properties

criteria: LLMStringEvaluator<EvalOutputType, BaseLanguageModel<any, BaseLanguageModelCallOptions>>

The criteria evaluator, which evaluates a model based on a custom set of criteria without any reference labels.

embedding_distance: StringEvaluator

Compare a prediction to a reference label using embedding distance.

labeled_criteria: LLMStringEvaluator<EvalOutputType, BaseLanguageModel<any, BaseLanguageModelCallOptions>>

The labeled criteria evaluator, which evaluates a model based on a custom set of criteria, with a reference label.

labeled_pairwise_string: LLMPairwiseStringEvaluator

The labeled pairwise string evaluator, which predicts the preferred prediction from between two models based on a ground truth reference label.

pairwise_embedding_distance: PairwiseStringEvaluator

Compare two predictions using embedding distance.

pairwise_string: LLMPairwiseStringEvaluator

The pairwise string evaluator, which predicts the preferred prediction from between two models.

trajectory: AgentTrajectoryEvaluator

The agent trajectory evaluator, which grades the agent's intermediate steps.

Generated using TypeDoc