`@codesoul-co/hypha-testing` / `evaluation`

Package index: @codesoul-co/hypha-testing
Source: packages/testing/src/evaluation.ts
Exports: 15

Using this module

Use the Evaluation module for using the public contracts and operations for this capability boundary. It exports 3 classes, 1 function, 10 interfaces, 1 type.

Import from the package entrypoint

import {
  DeterministicEvaluator,
  OutputContractValidator,
  TraceCompletenessEvaluator,
  validateJsonSchemaValue,
} from '@codesoul-co/hypha-testing';

import type {
  DeterministicEvaluationInput,
  DeterministicEvaluatorOptions,
  EvaluationCheckResult,
  EvaluationEventContext,
  EvaluationResult,
  EvaluationSummary,
  JsonSchemaValidationIssue,
  OutputContractValidationInput,
} from '@codesoul-co/hypha-testing';

// The complete export list is documented below.

Usage patterns

Use the 11 type/interface exports as static contracts in application code, adapters, or tests. Import them with import type; they do not exist at runtime.
The module exposes 3 classes as constructable runtime implementations. Each symbol entry lists its constructor and public methods.
The module exposes 1 function as direct operation entrypoints. Every overload, required/optional parameter, and return type is documented below.

Public exports

Symbol	Kind	Signature	Description
`DeterministicEvaluator`	class	`new DeterministicEvaluator(options?: DeterministicEvaluatorOptions): DeterministicEvaluator`	Deterministic Evaluator class with 3 public constructor or member entries; its exact declarations are listed below.
`OutputContractValidator`	class	`new OutputContractValidator(options?: { now?: () => string; }): OutputContractValidator`	Output Contract Validator class with 2 public constructor or member entries; its exact declarations are listed below.
`TraceCompletenessEvaluator`	class	`new TraceCompletenessEvaluator(options?: TraceCompletenessEvaluatorOptions): TraceCompletenessEvaluator`	Trace Completeness Evaluator class with 2 public constructor or member entries; its exact declarations are listed below.
`validateJsonSchemaValue`	function	`validateJsonSchemaValue(value: unknown, schema: JsonSchema, path?: string): JsonSchemaValidationIssue[]`	Validate JSON Schema Value function with 1 public call signature; parameters and return types are listed below.
`DeterministicEvaluationInput`	interface	`interface DeterministicEvaluationInput`	Deterministic Evaluation Input interface with 7 public fields or methods.
`DeterministicEvaluatorOptions`	interface	`interface DeterministicEvaluatorOptions`	Deterministic Evaluator Options interface with 9 public fields or methods.
`EvaluationCheckResult`	interface	`interface EvaluationCheckResult`	Evaluation Check Result interface with 7 public fields or methods.
`EvaluationEventContext`	interface	`interface EvaluationEventContext`	Evaluation Event Context interface with 5 public fields or methods.
`EvaluationResult`	interface	`interface EvaluationResult`	Evaluation Result interface with 10 public fields or methods.
`EvaluationSummary`	interface	`interface EvaluationSummary`	Evaluation Summary interface with 7 public fields or methods.
`JsonSchemaValidationIssue`	interface	`interface JsonSchemaValidationIssue`	JSON Schema Validation Issue interface with 4 public fields or methods.
`OutputContractValidationInput`	interface	`interface OutputContractValidationInput`	Output Contract Validation Input interface with 5 public fields or methods.
`TraceCompletenessEvaluatorOptions`	interface	`interface TraceCompletenessEvaluatorOptions`	Trace Completeness Evaluator Options interface with 3 public fields or methods.
`TraceCompletenessInput`	interface	`interface TraceCompletenessInput`	Trace Completeness Input interface with 6 public fields or methods.
`EvaluationStatus`	type	`type EvaluationStatus = 'passed' \| 'failed'`	Public type alias for Evaluation Status; the declaration contains its complete type expression.

`DeterministicEvaluator`

Deterministic Evaluator class with 3 public constructor or member entries; its exact declarations are listed below.

Kind: class
Import: import { DeterministicEvaluator } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export declare class DeterministicEvaluator {
    constructor(options?: DeterministicEvaluatorOptions);
    evaluate(input: DeterministicEvaluationInput): EvaluationSummary;
    evaluateAndRecord(input: DeterministicEvaluationInput): Promise<EvaluationSummary>;
}

Public members

Member	Kind	Signature	Description
`constructor`	constructor	`(options?: DeterministicEvaluatorOptions): DeterministicEvaluator`	Creates an instance of this class.
`evaluate`	method	`evaluate(input: DeterministicEvaluationInput): EvaluationSummary`	Public method; parameters and return type are shown in the signature.
`evaluateAndRecord`	method	`evaluateAndRecord(input: DeterministicEvaluationInput): Promise<EvaluationSummary>`	Public method; parameters and return type are shown in the signature.

`OutputContractValidator`

Output Contract Validator class with 2 public constructor or member entries; its exact declarations are listed below.

Kind: class
Import: import { OutputContractValidator } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export declare class OutputContractValidator {
    constructor(options?: {
            now?: () => string;
        });
    validate(input: OutputContractValidationInput): EvaluationResult;
}

Public members

Member	Kind	Signature	Description
`constructor`	constructor	`(options?: { now?: () => string; }): OutputContractValidator`	Creates an instance of this class.
`validate`	method	`validate(input: OutputContractValidationInput): EvaluationResult`	Public method; parameters and return type are shown in the signature.

`TraceCompletenessEvaluator`

Trace Completeness Evaluator class with 2 public constructor or member entries; its exact declarations are listed below.

Kind: class
Import: import { TraceCompletenessEvaluator } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export declare class TraceCompletenessEvaluator {
    constructor(options?: TraceCompletenessEvaluatorOptions);
    evaluate(input: TraceCompletenessInput): EvaluationResult;
}

Public members

Member	Kind	Signature	Description
`constructor`	constructor	`(options?: TraceCompletenessEvaluatorOptions): TraceCompletenessEvaluator`	Creates an instance of this class.
`evaluate`	method	`evaluate(input: TraceCompletenessInput): EvaluationResult`	Public method; parameters and return type are shown in the signature.

`validateJsonSchemaValue`

Validate JSON Schema Value function with 1 public call signature; parameters and return types are listed below.

Kind: function
Import: import { validateJsonSchemaValue } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export declare function validateJsonSchemaValue(value: unknown, schema: JsonSchema, path?: string): JsonSchemaValidationIssue[];

Call signature

text

validateJsonSchemaValue(value: unknown, schema: JsonSchema, path?: string): JsonSchemaValidationIssue[]

Parameters

Parameter	Type	Required	Description
`value`	`unknown`	Yes	Required parameter; accepted values are defined by the type column.
`schema`	`JsonSchema`	Yes	Required parameter; accepted values are defined by the type column.
`path`	`string`	No	Optional parameter; accepted values are defined by the type column.

Returns

Type: JsonSchemaValidationIssue[]
Description: The return contract is defined by the type shown above.

`DeterministicEvaluationInput`

Deterministic Evaluation Input interface with 7 public fields or methods.

Kind: interface
Import: import type { DeterministicEvaluationInput } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export interface DeterministicEvaluationInput {
    runId?: string;
    events?: FrameworkEvent[];
    output?: unknown;
    outputContracts?: OutputContractSpec[];
    traceSpecs?: TraceSpec[];
    evaluationSpecs?: EvaluationSpec[];
    metadata?: Record<string, unknown>;
}

Contract members

Member	Kind	Signature	Description
`evaluationSpecs`	property	`evaluationSpecs?: EvaluationSpec[]`	Public property; its type, readonly modifier and optionality are shown in the signature.
`events`	property	`events?: FrameworkEvent<unknown>[]`	Public property; its type, readonly modifier and optionality are shown in the signature.
`metadata`	property	`metadata?: Record<string, unknown>`	Public property; its type, readonly modifier and optionality are shown in the signature.
`output`	property	`output?: unknown`	Public property; its type, readonly modifier and optionality are shown in the signature.
`outputContracts`	property	`outputContracts?: OutputContractSpec[]`	Public property; its type, readonly modifier and optionality are shown in the signature.
`runId`	property	`runId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`traceSpecs`	property	`traceSpecs?: TraceSpec[]`	Public property; its type, readonly modifier and optionality are shown in the signature.

`DeterministicEvaluatorOptions`

Deterministic Evaluator Options interface with 9 public fields or methods.

Kind: interface
Import: import type { DeterministicEvaluatorOptions } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export interface DeterministicEvaluatorOptions {
    now?: () => string;
    outputValidator?: OutputContractValidator;
    traceEvaluator?: TraceCompletenessEvaluator;
    trace?: TraceRecorder;
    eventRunId?: string;
    sessionId?: string;
    workspaceId?: string;
    agentId?: string;
    metadata?: Record<string, unknown>;
}

Contract members

Member	Kind	Signature	Description
`agentId`	property	`agentId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`eventRunId`	property	`eventRunId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`metadata`	property	`metadata?: Record<string, unknown>`	Public property; its type, readonly modifier and optionality are shown in the signature.
`now`	method	`now?(): string`	Public method; parameters and return type are shown in the signature.
`outputValidator`	property	`outputValidator?: OutputContractValidator`	Public property; its type, readonly modifier and optionality are shown in the signature.
`sessionId`	property	`sessionId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`trace`	property	`trace?: TraceRecorder`	Public property; its type, readonly modifier and optionality are shown in the signature.
`traceEvaluator`	property	`traceEvaluator?: TraceCompletenessEvaluator`	Public property; its type, readonly modifier and optionality are shown in the signature.
`workspaceId`	property	`workspaceId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.

`EvaluationCheckResult`

Evaluation Check Result interface with 7 public fields or methods.

Kind: interface
Import: import type { EvaluationCheckResult } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export interface EvaluationCheckResult {
    id: string;
    status: EvaluationStatus;
    message: string;
    path?: string;
    expected?: unknown;
    actual?: unknown;
    metadata?: Record<string, unknown>;
}

Contract members

Member	Kind	Signature	Description
`actual`	property	`actual?: unknown`	Public property; its type, readonly modifier and optionality are shown in the signature.
`expected`	property	`expected?: unknown`	Public property; its type, readonly modifier and optionality are shown in the signature.
`id`	property	`id: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`message`	property	`message: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`metadata`	property	`metadata?: Record<string, unknown>`	Public property; its type, readonly modifier and optionality are shown in the signature.
`path`	property	`path?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`status`	property	`status: EvaluationStatus`	Public property; its type, readonly modifier and optionality are shown in the signature.

`EvaluationEventContext`

Evaluation Event Context interface with 5 public fields or methods.

Kind: interface
Import: import type { EvaluationEventContext } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export interface EvaluationEventContext {
    runId: string;
    sessionId?: string;
    workspaceId?: string;
    agentId?: string;
    metadata?: Record<string, unknown>;
}

Contract members

Member	Kind	Signature	Description
`agentId`	property	`agentId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`metadata`	property	`metadata?: Record<string, unknown>`	Public property; its type, readonly modifier and optionality are shown in the signature.
`runId`	property	`runId: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`sessionId`	property	`sessionId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`workspaceId`	property	`workspaceId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.

`EvaluationResult`

Evaluation Result interface with 10 public fields or methods.

Kind: interface
Import: import type { EvaluationResult } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export interface EvaluationResult {
    id: string;
    evaluatorId: string;
    type: string;
    status: EvaluationStatus;
    score: number;
    checks: EvaluationCheckResult[];
    runId?: string;
    startedAt: string;
    completedAt: string;
    metadata?: Record<string, unknown>;
}

Contract members

Member	Kind	Signature	Description
`checks`	property	`checks: EvaluationCheckResult[]`	Public property; its type, readonly modifier and optionality are shown in the signature.
`completedAt`	property	`completedAt: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`evaluatorId`	property	`evaluatorId: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`id`	property	`id: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`metadata`	property	`metadata?: Record<string, unknown>`	Public property; its type, readonly modifier and optionality are shown in the signature.
`runId`	property	`runId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`score`	property	`score: number`	Public property; its type, readonly modifier and optionality are shown in the signature.
`startedAt`	property	`startedAt: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`status`	property	`status: EvaluationStatus`	Public property; its type, readonly modifier and optionality are shown in the signature.
`type`	property	`type: string`	Public property; its type, readonly modifier and optionality are shown in the signature.

`EvaluationSummary`

Evaluation Summary interface with 7 public fields or methods.

Kind: interface
Import: import type { EvaluationSummary } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export interface EvaluationSummary {
    id: string;
    status: EvaluationStatus;
    score: number;
    results: EvaluationResult[];
    runId?: string;
    startedAt: string;
    completedAt: string;
}

Contract members

Member	Kind	Signature	Description
`completedAt`	property	`completedAt: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`id`	property	`id: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`results`	property	`results: EvaluationResult[]`	Public property; its type, readonly modifier and optionality are shown in the signature.
`runId`	property	`runId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`score`	property	`score: number`	Public property; its type, readonly modifier and optionality are shown in the signature.
`startedAt`	property	`startedAt: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`status`	property	`status: EvaluationStatus`	Public property; its type, readonly modifier and optionality are shown in the signature.

`JsonSchemaValidationIssue`

JSON Schema Validation Issue interface with 4 public fields or methods.

Kind: interface
Import: import type { JsonSchemaValidationIssue } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export interface JsonSchemaValidationIssue {
    path: string;
    message: string;
    expected?: unknown;
    actual?: unknown;
}

Contract members

Member	Kind	Signature	Description
`actual`	property	`actual?: unknown`	Public property; its type, readonly modifier and optionality are shown in the signature.
`expected`	property	`expected?: unknown`	Public property; its type, readonly modifier and optionality are shown in the signature.
`message`	property	`message: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`path`	property	`path: string`	Public property; its type, readonly modifier and optionality are shown in the signature.

`OutputContractValidationInput`

Output Contract Validation Input interface with 5 public fields or methods.

Kind: interface
Import: import type { OutputContractValidationInput } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export interface OutputContractValidationInput {
    contract: OutputContractSpec;
    output: unknown;
    runId?: string;
    evaluationId?: string;
    metadata?: Record<string, unknown>;
}

Contract members

Member	Kind	Signature	Description
`contract`	property	`contract: OutputContractSpec`	Public property; its type, readonly modifier and optionality are shown in the signature.
`evaluationId`	property	`evaluationId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`metadata`	property	`metadata?: Record<string, unknown>`	Public property; its type, readonly modifier and optionality are shown in the signature.
`output`	property	`output: unknown`	Public property; its type, readonly modifier and optionality are shown in the signature.
`runId`	property	`runId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.

`TraceCompletenessEvaluatorOptions`

Trace Completeness Evaluator Options interface with 3 public fields or methods.

Kind: interface
Import: import type { TraceCompletenessEvaluatorOptions } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export interface TraceCompletenessEvaluatorOptions {
    now?: () => string;
    enforceLifecyclePairs?: boolean;
    requireTerminalRun?: boolean;
}

Contract members

Member	Kind	Signature	Description
`enforceLifecyclePairs`	property	`enforceLifecyclePairs?: boolean`	Public property; its type, readonly modifier and optionality are shown in the signature.
`now`	method	`now?(): string`	Public method; parameters and return type are shown in the signature.
`requireTerminalRun`	property	`requireTerminalRun?: boolean`	Public property; its type, readonly modifier and optionality are shown in the signature.

`TraceCompletenessInput`

Trace Completeness Input interface with 6 public fields or methods.

Kind: interface
Import: import type { TraceCompletenessInput } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export interface TraceCompletenessInput {
    events: FrameworkEvent[];
    traceSpec?: TraceSpec;
    runId?: string;
    requiredEventTypes?: string[];
    evaluationId?: string;
    metadata?: Record<string, unknown>;
}

Contract members

Member	Kind	Signature	Description
`evaluationId`	property	`evaluationId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`events`	property	`events: FrameworkEvent<unknown>[]`	Public property; its type, readonly modifier and optionality are shown in the signature.
`metadata`	property	`metadata?: Record<string, unknown>`	Public property; its type, readonly modifier and optionality are shown in the signature.
`requiredEventTypes`	property	`requiredEventTypes?: string[]`	Public property; its type, readonly modifier and optionality are shown in the signature.
`runId`	property	`runId?: string`	Public property; its type, readonly modifier and optionality are shown in the signature.
`traceSpec`	property	`traceSpec?: TraceSpec`	Public property; its type, readonly modifier and optionality are shown in the signature.

`EvaluationStatus`

Public type alias for Evaluation Status; the declaration contains its complete type expression.

Kind: type
Import: import type { EvaluationStatus } from '@codesoul-co/hypha-testing';
Source module: evaluation

Declaration

text

export type EvaluationStatus = 'passed' | 'failed';

@codesoul-co/hypha-testing / evaluation ​

Using this module ​

Import from the package entrypoint ​

Usage patterns ​

Public exports ​

DeterministicEvaluator ​

Declaration ​

Public members ​

OutputContractValidator ​

Declaration ​

Public members ​

TraceCompletenessEvaluator ​

Declaration ​

Public members ​

validateJsonSchemaValue ​

Declaration ​

Call signature ​

Parameters ​

Returns ​

DeterministicEvaluationInput ​

Declaration ​

Contract members ​

DeterministicEvaluatorOptions ​

Declaration ​

Contract members ​

EvaluationCheckResult ​

Declaration ​

Contract members ​

EvaluationEventContext ​

Declaration ​

Contract members ​

EvaluationResult ​

Declaration ​

Contract members ​

EvaluationSummary ​

Declaration ​

Contract members ​

JsonSchemaValidationIssue ​

Declaration ​

Contract members ​

OutputContractValidationInput ​

Declaration ​

Contract members ​

TraceCompletenessEvaluatorOptions ​

Declaration ​

Contract members ​

TraceCompletenessInput ​

Declaration ​

Contract members ​

EvaluationStatus ​

Declaration ​

`@codesoul-co/hypha-testing` / `evaluation`

Using this module

Import from the package entrypoint

Usage patterns

Public exports

`DeterministicEvaluator`

Declaration

Public members

`OutputContractValidator`

Declaration

Public members

`TraceCompletenessEvaluator`

Declaration

Public members

`validateJsonSchemaValue`

Declaration

Call signature

Parameters

Returns

`DeterministicEvaluationInput`

Declaration

Contract members

`DeterministicEvaluatorOptions`

Declaration

Contract members

`EvaluationCheckResult`

Declaration

Contract members

`EvaluationEventContext`

Declaration

Contract members

`EvaluationResult`

Declaration

Contract members

`EvaluationSummary`

Declaration

Contract members

`JsonSchemaValidationIssue`

Declaration

Contract members

`OutputContractValidationInput`

Declaration

Contract members

`TraceCompletenessEvaluatorOptions`

Declaration

Contract members

`TraceCompletenessInput`

Declaration

Contract members

`EvaluationStatus`

Declaration