@codesoul-co/hypha-inference / backends

• Package index: @codesoul-co/hypha-inference
• Source: packages/inference/src/backends.ts
• Exports: 11

Using this module

Use the Backends module for using the public contracts and operations for this capability boundary. It exports 7 classes, 1 function, 3 interfaces.

Import from the package entrypoint

import {
  FetchInferenceBackendTransport,
  InferenceBackendRegistry,
  LlamaCppInferenceBackend,
  OllamaInferenceBackend,
  OpenAIAPIInferenceBackend,
  SGLangInferenceBackend,
  VLLMInferenceBackend,
  createDefaultInferenceBackendRegistry,
} from '@codesoul-co/hypha-inference';

import type {
  DefaultInferenceBackendRegistryOptions,
  HttpInferenceBackendConfig,
  InferenceBackendTransport,
} from '@codesoul-co/hypha-inference';

Usage patterns

• Use the 3 type/interface exports as static contracts in application code, adapters, or tests. Import them with import type; they do not exist at runtime.
• The module exposes 7 classes as constructable runtime implementations. Each symbol entry lists its constructor and public methods.
• The module exposes 1 function as direct operation entrypoints. Every overload, required/optional parameter, and return type is documented below.

Public exports

Symbol	Kind	Signature	Description
FetchInferenceBackendTransport	class	new FetchInferenceBackendTransport(): FetchInferenceBackendTransport	Fetch Inference Backend Transport class with 3 public constructor or member entries; its exact declarations are listed below.
InferenceBackendRegistry	class	new InferenceBackendRegistry(defaultBackendId?: string): InferenceBackendRegistry	Inference Backend Registry class with 6 public constructor or member entries; its exact declarations are listed below.
LlamaCppInferenceBackend	class	new LlamaCppInferenceBackend(config?: Partial<HttpInferenceBackendConfig>): LlamaCppInferenceBackend	Llama Cpp Inference Backend class with 6 public constructor or member entries; its exact declarations are listed below.
OllamaInferenceBackend	class	new OllamaInferenceBackend(config?: Partial<HttpInferenceBackendConfig>): OllamaInferenceBackend	Ollama Inference Backend class with 6 public constructor or member entries; its exact declarations are listed below.
OpenAIAPIInferenceBackend	class	new OpenAIAPIInferenceBackend(config?: Partial<HttpInferenceBackendConfig>): OpenAIAPIInferenceBackend	Open AIAPI Inference Backend class with 6 public constructor or member entries; its exact declarations are listed below.
SGLangInferenceBackend	class	new SGLangInferenceBackend(config?: Partial<HttpInferenceBackendConfig>): SGLangInferenceBackend	SG Lang Inference Backend class with 6 public constructor or member entries; its exact declarations are listed below.
VLLMInferenceBackend	class	new VLLMInferenceBackend(config?: Partial<HttpInferenceBackendConfig>): VLLMInferenceBackend	VLLM Inference Backend class with 6 public constructor or member entries; its exact declarations are listed below.
createDefaultInferenceBackendRegistry	function	createDefaultInferenceBackendRegistry(options?: DefaultInferenceBackendRegistryOptions): InferenceBackendRegistry	Create Default Inference Backend Registry function with 1 public call signature; parameters and return types are listed below.
DefaultInferenceBackendRegistryOptions	interface	interface DefaultInferenceBackendRegistryOptions	Default Inference Backend Registry Options interface with 6 public fields or methods.
HttpInferenceBackendConfig	interface	interface HttpInferenceBackendConfig	Http Inference Backend Config interface with 8 public fields or methods.
InferenceBackendTransport	interface	interface InferenceBackendTransport	Inference Backend Transport interface with 2 public fields or methods.

FetchInferenceBackendTransport

Fetch Inference Backend Transport class with 3 public constructor or member entries; its exact declarations are listed below.

• Kind: class
• Import: import { FetchInferenceBackendTransport } from '@codesoul-co/hypha-inference';
• Source module: backends

Declaration

export declare class FetchInferenceBackendTransport implements InferenceBackendTransport {
    postJson<TResponse = unknown>(url: string, body: unknown, headers?: Record<string, string>, timeoutMs?: number): Promise<TResponse>;
    streamJson<TResponse = unknown>(url: string, body: unknown, headers?: Record<string, string>, timeoutMs?: number): AsyncIterable<TResponse>;
}

Public members

Member	Kind	Signature	Description
constructor	constructor	(): FetchInferenceBackendTransport	Creates an instance of this class.
postJson	method	postJson<TResponse = unknown>(url: string, body: unknown, headers?: Record<string, string>, timeoutMs?: number): Promise<TResponse>	Public method; parameters and return type are shown in the signature.
streamJson	method	streamJson<TResponse = unknown>(url: string, body: unknown, headers?: Record<string, string>, timeoutMs?: number): AsyncIterable<TResponse>	Public method; parameters and return type are shown in the signature.

InferenceBackendRegistry

Inference Backend Registry class with 6 public constructor or member entries; its exact declarations are listed below.

• Kind: class
• Import: import { InferenceBackendRegistry } from '@codesoul-co/hypha-inference';
• Source module: backends

Declaration

export declare class InferenceBackendRegistry {
    constructor(defaultBackendId?: string);
    register(backend: InferenceBackend, options?: {
            default?: boolean;
        }): void;
    get(id: string): InferenceBackend | null;
    require(id: string): InferenceBackend;
    default(): InferenceBackend;
    list(): InferenceBackendRegistryEntry[];
}

Public members

Member	Kind	Signature	Description
constructor	constructor	(defaultBackendId?: string): InferenceBackendRegistry	Creates an instance of this class.
default	method	default(): InferenceBackend	Public method; parameters and return type are shown in the signature.
get	method	get(id: string): InferenceBackend | null	Public method; parameters and return type are shown in the signature.
list	method	list(): InferenceBackendRegistryEntry[]	Public method; parameters and return type are shown in the signature.
register	method	register(backend: InferenceBackend, options?: { default?: boolean; }): void	Public method; parameters and return type are shown in the signature.
require	method	require(id: string): InferenceBackend	Public method; parameters and return type are shown in the signature.

LlamaCppInferenceBackend

Llama Cpp Inference Backend class with 6 public constructor or member entries; its exact declarations are listed below.

• Kind: class
• Import: import { LlamaCppInferenceBackend } from '@codesoul-co/hypha-inference';
• Source module: backends

Declaration

export declare class LlamaCppInferenceBackend extends HttpInferenceBackend {
    constructor(config?: Partial<HttpInferenceBackendConfig>);
}

Public members

Member	Kind	Signature	Description
capabilities	method	capabilities(): InferenceBackendCapabilities	Public method; parameters and return type are shown in the signature.
constructor	constructor	(config?: Partial<HttpInferenceBackendConfig>): LlamaCppInferenceBackend	Creates an instance of this class.
id	property	readonly id: string	Public property; its type, readonly modifier and optionality are shown in the signature.
infer	method	infer(request: InferenceBackendRequest): Promise<InferenceBackendResponse>	Public method; parameters and return type are shown in the signature.
kind	property	readonly kind: InferenceBackendKind	Public property; its type, readonly modifier and optionality are shown in the signature.
stream	method	stream(request: InferenceBackendRequest): AsyncIterable<InferenceBackendResponse>	Public method; parameters and return type are shown in the signature.

OllamaInferenceBackend

Ollama Inference Backend class with 6 public constructor or member entries; its exact declarations are listed below.

• Kind: class
• Import: import { OllamaInferenceBackend } from '@codesoul-co/hypha-inference';
• Source module: backends

Declaration

export declare class OllamaInferenceBackend extends HttpInferenceBackend {
    constructor(config?: Partial<HttpInferenceBackendConfig>);
}

Public members

Member	Kind	Signature	Description
capabilities	method	capabilities(): InferenceBackendCapabilities	Public method; parameters and return type are shown in the signature.
constructor	constructor	(config?: Partial<HttpInferenceBackendConfig>): OllamaInferenceBackend	Creates an instance of this class.
id	property	readonly id: string	Public property; its type, readonly modifier and optionality are shown in the signature.
infer	method	infer(request: InferenceBackendRequest): Promise<InferenceBackendResponse>	Public method; parameters and return type are shown in the signature.
kind	property	readonly kind: InferenceBackendKind	Public property; its type, readonly modifier and optionality are shown in the signature.
stream	method	stream(request: InferenceBackendRequest): AsyncIterable<InferenceBackendResponse>	Public method; parameters and return type are shown in the signature.

OpenAIAPIInferenceBackend

Open AIAPI Inference Backend class with 6 public constructor or member entries; its exact declarations are listed below.

• Kind: class
• Import: import { OpenAIAPIInferenceBackend } from '@codesoul-co/hypha-inference';
• Source module: backends

Declaration

export declare class OpenAIAPIInferenceBackend extends HttpInferenceBackend {
    constructor(config?: Partial<HttpInferenceBackendConfig>);
}

Public members

Member	Kind	Signature	Description
capabilities	method	capabilities(): InferenceBackendCapabilities	Public method; parameters and return type are shown in the signature.
constructor	constructor	(config?: Partial<HttpInferenceBackendConfig>): OpenAIAPIInferenceBackend	Creates an instance of this class.
id	property	readonly id: string	Public property; its type, readonly modifier and optionality are shown in the signature.
infer	method	infer(request: InferenceBackendRequest): Promise<InferenceBackendResponse>	Public method; parameters and return type are shown in the signature.
kind	property	readonly kind: InferenceBackendKind	Public property; its type, readonly modifier and optionality are shown in the signature.
stream	method	stream(request: InferenceBackendRequest): AsyncIterable<InferenceBackendResponse>	Public method; parameters and return type are shown in the signature.

SGLangInferenceBackend

SG Lang Inference Backend class with 6 public constructor or member entries; its exact declarations are listed below.

• Kind: class
• Import: import { SGLangInferenceBackend } from '@codesoul-co/hypha-inference';
• Source module: backends

Declaration

export declare class SGLangInferenceBackend extends HttpInferenceBackend {
    constructor(config?: Partial<HttpInferenceBackendConfig>);
}

Public members

Member	Kind	Signature	Description
capabilities	method	capabilities(): InferenceBackendCapabilities	Public method; parameters and return type are shown in the signature.
constructor	constructor	(config?: Partial<HttpInferenceBackendConfig>): SGLangInferenceBackend	Creates an instance of this class.
id	property	readonly id: string	Public property; its type, readonly modifier and optionality are shown in the signature.
infer	method	infer(request: InferenceBackendRequest): Promise<InferenceBackendResponse>	Public method; parameters and return type are shown in the signature.
kind	property	readonly kind: InferenceBackendKind	Public property; its type, readonly modifier and optionality are shown in the signature.
stream	method	stream(request: InferenceBackendRequest): AsyncIterable<InferenceBackendResponse>	Public method; parameters and return type are shown in the signature.

VLLMInferenceBackend

VLLM Inference Backend class with 6 public constructor or member entries; its exact declarations are listed below.

• Kind: class
• Import: import { VLLMInferenceBackend } from '@codesoul-co/hypha-inference';
• Source module: backends

Declaration

export declare class VLLMInferenceBackend extends HttpInferenceBackend {
    constructor(config?: Partial<HttpInferenceBackendConfig>);
}

Public members

Member	Kind	Signature	Description
capabilities	method	capabilities(): InferenceBackendCapabilities	Public method; parameters and return type are shown in the signature.
constructor	constructor	(config?: Partial<HttpInferenceBackendConfig>): VLLMInferenceBackend	Creates an instance of this class.
id	property	readonly id: string	Public property; its type, readonly modifier and optionality are shown in the signature.
infer	method	infer(request: InferenceBackendRequest): Promise<InferenceBackendResponse>	Public method; parameters and return type are shown in the signature.
kind	property	readonly kind: InferenceBackendKind	Public property; its type, readonly modifier and optionality are shown in the signature.
stream	method	stream(request: InferenceBackendRequest): AsyncIterable<InferenceBackendResponse>	Public method; parameters and return type are shown in the signature.

createDefaultInferenceBackendRegistry

Create Default Inference Backend Registry function with 1 public call signature; parameters and return types are listed below.

• Kind: function
• Import: import { createDefaultInferenceBackendRegistry } from '@codesoul-co/hypha-inference';
• Source module: backends

Declaration

export declare function createDefaultInferenceBackendRegistry(options?: DefaultInferenceBackendRegistryOptions): InferenceBackendRegistry;

Call signature

createDefaultInferenceBackendRegistry(options?: DefaultInferenceBackendRegistryOptions): InferenceBackendRegistry

Parameters

Parameter	Type	Required	Description
options	DefaultInferenceBackendRegistryOptions	No	Optional parameter; accepted values are defined by the type column.

Returns

• Type: InferenceBackendRegistry
• Description: The return contract is defined by the type shown above.

DefaultInferenceBackendRegistryOptions

Default Inference Backend Registry Options interface with 6 public fields or methods.

• Kind: interface
• Import: import type { DefaultInferenceBackendRegistryOptions } from '@codesoul-co/hypha-inference';
• Source module: backends

Declaration

export interface DefaultInferenceBackendRegistryOptions {
    defaultBackendId?: string;
    ollama?: Partial<HttpInferenceBackendConfig>;
    sglang?: Partial<HttpInferenceBackendConfig>;
    vllm?: Partial<HttpInferenceBackendConfig>;
    llamaCpp?: Partial<HttpInferenceBackendConfig>;
    openaiApi?: Partial<HttpInferenceBackendConfig>;
}

Contract members

Member	Kind	Signature	Description
defaultBackendId	property	defaultBackendId?: string	Public property; its type, readonly modifier and optionality are shown in the signature.
llamaCpp	property	llamaCpp?: Partial<HttpInferenceBackendConfig>	Public property; its type, readonly modifier and optionality are shown in the signature.
ollama	property	ollama?: Partial<HttpInferenceBackendConfig>	Public property; its type, readonly modifier and optionality are shown in the signature.
openaiApi	property	openaiApi?: Partial<HttpInferenceBackendConfig>	Public property; its type, readonly modifier and optionality are shown in the signature.
sglang	property	sglang?: Partial<HttpInferenceBackendConfig>	Public property; its type, readonly modifier and optionality are shown in the signature.
vllm	property	vllm?: Partial<HttpInferenceBackendConfig>	Public property; its type, readonly modifier and optionality are shown in the signature.

HttpInferenceBackendConfig

Http Inference Backend Config interface with 8 public fields or methods.

• Kind: interface
• Import: import type { HttpInferenceBackendConfig } from '@codesoul-co/hypha-inference';
• Source module: backends

Declaration

export interface HttpInferenceBackendConfig {
    id?: string;
    baseUrl: string;
    endpoint: string;
    apiKey?: string;
    apiKeyEnv?: string;
    timeoutMs?: number;
    transport?: InferenceBackendTransport;
    capabilities?: Partial<InferenceBackendCapabilities>;
}

Contract members

Member	Kind	Signature	Description
apiKey	property	apiKey?: string	Public property; its type, readonly modifier and optionality are shown in the signature.
apiKeyEnv	property	apiKeyEnv?: string	Public property; its type, readonly modifier and optionality are shown in the signature.
baseUrl	property	baseUrl: string	Public property; its type, readonly modifier and optionality are shown in the signature.
capabilities	property	capabilities?: Partial<InferenceBackendCapabilities>	Public property; its type, readonly modifier and optionality are shown in the signature.
endpoint	property	endpoint: string	Public property; its type, readonly modifier and optionality are shown in the signature.
id	property	id?: string	Public property; its type, readonly modifier and optionality are shown in the signature.
timeoutMs	property	timeoutMs?: number	Public property; its type, readonly modifier and optionality are shown in the signature.
transport	property	transport?: InferenceBackendTransport	Public property; its type, readonly modifier and optionality are shown in the signature.

InferenceBackendTransport

Inference Backend Transport interface with 2 public fields or methods.

• Kind: interface
• Import: import type { InferenceBackendTransport } from '@codesoul-co/hypha-inference';
• Source module: backends

Declaration

export interface InferenceBackendTransport {
    postJson<TResponse = unknown>(url: string, body: unknown, headers?: Record<string, string>, timeoutMs?: number): Promise<TResponse>;
    streamJson?<TResponse = unknown>(url: string, body: unknown, headers?: Record<string, string>, timeoutMs?: number): AsyncIterable<TResponse>;
}

Contract members

Member	Kind	Signature	Description
postJson	method	postJson<TResponse = unknown>(url: string, body: unknown, headers?: Record<string, string>, timeoutMs?: number): Promise<TResponse>	Public method; parameters and return type are shown in the signature.
streamJson	method	streamJson?<TResponse = unknown>(url: string, body: unknown, headers?: Record<string, string>, timeoutMs?: number): AsyncIterable<TResponse>	Public method; parameters and return type are shown in the signature.