Plugin API¶

Core plugin lifecycle: BasePlugin, the manifest contract, optional interfaces (discovery, notifier, detection).

camera_ui_sdk.plugin ¶

APIListener `module-attribute` ¶

APIListener = Callable[[], None] | Callable[[], Awaitable[None]]

Listener for plugin lifecycle events. Coroutine functions are awaited.

AssistantToolProperty `module-attribute` ¶

AssistantToolProperty = AssistantToolStringProperty | AssistantToolNumberProperty | AssistantToolBooleanProperty | AssistantToolArrayProperty | AssistantToolObjectProperty

Property schema of a tool input: the JSON Schema draft-07 subset every supported model provider accepts unchanged.

AssistantModelChunk `module-attribute` ¶

AssistantModelChunk = AssistantModelTextChunk | AssistantModelToolCallChunk | AssistantModelUsageChunk | AssistantModelDoneChunk

A piece of the answer: text, a tool call, token counts, or the end.

AssistantModelContent `module-attribute` ¶

AssistantModelContent = AssistantModelText | AssistantModelImage

Part of a message: text or an image.

PROTOCOL_LEVEL `module-attribute` ¶

PROTOCOL_LEVEL = 2

Compatibility level of the plugin surface: the plugin-facing API and the plugin wire protocol. Bumped only on breaking changes, never for additive features. The CLI stamps the level a plugin was built against into its bundle (cameraui.protocolLevel in the bundle package.json); the server compares that stamp with its own level and refuses to start plugins outside its supported range.

PythonVersion `module-attribute` ¶

PythonVersion = Literal['3.11', '3.12']

Python interpreter major.minor version a Python plugin requires. The host ensures a matching interpreter exists in its venv pool before launching the plugin; Node and Go plugins ignore this field.

PluginInterfaces `module-attribute` ¶

PluginInterfaces = MotionDetectionInterface | ObjectDetectionInterface | AudioDetectionInterface | FaceDetectionInterface | LicensePlateDetectionInterface | ClassifierDetectionInterface | ClipDetectionInterface | PersonEmbeddingInterface | SegmentationInterface | DiscoveryProvider | AssistantToolProvider

Union of all optional plugin interfaces.

OAuthStatus `module-attribute` ¶

OAuthStatus = Literal['disconnected', 'awaiting_user', 'polling', 'connected', 'error']

Lifecycle phase of an OAuth provider connection, carried in OAuthState.status.

API_EVENT ¶

Bases: Enum

Lifecycle events emitted on the PluginAPI EventEmitter.

Plugins subscribe with api.on(API_EVENT.X, handler) to react to host-driven phase changes.

FINISH_LAUNCHING `class-attribute` `instance-attribute` ¶

FINISH_LAUNCHING = 'finishLaunching'

Emitted once after every assigned camera is wired up and configureCameras() returned. Start timers and warm-ups here.

SHUTDOWN `class-attribute` `instance-attribute` ¶

SHUTDOWN = 'shutdown'

Emitted when the host tears the plugin down. Release files, sockets, timers and child processes now.

PluginAPI ¶

Bases: Protocol

The PluginAPI is injected into the plugin at runtime and exposes the system services the plugin is allowed to talk to. It also acts as an EventEmitter for plugin lifecycle events (see :class:API_EVENT).

Example

class MyPlugin(BasePlugin):
    async def configureCameras(self, cameras):
        ffmpeg = await self.api.coreManager.getFFmpegPath()

coreManager `property` ¶

coreManager: CoreManager

System-level operations: the FFmpeg path and the server addresses used for media URLs (HTTP/RTSP).

deviceManager `property` ¶

deviceManager: DeviceManager

Owns the camera devices assigned to this plugin and publishes camera-state changes.

sensorManager `property` ¶

sensorManager: SensorManager

Registers standalone sensors: entities of their own, persisted across restarts, assignable to cameras by the user.

downloadManager `property` ¶

downloadManager: DownloadManager

Mints token-protected download URLs for files the plugin exposes to the UI (clip exports, snapshots).

notificationManager `property` ¶

notificationManager: NotificationManager

Publishes notifications to every installed notifier and the in-app UI. Requires :attr:PluginCapability.PublishNotifications.

storagePath `property` ¶

storagePath: str

Absolute path to the plugin's writable storage directory, created and cleaned up by the host.

on ¶

on(event: API_EVENT, f: APIListener) -> Any

Subscribe to a lifecycle event. Returns self for chaining.

once ¶

once(event: API_EVENT, f: APIListener) -> Any

Subscribe to a lifecycle event for one delivery only. Returns self for chaining.

off ¶

off(event: API_EVENT, f: APIListener) -> None

Remove a previously registered listener (alias of :meth:removeListener).

removeListener ¶

removeListener(event: API_EVENT, f: APIListener) -> None

Remove a previously registered listener.

removeAllListeners ¶

removeAllListeners(event: API_EVENT | None = None) -> None

Remove every listener for event, or every listener when no event is given.

AssistantToolArrayProperty ¶

Bases: TypedDict

Array argument of a tool input.

type `instance-attribute` ¶

type: Literal['array']

Always array.

items `instance-attribute` ¶

items: AssistantToolProperty

Schema of each element.

description `instance-attribute` ¶

description: NotRequired[str]

What the argument means, written for the model.

minItems `instance-attribute` ¶

minItems: NotRequired[int]

Fewest elements allowed.

maxItems `instance-attribute` ¶

maxItems: NotRequired[int]

Most elements allowed.

AssistantToolBooleanProperty ¶

Bases: TypedDict

Boolean argument of a tool input.

type `instance-attribute` ¶

type: Literal['boolean']

Always boolean.

description `instance-attribute` ¶

description: NotRequired[str]

What the argument means, written for the model.

AssistantToolContext ¶

Bases: TypedDict

Who is asking and in which language. Passed to every tool call; the host has already enforced adminOnly and approval, the fields are information.

userId `instance-attribute` ¶

userId: str

ID of the user talking to the assistant.

role `instance-attribute` ¶

role: Literal['user', 'admin', 'master']

Role of the user.

language `instance-attribute` ¶

language: str

BCP 47 language tag of the conversation, for localized text results.

timezone `instance-attribute` ¶

timezone: str

IANA time zone of the user, for parsing and formatting times.

threadId `instance-attribute` ¶

threadId: NotRequired[str]

ID of the conversation the tool is called from. Absent for scheduled runs and MCP clients.

AssistantToolImage ¶

Bases: TypedDict

An image a tool returns alongside its text. Forwarded to the model only when the instance allows images to leave the machine, so content should still describe what the image shows.

data `instance-attribute` ¶

data: str

Base64-encoded image bytes.

mimeType `instance-attribute` ¶

mimeType: Literal['image/jpeg', 'image/png', 'image/webp']

MIME type of data.

caption `instance-attribute` ¶

caption: NotRequired[str]

Short caption shown to the user next to the image.

AssistantToolNumberProperty ¶

Bases: TypedDict

Numeric argument of a tool input.

type `instance-attribute` ¶

type: Literal['number', 'integer']

number or integer.

description `instance-attribute` ¶

description: NotRequired[str]

What the argument means, written for the model.

minimum `instance-attribute` ¶

minimum: NotRequired[float]

Smallest allowed value.

maximum `instance-attribute` ¶

maximum: NotRequired[float]

Largest allowed value.

AssistantToolObjectProperty ¶

Bases: TypedDict

Object argument of a tool input.

type `instance-attribute` ¶

type: Literal['object']

Always object.

properties `instance-attribute` ¶

properties: dict[str, AssistantToolProperty]

Named members of the object.

description `instance-attribute` ¶

description: NotRequired[str]

What the argument means, written for the model.

required `instance-attribute` ¶

required: NotRequired[list[str]]

Members the model has to provide.

AssistantToolProvider ¶

Bases: Protocol

Implemented by plugins that list PluginInterface.AssistantTools in their contract. The host asks for the tool list once after the plugin is ready and dispatches calls by name.

Example

class MyPlugin(BasePlugin):
    async def assistantTools(self) -> list[AssistantToolSpec]:
        return [query_events]

    async def callAssistantTool(
        self, name: str, input: dict[str, Any], ctx: AssistantToolContext
    ) -> AssistantToolResult:
        if name == "query_events":
            return {"content": await self.store.query(input["from"], input["to"])}
        return {"error": f"Unknown tool {name}"}

assistantTools `async` ¶

assistantTools() -> list[AssistantToolSpec]

List the tools this plugin offers. Called once per plugin start.

Returns:

Type	Description
`list[AssistantToolSpec]`	The tool specs, empty when the plugin currently has none.

callAssistantTool `async` ¶

callAssistantTool(name: str, input: dict[str, Any], ctx: AssistantToolContext) -> AssistantToolResult

Run one tool. Validate input against the declared schema, the host does not.

Parameters:

Name	Type	Description	Default
`name`	`str`	Tool name as declared in `AssistantToolSpec["name"]`.	required
`input`	`dict[str, Any]`	Parsed arguments the model produced.	required
`ctx`	`AssistantToolContext`	Who is asking and in which language.	required

Returns:

Type	Description
`AssistantToolResult`	Text, JSON or images for the model, or an `error` it may repair.

AssistantToolReference ¶

Bases: TypedDict

Something in the app a tool result points at. The chat renders it as a button that opens the matching view: the recording player for an event, the episode player for an episode, the camera page for a camera, or a download button for a file the download manager serves.

kind `instance-attribute` ¶

kind: Literal['event', 'episode', 'camera', 'download']

What the id names.

id `instance-attribute` ¶

id: str

Id of the event, episode or camera, or the download token.

label `instance-attribute` ¶

label: NotRequired[str]

Short label for the button, for example the camera name and time.

cameraId `instance-attribute` ¶

cameraId: NotRequired[str]

Camera the event belongs to, required for event.

timestamp `instance-attribute` ¶

timestamp: NotRequired[int]

Start of the event or episode in milliseconds since the epoch, required for event.

url `instance-attribute` ¶

url: NotRequired[str]

Download path from the download manager, required for download; the chat shows a download button labelled with label.

AssistantToolResult ¶

Bases: TypedDict

Result of a tool call. Return error for problems the model can repair by asking differently (unknown camera, empty range); raise only for failures that should end the run.

content `instance-attribute` ¶

content: NotRequired[str | dict[str, Any] | list[Any]]

Text or JSON the model reads. The host serializes JSON.

images `instance-attribute` ¶

images: NotRequired[list[AssistantToolImage]]

Images to show and, when allowed, to send to the model.

references `instance-attribute` ¶

references: NotRequired[list[AssistantToolReference]]

Things the chat should offer to open, at most a handful per result.

error `instance-attribute` ¶

error: NotRequired[str]

User-level problem the model may repair. Replaces content.

AssistantToolSchema ¶

Bases: TypedDict

Input schema of a tool: always an object schema. The model fills the properties, the host passes them to callAssistantTool as parsed JSON.

type `instance-attribute` ¶

type: Literal['object']

Always object.

properties `instance-attribute` ¶

properties: dict[str, AssistantToolProperty]

Named arguments the model may pass.

required `instance-attribute` ¶

required: NotRequired[list[str]]

Arguments the model has to provide.

additionalProperties `instance-attribute` ¶

additionalProperties: NotRequired[Literal[False]]

Set to False to reject arguments outside properties.

AssistantToolSpec ¶

Bases: TypedDict

One tool a plugin offers to the assistant. The host prefixes name with the plugin name, so names only have to be unique within the plugin.

Example

query_events = define_assistant_tool(
    {
        "name": "query_events",
        "description": "List recorded events for a time range, optionally filtered by camera and label.",
        "inputSchema": {
            "type": "object",
            "properties": {
                "from": {
                    "type": "string",
                    "format": "date-time",
                    "description": "Start of the range, ISO 8601",
                },
                "to": {
                    "type": "string",
                    "format": "date-time",
                    "description": "End of the range, ISO 8601",
                },
                "camera": {"type": "string", "description": "Camera name, all cameras when omitted"},
            },
            "required": ["from", "to"],
        },
    }
)

name `instance-attribute` ¶

name: str

Tool name in snake_case, unique within the plugin.

description `instance-attribute` ¶

description: str

What the tool does and when to use it, written for the model. One or two sentences, name units and defaults.

inputSchema `instance-attribute` ¶

inputSchema: AssistantToolSchema

Object schema of the arguments (see :class:AssistantToolSchema).

approval `instance-attribute` ¶

approval: NotRequired[bool]

The user confirms every call before it runs. Set it on anything that changes state.

adminOnly `instance-attribute` ¶

adminOnly: NotRequired[bool]

Hidden from users without the admin role.

timeoutMs `instance-attribute` ¶

timeoutMs: NotRequired[int]

Time the host waits for callAssistantTool, in milliseconds. Default 60000.

lazy `instance-attribute` ¶

lazy: NotRequired[bool]

Offered through tool discovery instead of on every turn. Use for rarely needed tools.

AssistantToolStringProperty ¶

Bases: TypedDict

String argument of a tool input.

type `instance-attribute` ¶

type: Literal['string']

Always string.

description `instance-attribute` ¶

description: NotRequired[str]

What the argument means, written for the model.

enum `instance-attribute` ¶

enum: NotRequired[list[str]]

Allowed values.

format `instance-attribute` ¶

format: NotRequired[Literal['date-time', 'date', 'time']]

Expected text format.

AssistantModelContext ¶

Bases: TypedDict

Who the answer is for and when to stop writing it.

userId `instance-attribute` ¶

userId: NotRequired[str]

ID of the user the answer is for. Absent for scheduled runs and plugin requests.

language `instance-attribute` ¶

language: str

BCP 47 language tag the answer should use.

timeoutMs `instance-attribute` ¶

timeoutMs: int

Milliseconds the host waits before it drops the answer.

AssistantModelDoneChunk ¶

Bases: TypedDict

End of the answer, yielded exactly once.

type `instance-attribute` ¶

type: Literal['done']

Always done.

finish `instance-attribute` ¶

finish: Literal['stop', 'tool_calls', 'length', 'error']

Why the model stopped.

message `instance-attribute` ¶

message: NotRequired[str]

What went wrong, with error.

AssistantModelImage ¶

Bases: TypedDict

Image part of a message. Arrives only for a model whose spec says vision, and only when the instance allows pictures to reach the model.

type `instance-attribute` ¶

type: Literal['image']

Always image.

data `instance-attribute` ¶

data: str

Base64-encoded image bytes.

mimeType `instance-attribute` ¶

mimeType: Literal['image/jpeg', 'image/png', 'image/webp']

MIME type of data.

AssistantModelMessage ¶

Bases: TypedDict

One message of the conversation. tool messages carry the result of the call with the matching toolCallId.

role `instance-attribute` ¶

role: Literal['user', 'assistant', 'tool']

Who wrote it.

content `instance-attribute` ¶

content: list[AssistantModelContent]

Text and images of the message.

toolCalls `instance-attribute` ¶

toolCalls: NotRequired[list[AssistantModelToolCall]]

Tool calls the assistant asked for, on assistant messages.

toolCallId `instance-attribute` ¶

toolCallId: NotRequired[str]

Call this message answers, on tool messages.

AssistantModelProvider ¶

Bases: Protocol

Implemented by plugins that list PluginInterface.AssistantModels in their contract. Such a plugin is a bridge to a model the host cannot reach itself: one without an HTTP API, one behind an unusual authentication, or one that lives on the machine the plugin runs on. The plugin holds no conversation state, every request carries the whole conversation.

Example

class ApplePlugin(BasePlugin):
    async def assistantModels(self) -> list[AssistantModelSpec]:
        return [
            {
                "id": "apple-on-device",
                "name": "Apple on-device",
                "contextTokens": 8192,
                "vision": True,
                "toolCalling": True,
                "structuredOutput": True,
            }
        ]

    async def assistantGenerate(
        self, request: AssistantModelRequest, ctx: AssistantModelContext
    ) -> AsyncIterator[AssistantModelChunk]:
        async for delta in self.session.stream(request, ctx["timeoutMs"]):
            yield {"type": "text", "delta": delta}
        yield {"type": "done", "finish": "stop"}

assistantModels `async` ¶

assistantModels() -> list[AssistantModelSpec]

List the models this plugin offers. Called after the plugin is ready and whenever the settings page is opened.

Returns:

Type	Description
`list[AssistantModelSpec]`	The specs, empty while the plugin has no usable model (no hardware,
`list[AssistantModelSpec]`	no permission yet).

assistantModelStatus `async` ¶

assistantModelStatus() -> AssistantModelStatus

Report whether the models can answer right now. The host asks before it offers them and while the settings page is open, so a model that is still downloading shows up as busy instead of missing. Leave it out when the models are usable as soon as the plugin runs.

Returns:

Type	Description
`AssistantModelStatus`	Readiness, with a line and a progress while it loads.

assistantGenerate ¶

assistantGenerate(request: AssistantModelRequest, ctx: AssistantModelContext) -> AsyncIterator[AssistantModelChunk]

Answer one request. The host reads chunks until done, then either returns the text to the user or runs the tool calls and asks again with their results in messages.

Parameters:

Name	Type	Description	Default
`request`	`AssistantModelRequest`	Conversation, tools and schema for this answer.	required
`ctx`	`AssistantModelContext`	Who asks, in which language, and how long the host waits.	required

Returns:

Type	Description
`AsyncIterator[AssistantModelChunk]`	The answer in pieces.

AssistantModelRequest ¶

Bases: TypedDict

One request to a plugin model: the conversation so far plus what the model may do with it.

modelId `instance-attribute` ¶

modelId: str

Model id from AssistantModelSpec["id"].

system `instance-attribute` ¶

system: list[str]

Instructions that precede the conversation.

messages `instance-attribute` ¶

messages: list[AssistantModelMessage]

The conversation, oldest first. The host has already trimmed it to contextTokens.

tools `instance-attribute` ¶

tools: list[AssistantModelTool]

Tools the model may call. Empty when the run allows none.

outputSchema `instance-attribute` ¶

outputSchema: NotRequired[dict[str, Any]]

JSON Schema the answer has to match. Only set for models whose spec says structuredOutput.

maxOutputTokens `instance-attribute` ¶

maxOutputTokens: NotRequired[int]

Upper bound for the answer in tokens, when the host has one.

AssistantModelSpec ¶

Bases: TypedDict

One model a plugin offers to the assistant. The user picks it under Settings, Assistant like any other model; the host never sees a key or an endpoint of it.

Example

on_device: AssistantModelSpec = {
    "id": "apple-on-device",
    "name": "Apple on-device",
    "contextTokens": 8192,
    "vision": True,
    "toolCalling": True,
    "structuredOutput": True,
    "note": "Runs on this Mac, nothing leaves the house.",
}

id `instance-attribute` ¶

id: str

Model id, unique within the plugin and stable across restarts: the settings store it.

name `instance-attribute` ¶

name: str

Name shown in the model list.

contextTokens `instance-attribute` ¶

contextTokens: int

Hard context limit of the model in tokens. The host plans prompt, tools and history with it.

vision `instance-attribute` ¶

vision: bool

The model reads images. Without it the host sends text only.

toolCalling `instance-attribute` ¶

toolCalling: bool

The model calls tools. Without it the assistant answers from the conversation alone.

structuredOutput `instance-attribute` ¶

structuredOutput: bool

The model honors a JSON schema. Without it the host asks for free text and parses defensively.

toolRouting `instance-attribute` ¶

toolRouting: NotRequired[bool]

The host picks the tools for a question with a short call of its own and puts only those in front of the model. For a small model that calls the tools it sees but does not work through a catalog of fifty.

note `instance-attribute` ¶

note: NotRequired[str]

One line shown next to the model, for example where it runs or what it costs.

AssistantModelStatus ¶

Bases: TypedDict

Whether the plugin can answer right now. A model that has to be downloaded or loaded first reports ready: False with a line the settings page shows, and a progress while it knows one.

Example

{"ready": False, "message": "macOS is downloading the model", "progress": 0.62}

ready `instance-attribute` ¶

ready: bool

The models of this plugin can answer right now.

message `instance-attribute` ¶

message: NotRequired[str]

One line for the user: what the plugin is waiting for, or what it is doing.

progress `instance-attribute` ¶

progress: NotRequired[float]

How far the loading got, between 0 and 1.

AssistantModelText ¶

Bases: TypedDict

Text part of a message.

type `instance-attribute` ¶

type: Literal['text']

Always text.

text `instance-attribute` ¶

text: str

The text itself.

AssistantModelTextChunk ¶

Bases: TypedDict

A piece of the answer text.

type `instance-attribute` ¶

type: Literal['text']

Always text.

delta `instance-attribute` ¶

delta: str

The text generated since the last chunk.

AssistantModelTool ¶

Bases: TypedDict

A tool the model may call in this request.

name `instance-attribute` ¶

name: str

Name the model uses in its call.

description `instance-attribute` ¶

description: str

What the tool does, written for the model.

inputSchema `instance-attribute` ¶

inputSchema: dict[str, Any]

JSON Schema object of the arguments.

AssistantModelToolCall ¶

Bases: TypedDict

A tool call: which tool with which arguments.

id `instance-attribute` ¶

id: str

Id the host uses to match the result, unique within the request.

name `instance-attribute` ¶

name: str

Name of the tool as declared in AssistantModelRequest["tools"].

arguments `instance-attribute` ¶

arguments: dict[str, Any]

Arguments as parsed JSON, matching the tool's input schema.

AssistantModelToolCallChunk ¶

Bases: TypedDict

A tool call the model asks for, with complete arguments.

type `instance-attribute` ¶

type: Literal['tool_call']

Always tool_call.

call `instance-attribute` ¶

call: AssistantModelToolCall

The call itself.

AssistantModelUsageChunk ¶

Bases: TypedDict

What the answer consumed.

type `instance-attribute` ¶

type: Literal['usage']

Always usage.

promptTokens `instance-attribute` ¶

promptTokens: int

Tokens the request consumed.

completionTokens `instance-attribute` ¶

completionTokens: int

Tokens the answer consumed.

PluginCapability ¶

Bases: StrEnum

Permission a plugin requests so it can call a host-provided system feature. Each capability gates one outgoing SDK call. Calls without the matching capability are rejected by the host.

PublishNotifications `class-attribute` `instance-attribute` ¶

PublishNotifications = 'publishNotifications'

Allows api.notificationManager.publish. Without it the host drops published notifications and logs an error.

PluginContract ¶

Bases: TypedDict

Manifest contract a plugin declares so the host knows what it does and what it needs at load time. Validated before the plugin is started.

name `instance-attribute` ¶

name: str

Stable, unique identifier: registry key, log prefix and storage namespace.

role `instance-attribute` ¶

role: PluginRole

Role of the plugin (see :class:PluginRole).

provides `instance-attribute` ¶

provides: list[SensorType]

Sensor types the plugin produces. Empty for hubs and pure camera-controllers, required for sensor providers.

consumes `instance-attribute` ¶

consumes: list[SensorType]

Sensor types the plugin reads from other plugins (e.g. a face plugin consuming camera video frames).

interfaces `instance-attribute` ¶

interfaces: list[PluginInterface]

Capability flags the plugin implements (see :class:PluginInterface).

capabilities `instance-attribute` ¶

capabilities: NotRequired[list[PluginCapability]]

Permissions the plugin requests to call host system features (see :class:PluginCapability).

pythonVersion `instance-attribute` ¶

pythonVersion: NotRequired[PythonVersion]

Required Python interpreter version for Python plugins. Ignored by Node and Go plugins.

dependencies `instance-attribute` ¶

dependencies: NotRequired[list[str]]

Extra dependencies installed into the plugin's runtime (Go module paths, PyPI or npm names).

PluginInfo ¶

Bases: TypedDict

Lightweight handle identifying an installed plugin, used in RPC payloads and managers to refer to the plugin without shipping its full state.

id `instance-attribute` ¶

id: str

Unique runtime ID assigned by the host (stable across restarts).

name `instance-attribute` ¶

name: str

Plugin package name (matches PluginContract.name).

contract `instance-attribute` ¶

contract: PluginContract

Full contract the plugin was loaded with.

running `instance-attribute` ¶

running: NotRequired[bool]

True while the plugin's process is started; set by getPluginsByInterface.

PluginInterface ¶

Bases: StrEnum

Capability flags a plugin advertises in its contract.

The host uses these to decide which RPC handlers to wire up and which UI affordances to show.

MotionDetection `class-attribute` `instance-attribute` ¶

MotionDetection = 'MotionDetection'

Implements MotionDetectionInterface (video-based motion detection).

ObjectDetection `class-attribute` `instance-attribute` ¶

ObjectDetection = 'ObjectDetection'

Implements ObjectDetectionInterface (e.g. person, vehicle, animal).

AudioDetection `class-attribute` `instance-attribute` ¶

AudioDetection = 'AudioDetection'

Implements AudioDetectionInterface (event/keyword audio detection).

FaceDetection `class-attribute` `instance-attribute` ¶

FaceDetection = 'FaceDetection'

Implements FaceDetectionInterface (face localisation + embeddings). Matching against enrolled faces happens in the NVR.

LicensePlateDetection `class-attribute` `instance-attribute` ¶

LicensePlateDetection = 'LicensePlateDetection'

Implements LicensePlateDetectionInterface (plate localisation + OCR).

ClassifierDetection `class-attribute` `instance-attribute` ¶

ClassifierDetection = 'ClassifierDetection'

Implements ClassifierDetectionInterface (generic image classification emitting attribute/label pairs).

ClipDetection `class-attribute` `instance-attribute` ¶

ClipDetection = 'ClipDetection'

Implements ClipDetectionInterface (CLIP image and text embeddings used for semantic search).

FaceEmbedding `class-attribute` `instance-attribute` ¶

FaceEmbedding = 'FaceEmbedding'

Implements FaceEmbeddingInterface (turns a face crop into a vector). Matching against enrolled faces happens in the NVR.

PersonEmbedding `class-attribute` `instance-attribute` ¶

PersonEmbedding = 'PersonEmbedding'

Implements PersonEmbeddingInterface (turns the crop of a person into a vector of their appearance). Searching happens in the NVR.

Segmentation `class-attribute` `instance-attribute` ¶

Segmentation = 'Segmentation'

Implements SegmentationInterface (outlines the object at a box in a picture).

DiscoveryProvider `class-attribute` `instance-attribute` ¶

DiscoveryProvider = 'DiscoveryProvider'

Implements DiscoveryProvider (network scan + adoption). Only valid for camera-controlling roles.

SensorDiscovery `class-attribute` `instance-attribute` ¶

SensorDiscovery = 'SensorDiscovery'

Implements SensorDiscoveryProvider (external sensor inventory the user adopts from).

NVR `class-attribute` `instance-attribute` ¶

NVR = 'NVR'

Implements NVRInterface (events and recordings). Exactly one plugin per host fills this role at runtime.

Notifier `class-attribute` `instance-attribute` ¶

Notifier = 'Notifier'

Implements NotifierInterface, so the NotificationManager can dispatch notifications to this plugin.

OAuthCapable `class-attribute` `instance-attribute` ¶

OAuthCapable = 'OAuthCapable'

Implements the OAuthCapable base interface plus at least one of the flow sub-interfaces below.

OAuthDeviceFlow `class-attribute` `instance-attribute` ¶

OAuthDeviceFlow = 'OAuthDeviceFlow'

Implements OAuthDeviceFlowCapable (RFC 8628 Device Authorization Grant).

OAuthAuthCodeFlow `class-attribute` `instance-attribute` ¶

OAuthAuthCodeFlow = 'OAuthAuthCodeFlow'

Implements OAuthAuthCodeFlowCapable (Authorization Code Flow + PKCE).

OAuthClientCredentials `class-attribute` `instance-attribute` ¶

OAuthClientCredentials = 'OAuthClientCredentials'

Implements OAuthClientCredentialsCapable (user-supplied client_id + client_secret).

AssistantTools `class-attribute` `instance-attribute` ¶

AssistantTools = 'AssistantTools'

Implements AssistantToolProvider, so the assistant can offer the plugin's tools to the model.

AssistantModels `class-attribute` `instance-attribute` ¶

AssistantModels = 'AssistantModels'

Implements AssistantModelProvider, so its models appear in the assistant settings.

PluginRole ¶

Bases: StrEnum

Role a plugin plays in the system. The role decides which lifecycle hooks the host invokes and which contract validations apply.

Hub `class-attribute` `instance-attribute` ¶

Hub = 'hub'

Cross-camera aggregator (smart-home bridge, recorder). Owns no cameras and provides no sensors.

SensorProvider `class-attribute` `instance-attribute` ¶

SensorProvider = 'sensorProvider'

Adds sensors to cameras owned by other plugins, for example a detector running on foreign video frames.

CameraController `class-attribute` `instance-attribute` ¶

CameraController = 'cameraController'

Manages cameras and their media streams: stream URLs, PTZ, snapshots. Provides no sensors for foreign cameras.

CameraAndSensorProvider `class-attribute` `instance-attribute` ¶

CameraAndSensorProvider = 'cameraAndSensorProvider'

Manages cameras and exposes sensors, on its own cameras and, with consumes set, on foreign ones.

Service `class-attribute` `instance-attribute` ¶

Service = 'service'

Serves camera.ui itself, for example with a language model. Touches no cameras and no sensors, and is never assigned to one.

AudioDetectionInterface ¶

Bases: Protocol

Implemented by plugins that perform audio event or keyword detection.

testAudioDetection `async` ¶

testAudioDetection(audio_data: bytes, metadata: AudioMetadata, config: dict[str, Any]) -> AudioDetectionPluginResponse | None

Run detection on an audio buffer captured by the UI test panel; metadata carries the input MIME type.

detectAudio `async` ¶

detectAudio(audio: AudioFrameData, config: dict[str, Any] | None = None) -> AudioDetectionPluginResponse | None

Run detection on a pre-decoded audio frame. Called from automation / benchmark pipelines.

audioDetectionSettings `async` ¶

audioDetectionSettings() -> list[JsonSchema] | None

Return the JSON schema used to render the audio-detection settings form in the UI, or None for no schema.

AudioDetectionPluginResponse ¶

Bases: TypedDict

Result of an audio detection run.

detected `instance-attribute` ¶

detected: bool

True when the run produced at least one detection.

detections `instance-attribute` ¶

detections: list[Detection]

Detected audio events.

decibels `instance-attribute` ¶

decibels: NotRequired[float]

Loudness of the analysed buffer in dBFS.

AudioMetadata ¶

Bases: TypedDict

Audio metadata passed to audio detector test methods.

mimeType `instance-attribute` ¶

mimeType: Literal['audio/mpeg', 'audio/wav', 'audio/ogg']

Container format of the audio buffer.

BasePlugin ¶

BasePlugin(logger: LoggerService, api: PluginAPI, storage: DeviceStorage[StorageT])

Bases: ABC, Generic[StorageT]

Base class every plugin extends.

It wires up the three dependencies the host injects (logger, PluginAPI, DeviceStorage) and declares the lifecycle methods the host calls on the plugin.

The host calls :meth:configureCameras once at startup with every camera already assigned to this plugin, then :meth:onCameraAdded / :meth:onCameraReleased as the user adds or removes cameras at runtime. StorageT types storage.values so plugin code gets autocompletion for its own settings shape.

Example

class MyPlugin(BasePlugin[MyStorageValues]):
    async def configureCameras(self, cameras: list[CameraDevice]) -> None:
        self.model_path = self.storage.values.get("model_path")
        for camera in cameras:
            await self.onCameraAdded(camera)

    async def onCameraAdded(self, camera: CameraDevice) -> None:
        self.state[camera.id] = await self.attach(camera)

    async def onCameraReleased(self, camera_id: str) -> None:
        self.state.pop(camera_id, None)

storage_schema `property` ¶

storage_schema: list[JsonSchema]

Override to register a JSON schema for the plugin-level settings form rendered in the UI. Default: no schema.

configureCameras `abstractmethod` `async` ¶

configureCameras(cameras: list[CameraDevice]) -> None

Called once on startup with every camera already assigned to this plugin. Attach handlers, open vendor sessions, warm up models here. Raising aborts plugin startup.

Parameters:

Name	Type	Description	Default
`cameras`	`list[CameraDevice]`	Cameras already assigned to this plugin.	required

onCameraAdded `abstractmethod` `async` ¶

onCameraAdded(camera: CameraDevice) -> None

Called whenever a camera is assigned to this plugin at runtime, after a discovery adoption (:meth:DiscoveryProvider.onAdoptCamera) or after the user re-assigns an existing camera. Set up the same per-camera state as in :meth:configureCameras.

Parameters:

Name	Type	Description	Default
`camera`	`CameraDevice`	The camera device that was added.	required

onCameraReleased `abstractmethod` `async` ¶

onCameraReleased(cameraId: str) -> None

Called when a camera is unassigned from this plugin or deleted from the system. Release per-camera resources (sessions, timers, decoders) before returning.

Parameters:

Name	Type	Description	Default
`cameraId`	`str`	ID of the camera that was released.	required

configureSensors `async` ¶

configureSensors(sensors: list[SensorLike]) -> None

Called once on startup with every sensor this plugin may consume: sensors whose type is listed in contract.consumes and that are exposed. Each sensor carries type, assignedCameraIds and connected, so consumers decide rendering purely from that data. Optional, only bridge plugins override it.

Parameters:

Name	Type	Description	Default
`sensors`	`list[SensorLike]`	Consumable sensors known at startup.	required

onSensorAdded `async` ¶

onSensorAdded(sensor: SensorLike) -> None

Called when a sensor enters this plugin's consumable view at runtime: it was created, became exposed, or its type became consumable.

Parameters:

Name	Type	Description	Default
`sensor`	`SensorLike`	The sensor that appeared.	required

onSensorReleased `async` ¶

onSensorReleased(sensorId: str) -> None

Called when a sensor permanently leaves the consumable view: it was deleted or unexposed. Plugin connectivity does NOT fire this, watch sensor.onConnectedChanged for that.

Parameters:

Name	Type	Description	Default
`sensorId`	`str`	Persistent id of the sensor that left.	required

ClassifierDetectionInterface ¶

Bases: Protocol

Implemented by plugins that run a generic image classifier and emit attribute/label pairs (e.g. weather, scene, activity).

testClassifierDetection `async` ¶

testClassifierDetection(image_data: bytes, metadata: ImageMetadata, config: dict[str, Any]) -> ClassifierDetectionPluginResponse | None

Run classification on a single image captured by the UI test panel and return the result for preview rendering.

detectClassifications `async` ¶

detectClassifications(frame: VideoFrameData, config: dict[str, Any] | None = None) -> ClassifierDetectionPluginResponse | None

Run classification on a pre-decoded video frame.

classifierDetectionSettings `async` ¶

classifierDetectionSettings() -> list[JsonSchema] | None

Return the JSON schema for the classifier-detection settings form in the UI, or None for no schema.

ClassifierDetectionPluginResponse ¶

Bases: TypedDict

Result of a classifier detection run.

detected `instance-attribute` ¶

detected: bool

True when the run produced at least one classification.

detections `instance-attribute` ¶

detections: list[ClassifierDetection]

Attribute/label pairs the classifier emitted.

ClipDetectionInterface ¶

Bases: Protocol

Implemented by plugins that generate CLIP image and text embeddings used for semantic search over recorded events.

testClipEmbedding `async` ¶

testClipEmbedding(image_data: bytes, metadata: ImageMetadata, config: dict[str, Any]) -> ClipDetectionPluginResponse | None

Run the CLIP image branch on a single image captured by the UI test panel.

detectClipEmbedding `async` ¶

detectClipEmbedding(frame: VideoFrameData, config: dict[str, Any] | None = None) -> ClipDetectionPluginResponse | None

Run the CLIP image branch on a pre-decoded video frame.

embedImages `async` ¶

embedImages(images: list[bytes], config: dict[str, Any] | None = None) -> list[ClipDetectionPluginResponse | None]

Run the CLIP image branch over a batch of encoded images (JPEG/PNG).

One result per input in the same order, None where decoding or embedding failed. Meant for re-indexing stored images after an embedding-model change.

getTextEmbedding `async` ¶

getTextEmbedding(text: str) -> ClipTextEmbeddingResult

Run the CLIP text branch and return a vector usable for semantic-search queries against stored image embeddings.

getTextEmbeddings `async` ¶

getTextEmbeddings(text: str) -> list[ClipTextEmbeddingResult]

Run the CLIP text branch once per embedding space the plugin can currently serve.

The configured search model comes first. Lets semantic search also cover embeddings produced by an older model during a transition.

clipSettings `async` ¶

clipSettings() -> list[JsonSchema] | None

Return the JSON schema for the CLIP settings form in the UI, or None for no schema.

ClipDetectionPluginResponse ¶

Bases: TypedDict

Result of a CLIP image embedding run.

embeddings `instance-attribute` ¶

embeddings: list[ClipEmbedding]

Embedding vectors generated for the input.

embeddingModel `instance-attribute` ¶

embeddingModel: str

Model that produced the embeddings; consumers must not mix models.

scoreBand `instance-attribute` ¶

scoreBand: list[float]

[floor, ceiling] of raw text-image cosine scores for this model; consumers map scores to a 0..1 relevance scale and treat a missing band as score 0.

ClipTextEmbeddingResult ¶

Bases: TypedDict

Result of a CLIP text embedding request.

embedding `instance-attribute` ¶

embedding: list[float]

Embedding vector for the query text.

embeddingModel `instance-attribute` ¶

embeddingModel: str

Model that produced the embedding; consumers must not mix models.

scoreBand `instance-attribute` ¶

scoreBand: list[float]

[floor, ceiling] of raw text-image cosine scores for this model; consumers map scores to a 0..1 relevance scale and treat a missing band as score 0.

DiscoveryProvider ¶

Bases: Protocol

Implemented by plugins that can scan the network for new cameras and adopt them. Only plugins with a camera-controlling role (CameraController or CameraAndSensorProvider) are queried for discovery.

onDiscoverCameras `async` ¶

onDiscoverCameras() -> list[DiscoveredCamera]

Scan the network and return the cameras the plugin can offer for adoption. Called by the host on demand (UI rescan button) or on a polling schedule.

Returns:

Type	Description
`list[DiscoveredCamera]`	Cameras currently discoverable by this plugin.

onGetCameraSettings `async` ¶

onGetCameraSettings(camera: DiscoveredCamera) -> list[JsonSchemaWithoutCallbacks]

Return a JSON schema describing the form fields (credentials, transport options, ...) the user must fill in to adopt this discovered camera.

Parameters:

Name	Type	Description	Default
`camera`	`DiscoveredCamera`	The discovered camera the user is about to adopt.	required

Returns:

Type	Description
`list[JsonSchemaWithoutCallbacks]`	Schema for the adoption form.

onAdoptCamera `async` ¶

onAdoptCamera(camera: DiscoveredCamera, cameraSettings: dict[str, object]) -> CameraConfig

Probe the device with the user-provided settings and return the camera configuration the host should persist. The host then creates the camera and invokes :meth:BasePlugin.onCameraAdded on the plugin.

Parameters:

Name	Type	Description	Default
`camera`	`DiscoveredCamera`	The discovered camera being adopted.	required
`cameraSettings`	`dict[str, object]`	Values entered into the adoption form.	required

Returns:

Type	Description
`CameraConfig`	Final camera configuration for the host to persist.

FaceDetectionInterface ¶

Bases: Protocol

Implemented by plugins that locate faces and emit per-face embeddings. The NVR owns matching against enrolled faces, the plugin only emits raw detections and embeddings.

testFaceDetection `async` ¶

testFaceDetection(image_data: bytes, metadata: ImageMetadata, config: dict[str, Any]) -> FaceDetectionPluginResponse | None

Run face detection on a single image captured by the UI test panel and return the result for preview rendering.

A number in config["threshold"] is the lowest confidence to return, without it the plugin's default applies.

detectFaces `async` ¶

detectFaces(frame: VideoFrameData, config: dict[str, Any] | None = None) -> FaceDetectionPluginResponse | None

Run face detection on a pre-decoded video frame.

faceDetectionSettings `async` ¶

faceDetectionSettings() -> list[JsonSchema] | None

Return the JSON schema for the face-detection settings form in the UI, or None for no schema.

FaceDetectionPluginResponse ¶

Bases: TypedDict

Result of a face detection run.

detected `instance-attribute` ¶

detected: bool

True when the run produced at least one detection.

detections `instance-attribute` ¶

detections: list[FaceDetection]

Located faces. Vectors come from a face-embedding plugin, not from here.

FaceEmbeddingInterface ¶

Bases: Protocol

Implemented by plugins that turn a face crop into an embedding vector.

Split from face detection so the two can run on different hosts: locating a face is cheap, embedding it is not.

embedFaceImages `async` ¶

embedFaceImages(images: list[bytes], config: dict[str, Any] | None = None, landmarks: list[list[Point] | None] | None = None) -> list[FaceEmbeddingPluginResponse | None]

Embed a batch of encoded images (JPEG/PNG), each showing one face.

One result per input in the same order. An empty embedding means the picture holds no face this model can use, None that the plugin could not run at all: the caller may drop such a picture, so the two must not be mixed up. Meant for re-embedding stored pictures after an embedding-model change. landmarks holds the points an earlier result returned for the same picture, one entry per image: with them the face is not searched again.

faceEmbeddingSettings `async` ¶

faceEmbeddingSettings() -> list[JsonSchema] | None

Return the JSON schema for the face-embedding settings form in the UI, or None for no schema.

FaceEmbeddingPluginResponse ¶

Bases: TypedDict

Result of a face embedding run on a single image.

embedding `instance-attribute` ¶

embedding: list[float]

Embedding vector for the face, empty when no face could be embedded.

embeddingModel `instance-attribute` ¶

embeddingModel: str

Model that produced the embedding; consumers must not mix models.

landmarks `instance-attribute` ¶

landmarks: NotRequired[list[Point]]

The five points the face was aligned on, in 0 - 1 of the input image: right eye, left eye, nose, right and left mouth corner.

quality `instance-attribute` ¶

quality: NotRequired[float]

How sure the model is that those points sit on a face (0 - 1).

ImageMetadata ¶

Bases: TypedDict

Image metadata passed to detector test methods.

width `instance-attribute` ¶

width: int

Image width in pixels.

height `instance-attribute` ¶

height: int

Image height in pixels.

LicensePlateDetectionInterface ¶

Bases: Protocol

Implemented by plugins that locate license plates and run OCR on them.

testLicensePlateDetection `async` ¶

testLicensePlateDetection(image_data: bytes, metadata: ImageMetadata, config: dict[str, Any]) -> LicensePlateDetectionPluginResponse | None

Run detection on a single image captured by the UI test panel and return the result for preview rendering.

A number in config["threshold"] is the lowest confidence to return, without it the plugin's default applies.

detectLicensePlates `async` ¶

detectLicensePlates(frame: VideoFrameData, config: dict[str, Any] | None = None) -> LicensePlateDetectionPluginResponse | None

Run detection on a pre-decoded video frame.

licensePlateDetectionSettings `async` ¶

licensePlateDetectionSettings() -> list[JsonSchema] | None

Return the JSON schema for the license-plate-detection settings form in the UI, or None for no schema.

LicensePlateDetectionPluginResponse ¶

Bases: TypedDict

Result of a license plate detection run.

detected `instance-attribute` ¶

detected: bool

True when the run produced at least one detection.

detections `instance-attribute` ¶

detections: list[LicensePlateDetection]

Detected plates with their OCR text.

MotionDetectionInterface ¶

Bases: Protocol

Implemented by plugins that perform video-based motion detection. The host invokes :meth:testMotionDetection from the UI test panel and :meth:detectMotion from automation / benchmark pipelines.

testMotionDetection `async` ¶

testMotionDetection(video_data: bytes, config: dict[str, Any]) -> MotionDetectionPluginResponse | None

Run detection on a raw video buffer captured by the UI test panel and return the result for preview rendering.

detectMotion `async` ¶

detectMotion(frames: list[VideoFrameData], config: dict[str, Any] | None = None) -> MotionDetectionPluginResponse | None

Run detection on already-decoded frames, supplied by automation / benchmark pipelines to avoid re-encoding.

motionDetectionSettings `async` ¶

motionDetectionSettings() -> list[JsonSchema] | None

Return the JSON schema used to render the motion-detection settings form in the UI, or None for no schema.

MotionDetectionPluginResponse ¶

Bases: TypedDict

Result of a motion detection run.

detected `instance-attribute` ¶

detected: bool

True when the run produced at least one detection.

detections `instance-attribute` ¶

detections: list[Detection]

Motion regions found in the input.

videoData `instance-attribute` ¶

videoData: NotRequired[bytes]

Annotated re-encoded clip for the UI test panel, when the plugin renders one.

ObjectDetectionInterface ¶

Bases: Protocol

Implemented by plugins that perform object detection (person, vehicle, animal, ...).

testObjectDetection `async` ¶

testObjectDetection(image_data: bytes, metadata: ImageMetadata, config: dict[str, Any]) -> ObjectDetectionPluginResponse | None

Run detection on a single image captured by the UI test panel; metadata carries the image dimensions.

A number in config["threshold"] is the lowest confidence to return, without it the plugin's default applies.

detectObjects `async` ¶

detectObjects(frame: VideoFrameData, config: dict[str, Any] | None = None) -> ObjectDetectionPluginResponse | None

Run detection on a pre-decoded video frame. Called from automation / benchmark pipelines.

objectDetectionSettings `async` ¶

objectDetectionSettings() -> list[JsonSchema] | None

Return the JSON schema used to render the object-detection settings form in the UI, or None for no schema.

ObjectDetectionPluginResponse ¶

Bases: TypedDict

Result of an object detection run.

detected `instance-attribute` ¶

detected: bool

True when the run produced at least one detection.

detections `instance-attribute` ¶

detections: list[Detection]

Detected objects with label, score and bounding box.

PersonEmbeddingInterface ¶

Bases: Protocol

Implemented by plugins that turn the crop of a person into a vector of their appearance.

The NVR stores and searches the vectors, the plugin only emits them.

embedPersonImages `async` ¶

embedPersonImages(images: list[bytes], config: dict[str, Any] | None = None) -> list[PersonEmbeddingPluginResponse | None]

Embed a batch of encoded images (JPEG/PNG), each showing one person cut tight around their box.

One result per input in the same order. An empty embedding means the picture could not be used, None that the plugin could not run at all. Meant for searching by a picture the user picked.

personEmbeddingSettings `async` ¶

personEmbeddingSettings() -> list[JsonSchema] | None

Return the JSON schema for the person-embedding settings form in the UI, or None for no schema.

PersonEmbeddingPluginResponse ¶

Bases: TypedDict

Result of a person embedding run on a single image.

embedding `instance-attribute` ¶

embedding: list[float]

Embedding vector for the person, empty when the picture could not be embedded.

embeddingModel `instance-attribute` ¶

embeddingModel: str

Model that produced the embedding; consumers must not mix models.

SegmentationImage ¶

Bases: TypedDict

A picture to outline an object in, with the object's box.

image `instance-attribute` ¶

image: bytes

Encoded image (JPEG/PNG).

box `instance-attribute` ¶

box: BoundingBox

Box of the object to outline, normalized to the image.

SegmentationInterface ¶

Bases: Protocol

Implemented by plugins that outline objects: a mask that separates an object from its background.

The frame-based side is the segmenter sensor.

segmentImages `async` ¶

segmentImages(images: list[SegmentationImage], config: dict[str, Any] | None = None) -> list[SegmentationPluginResponse | None]

Outline the object at box in each picture.

One result per input in the same order, None when the plugin could not run at all. config holds the values of the segmentation settings form.

segmentationSettings `async` ¶

segmentationSettings() -> list[JsonSchema] | None

Return the JSON schema for the segmentation settings form in the UI, or None for no schema.

SegmentationPluginResponse ¶

Bases: TypedDict

Result of a segmentation run on a single image.

mask `instance-attribute` ¶

mask: NotRequired[ObjectMask]

The object's outline, missing when the model found no object at the box.

SensorDiscoveryProvider ¶

Bases: Protocol

Implemented by sensor-providing plugins that face an external inventory (Home Assistant entities, vendor accessories) the user picks from instead of the plugin importing everything. Declare PluginInterface.SensorDiscovery in the contract.

The host owns the adoption: it lists what the plugin discovers, creates the sensor record when the user adopts, hands the plugin its adopted sensors on every start and tells it when the user deletes one. The plugin keeps no list of its own.

Identity is the source's stable id (DiscoveredSensor.id), never an address; the same id means the same sensor across restarts and renames.

onDiscoverSensors `async` ¶

onDiscoverSensors() -> list[DiscoveredSensor]

Return every sensor the source currently offers. The host drops the ones already adopted, the plugin does not filter. Called on demand (Sensors page, rescan) and on a polling schedule while the page is open.

Returns:

Type	Description
`list[DiscoveredSensor]`	Sensors currently discoverable by this plugin.

configureAdoptedSensors `async` ¶

configureAdoptedSensors(sensors: list[AdoptedSensor]) -> list[Sensor[Any, Any, Any]]

Called once at startup, right after configureCameras(), with every sensor the user adopted from this plugin. Build and return one runtime sensor per record, always, from the record's type and name alone; the host binds each returned sensor to its record by nativeId. Report what the source looks like on the sensor itself (setSourceState, setAddress) as soon as you know: a record the source no longer has gets setSourceState("removed"), it is never dropped here.

Parameters:

Name	Type	Description	Default
`sensors`	`list[AdoptedSensor]`	The adopted sensors of this plugin.	required

Returns:

Type	Description
`list[Sensor[Any, Any, Any]]`	One runtime sensor per adopted record.

onSensorAdopted `async` ¶

onSensorAdopted(sensor: AdoptedSensor) -> Sensor[Any, Any, Any]

The user adopted a discovered sensor and the host created its record. Build and return the runtime sensor for it, same as one entry of configureAdoptedSensors().

Parameters:

Name	Type	Description	Default
`sensor`	`AdoptedSensor`	The adopted sensor.	required

Returns:

Type	Description
`Sensor[Any, Any, Any]`	The runtime sensor to bind to the record.

onSensorUnadopted `async` ¶

onSensorUnadopted(nativeId: str) -> None

The user deleted an adopted sensor. The host has already unbound the runtime sensor; drop whatever the plugin still holds for it. The entity shows up as discovered again on the next scan.

Parameters:

Name	Type	Description	Default
`nativeId`	`str`	The `DiscoveredSensor.id` the adoption used.	required

ServicePlugin ¶

ServicePlugin(logger: LoggerService, api: PluginAPI, storage: DeviceStorage[StorageT])

Bases: BasePlugin[StorageT]

Base class for a plugin with the role :attr:PluginRole.Service: it serves camera.ui itself and never touches cameras, so the camera lifecycle hooks are already implemented as no-ops.

Example

class MyModels(ServicePlugin):
    async def assistantModels(self) -> list[AssistantModelSpec]:
        return [
            {
                "id": "local",
                "name": "Local model",
                "contextTokens": 8192,
                "vision": False,
                "toolCalling": False,
                "structuredOutput": True,
            }
        ]

configureCameras `async` ¶

configureCameras(cameras: list[CameraDevice]) -> None

Does nothing, a service plugin owns no cameras.

onCameraAdded `async` ¶

onCameraAdded(camera: CameraDevice) -> None

Does nothing, a service plugin owns no cameras.

onCameraReleased `async` ¶

onCameraReleased(cameraId: str) -> None

Does nothing, a service plugin owns no cameras.

Notification ¶

Bases: TypedDict

Payload published via api.notificationManager.publish or routed by the host. Plugins fill the user-visible fields; the host stamps the message id, timestamp and source identifier on receive.

title `instance-attribute` ¶

title: str

Headline shown by every notifier.

subtitle `instance-attribute` ¶

subtitle: NotRequired[str]

Optional second bold line, honoured natively on iOS; other notifiers may fold it into the body.

body `instance-attribute` ¶

body: NotRequired[str]

Optional secondary text.

severity `instance-attribute` ¶

severity: NotRequired[Severity]

Drives DND / Critical-Alerts behaviour and Quiet-Hours bypass. Defaults to :attr:Severity.Info.

tag `instance-attribute` ¶

tag: NotRequired[str]

Collapse-key (e.g. motion:cam-1). The host replaces an older entry with the same tag in the in-app list. Delivery is not throttled: every publish is sent. Notifiers may map it to a platform collapse-id.

thumbnail `instance-attribute` ¶

thumbnail: NotRequired[bytes]

Optional inline JPEG attached to the notification.

imageUrl `instance-attribute` ¶

imageUrl: NotRequired[str]

Publicly-fetchable URL to a rich image (e.g. a detection snapshot). Preferred over inline thumbnail bytes when a URL is available; empty renders text-only.

videoUrl `instance-attribute` ¶

videoUrl: NotRequired[str]

Publicly-fetchable URL to a short MP4 clip. Notifiers that can render video (iOS attachments) prefer it over imageUrl; everything else ignores it, so always send the image alongside. Keep clips small: the receiving phone downloads inside a tight OS budget.

deepLink `instance-attribute` ¶

deepLink: NotRequired[str]

Router-relative path for mobile / web tap-handlers (e.g. /cameras/cam-1). No host, no scheme.

data `instance-attribute` ¶

data: NotRequired[dict[str, str]]

Plugin-specific context (cameraId, eventId, plugin-defined keys), string values only.

adminOnly `instance-attribute` ¶

adminOnly: NotRequired[bool]

Restricts delivery to users with the master or admin role. Use it for operational alerts (camera offline, disk full, plugin failures) so they don't reach guests the instance is merely shared with. Defaults to False.

silent `instance-attribute` ¶

silent: NotRequired[bool]

Delivers without sound, vibration or badge increment: meant for publishes that replace an earlier notification with the same tag (e.g. a richer description superseding the initial alert). The banner still updates. Ignored when severity is :attr:Severity.Critical. Defaults to False.

NotifierDevice ¶

Bases: TypedDict

A push-target managed by a notifier plugin (one phone, one chat, ...).

Devices are owned by the plugin that registered them; the manager queries plugins for their device list rather than maintaining a shared registry.

id `instance-attribute` ¶

id: str

Plugin-assigned device id, unique within the notifier.

ownerUserId `instance-attribute` ¶

ownerUserId: str

User the device belongs to.

name `instance-attribute` ¶

name: str

Display name shown in the UI.

active `instance-attribute` ¶

active: bool

False while the user has muted this device; the manager skips it.

metadata `instance-attribute` ¶

metadata: NotRequired[dict[str, Any]]

Plugin-specific extras (push tokens, chat ids, platform hints).

NotifierInterface ¶

Bases: Protocol

Implemented by plugins that deliver notifications.

The NotificationManager invokes these methods over RPC. Plugins own their device storage, the manager never persists devices itself.

get_devices `async` ¶

get_devices(owner_user_ids: list[str]) -> list[NotifierDevice]

Return the devices this notifier knows for the given users, each carrying its ownerUserId. Return [] when the notifier is unavailable (e.g. invalid license). Called often, keep it cheap.

get_device `async` ¶

get_device(device_id: str) -> NotifierDevice | None

Return a single device by id, or None if not found.

send_notification `async` ¶

send_notification(device_ids: list[str], n: Notification) -> None

Deliver a notification to the given devices in one call. Errors are logged, a failing notifier never aborts the fan-out.

register_device `async` ¶

register_device(owner_user_id: str, input: dict[str, Any]) -> NotifierDevice

Create a new device. input is plugin-specific JSON the manager forwards opaquely.

revoke_device `async` ¶

revoke_device(device_id: str) -> None

Permanently remove a device. Called when the user revokes it through their notifier-specific UI.

update_device `async` ¶

update_device(device_id: str, patch: dict[str, Any]) -> NotifierDevice | None

Mutate name / active on an existing device. Return None if the id isn't ours so the manager can probe the next plugin.

notificationSettings `async` ¶

notificationSettings() -> list[JsonSchema] | None

Return the JSON schema used to render the notifier's settings form in the UI, or None for no schema.

Severity ¶

Bases: StrEnum

Classifies how urgent a Notification is.

Notifiers map this to platform-specific delivery characteristics; the host bypasses user-configured Quiet Hours for Critical.

Info `class-attribute` `instance-attribute` ¶

Info = 'info'

Standard notification, default delivery (sound + banner).

Warn `class-attribute` `instance-attribute` ¶

Warn = 'warn'

Heightened attention; notifiers may use a different sound or colour.

Error `class-attribute` `instance-attribute` ¶

Error = 'error'

Failure or action-required notification.

Critical `class-attribute` `instance-attribute` ¶

Critical = 'critical'

Highest-priority delivery on supporting notifiers; bypasses Quiet Hours.

OAuthAuthCodeFlowCapable ¶

Bases: OAuthCapable, Protocol

Implemented by plugins that use the Authorization Code Flow with PKCE. The host opens the auth URL and forwards the IdP redirect's code+state to completeAuthCodeFlow.

startAuthCodeFlow `async` ¶

startAuthCodeFlow(scope: list[str]) -> OAuthState

Build the authorization URL for the given scopes; return the awaiting-user state (authUrl set).

completeAuthCodeFlow `async` ¶

completeAuthCodeFlow(code: str, state: str) -> OAuthState

Exchange the IdP-returned code for tokens after validating state.

cancelAuthCodeFlow `async` ¶

cancelAuthCodeFlow() -> None

Abort an in-progress authorization-code flow.

OAuthCapable ¶

Bases: Protocol

Base interface every OAuth-capable plugin implements, alongside at least one flow sub-interface. IdP-agnostic: the plugin brings its own endpoint config and knows nothing about the host's internals.

getOAuthMetadata `async` ¶

getOAuthMetadata() -> OAuthMetadata

Return IdP display info, scope descriptions and the implemented flow sub-interfaces.

getOAuthState `async` ¶

getOAuthState() -> OAuthState

Return a snapshot of the current lifecycle state; the host polls this to mirror progress.

disconnect `async` ¶

disconnect() -> None

Revoke the current grant at the IdP and clear stored tokens.

OAuthClientCredentialsCapable ¶

Bases: OAuthCapable, Protocol

Implemented by plugins that authenticate with a user-supplied client_id + client_secret (no user redirect). The plugin validates by fetching a token immediately.

configureClientCredentials `async` ¶

configureClientCredentials(client_id: str, client_secret: str) -> OAuthState

Store the supplied credentials and fetch an initial token to validate them.

OAuthDeviceFlowCapable ¶

Bases: OAuthCapable, Protocol

Implemented by plugins whose IdP supports the RFC 8628 Device Authorization Grant. The plugin polls the IdP internally; the host only polls getOAuthState to mirror progress.

startDeviceFlow `async` ¶

startDeviceFlow(scope: list[str]) -> OAuthState

Request a device code for the given scopes and begin polling; return the awaiting-user state.

cancelDeviceFlow `async` ¶

cancelDeviceFlow() -> None

Abort an in-progress device flow.

OAuthMetadata ¶

Bases: TypedDict

Informational data the host renders in the connect dialog.

idpDisplayName `instance-attribute` ¶

idpDisplayName: str

Human name of the identity provider, e.g. cameraui.com, Spotify.

scopeDescriptions `instance-attribute` ¶

scopeDescriptions: dict[str, str]

Maps each scope to a human-readable description.

supportedFlows `instance-attribute` ¶

supportedFlows: list[PluginInterface]

Flow sub-interfaces the plugin implements, so the host knows which affordance to render.

OAuthProviderConfig ¶

Bases: TypedDict

Points the plugin's OAuth manager at an identity provider.

preset `instance-attribute` ¶

preset: NotRequired[str]

Built-in IdP endpoint set, e.g. cameraui.com. When unset, the explicit endpoints are used.

deviceAuthUrl `instance-attribute` ¶

deviceAuthUrl: NotRequired[str]

Device-authorization endpoint (used when preset is unset).

authUrl `instance-attribute` ¶

authUrl: NotRequired[str]

Authorization endpoint (used when preset is unset).

tokenUrl `instance-attribute` ¶

tokenUrl: NotRequired[str]

Token endpoint (used when preset is unset).

revokeUrl `instance-attribute` ¶

revokeUrl: NotRequired[str]

Revocation endpoint (used when preset is unset).

OAuthProviderDeclaration ¶

Bases: TypedDict

One provider a plugin integrates with. A single-provider plugin declares exactly one.

id `instance-attribute` ¶

id: str

Plugin-local provider identifier (storage-key dimension for multi-provider plugins).

provider `instance-attribute` ¶

provider: OAuthProviderConfig

IdP endpoint configuration.

clientId `instance-attribute` ¶

clientId: str

OAuth client id the plugin authenticates as.

scopes `instance-attribute` ¶

scopes: list[str]

Scopes requested for this provider.

required `instance-attribute` ¶

required: NotRequired[bool]

Whether the provider is mandatory for the plugin to function.

description `instance-attribute` ¶

description: NotRequired[str]

One-line UI hint shown alongside the connect button.

OAuthState ¶

Bases: TypedDict

Snapshot of a provider connection's lifecycle. Lives in the plugin and is the source of truth for both the host UI and downstream plugin code that needs a token. The host polls it via getOAuthState while a flow runs.

status `instance-attribute` ¶

status: OAuthStatus

Current lifecycle phase.

userCode `instance-attribute` ¶

userCode: NotRequired[str]

Device-flow user code shown to the user (set while awaiting_user).

verificationUri `instance-attribute` ¶

verificationUri: NotRequired[str]

Device-flow verification URI the user opens (set while awaiting_user).

verificationUriComplete `instance-attribute` ¶

verificationUriComplete: NotRequired[str]

Verification URI with the user code embedded, rendered as a QR code.

authUrl `instance-attribute` ¶

authUrl: NotRequired[str]

Authorization-code-flow URL the browser must open (set while awaiting_user).

userEmail `instance-attribute` ¶

userEmail: NotRequired[str]

Connected account email (set while connected).

connectedAt `instance-attribute` ¶

connectedAt: NotRequired[int]

Unix timestamp the grant was established (set while connected).

scopesGranted `instance-attribute` ¶

scopesGranted: NotRequired[list[str]]

Scopes granted by the IdP (set while connected).

errorCode `instance-attribute` ¶

errorCode: NotRequired[str]

OAuth error code (set while error): access_denied | expired_token | server_error.

errorMessage `instance-attribute` ¶

errorMessage: NotRequired[str]

Human-readable error detail (set while error).

define_assistant_tool ¶

define_assistant_tool(spec: AssistantToolSpec) -> AssistantToolSpec

Identity helper that types a tool spec, so a typo in a property type is a type-check error instead of a runtime surprise.

Parameters:

Name	Type	Description	Default
`spec`	`AssistantToolSpec`	The tool spec.	required

Returns:

Type	Description
`AssistantToolSpec`	The same spec, unchanged.

Example

set_favorite = define_assistant_tool(
    {
        "name": "set_favorite",
        "description": "Mark or unmark a recorded event as favorite.",
        "inputSchema": {
            "type": "object",
            "properties": {"eventId": {"type": "string"}, "favorite": {"type": "boolean"}},
            "required": ["eventId", "favorite"],
        },
        "approval": True,
    }
)

can_create_cameras ¶

can_create_cameras(contract: PluginContract) -> bool

Report whether the plugin can create cameras (role is CameraController or CameraAndSensorProvider). Used to gate camera-creating operations such as DiscoveryProvider adoption.

Parameters:

Name	Type	Description	Default
`contract`	`PluginContract`	Plugin contract to inspect.	required

Returns:

Type	Description
`bool`	True if the plugin may create cameras.

Example

if can_create_cameras(contract):
    enable_adoption()

can_provide_sensors_to_any_cameras ¶

can_provide_sensors_to_any_cameras(contract: PluginContract) -> bool

Report whether the plugin is allowed to add sensors to cameras owned by other plugins (true for SensorProvider and CameraAndSensorProvider). Hub and pure CameraController plugins only see their own cameras.

Parameters:

Name	Type	Description	Default
`contract`	`PluginContract`	Plugin contract to inspect.	required

Returns:

Type	Description
`bool`	True if the plugin may attach sensors to any camera.

Example

if can_provide_sensors_to_any_cameras(contract):
    list_all_cameras()

get_contract_validation_errors ¶

get_contract_validation_errors(contract: object) -> list[str]

Check the structural validity of an unknown contract object: required fields present, enum values inside the accepted sets. Returns one human-readable error per problem found, empty when the contract is valid.

Parameters:

Name	Type	Description	Default
`contract`	`object`	Untrusted candidate contract (e.g. parsed manifest JSON).	required

Returns:

Type	Description
`list[str]`	Error messages, empty if the contract is valid.

Example

errors = get_contract_validation_errors(my_contract)
if errors:
    print(f"Invalid contract: {errors}")

has_capability ¶

has_capability(contract: PluginContract, cap: PluginCapability) -> bool

Report whether the plugin requested the given capability.

Parameters:

Name	Type	Description	Default
`contract`	`PluginContract`	Plugin contract to inspect.	required
`cap`	`PluginCapability`	Capability to check (e.g. :attr:`PluginCapability.PublishNotifications`).	required

Returns:

Type	Description
`bool`	True if `cap` is listed in the contract's `capabilities`.

Example

if has_capability(contract, PluginCapability.PublishNotifications):
    allow_publish()

has_interface ¶

has_interface(contract: PluginContract, iface: PluginInterface) -> bool

Report whether the plugin implements the given capability.

Parameters:

Name	Type	Description	Default
`contract`	`PluginContract`	Plugin contract to inspect.	required
`iface`	`PluginInterface`	Interface to check (e.g. :attr:`PluginInterface.DiscoveryProvider`).	required

Returns:

Type	Description
`bool`	True if `iface` is listed in the contract's `interfaces`.

Example

if has_interface(contract, PluginInterface.DiscoveryProvider):
    start_scan()

is_hub ¶

is_hub(contract: PluginContract) -> bool

Report whether the plugin's role is Hub (a cross-camera aggregator such as a smart-home bridge or recorder, which owns no cameras of its own).

Parameters:

Name	Type	Description	Default
`contract`	`PluginContract`	Plugin contract to inspect.	required

Returns:

Type	Description
`bool`	True if the role is :attr:`PluginRole.Hub`.

Example

if is_hub(contract):
    skip_local_discovery()

is_service ¶

is_service(contract: PluginContract) -> bool

Report whether the plugin only serves camera.ui itself (role Service), which means it never appears in a camera's plugin list.

Parameters:

Name	Type	Description	Default
`contract`	`PluginContract`	Plugin contract to inspect.	required

Returns:

Type	Description
`bool`	True if the role is :attr:`PluginRole.Service`.

Example

if is_service(contract):
    skip_camera_assignment()

validate_contract_consistency ¶

validate_contract_consistency(contract: PluginContract, plugin_name: str | None = None) -> None

Enforce role-specific consistency rules on top of the structural check (e.g. SensorProvider plugins must declare at least one provided sensor; Hub plugins cannot expose sensors). Raises on the first violation.

Parameters:

Name	Type	Description	Default
`contract`	`PluginContract`	Already-structurally-valid contract.	required
`plugin_name`	`str \| None`	Optional plugin name; used to prefix error messages.	`None`

Raises:

Type	Description
`ValueError`	When the contract violates a role-specific rule.

Example

validate_contract_consistency(contract, "my-plugin")

Plugin API¶

camera_ui_sdk.plugin ¶

APIListener module-attribute ¶

AssistantToolProperty module-attribute ¶

AssistantModelChunk module-attribute ¶

AssistantModelContent module-attribute ¶

PROTOCOL_LEVEL module-attribute ¶

PythonVersion module-attribute ¶

PluginInterfaces module-attribute ¶

OAuthStatus module-attribute ¶

API_EVENT ¶

FINISH_LAUNCHING class-attribute instance-attribute ¶

SHUTDOWN class-attribute instance-attribute ¶

PluginAPI ¶

coreManager property ¶

deviceManager property ¶

sensorManager property ¶

downloadManager property ¶

notificationManager property ¶

storagePath property ¶

on ¶

once ¶

off ¶

removeListener ¶

removeAllListeners ¶

AssistantToolArrayProperty ¶

type instance-attribute ¶

items instance-attribute ¶

description instance-attribute ¶

minItems instance-attribute ¶

maxItems instance-attribute ¶

AssistantToolBooleanProperty ¶

type instance-attribute ¶

description instance-attribute ¶

AssistantToolContext ¶

userId instance-attribute ¶

role instance-attribute ¶

language instance-attribute ¶

timezone instance-attribute ¶

threadId instance-attribute ¶

AssistantToolImage ¶

data instance-attribute ¶

mimeType instance-attribute ¶

caption instance-attribute ¶

AssistantToolNumberProperty ¶

type instance-attribute ¶

description instance-attribute ¶

minimum instance-attribute ¶

maximum instance-attribute ¶

AssistantToolObjectProperty ¶

type instance-attribute ¶

properties instance-attribute ¶

description instance-attribute ¶

required instance-attribute ¶

AssistantToolProvider ¶

assistantTools async ¶

callAssistantTool async ¶

AssistantToolReference ¶

kind instance-attribute ¶

id instance-attribute ¶

label instance-attribute ¶

cameraId instance-attribute ¶

timestamp instance-attribute ¶

url instance-attribute ¶

AssistantToolResult ¶

content instance-attribute ¶

images instance-attribute ¶

references instance-attribute ¶

error instance-attribute ¶

AssistantToolSchema ¶

type instance-attribute ¶

properties instance-attribute ¶

required instance-attribute ¶

additionalProperties instance-attribute ¶

AssistantToolSpec ¶

name instance-attribute ¶

description instance-attribute ¶

inputSchema instance-attribute ¶

approval instance-attribute ¶

adminOnly instance-attribute ¶

APIListener `module-attribute` ¶

AssistantToolProperty `module-attribute` ¶

AssistantModelChunk `module-attribute` ¶

AssistantModelContent `module-attribute` ¶

PROTOCOL_LEVEL `module-attribute` ¶

PythonVersion `module-attribute` ¶

PluginInterfaces `module-attribute` ¶

OAuthStatus `module-attribute` ¶

FINISH_LAUNCHING `class-attribute` `instance-attribute` ¶

SHUTDOWN `class-attribute` `instance-attribute` ¶

coreManager `property` ¶

deviceManager `property` ¶

sensorManager `property` ¶

downloadManager `property` ¶

notificationManager `property` ¶

storagePath `property` ¶

type `instance-attribute` ¶

items `instance-attribute` ¶

description `instance-attribute` ¶

minItems `instance-attribute` ¶

maxItems `instance-attribute` ¶

type `instance-attribute` ¶

description `instance-attribute` ¶

userId `instance-attribute` ¶

role `instance-attribute` ¶

language `instance-attribute` ¶

timezone `instance-attribute` ¶

threadId `instance-attribute` ¶

data `instance-attribute` ¶

mimeType `instance-attribute` ¶

caption `instance-attribute` ¶

type `instance-attribute` ¶

description `instance-attribute` ¶

minimum `instance-attribute` ¶

maximum `instance-attribute` ¶

type `instance-attribute` ¶

properties `instance-attribute` ¶

description `instance-attribute` ¶

required `instance-attribute` ¶

assistantTools `async` ¶

callAssistantTool `async` ¶

kind `instance-attribute` ¶

id `instance-attribute` ¶

label `instance-attribute` ¶

cameraId `instance-attribute` ¶

timestamp `instance-attribute` ¶

url `instance-attribute` ¶

content `instance-attribute` ¶

images `instance-attribute` ¶

references `instance-attribute` ¶

error `instance-attribute` ¶

type `instance-attribute` ¶

properties `instance-attribute` ¶

required `instance-attribute` ¶

additionalProperties `instance-attribute` ¶

name `instance-attribute` ¶

description `instance-attribute` ¶

inputSchema `instance-attribute` ¶

approval `instance-attribute` ¶

adminOnly `instance-attribute` ¶

timeoutMs `instance-attribute` ¶

lazy `instance-attribute` ¶

type `instance-attribute` ¶

description `instance-attribute` ¶

enum `instance-attribute` ¶

format `instance-attribute` ¶

userId `instance-attribute` ¶

language `instance-attribute` ¶

timeoutMs `instance-attribute` ¶

type `instance-attribute` ¶

finish `instance-attribute` ¶

message `instance-attribute` ¶

type `instance-attribute` ¶

data `instance-attribute` ¶

mimeType `instance-attribute` ¶

role `instance-attribute` ¶

content `instance-attribute` ¶

toolCalls `instance-attribute` ¶

toolCallId `instance-attribute` ¶

assistantModels `async` ¶