Plugin Guide

A camera.ui plugin is a Node package the host loads at runtime to extend cameras with new capabilities — a detection model, a vendor camera integration, a notifier, a smart-home bridge. This guide is the single reference for shipping one. You should be comfortable with TypeScript and have the SDK installed (npm install @camera.ui/sdk).

1. Plugin anatomy

A plugin is a folder with two files the host expects:

• contract.ts — the manifest. A static object describing what the plugin is.
• index.ts — the runtime. A class extending BasePlugin, exported as default.

The minimal compilable plugin:

// contract.ts
import { PluginRole } from '@camera.ui/sdk';
import type { PluginContract } from '@camera.ui/sdk';

export const contract: PluginContract = {
  name: 'My Plugin',
  role: PluginRole.SensorProvider,
  provides: [],
  consumes: [],
  interfaces: [],
};

export default contract;

// index.ts
import { BasePlugin } from '@camera.ui/sdk';
import type { CameraDevice } from '@camera.ui/sdk';

export default class MyPlugin extends BasePlugin {
  async configureCameras(_cameras: CameraDevice[]): Promise<void> {}
  async onCameraAdded(_camera: CameraDevice): Promise<void> {}
  async onCameraReleased(_cameraId: string): Promise<void> {}
}

The host instantiates the class with three arguments: a LoggerService, a PluginAPI, and a typed DeviceStorage. Everything else — sensors, discovery, schemas — is opt-in.

2. The contract

PluginContract is the static manifest the host reads before starting the plugin. The fields:

• name — stable identifier; doubles as log prefix and storage namespace.
• role — what the plugin does at the highest level (see table below).
• provides — sensor types the plugin attaches to cameras (e.g. [SensorType.Motion]).
• consumes — sensor types the plugin reads from other plugins.
• interfaces — capability flags (DiscoveryProvider, Notifier, detection types, …).

Role	Use when
SensorProvider	You add detection or smart-home sensors to existing cameras (motion plugin, classifier, contact sensor).
CameraController	You bring your own cameras and own their streams (RTSP, ONVIF, vendor SDK), but produce no sensors.
CameraAndSensorProvider	You bring your own cameras AND want to attach sensors to them. Most vendor integrations land here.
Hub	Cloud-service integration that owns its cameras end-to-end via a vendor account, OR a bridge plugin that consumes other plugins' sensors and forwards them to an external system (HomeKit, MQTT, automations) or implements a Notifier.

Examples:

// Detection plugin attaching motion sensors
{ name: 'My Motion', role: PluginRole.SensorProvider,
  provides: [SensorType.Motion], consumes: [], interfaces: [PluginInterface.MotionDetection] }

// ONVIF-style integration with discovery
{ name: 'ACME Cameras', role: PluginRole.CameraAndSensorProvider,
  provides: [SensorType.Motion], consumes: [], interfaces: [PluginInterface.DiscoveryProvider] }

// HomeKit bridge consuming sensor state from other plugins
{ name: 'HomeKit Bridge', role: PluginRole.Hub,
  provides: [], consumes: [SensorType.Motion, SensorType.Doorbell], interfaces: [] }

// A language model for the assistant, nothing camera-shaped
{ name: 'Apple LLM', role: PluginRole.Service,
  provides: [], consumes: [], interfaces: [PluginInterface.AssistantModels] }

PluginRole.Service is the role for a plugin that serves camera.ui itself instead of cameras: it provides and consumes no sensors and never shows up in a camera's plugin list. Extend ServicePlugin instead of BasePlugin and the camera lifecycle hooks are already there as no-ops.

The SDK also carries a plugin protocol level (PROTOCOL_LEVEL). The CLI stamps the level your plugin was built against into the bundle, and the server refuses to start a plugin whose level it does not support, telling the user whether the plugin or the server needs an update. You never set it yourself: rebuilding against the current SDK is all it takes.

3. The plugin class

BasePlugin<TStorage> is generic over your storage shape so this.storage.values.X is typed. Its constructor takes the three host-injected dependencies in order: logger, api, storage.

import { API_EVENT, BasePlugin } from '@camera.ui/sdk';
import type { CameraDevice, DeviceStorage, LoggerService, PluginAPI } from '@camera.ui/sdk';

interface MyStorage {
  pollIntervalSec: number;
}

export default class MyPlugin extends BasePlugin<MyStorage> {
  private state = new Map<string, MyCameraState>();

  constructor(logger: LoggerService, api: PluginAPI, storage: DeviceStorage<MyStorage>) {
    super(logger, api, storage);
    this.api.on(API_EVENT.SHUTDOWN, () => this.shutdown());
  }

  async configureCameras(cameras: CameraDevice[]): Promise<void> {
    for (const camera of cameras) await this.attach(camera);
  }

  async onCameraAdded(camera: CameraDevice): Promise<void> {
    await this.attach(camera);
  }

  async onCameraReleased(cameraId: string): Promise<void> {
    this.state.get(cameraId)?.dispose();
    this.state.delete(cameraId);
  }

  private async attach(_camera: CameraDevice): Promise<void> { /* ... */ }
  private shutdown(): void { /* drop timers, close sockets */ }
}

Things to internalize:

• configureCameras runs once at startup with the cameras already assigned to the plugin. A rejection aborts plugin startup.
• onCameraAdded runs whenever the user assigns a new camera at runtime. Set up the same per-camera state as configureCameras.
• onCameraReleased runs when a camera is removed or reassigned. Drop timers, close vendor sessions.
• A Map<cameraId, X> is the conventional pattern for per-camera state — cheap to look up, trivial to clean up.
• API_EVENT.SHUTDOWN fires when the host tears the plugin down (reload, server stop). Listeners must release everything synchronously enough for the host to stop the process. There is also API_EVENT.FINISH_LAUNCHING, fired once after configureCameras returns — useful for kicking off background work that should wait until the camera set is stable.

4. Sensors

A sensor is the smallest smart-home unit in camera.ui. Detection sensors push results from analyzing video; control sensors expose user-toggleable hardware (lights, sirens, locks); event sensors fire one-shot triggers (doorbell).

Every sensor is a persisted entity of its own. The plugin supplies the durable identity, everything else belongs to the user: camera assignments, display name and whether the sensor is exported to consumers. Where a sensor comes from decides who owns it:

• camera.addSensor(sensor) — the sensor belongs to this camera's hardware (spotlight, siren, battery, PTZ). The host locks the assignment to the camera; users cannot re-assign it. Pass a nativeId (e.g. the upstream device id) in the constructor options so the host can reconcile the sensor across restarts by (pluginId, nativeId). Without it, identity falls back to (type, name) and a rename creates a new sensor.
• Standalone sensors (smart plugs, hubs, smart-home entities) — the plugin never registers these. One exists once the user adopted it from what the plugin offers, and the host hands it back to the plugin on every start through the SensorDiscoveryProvider hooks (see 6.4). The user assigns it to zero or more cameras in the UI.

For detection, subclass the matching *DetectorSensor and implement the detect method. The host pushes one frame at the configured rate:

import { MotionDetectorSensor } from '@camera.ui/sdk';
import type { MotionResult, VideoFrameData } from '@camera.ui/sdk';

class MyMotionSensor extends MotionDetectorSensor {
  constructor() {
    super('My Motion');
  }

  async detectMotion(_frame: VideoFrameData): Promise<MotionResult> {
    // analyze frame.data, return detections
    return { detected: false, detections: [] };
  }
}

Then attach it from the plugin's attach() helper:

const sensor = new MyMotionSensor();
await camera.addSensor(sensor);

Other detector base classes follow the same shape but expect a batch (frames: VideoFrameData[]) and an abstract modelSpec getter: ObjectDetectorSensor (detectObjects(frame) — singular here), FaceDetectorSensor.detectFaces(frames), LicensePlateDetectorSensor.detectLicensePlates(frames), AudioDetectorSensor.detectAudio(audio), ClassifierDetectorSensor.detectClassifications(frames), ClipDetectorSensor.detectEmbeddings(frames). Smart-home sensors expose semantic methods instead — e.g. LightControl gives you setOn() / setOff() / setBrightness(value), ContactSensor gives you setDetected(value), DoorbellTrigger gives you trigger(). You construct them, register them, and then call those methods when your hardware reports a change.

The host may send up to three frames to detectObjects at once, so the method must not keep per-call state on the sensor. A detector that needs one call at a time sets _rpcConcurrency = {} in its class.

A standalone sensor works exactly the same once it runs, only nobody registers it: you build it from the adopted record the host hands you and return it, and the host binds it to the record by nativeId:

import { LockControl } from '@camera.ui/sdk';
import type { AdoptedSensor } from '@camera.ui/sdk';

async onSensorAdopted(record: AdoptedSensor) {
  return new LockControl(record.name, { nativeId: record.nativeId, address: record.address });
}

Every sensor also has a lifecycle pair: onStart() runs once the sensor is registered and its storage is ready — start pollers, subscriptions, timers there. onStop() is the counterpart and runs on removal, plugin shutdown and cleanup:

class PollingContact extends ContactSensor {
  private timer?: NodeJS.Timeout;

  protected override onStart(): void {
    this.timer = setInterval(() => this.poll(), 10_000);
  }

  protected override onStop(): void {
    clearInterval(this.timer);
  }
}

removeSensor only unregisters the runtime instance — the persisted entity stays and shows as disconnected until the user deletes it in the UI. The host removes camera-bound sensors automatically when a camera is released; your onCameraReleased hook just needs to drop your reference.

5. Storage and configuration schema

User-facing settings live in storage. Storage is split into two scopes — plugin-level and sensor-level — each with its own storageSchema getter that returns a JSON schema array. The host renders the schemas as form fields and persists the values; you read them via storage.values.X.

Plugin-level schemas appear on the plugin's settings page. Override the getter on your BasePlugin subclass.

override get storageSchema(): JsonSchema[] {
  return [
    {
      type: 'number',
      key: 'pollIntervalSec',
      title: 'Poll interval (seconds)',
      description: 'How often background work runs',
      defaultValue: 30,
      minimum: 5,
      maximum: 300,
      step: 5,
      store: true,
      required: true,
      onSet: async (newValue, oldValue) => {
        this.logger.log(`Poll interval ${oldValue} -> ${newValue}`);
        this.reschedule();
      },
    },
  ];
}

Sensor-level schemas appear on the camera detail page next to that one sensor. Override the getter on your Sensor subclass.

class ConfigurableMotion extends MotionDetectorSensor<{ sensitivity: number; mode: 'fast' | 'accurate' }> {
  override get storageSchema(): JsonSchema[] {
    return [
      {
        type: 'number',
        key: 'sensitivity',
        title: 'Sensitivity',
        description: 'Higher = trigger on smaller motion',
        defaultValue: 50,
        minimum: 0, maximum: 100, step: 1,
        store: true,
        onSet: async () => this.reconfigure(),
      },
      {
        type: 'string',
        key: 'mode',
        title: 'Mode',
        description: 'Trade-off between speed and accuracy',
        defaultValue: 'fast',
        enum: ['fast', 'accurate'],
        store: true,
      },
      {
        type: 'button',
        key: 'reset',
        title: 'Reset to defaults',
        description: 'Restore sensitivity / mode',
        color: 'danger',
        onSet: async () => {
          await this.storage.setValue('sensitivity', 50);
          await this.storage.setValue('mode', 'fast');
        },
      },
    ];
  }

  async detectMotion(_frame: VideoFrameData): Promise<MotionResult> {
    const sensitivity = this.storage.values.sensitivity;
    const _mode = this.storage.values.mode;
    void sensitivity;
    return { detected: false, detections: [] };
  }

  private reconfigure(): void { /* re-warm caches */ }
}

Field types:

• number — slider/input with optional minimum, maximum, step.
• string — text input; add enum: [...] for a dropdown, format: 'password' to mask, format: 'image' / 'qrCode' for media display.
• boolean — toggle.
• button — fires onSet() on click; stores no value. Useful for actions like "Test connection" or "Reset".

onSet(newValue, oldValue) runs after the host has persisted the new value. Use it to re-warm caches, restart sessions, or anything else that depends on the changed setting. It's async, but the host doesn't block UI on it — keep work scoped to the plugin.

For the full schema reference (conditional visibility, submit handlers with toast feedback, array fields), see src/storage/index.ts in the SDK.

6. Optional interfaces

BasePlugin covers the lifecycle. Specific capabilities are unlocked by implementing one of the optional interfaces and listing it in contract.interfaces. The rest of this section shows each one with a working snippet.

6.1 DiscoveryProvider

Let users scan and adopt cameras. Three methods: onDiscoverCameras returns adoption candidates, onGetCameraSettings returns the schema for the adoption form, onAdoptCamera resolves the form values into a CameraConfig for the host to persist. Available only for camera-controlling roles (CameraController, CameraAndSensorProvider).

import { API_EVENT, BasePlugin } from '@camera.ui/sdk';
import type {
  CameraConfig, CameraDevice, DeviceStorage, DiscoveredCamera, DiscoveryProvider,
  JsonSchemaWithoutCallbacks, LoggerService, PluginAPI,
} from '@camera.ui/sdk';

interface FakeDevice { id: string; name: string; manufacturer: string; model: string; host: string; }
const FAKE_DEVICES: FakeDevice[] = [
  { id: 'fake-001', name: 'Front Door', manufacturer: 'ACME', model: 'X1', host: '192.0.2.10' },
];

export default class CameraProvider extends BasePlugin implements DiscoveryProvider {
  private cameras = new Map<string, CameraDevice>();

  constructor(logger: LoggerService, api: PluginAPI, storage: DeviceStorage) {
    super(logger, api, storage);
    this.api.on(API_EVENT.SHUTDOWN, () => this.cameras.clear());
  }

  async configureCameras(cameras: CameraDevice[]): Promise<void> {
    for (const c of cameras) this.cameras.set(c.id, c);
  }
  async onCameraAdded(camera: CameraDevice): Promise<void> { this.cameras.set(camera.id, camera); }
  async onCameraReleased(cameraId: string): Promise<void> { this.cameras.delete(cameraId); }

  async onDiscoverCameras(): Promise<DiscoveredCamera[]> {
    const adopted = new Set(Array.from(this.cameras.values()).map((c) => c.nativeId));
    return FAKE_DEVICES
      .filter((d) => !adopted.has(d.id))
      .map(({ id, name, manufacturer, model }) => ({ id, name, manufacturer, model }));
  }

  async onGetCameraSettings(_camera: DiscoveredCamera): Promise<JsonSchemaWithoutCallbacks[]> {
    return [
      { type: 'string', key: 'username', title: 'Username', description: '', required: true },
      { type: 'string', key: 'password', title: 'Password', description: '', format: 'password', required: true },
    ];
  }

  async onAdoptCamera(camera: DiscoveredCamera, settings: Record<string, unknown>): Promise<CameraConfig> {
    const device = FAKE_DEVICES.find((d) => d.id === camera.id);
    if (!device) throw new Error(`Unknown device: ${camera.id}`);
    const u = encodeURIComponent(String(settings.username ?? ''));
    const p = encodeURIComponent(String(settings.password ?? ''));
    return {
      name: device.name,
      nativeId: device.id,
      info: { manufacturer: device.manufacturer, model: device.model },
      sources: [{
        name: 'main',
        role: 'high-resolution',
        urls: [`rtsp://${u}:${p}@${device.host}/stream0`],
        useForSnapshot: true,
        hotMode: true,
        preload: true,
      }],
    };
  }
}

For asynchronous discovery (cloud OAuth callbacks, mDNS bursts), you can also push candidates directly into the UI without waiting for the next poll: await api.deviceManager.pushDiscoveredCameras([...]).

6.2 NotifierInterface

Register as a notification target so the host's NotificationManager can dispatch through you. The plugin owns its device list — the manager queries through these methods rather than maintaining a shared registry.

import { API_EVENT, BasePlugin } from '@camera.ui/sdk';
import type {
  CameraDevice, DeviceStorage, LoggerService, Notification,
  NotifierDevice, NotifierInterface, PluginAPI,
} from '@camera.ui/sdk';

export default class MyNotifier extends BasePlugin implements NotifierInterface {
  private devices: NotifierDevice[] = [];

  constructor(logger: LoggerService, api: PluginAPI, storage: DeviceStorage) {
    super(logger, api, storage);
    this.api.on(API_EVENT.SHUTDOWN, () => this.devices.splice(0));
  }

  async configureCameras(_cameras: CameraDevice[]): Promise<void> {}
  async onCameraAdded(_camera: CameraDevice): Promise<void> {}
  async onCameraReleased(_cameraId: string): Promise<void> {}

  async getDevices(ownerUserId: string): Promise<NotifierDevice[]> {
    return this.devices.filter((d) => d.ownerUserId === ownerUserId);
  }

  async getDevice(deviceId: string): Promise<NotifierDevice | null> {
    return this.devices.find((d) => d.id === deviceId) ?? null;
  }

  async sendNotification(deviceId: string, n: Notification): Promise<void> {
    const device = this.devices.find((d) => d.id === deviceId);
    if (!device?.active) return;
    this.logger.log(`[${device.name}] ${n.title}: ${n.body ?? ''}`);
  }

  async registerDevice(ownerUserId: string, input: Record<string, unknown>): Promise<NotifierDevice> {
    const device: NotifierDevice = {
      id: crypto.randomUUID(),
      ownerUserId,
      type: String(input.type ?? 'mobile'),
      name: String(input.name ?? 'Unnamed device'),
      active: true,
    };
    this.devices.push(device);
    return device;
  }

  async revokeDevice(deviceId: string): Promise<void> {
    const idx = this.devices.findIndex((d) => d.id === deviceId);
    if (idx >= 0) this.devices.splice(idx, 1);
  }

  async updateDevice(deviceId: string, patch: Record<string, unknown>): Promise<NotifierDevice | null> {
    const device = this.devices.find((d) => d.id === deviceId);
    if (!device) return null;
    if (typeof patch.name === 'string') device.name = patch.name;
    if (typeof patch.active === 'boolean') device.active = patch.active;
    return device;
  }
}

Notification.tag is a collapse key for dedup at both the manager and notifier level — multiple events with the same tag inside the throttle window collapse into one notification. Notification.severity is 'info' | 'warn' | 'error' | 'critical'; map critical to whatever DND-bypass mechanism your platform offers.

6.3 Detection interfaces

The seven detection capabilities follow a single pattern. Each interface has:

• A required test* method invoked by the UI when the user uploads a clip/image and clicks "Test" — it accepts raw media bytes plus metadata and returns the same result shape the per-frame detector would.
• An optional pre-processed detect* method invoked by automations and benchmarks — it accepts already-decoded frames.
• An optional *Settings() method that returns a schema for the detection configuration UI.

All seven (MotionDetectionInterface, ObjectDetectionInterface, AudioDetectionInterface, FaceDetectionInterface, LicensePlateDetectionInterface, ClassifierDetectionInterface, ClipDetectionInterface) share that shape. ClipDetectionInterface additionally requires getTextEmbedding(text) for semantic search.

Motion as a worked example:

import type {
  MotionDetectionInterface, MotionDetectionPluginResponse,
  VideoFrameData, JsonSchema,
} from '@camera.ui/sdk';

export default class MotionPlugin extends BasePlugin implements MotionDetectionInterface {
  // ... lifecycle methods omitted ...

  async testMotionDetection(
    videoData: Buffer | Uint8Array,
    _config: Record<string, unknown>,
  ): Promise<MotionDetectionPluginResponse | undefined> {
    const detections = await this.runOnEncodedClip(videoData);
    return { detected: detections.length > 0, detections };
  }

  async detectMotion(
    frames: VideoFrameData[],
    _config?: Record<string, unknown>,
  ): Promise<MotionDetectionPluginResponse | undefined> {
    const detections = await this.runOnFrames(frames);
    return { detected: detections.length > 0, detections };
  }

  async motionDetectionSettings(): Promise<JsonSchema[] | undefined> {
    return [
      { type: 'number', key: 'minArea', title: 'Min area (%)', description: '',
        defaultValue: 1, minimum: 0, maximum: 100, step: 1, store: true },
    ];
  }

  private async runOnEncodedClip(_v: Buffer | Uint8Array): Promise<[]> { return []; }
  private async runOnFrames(_f: VideoFrameData[]): Promise<[]> { return []; }
}

The image-based detection interfaces (ObjectDetectionInterface, FaceDetectionInterface, LicensePlateDetectionInterface, ClassifierDetectionInterface, ClipDetectionInterface) take an extra metadata: ImageMetadata argument with width / height. The audio interface takes metadata: AudioMetadata with the mimeType. Otherwise the wiring is identical to the motion example above — add the matching PluginInterface.X flag to the contract and implement the test* / optional detect* / optional settings trio.

A detection plugin almost always implements both halves: the appropriate *DetectorSensor subclass (Section 4) for the live pipeline, AND the matching *DetectionInterface here for UI test dialogs and ad-hoc benchmarks.

6.4 SensorDiscoveryProvider

Let users pick sensors from an inventory the plugin faces (Home Assistant entities, a vendor account) instead of importing all of it. List PluginInterface.SensorDiscovery in the contract. The host owns the adoption: it lists what you discover on the Sensors page, creates the record when the user adopts, hands you your adopted sensors on every start and tells you when the user deletes one. The plugin keeps no list of its own and persists nothing about adoption.

Four methods:

• onDiscoverSensors returns everything the source currently offers. Don't drop what is already adopted, the host filters.
• configureAdoptedSensors runs once at startup, right after configureCameras, with every record the user adopted from this plugin. Return one sensor per record, always, built from the record's type and name with nativeId: record.nativeId; that is what the host binds by. A record the source no longer knows still gets its sensor, marked setSourceState('removed').
• onSensorAdopted runs when the user adopts a discovered sensor. Same as one entry of configureAdoptedSensors.
• onSensorUnadopted runs after the user deleted the sensor. The host has already unbound it, drop what you still hold.

Identity is the source's stable id, never its address. DiscoveredSensor.id becomes the record's nativeId, and the sensor keeps its record, assignments, automations and history for as long as that id stays the same. A Home Assistant entity-registry id or an MQTT unique_id qualifies; an entity_id does not, it changes on rename and would leave an orphan plus a new sensor behind. Carry the mutable address in address, it is shown next to the name.

The sensor reports its source on itself: setSourceState('connected' | 'unavailable' | 'removed'), and setAddress(address) after a rename. Neither deletes anything. An adopted sensor is deleted by the user only.

import { BasePlugin, ContactSensor, LockControl, SensorType } from '@camera.ui/sdk';
import type { AdoptedSensor, DiscoveredSensor, Sensor, SensorDiscoveryProvider } from '@camera.ui/sdk';

export default class SmartHome extends BasePlugin implements SensorDiscoveryProvider {
  private sensors = new Map<string, Sensor<any>>();

  async onDiscoverSensors(): Promise<DiscoveredSensor[]> {
    const entities = await this.hub.listEntities();
    return entities.map((e) => ({
      id: e.registryId, // stable, survives a rename
      address: e.entityId, // mutable, display only
      name: e.name,
      type: e.domain === 'lock' ? SensorType.Lock : SensorType.Contact,
      room: e.area,
    }));
  }

  async configureAdoptedSensors(records: AdoptedSensor[]): Promise<Sensor<any>[]> {
    return records.map((record) => this.build(record));
  }

  async onSensorAdopted(record: AdoptedSensor): Promise<Sensor<any>> {
    return this.build(record);
  }

  async onSensorUnadopted(nativeId: string): Promise<void> {
    this.sensors.delete(nativeId);
  }

  private build(record: AdoptedSensor): Sensor<any> {
    const sensor = record.type === SensorType.Lock
      ? new LockControl(record.name, { nativeId: record.nativeId, address: record.address })
      : new ContactSensor(record.name, { nativeId: record.nativeId, address: record.address });
    this.sensors.set(record.nativeId, sensor);
    this.hub.watch(record.nativeId, (entity) => {
      if (!entity) {
        sensor.setSourceState('removed');
        return;
      }
      sensor.setAddress(entity.entityId);
      sensor.setSourceState(entity.available ? 'connected' : 'unavailable');
    });
    return sensor;
  }
}

7. Logging

this.logger is a LoggerService. The methods are log, warn, error, success, debug, trace, and attention — each accepts a list of arguments joined with spaces by the host. debug and trace are gated by host log level.

this.logger.log('Plugin started');
this.logger.success(`Connected to vendor cloud as ${user}`);
this.logger.warn('Falling back to substream');
this.logger.error('Adopt failed:', err);

Every CameraDevice exposes camera.logger — same interface, but the output is prefixed with the camera name. Prefer it over this.logger whenever the message is about a specific camera:

async onCameraAdded(camera: CameraDevice): Promise<void> {
  camera.logger.log('attached');
}

8. Consuming sensors from other plugins

Bridge plugins (HomeKit, MQTT, automations) don't look at cameras to find sensors — they receive a consumable view: every sensor whose type is listed in contract.consumes and that the user exported. The host delivers it through three optional BasePlugin hooks:

• configureSensors(sensors) — once at startup with the view known at that point.
• onSensorAdded(sensor) — a sensor entered the view at runtime: it was created, became exposed, or its type became consumable.
• onSensorReleased(sensorId) — a sensor permanently left the view: deleted or unexposed. Plugin connectivity does NOT fire this; watch sensor.onConnectedChanged for that.

Each sensor arrives as a SensorLike: a read-only view with type, displayName, connected, assignedCameraIds, assignmentLocked, plus getValue() / getValues() and the observables onPropertyChanged, onConnectedChanged, onAssignmentChanged. Consumers decide rendering purely from that data — e.g. a light with assignmentLocked is camera hardware, an unassigned contact sensor is a standalone device.

A complete Hub consumer that reacts to motion and doorbell state:

import { API_EVENT, BasePlugin, MotionProperty, SensorType } from '@camera.ui/sdk';
import type {
  CameraDevice, DeviceStorage, Disposable, LoggerService, PluginAPI, SensorLike,
} from '@camera.ui/sdk';

export default class HubConsumer extends BasePlugin {
  private subs = new Map<string, Disposable[]>();

  constructor(logger: LoggerService, api: PluginAPI, storage: DeviceStorage) {
    super(logger, api, storage);
    this.api.on(API_EVENT.SHUTDOWN, () => this.disposeAll());
  }

  async configureCameras(_cameras: CameraDevice[]): Promise<void> {}
  async onCameraAdded(_camera: CameraDevice): Promise<void> {}
  async onCameraReleased(_cameraId: string): Promise<void> {}

  async configureSensors(sensors: SensorLike[]): Promise<void> {
    for (const sensor of sensors) this.bind(sensor);
  }

  async onSensorAdded(sensor: SensorLike): Promise<void> {
    this.bind(sensor);
  }

  async onSensorReleased(sensorId: string): Promise<void> {
    for (const sub of this.subs.get(sensorId) ?? []) sub.dispose();
    this.subs.delete(sensorId);
  }

  private bind(sensor: SensorLike): void {
    const props = sensor.onPropertyChanged.subscribe(({ property, value }) => {
      if (sensor.type === SensorType.Motion && property === MotionProperty.Detected && value) {
        this.logger.log(`${sensor.displayName}: motion started`);
      }
    });

    const conn = sensor.onConnectedChanged.subscribe((connected) => {
      this.logger.log(`${sensor.displayName}: ${connected ? 'online' : 'offline'}`);
    });

    this.subs.set(sensor.id, [props, conn]);
  }

  private disposeAll(): void {
    for (const list of this.subs.values()) for (const sub of list) sub.dispose();
    this.subs.clear();
  }
}

Two things to notice:

• The bridge keeps one Disposable[] per sensor and disposes it in onSensorReleased. subscribe() keeps the subscription alive until you call .dispose().
• The bind happens in both configureSensors AND onSensorAdded for sensors that show up after startup. Same shape as the camera lifecycle.

For direct plugin-to-plugin RPC (e.g. asking a face plugin to compute embeddings on demand), use api.coreManager.connectToPlugin(name). It returns a typed proxy of the target plugin including any optional interfaces it implements:

const face = await this.api.coreManager.connectToPlugin('Face Plugin');
const result = await face?.testFaceDetection(jpegBytes, { width: 640, height: 480 }, {});

Use api.coreManager.getPluginsByInterface(PluginInterface.FaceDetection) to discover candidate plugins by capability rather than by name. A plugin in that list can be installed but not started; skip it when its running is false. A call to a method no process serves is retried for about 3.5 s, because the plugin may be starting. To probe an optional method without that wait, connect with connectToPlugin(name, { noRetry: true }); the call then fails at once, so wrap it in try.

To use one of the language models configured in camera.ui, call api.coreManager.assistantAsk(). The admin allows the plugin under Settings, Assistant, Plugin access and picks the model; camera.ui runs the request, so the plugin never holds a key. assistantAccess() tells you whether the plugin is allowed, and the assistantModelChanged core event fires when that changes:

const access = await this.api.coreManager.assistantAccess();
if (access.allowed) {
  const answer = await this.api.coreManager.assistantAsk({
    system: 'Answer with one word.',
    prompt: 'Is there a person in this picture?',
    images: [{ data: jpegBytes, mimeType: 'image/jpeg' }],
  });
  const text = answer.ok ? answer.text : undefined;
}

A plugin can also go the other way and bring a model: list PluginInterface.AssistantModels in the contract and implement AssistantModelProvider. Its models then appear in the provider list of the assistant settings, with no key and no address to fill in. Worth it for a model camera.ui cannot reach itself: one without an HTTP API, one behind an unusual authentication, or one that only runs on the machine the plugin runs on (a Mac worker, a GPU box).

async assistantModels(): Promise<AssistantModelSpec[]> {
  return [{ id: 'local', name: 'Local model', contextTokens: 8192, vision: true, toolCalling: true, structuredOutput: true, note: 'Runs on this machine.' }];
}

async *assistantGenerate(request: AssistantModelRequest, ctx: AssistantModelContext): AsyncGenerator<AssistantModelChunk> {
  for await (const delta of this.session.stream(request, ctx.timeoutMs)) {
    yield { type: 'text', delta };
  }
  yield { type: 'usage', promptTokens: 120, completionTokens: 40 };
  yield { type: 'done', finish: 'stop' };
}

Every request carries the whole conversation, the plugin keeps no state. contextTokens is not decoration: camera.ui plans prompt, tools and history with it, so a wrong number ends in truncated requests. A tool call is yielded once its arguments are complete ({ type: 'tool_call', call }), the host runs the tool and asks again with the result in messages. A small model that calls the tools it sees but gets lost in a long catalog sets toolRouting: true: camera.ui then picks the tools for each question with a short call of its own and offers only those. The user can switch it per model.