camera.ui Node SDK

Appearance

@camera.ui/sdk / sensor / AudioDetectorSensor

Abstract Class: AudioDetectorSensor<TStorage> ​

Defined in: sensor/audio.ts:200

Audio detector that receives audio frames from the backend pipeline. Extend this class and implement detectAudio to classify audio events. The backend resamples and buffers audio to match modelSpec before each call.

Extends ​

Type Parameters ​

TStorage ​

TStorage extends object = Record<string, any>

Constructors ​

Constructor ​

new AudioDetectorSensor<TStorage>(name?, options?): AudioDetectorSensor<TStorage>

Defined in: sensor/audio.ts:81

Parameters ​

name? ​

string = 'Audio Sensor'

options? ​

SensorOptions

Returns ​

AudioDetectorSensor<TStorage>

Inherited from ​

AudioSensor.constructor

Properties ​

_requiresFrames ​

_requiresFrames: boolean = true

Defined in: sensor/audio.ts:201

Overrides ​

AudioSensor._requiresFrames

category ​

readonly category: Sensor = SensorCategory.Sensor

Defined in: sensor/audio.ts:77

Inherited from ​

AudioSensor.category

modelSpec ​

abstract readonly modelSpec: AudioModelSpec

Defined in: sensor/audio.ts:203

name ​

readonly name: string

Defined in: sensor/base.ts:223

Inherited from ​

AudioSensor.name

onAssignmentChanged ​

readonly onAssignmentChanged: Observable<readonly string[]>

Defined in: sensor/base.ts:250

Inherited from ​

AudioSensor.onAssignmentChanged

onCapabilitiesChanged ​

readonly onCapabilitiesChanged: Observable<string[]>

Defined in: sensor/base.ts:242

Inherited from ​

AudioSensor.onCapabilitiesChanged

onConnectedChanged ​

readonly onConnectedChanged: Observable<boolean>

Defined in: sensor/base.ts:252

Inherited from ​

AudioSensor.onConnectedChanged

onPropertyChanged ​

readonly onPropertyChanged: Observable<PropertyChangeOf<AudioSensorProperties>>

Defined in: sensor/base.ts:240

Inherited from ​

AudioSensor.onPropertyChanged

type ​

readonly type: Audio = SensorType.Audio

Defined in: sensor/audio.ts:76

Inherited from ​

AudioSensor.type

Accessors ​

assignedCameraIds ​

Get Signature ​

get assignedCameraIds(): readonly string[]

Defined in: sensor/base.ts:293

Returns ​

readonly string[]

Inherited from ​

AudioSensor.assignedCameraIds

assignmentLocked ​

Get Signature ​

get assignmentLocked(): boolean

Defined in: sensor/base.ts:297

Returns ​

boolean

Inherited from ​

AudioSensor.assignmentLocked

capabilities ​

Get Signature ​

get capabilities(): TCapability[]

Defined in: sensor/base.ts:316

Returns ​

TCapability[]

Set Signature ​

set capabilities(value): void

Defined in: sensor/base.ts:321

Set capabilities, deduplicated, and notify backend plus local listeners.

Parameters ​
value ​

TCapability[]

Returns ​

void

Inherited from ​

AudioSensor.capabilities

connected ​

Get Signature ​

get connected(): boolean

Defined in: sensor/base.ts:289

Returns ​

boolean

Inherited from ​

AudioSensor.connected

decibels ​

Get Signature ​

get decibels(): number

Defined in: sensor/audio.ts:99

Returns ​

number

Inherited from ​

AudioSensor.decibels

detected ​

Get Signature ​

get detected(): boolean

Defined in: sensor/audio.ts:91

Returns ​

boolean

Inherited from ​

AudioSensor.detected

detections ​

Get Signature ​

get detections(): Detection[]

Defined in: sensor/audio.ts:95

Returns ​

Detection[]

Inherited from ​

AudioSensor.detections

displayName ​

Get Signature ​

get displayName(): string

Defined in: sensor/base.ts:281

Returns ​

string

Inherited from ​

AudioSensor.displayName

id ​

Get Signature ​

get id(): string

Defined in: sensor/base.ts:269

Returns ​

string

Inherited from ​

AudioSensor.id

isAssigned ​

Get Signature ​

get isAssigned(): boolean

Defined in: sensor/base.ts:285

Returns ​

boolean

Inherited from ​

AudioSensor.isAssigned

nativeId ​

Get Signature ​

get nativeId(): string | undefined

Defined in: sensor/base.ts:273

Returns ​

string | undefined

Inherited from ​

AudioSensor.nativeId

origin ​

Get Signature ​

get origin(): string | undefined

Defined in: sensor/base.ts:277

Returns ​

string | undefined

Inherited from ​

AudioSensor.origin

pluginId ​

Get Signature ​

get pluginId(): string | undefined

Defined in: sensor/base.ts:301

Returns ​

string | undefined

Inherited from ​

AudioSensor.pluginId

props ​

Get Signature ​

get protected props(): Readonly<TProperties>

Defined in: sensor/base.ts:546

Returns ​

Readonly<TProperties>

Inherited from ​

AudioSensor.props

storage ​

Get Signature ​

get storage(): DeviceStorage<TStorage>

Defined in: sensor/base.ts:305

Returns ​

DeviceStorage<TStorage>

Inherited from ​

AudioSensor.storage

storageSchema ​

Get Signature ​

get storageSchema(): JsonSchema[]

Defined in: sensor/base.ts:312

Returns ​

JsonSchema[]

Inherited from ​

AudioSensor.storageSchema

Methods ​

clearDetections() ​

clearDetections(): void

Defined in: sensor/audio.ts:140

Explicitly clear audio detection state (detected = false, detections = []).

Returns ​

void

Example ​

ts
sensor.clearDetections();

Inherited from ​

AudioSensor.clearDetections

detectAudio() ​

abstract detectAudio(audio): Promise<AudioResult>

Defined in: sensor/audio.ts:206

Analyze a single audio frame for events. Called by the backend at the configured cadence.

Parameters ​

audio ​

AudioFrameData

Returns ​

Promise<AudioResult>

getValue() ​

Call Signature ​

getValue<K>(property): AudioSensorProperties[K] | undefined

Defined in: sensor/base.ts:345

Get the current value of a sensor property. Calling the generic overload with a property enum value gives a properly typed result.

Type Parameters ​
K ​

K extends keyof AudioSensorProperties

Parameters ​
property ​

K

Returns ​

AudioSensorProperties[K] | undefined

Inherited from ​

AudioSensor.getValue

Call Signature ​

getValue(property): unknown

Defined in: sensor/base.ts:346

Get the current value of a sensor property. Calling the generic overload with a property enum value gives a properly typed result.

Parameters ​
property ​

string

Returns ​

unknown

Inherited from ​

AudioSensor.getValue

getValues() ​

getValues(): Readonly<TProperties>

Defined in: sensor/base.ts:362

Get a read-only snapshot of all property values.

Returns ​

Readonly<TProperties>

Shallow copy of every property currently held by the sensor.

Example ​

ts
const snapshot = sensor.getValues();
console.log(snapshot);

Inherited from ​

AudioSensor.getValues

hasCapability() ​

hasCapability(capability): boolean

Defined in: sensor/base.ts:388

Check whether the sensor advertises a capability.

Parameters ​

capability ​

string

Capability flag to look for.

Returns ​

boolean

True if the sensor currently advertises it.

Example ​

ts
const dimmable = sensor.hasCapability('brightness');

Inherited from ​

AudioSensor.hasCapability

onStart() ​

protected onStart(): void | Promise<void>

Defined in: sensor/base.ts:651

Lifecycle hook, called once the sensor is registered and live (storage and RPC are wired up). Override it to start work whose lifetime matches the sensor's: polling loops, event subscriptions, timers.

Errors thrown here are swallowed, not logged. Handle failures inside the override. Paired 1:1 with onStop, which runs on removal, plugin shutdown or cleanup.

Returns ​

void | Promise<void>

Example ​

ts
protected override onStart(): void {
  this._timer = setInterval(() => this.poll(), 5_000);
}

Inherited from ​

AudioSensor.onStart

onStop() ​

protected onStop(): void | Promise<void>

Defined in: sensor/base.ts:664

Counterpart of onStart: tear down whatever it started, such as timers, subscriptions and external resources.

Returns ​

void | Promise<void>

Example ​

ts
protected override onStop(): void {
  if (this._timer) clearInterval(this._timer);
}

Inherited from ​

AudioSensor.onStop

reportDetections() ​

reportDetections(detected, detections?): void

Defined in: sensor/audio.ts:124

Report detected audio events.

  • reportDetections(true): audio detected without specifics. The SDK synthesizes a single full-frame 'audio' detection.
  • reportDetections(true, [...]): audio detected with explicit detections.
  • reportDetections(false): clear.

Parameters ​

detected ​

boolean

Whether an audio event is currently detected.

detections? ​

Detection[]

Optional explicit detections produced for this event.

Returns ​

void

Example ​

ts
import type { Detection } from '@camera.ui/sdk';
sensor.reportDetections(true, [
  { label: 'glass_break', confidence: 0.91, box: { x: 0, y: 0, width: 1, height: 1 } } satisfies Detection,
]);
sensor.reportDetections(false);

Inherited from ​

AudioSensor.reportDetections

setDecibels() ​

setDecibels(value): void

Defined in: sensor/audio.ts:154

Update the current audio level (in decibels).

Parameters ​

value ​

number

Audio level in decibels.

Returns ​

void

Example ​

ts
sensor.setDecibels(72);

Inherited from ​

AudioSensor.setDecibels

setDisplayName() ​

setDisplayName(value): void

Defined in: sensor/base.ts:337

Set the display name (the only mutable identifier on a sensor).

Parameters ​

value ​

string

Human-readable label shown in the UI.

Returns ​

void

Example ​

ts
sensor.setDisplayName('Front Door Motion');

Inherited from ​

AudioSensor.setDisplayName

updateModelSpec() ​

updateModelSpec(): void

Defined in: sensor/base.ts:565

Re-announces the model spec, e.g. once the models finished loading.

The spec a detector reports at registration is what it knows at that moment: models that load in the background are missing from it. Call this after loading, the server replaces its copy.

Returns ​

void

Example ​

typescript
async onStart(): Promise<void> {
  await this.loadModel();
  this.updateModelSpec();
}

Inherited from ​

AudioSensor.updateModelSpec