Camera

What a camera contributes to the perception output: its video, live and as recorded into the history.

The pictures are passed on as the camera encoded them and decoded by whoever shows them, so a device needs no decoder of its own to record what a camera saw.

Video

One message of a camera’s video: how the pictures are encoded, or one of them.

Table 1. Available fields in blickfeld.percept_processing.data.Camera.Video
Field	Type	Default	Unit
format	Format	-	-
format	How the pictures that follow are encoded.
picture	Picture	-	-
picture	One encoded picture.

Format

How a camera’s video is encoded, which is what a decoder is configured with.

Table 2. Available fields in blickfeld.percept_processing.data.Camera.Video.Format
Field	Type	Default	Unit
codec	string	-	-
codec	RFC 6381 codec string, e.g. `avc1.4d4016`, as WebCodecs' `VideoDecoderConfig.codec` takes it.
binary	bytes	-	-
binary	Decoder configuration the codec defines, as WebCodecs' `VideoDecoderConfig.description` takes it. For H.264 this is the `AVCDecoderConfigurationRecord` (ISO/IEC 14496-15) carrying the SPS and PPS.
width	uint32	-	px
width	Width of the picture.
height	uint32	-	px
height	Height of the picture.

Picture

One encoded picture of a camera’s video.

Table 3. Available fields in blickfeld.percept_processing.data.Camera.Video.Picture
Field	Type	Default	Unit
timestamp	uint64	-	ns
timestamp	Unix timestamp at which the camera captured the picture, from its own clock where it reports one, and the time the device received it otherwise.
keyframe	bool	-	-
keyframe	Whether the picture decodes on its own. A decoder has to start at one of these.
binary	bytes	-	-
binary	The encoded picture. For H.264 these are its NAL units, each prefixed with its length as 4 byte big endian (`avc` format).

Segment

A run of a camera’s video between one keyframe and the next.

The segments of consecutive history frames continue each other, so a camera’s video over a range of frames is its segments one after the other. The first segment of a capture window starts at a keyframe, which is where decoding the video of that window starts.

Table 4. Available fields in blickfeld.percept_processing.data.Camera.Video.Segment
Field	Type	Default	Unit
serial_number	string	-	-
serial_number	`config.DataSource.Camera.serial_number` of the camera it was recorded by.
object_uuid	optional string	-	-
object_uuid	UUID of the DetectedObject the camera was tracking while it was recorded, if any.
format	Format	-	-
format	How the pictures are encoded.
pictures	repeated Picture	-	-
pictures	The pictures in the order they were captured.
position	optional percept_pipeline.config.DataSource.Camera.Ptz.Position	-	-
position	Where the camera was pointing while it recorded them. A picture on its own does not say which way the camera was facing, so replaying one cannot show where in the scene it was looking. Recorded beside the pictures, the aim can be drawn with them - and followed as a replay runs, rather than being stuck wherever the camera happens to point now. Unset for a camera that cannot be aimed.

Segments

The video one or more cameras took since the previous frame of the history, up to the moment of this one.

Table 5. Available fields in blickfeld.percept_processing.data.Camera.Video.Segments
Field	Type	Default	Unit
segments	repeated Segment	-	-
segments	Segments in the order they were recorded, for any camera.