viam.components.camera

Submodules

Package Contents

Classes

RawImage

DEPRECATED Use ViamImage instead

ViamImage

A native implementation of an image.

DistortionParameters

Abstract base class for protocol messages.

IntrinsicParameters

Abstract base class for protocol messages.

Camera

Camera represents any physical hardware that can capture frames.

class viam.components.camera.RawImage[source]

Bases: NamedTuple

DEPRECATED Use ViamImage instead

A raw bytes representation of an image.

A RawImage should be returned instead of a PIL Image instance under one of the following conditions

  1. The requested mime type has the LAZY_SUFFIX string appended to it

2) The requested mime type is not supported for decoding/encoding by Viam’s Python SDK

data: bytes

The raw data of the image

mime_type: str

The mimetype of the image

close()[source]

Close the image and release resources. For RawImage, this is a noop.

bytes_to_depth_array() List[List[int]][source]

Decode the data of an image that has the custom depth MIME type image/vnd.viam.dep into a standard representation.

Raises:

NotSupportedError – Raised if given an image that is not of MIME type image/vnd.viam.dep.

Returns:

The standard representation of the image.

Return type:

List[List[int]]

class viam.components.camera.ViamImage(data: bytes, mime_type: CameraMimeType)[source]

A native implementation of an image.

Provides the raw data and the mime type, as well as lazily loading and caching the PIL.Image representation.

property data: bytes

The raw bytes of the image

property mime_type: CameraMimeType

The mime type of the image

property image: PIL.Image.Image | None

The PIL.Image representation of the image. If the mime type is not supported, this will be None.

close()[source]

Close the image and release resources.

bytes_to_depth_array() List[List[int]][source]

Decode the data of an image that has the custom depth MIME type image/vnd.viam.dep into a standard representation.

Raises:

NotSupportedError – Raised if given an image that is not of MIME type image/vnd.viam.dep.

Returns:

The standard representation of the image.

Return type:

List[List[int]]

class viam.components.camera.DistortionParameters(*, model: str = ..., parameters: collections.abc.Iterable[float] | None = ...)

Bases: google.protobuf.message.Message

Abstract base class for protocol messages.

Protocol message classes are almost always generated by the protocol compiler. These generated types subclass Message and implement the methods shown below.

property parameters: google.protobuf.internal.containers.RepeatedScalarFieldContainer[float]
model: str
class viam.components.camera.IntrinsicParameters(*, width_px: int = ..., height_px: int = ..., focal_x_px: float = ..., focal_y_px: float = ..., center_x_px: float = ..., center_y_px: float = ...)

Bases: google.protobuf.message.Message

Abstract base class for protocol messages.

Protocol message classes are almost always generated by the protocol compiler. These generated types subclass Message and implement the methods shown below.

width_px: int
height_px: int
focal_x_px: float
focal_y_px: float
center_x_px: float
center_y_px: float
class viam.components.camera.Camera(name: str)[source]

Bases: viam.components.component_base.ComponentBase

Camera represents any physical hardware that can capture frames.

This acts as an abstract base class for any drivers representing specific camera implementations. This cannot be used on its own. If the __init__() function is overridden, it must call the super().__init__() function.

SUBTYPE: Final
Properties: TypeAlias
abstract async get_image(mime_type: str = '', *, extra: Dict[str, Any] | None = None, timeout: float | None = None, **kwargs) PIL.Image.Image | viam.components.camera.RawImage[source]

Get the next image from the camera as an Image or RawImage. Be sure to close the image when finished.

NOTE: If the mime type is image/vnd.viam.dep you can use viam.media.video.ViamImage.bytes_to_depth_array() to convert the data to a standard representation.

my_camera = Camera.from_robot(robot=robot, name="my_camera")

# Assume "frame" has a mime_type of "image/vnd.viam.dep"
frame = await my_camera.get_image(mime_type = CameraMimeType.VIAM_RAW_DEPTH)

# Convert "frame" to a standard 2D image representation.
# Remove the 1st 3x8 bytes and reshape the raw bytes to List[List[Int]].
standard_frame = frame.bytes_to_depth_array()
Parameters:

mime_type (str) – The desired mime type of the image. This does not guarantee output type

Returns:

The frame

Return type:

Image | RawImage

abstract async get_images(*, timeout: float | None = None, **kwargs) Tuple[List[viam.media.video.NamedImage], viam.proto.common.ResponseMetadata][source]

Get simultaneous images from different imagers, along with associated metadata. This should not be used for getting a time series of images from the same imager.

my_camera = Camera.from_robot(robot=robot, name="my_camera")

images, metadata = await my_camera.get_images()
img0 = images[0].image
timestamp = metadata.captured_at
Returns:

  • List[NamedImage]: The list of images returned from the camera system.

  • ResponseMetadata: The metadata associated with this response

Return type:

Tuple[List[NamedImage], ResponseMetadata]

abstract async get_point_cloud(*, extra: Dict[str, Any] | None = None, timeout: float | None = None, **kwargs) Tuple[bytes, str][source]

Get the next point cloud from the camera. This will be returned as bytes with a mimetype describing the structure of the data. The consumer of this call should encode the bytes into the formatted suggested by the mimetype.

To deserialize the returned information into a numpy array, use the Open3D library.

import numpy as np
import open3d as o3d

data, _ = await camera.get_point_cloud()

# write the point cloud into a temporary file
with open("/tmp/pointcloud_data.pcd", "wb") as f:
    f.write(data)
pcd = o3d.io.read_point_cloud("/tmp/pointcloud_data.pcd")
points = np.asarray(pcd.points)
Returns:

The pointcloud data. str: The mimetype of the pointcloud (e.g. PCD).

Return type:

bytes

abstract async get_properties(*, timeout: float | None = None, **kwargs) Properties[source]

Get the camera intrinsic parameters and camera distortion parameters

my_camera = Camera.from_robot(robot=robot, name="my_camera")

properties = await my_camera.get_properties()
Returns:

The properties of the camera

Return type:

Properties

classmethod from_robot(robot: viam.robot.client.RobotClient, name: str) typing_extensions.Self

Get the component named name from the provided robot.

Parameters:
  • robot (RobotClient) – The robot

  • name (str) – The name of the component

Returns:

The component, if it exists on the robot

Return type:

Self

abstract async do_command(command: Mapping[str, ValueTypes], *, timeout: float | None = None, **kwargs) Mapping[str, ValueTypes]

Send/Receive arbitrary commands to the Resource

Parameters:

command (Mapping[str, ValueTypes]) – The command to execute

Raises:

NotImplementedError – Raised if the Resource does not support arbitrary commands

Returns:

Result of the executed command

Return type:

Mapping[str, ValueTypes]

async get_geometries(*, extra: Dict[str, Any] | None = None, timeout: float | None = None) List[viam.proto.common.Geometry]

Get all geometries associated with the Component, in their current configuration, in the frame of the Component.

Returns:

The geometries associated with the Component.

Return type:

List[Geometry]

classmethod get_resource_name(name: str) viam.proto.common.ResourceName

Get the ResourceName for this Resource with the given name

Parameters:

name (str) – The name of the Resource

get_operation(kwargs: Mapping[str, Any]) viam.operations.Operation

Get the Operation associated with the currently running function.

When writing custom resources, you should get the Operation by calling this function and check to see if it’s cancelled. If the Operation is cancelled, then you can perform any necessary (terminating long running tasks, cleaning up connections, etc. ).

Parameters:

kwargs (Mapping[str, Any]) – The kwargs object containing the operation

Returns:

The operation associated with this function

Return type:

viam.operations.Operation

async close()

Safely shut down the resource and prevent further use.

Close must be idempotent. Later configuration may allow a resource to be “open” again. If a resource does not want or need a close function, it is assumed that the resource does not need to retun errors when future non-Close methods are called.