Skip to content

Documentation: PolicyRuntime in Runtime API Reference #176

Description

@intel-ohara

The document doesn't mention, but 'task' seems necessary.

When I run the script,

from physicalai.capture import UVCCamera
from physicalai.capture import RealSenseCamera
from physicalai.inference import InferenceModel
from physicalai.runtime import PolicyRuntime, SyncExecution
from physicalai.robot import SO101

#policy = InferenceModel("./SmolVLA/exports/openvino", device="GPU")
policy = InferenceModel("./SmolVLA/exports/openvino")

cameras = {
#    "bird view": UVCCamera(device="/dev/video32", width=960, height=720, fps=30), # v4l2-ctl --list-devices
    "bird view": UVCCamera(device="/dev/video40", width=1280, height=720, fps=30), # v4l2-ctl --list-devices
#    "side view": RealSenseCamera(serial_number="013222072639", width=640, height=480, fps=30),
}

robot = SO101(port="/dev/ttyACM1", calibration="./Bob.json", role="follower")
#robot.connect()

runtime = PolicyRuntime(
    fps=30,
    robot=robot,
    model=policy,
    cameras=cameras,
    execution=SyncExecution(),
#    task='pick pen and put it on cup',
)

with runtime:
    runtime.run(duration_s=60)

I got the error below.

Traceback (most recent call last):
File "/home/intel/openvino-physical-ai/test_inference.py", line 29, in
runtime.run(duration_s=60)
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/runtime.py", line 313, in run
self._warmup_with_retry()
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/runtime.py", line 590, in _warmup_with_retry
self._execution.warmup(sample_obs)
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/runtime/execution.py", line 107, in warmup
actions = self._model.predict_action_chunk(sample_observation)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/model.py", line 253, in predict_action_chunk
outputs = self(observation)
^^^^^^^^^^^^^^^^^
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/model.py", line 203, in call
inputs = preprocessor(inputs)
^^^^^^^^^^^^^^^^^^^^
File "/home/intel/openvino-physical-ai/.venv/lib/python3.12/site-packages/physicalai/inference/preprocessors/new_line.py", line 33, in call
batch_tasks = inputs[TASK]
~~~~~~^^^^^^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions