Camera Operator
ACM Multimedia 2026 Camera-ready article

Camera Operator:
Object-Grounded Camera Trajectory Generation from Text and 3D Bounding Box Sequences

Language specifies sequence-level cinematographic intent. Dynamic 3D target boxes provide the frame-wise geometry needed to preserve framing, visibility, and viewing direction.

Zhongyuan Hu1 · Yue Ma2,* · Jiangming Wang3 · Ronghui Li1 · Xiu Li1,†

1Tsinghua University · 2Hong Kong University of Science and Technology · 3Sun Yat-Sen University
* Project lead. † Corresponding author.

Paper DOI pending Project repository Code · coming soon Model · coming soon Dataset
Examples of target object trajectories, generated camera trajectories, and downstream video-generation frames.
Top: a camera-motion instruction and a dynamic 3D target box sequence condition the generated camera trajectory. Bottom: the resulting trajectory can serve as camera control for downstream video generation; projected boxes can additionally provide object-track guidance. Figure 1 of the CC BY 4.0 camera-ready article, converted for web display.

Core idea

Intent and feasibility play different roles.

Camera motion is underdetermined: a single scene can admit many valid shots. The geometric relation losses define admissible regions instead of requiring exact relation matching to one reference path. Language selects the kind of shot, while target geometry keeps the shot viable over time.

01

Language selects intent

A natural-language instruction describes high-level motion such as orbiting, following, approaching, or changing viewing angle.

02

3D boxes ground each frame

Target location alone is not enough. A time-varying box also exposes extent and orientation—the cues needed for scale, composition, and look-at control.

03

Flow matching exposes a clean estimate

An intermediate clean trajectory estimate makes projection-space relation supervision available throughout training, not only after generation.

Inputs Camera instruction + dynamic 3D target boxes
Generator Geometry-conditioned flow matching
Output Object-grounded 6-DoF camera trajectory

Method

Geometry-conditioned flow matching

Text and box-state encoders provide complementary conditions to a trajectory transformer. Framing, visibility, look-at, velocity, and acceleration objectives act on the estimated clean trajectory in a shared geometric frame.

Camera Operator architecture showing text and 3D box conditioning, a flow transformer, clean trajectory estimation, and geometric supervision.
Camera Operator architecture. Figure 4 of the CC BY 4.0 camera-ready article, converted for web display.

Qualitative behavior

Preserving the camera–target relation

The paper evaluates text–trajectory alignment, trajectory distribution, and camera–target relation preservation under its reported protocol. The qualitative comparison below illustrates the failure modes that motivate full 3D target grounding.

A three-panel trajectory visualization comparing Camera Operator, GenDoP, and E.T.; the panels annotate camera poses, target boxes, gaze drift, and composition drift.
Representative relation-preservation comparison; see the paper for the complete evaluation. Figure 5 of the CC BY 4.0 camera-ready article, converted for web display.
Read the evaluation

Downstream use

From trajectories to camera-controlled video

Camera Operator produces camera motion, not RGB video. Its predicted trajectory can be supplied to a downstream camera-controlled video generator, while projected target boxes can support object-motion guidance.

Two downstream video-generation examples paired with predicted camera and target trajectories.
Downstream examples reported in the paper. Appearance is specified by the content prompt; Camera Operator supplies the camera trajectory. Figure 6 of the CC BY 4.0 camera-ready article, converted for web display.

Dataset

BlockCam

The paper introduces aligned language, dynamic 3D target boxes, and camera trajectories. The publication benchmark and the public synthetic-only artifact have different memberships, so their status is stated separately.

Described in the paper

BlockCam benchmark

41K sequences

12K real-world + 29K synthetic · 11.2M frames

A mixed real/synthetic benchmark described by the camera-ready article. It is the data scope associated with the reported paper protocol.

Public release

Synthetic-only annotations

37,499 records

A repaired-and-audited synthetic-only release with 150 frames per record. It contains no source video, rendered media, REAL annotations, Unreal project, or Fab asset.

Its membership and splits differ from the accepted-paper protocol; it is not presented as a drop-in reproduction of the main table.

The audited synthetic-only annotation release is public.

It passed the declared geometry, templated-text consistency, split, archive, and remote round-trip checks. The Hugging Face release is distributed under CC BY 4.0 with its Dataset Card, notices, and checksums.

Code & model · coming soon Hugging Face

Scope

What Camera Operator currently addresses

In scope

  • Offline camera trajectory generation
  • One designated target subject
  • A known time-varying 3D target box sequence
  • Single-shot clips with continuous target visibility

Future extensions

  • Partial or noisy observations and target re-acquisition
  • Multi-target composition and target switching
  • Shot transitions and broader language
  • Camera-independent normalization and online planning

Citation

Cite Camera Operator

The DOI has been assigned by ACM and will resolve after Digital Library activation.

@inproceedings{hu2026cameraoperator,
  author    = {Hu, Zhongyuan and Ma, Yue and Wang, Jiangming and
               Li, Ronghui and Li, Xiu},
  title     = {Camera Operator: Object-Grounded Camera Trajectory
               Generation from Text and 3D Bounding Box Sequences},
  booktitle = {Proceedings of the 34th ACM International Conference
               on Multimedia},
  year      = {2026},
  doi       = {10.1145/3767308.3835459}
}