EdgeFirst Camera Node

Production-ready ROS2-compatible camera publisher with zero-copy DMA, hardware-accelerated encoding, and EdgeFirst Perception integration

The EdgeFirst Camera Node is a high-performance camera service designed for edge AI perception systems. It interfaces directly with camera hardware through V4L2, provides zero-copy DMA buffer access for ultra-low latency pipelines, and publishes ROS2-compatible messages over Zenoh for seamless integration with robotics and vision ecosystems.

Key Features:

• 🎥 ROS2 Compatibility - Standard sensor_msgs interfaces for drop-in integration
• ⚡ Zero-Copy DMA - Direct memory access for latency-critical vision pipelines
• 🚀 Hardware Acceleration - NXP G2D format conversion and H.264 encoding
• 📦 Multi-Format Output - Raw DMA, JPEG, and H.264 streams from single capture
• 🔧 Flexible Transport - Native Zenoh with optional ROS2 bridge compatibility
• 📊 Production Ready - Tracy profiling, journald logging, comprehensive error handling
• 🎬 4K Support - Automatic tiling for high-resolution cameras

The EdgeFirst Camera Node is the sensor interface layer of the EdgeFirst Perception Middleware—a modular software stack for edge AI vision systems.

graph LR
    subgraph "Sensor Inputs"
        Camera["📷 Camera Service<br/>(THIS REPO)"]
        Radar["📡 Radar"]
        LiDAR["🔬 LiDAR"]
        IMU["⚖️ IMU"]
    end
    
    subgraph "EdgeFirst Perception Middleware"
        Zenoh["Zenoh Messaging<br/>(ROS2 CDR)"]
    end
    
    subgraph "Outputs & Applications"
        Vision["🤖 Vision Models"]
        Fusion["🔗 Fusion Models"]
        Recording["💾 Recording<br/>(MCAP)"]
        WebUI["🌐 Web UI<br/>(HTTPS)"]
    end
    
    Camera --> Zenoh
    Radar --> Zenoh
    LiDAR --> Zenoh
    IMU --> Zenoh
    
    Zenoh --> Vision
    Zenoh --> Fusion
    Zenoh --> Recording
    Zenoh --> WebUI
    
    style Camera fill:#f96,stroke:#333,stroke-width:3px

What the Camera Node Does:

• Captures frames from hardware cameras (MIPI CSI-2, USB UVC)
• Processes frames with hardware-accelerated format conversion and scaling
• Encodes video streams (JPEG for visualization, H.264 for recording)
• Publishes to Zenoh topics using ROS2 message formats
• Provides zero-copy DMA access for low-latency vision model inference

Integration Points:

• Vision Models: Consume raw DMA buffers for real-time object detection, classification, segmentation
• Fusion Models: Combine camera data with radar/LiDAR for multi-modal perception
• Recording: MCAP format recording for dataset collection and replay
• Web UI: Live JPEG/H.264 streaming for monitoring and debugging
• Custom Applications: ROS2-compatible message access via zenoh-bridge-dds

Learn More: EdgeFirst Perception Documentation

Hardware:

• NXP i.MX8M Plus based platform (Maivin, Raivin) or compatible ARM64 device
• Camera: MIPI CSI-2 or USB UVC camera
• Minimum: 2GB RAM, quad-core ARM Cortex-A53

Software:

• Linux kernel 5.10+ with V4L2 support
• Rust 1.90.0 or later (for building from source)
• OR: Pre-built binaries from GitHub Releases

Option 1: Pre-built Binary (Recommended)

# Download latest release for ARM64
wget https://github.com/EdgeFirstAI/camera/releases/latest/download/edgefirst-camera-linux-aarch64

# Make executable
chmod +x edgefirst-camera-linux-aarch64

# Run with default settings
./edgefirst-camera-linux-aarch64 --jpeg --h264

Option 2: Build from Source

# Clone repository
git clone https://github.com/EdgeFirstAI/camera.git
cd camera

# Build release binary
cargo build --release

# Run
./target/release/edgefirst-camera --jpeg --h264

Option 3: Cross-Compile for ARM64

# Add ARM64 target
rustup target add aarch64-unknown-linux-gnu

# Install cross-compilation toolchain
# (Debian/Ubuntu)
sudo apt-get install gcc-aarch64-linux-gnu

# Build for ARM64
cargo build --release --target aarch64-unknown-linux-gnu

# Binary at: target/aarch64-unknown-linux-gnu/release/edgefirst-camera

Single Camera with JPEG and H.264 Streams:

edgefirst-camera \
  --camera /dev/video0 \
  --camera-size 1920 1080 \
  --jpeg \
  --h264

4K Camera with Automatic Tiling:

# Automatically splits 4K into 4x 1080p H.264 tiles
edgefirst-camera \
  --camera /dev/video0 \
  --camera-size 3840 2160 \
  --h264-tiles

Zero-Copy DMA for Vision Models:

# Publishes CameraFrame (embedded Tensor + dma-buf planes) on camera/frame
# Consume these in your vision model for lowest latency
edgefirst-camera --camera /dev/video0

Custom Topics and Configuration:

edgefirst-camera \
  --camera /dev/video1 \
  --frame-topic camera/front/frame \
  --jpeg-topic camera/front/jpeg \
  --h264-topic camera/front/h264 \
  --mirror both \
  --stream-size 640 480

The camera node publishes standard ROS2 message types using CDR serialization, ensuring seamless integration with existing ROS2 ecosystems:

ROS2 Bridge Integration:

# Start camera node
edgefirst-camera --jpeg --h264 &

# Start zenoh-bridge-dds for ROS2 compatibility
zenoh-bridge-dds

# View with ROS2 tools
ros2 topic list
ros2 topic echo /camera/info
rqt_image_view  # View JPEG stream

For latency-critical vision pipelines, the camera node provides direct DMA buffer access:

// Vision model consuming CameraFrame tensors (pseudo-code)
subscriber.on_message(|payload| {
    let cf = CameraFrame::from_cdr(payload)?;
    let t = cf.tensor();
    let plane = t.plane_at(0)?;
    // t.pid() + plane.handle: dma-buf fd in the producer
    // t.shape(): [height, width]; t.format(): "YUYV" / "NV12" / ...
    // plane.offset / stride / size / used: memory layout

    let local_fd = pidfd_getfd(t.pid(), plane.handle as RawFd)?;
    let frame = map_dma_buffer(local_fd, plane.offset, plane.size)?;
    model.infer(frame)?;
});

Performance Benefits:

• 5-10ms latency vs 50-100ms with traditional copy-based approaches
• Zero memory copies from camera to inference
• Reduced CPU load - no memcpy overhead
• Lower memory bandwidth - critical for edge devices

NXP i.MX8 G2D Engine:

• Format conversion: YUYV → NV12, RGB, RGBA (hardware accelerated)
• Scaling: 4K → 1080p, arbitrary resolutions
• Rotation: 90°, 180°, 270° hardware rotation
• Mirroring: Horizontal, vertical, both

H.264 Hardware Encoder:

• Real-time encoding up to 4K @ 30 FPS
• Configurable bitrate (5/25/50/100 Mbps)
• Automatic tiling for resolutions > 1080p
• Tile-specific FPS control to reduce artifacts

For cameras with resolutions exceeding 1080p, the camera node automatically splits frames into 1920×1080 tiles for hardware encoding:

graph TB
    subgraph "4K Camera (3840×2160)"
        TL["Top-Left<br/>1920×1080<br/>camera/h264/tl"]
        TR["Top-Right<br/>1920×1080<br/>camera/h264/tr"]
        BL["Bottom-Left<br/>1920×1080<br/>camera/h264/bl"]
        BR["Bottom-Right<br/>1920×1080<br/>camera/h264/br"]
    end
    
    Camera["4K Camera"] --> TL
    Camera --> TR
    Camera --> BL
    Camera --> BR
    
    TL --> Encode1["H.264 Encoder 1"]
    TR --> Encode2["H.264 Encoder 2"]
    BL --> Encode3["H.264 Encoder 3"]
    BR --> Encode4["H.264 Encoder 4"]

Why Tiling?

• Hardware Limitation: NXP H.264 encoder max resolution is 1920×1080
• Parallel Processing: 4 encoder instances for higher throughput
• Artifact Reduction: Lower FPS per tile reduces compression artifacts at 4K
• Bandwidth Control: Disable specific tiles if not needed

edgefirst-camera --help

Essential Options:

• --camera <PATH> - Camera device path (default: /dev/video3)
• --camera-size <WIDTH> <HEIGHT> - Capture resolution (default: 1920 1080)
• --stream-size <WIDTH> <HEIGHT> - Output resolution for JPEG/H264 (default: 1920 1080)
• --mirror <none|horizontal|vertical|both> - Mirror camera image (default: both)

Output Formats:

• --jpeg - Enable JPEG streaming
• --h264 - Enable H.264 streaming
• --h264-tiles - Enable 4K tiling (auto-enabled for >1080p cameras)

Topic Configuration:

• --frame-topic <TOPIC> - CameraFrame topic (default: camera/frame)
• --info-topic <TOPIC> - CameraInfo topic (default: camera/info)
• --jpeg-topic <TOPIC> - JPEG topic (default: camera/jpeg)
• --h264-topic <TOPIC> - H264 topic (default: camera/h264)

Performance Tuning:

• --h264-bitrate <auto|mbps5|mbps25|mbps50|mbps100> - H264 bitrate (default: auto)
• --h264-tiles-fps <FPS> - FPS limit for 4K tiles (default: 15)

Recording and Replay:

• --record <PATH> - Record the live H.264 stream to <PATH> as raw Annex-B. Writes a <PATH>.json sidecar alongside. Requires --h264.
• --replay <PATH> - Replay a previously recorded file instead of opening a camera. Requires the matching .json sidecar.
• --replay-loop - Loop replay back to the start on EOF. CameraFrame.seq keeps incrementing across loops.
• --replay-fps <N> - Override playback rate (defaults to the sidecar's recorded fps).

Zenoh Configuration:

• --zenoh-config <FILE> - Zenoh config file (JSON or JSON5)
• --connect <ENDPOINT> - Connect to Zenoh router
• --listen <ENDPOINT> - Listen for Zenoh connections
• --mode <peer|client|router> - Zenoh participant mode (default: peer)

Debugging:

• --tracy - Enable Tracy profiler integration
• --tokio-console - Enable Tokio console for async debugging

See full options: edgefirst-camera --help

All command-line flags can be set via environment variables with EDGEFIRST_CAMERA_ prefix:

export EDGEFIRST_CAMERA_CAMERA=/dev/video0
export EDGEFIRST_CAMERA_MIRROR=both
export EDGEFIRST_CAMERA_JPEG=true
export EDGEFIRST_CAMERA_H264=true

edgefirst-camera  # Uses environment configuration

For complex deployments, use a Zenoh configuration file:

{
  mode: "client",
  connect: {
    endpoints: ["tcp/192.168.1.100:7447"]
  },
  scouting: {
    multicast: {
      enabled: false
    }
  }
}

edgefirst-camera --zenoh-config /etc/edgefirst/zenoh.json5

The camera node ships built-in recording and replay of the H.264 stream, intended for reproducible tests and demos. Recordings are plain raw Annex-B bitstream files — power-loss resilient, playable in VLC / mpv / ffplay out of the box.

# Record 30 seconds of H.264 to a file (requires --h264).
edgefirst-camera --camera /dev/video0 --h264 --record capture.h264

Writes two files:

• capture.h264 — raw H.264 Annex-B bitstream, appended frame-by-frame. Flushed on every keyframe so a crash loses at most one GOP (~1 s).
• capture.json — sidecar metadata. Written once at startup, holds colorimetry + the exact sensor_msgs/CameraInfo and /tf_static payloads that would have been published live. Stateless; no per-frame data.

The tap is inside the h264 encode thread and runs before the Zenoh publish, so the recorder captures every frame even if Zenoh drops a publish under congestion.

# Replay the recorded pair — camera device is not opened.
edgefirst-camera --replay capture.h264

# Loop indefinitely; seq keeps incrementing across loop boundaries.
edgefirst-camera --replay capture.h264 --replay-loop

# Override playback rate.
edgefirst-camera --replay capture.h264 --replay-fps 15

Replay publishes on the same topics and schemas as live capture — subscribers see identical on-wire data. /camera/info, /tf_static, and the CameraFrame tensor colorimetry fields come from the sidecar, not from CLI flags (any --cam-info-path, --cam-tf-*, or --base-frame-id flags supplied at replay time are ignored with a warning).

The camera/h264 topic forwards the recorded Annex-B bytes verbatim — no re-encode. camera/frame is built from the decoded NV12 frame.

Not supported with --replay: --jpeg, --h264-tiles (recorded files carry only the main H.264 stream). Use the live path if you need JPEG or tile output.

The raw .h264 file is directly playable by any standard tool:

vlc capture.h264
mpv --demuxer=h264 capture.h264
ffplay -f h264 -framerate 30 capture.h264
ffmpeg -i capture.h264 -c copy capture.mp4   # remux to MP4, no re-encode

See ARCHITECTURE.md § Record and Replay for the sidecar JSON schema and internals.

The camera node includes Tracy profiler integration for performance analysis. See CONTRIBUTING.md for setup instructions.

Quick Start:

# Run with Tracy profiler enabled
edgefirst-camera --tracy --camera /dev/video0 --jpeg --h264

# Connect Tracy profiler GUI to analyze frame timing
# Download Tracy: https://github.com/wolfpld/tracy/releases

MIPI CSI-2 Cameras (primary):

• Directly connected via CSI interface
• Lowest latency, highest performance
• V4L2 device typically /dev/video0-3

USB UVC Cameras (secondary):

• Standard USB webcams
• V4L2 device typically /dev/video4+
• Slightly higher latency

Supported Formats:

• YUYV (preferred for hardware acceleration)
• NV12, RGB, RGBA

# Clone repository
git clone https://github.com/EdgeFirstAI/camera.git
cd camera

# Build with all features
cargo build --release

# Run tests
cargo test

# Run benchmarks (requires hardware)
cargo bench

# Generate documentation
cargo doc --no-deps --open

camera/
├── src/
│   ├── main.rs          # Main loop, Zenoh publishers, thread coordination
│   ├── image.rs         # DMA allocation, G2D operations, JPEG encoding
│   ├── video.rs         # H264 encoding, 4K tiling
│   ├── args.rs          # CLI argument parsing
│   └── lib.rs           # Public library interface
├── tests/               # Integration tests
├── benches/             # Performance benchmarks
├── Cargo.toml           # Project dependencies
└── README.md            # This file

See also:

• ARCHITECTURE.md - Detailed architecture and design decisions
• CONTRIBUTING.md - Contribution guidelines and development workflow
• SECURITY.md - Security policy and vulnerability reporting

Problem: "No such device" error

# List available cameras
v4l2-ctl --list-devices

# Try different device path
edgefirst-camera --camera /dev/video0

Problem: "Permission denied" accessing camera

# Add user to video group
sudo usermod -a -G video $USER
# Log out and back in for changes to take effect

# Or run with elevated permissions (not recommended for production)
sudo edgefirst-camera

Problem: Low FPS or frame drops

# Check CPU usage
htop

# Reduce output formats to decrease load
edgefirst-camera --jpeg  # Only JPEG, no H264

# For 4K cameras, ensure tiling is enabled
edgefirst-camera --camera-size 3840 2160 --h264-tiles

Problem: JPEG/H264 streams not appearing in ROS2

# Ensure zenoh-bridge-dds is running
zenoh-bridge-dds &

# Check Zenoh can see publishers
zenoh-cli query "**"

# Verify ROS2 topics
ros2 topic list | grep camera

Problem: DMA buffers not working with custom application

• Ensure your application has access to DMA heap (/dev/dma_heap/linux,cma)
• File descriptor passing requires Unix domain sockets or shared memory
• Check your Zenoh session configuration allows large payloads

# Set log level
RUST_LOG=debug edgefirst-camera

# Filter specific module
RUST_LOG=edgefirst_camera::image=trace edgefirst-camera

# View systemd journal logs
journalctl -u edgefirst-camera -f

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Third-Party Components: See NOTICE for required attributions.

We welcome contributions! Please see CONTRIBUTING.md for:

• Code style guidelines
• Development workflow
• Pull request process
• Testing requirements

Found a bug? Open an issue

Have a feature request? Start a discussion

Community Resources:

• 📚 Documentation: https://doc.edgefirst.ai/test/perception/
• 💬 Discussions: https://github.com/orgs/EdgeFirstAI/discussions
• 🐛 Issues: https://github.com/EdgeFirstAI/camera/issues

Commercial Support:

• EdgeFirst Studio: Integrated deployment, monitoring, and management
• Professional Services: Training, custom development, enterprise support
• Contact: [email protected]

Built with ❤️ by the EdgeFirst team at Au-Zone Technologies

For questions or support, see our Contributing Guide or open an issue on GitHub.

Powered by:

• Zenoh - Efficient pub/sub middleware
• Tokio - Async runtime
• ROS2 CDR - Message serialization
• NXP i.MX8 - Hardware acceleration platform