Skip to content
RFL_GLOBAL
中文
  • MAC-EGO3D // CVPR 2025
  • MULTI-AGENT
  • LOOP CLOSURE

SYNTHESIS

MANY EYES, ONE MAP

Single-robot SLAM breaks down when mapping large spaces. Multi-agent teams have overlapping views but no standardized way to merge reconstructions. SYNTHESIS is collaborative SLAM built on MAC-Ego3D (CVPR 2025). Multiple robots, one shared 3D map, with real-time loop closure detection and globally consistent trajectory optimization.

MODULE STATUS: ACTIVE

Agent Support

5+

DIVISION
ANIMA
WAVE
W3
DOMAIN
ACTION
WAVE 3 // ANIMA SUITE
COLLABORATIVE SLAM
SYNTHESIS // W3 // 072/079
01THE CHALLENGE

ONE ROBOT IS NOT ENOUGH

Single-robot SLAM breaks down when mapping large spaces, narrow corridors, or GPS-denied environments. Multi-agent teams have overlapping views but no standardized way to merge egocentric reconstructions into a consistent global map.

Collaborative SLAM requires solving loop closure at scale across agents, handling communication delays, and maintaining consistency. Most systems require centralized fusion (single point of failure) or expensive consensus algorithms. SYNTHESIS provides the clean solution.

02THE SOLUTION

WHAT SYNTHESIS DELIVERS

SYNTHESIS is a containerized multi-agent collaborative SLAM service built on MAC-Ego3D (CVPR 2025). Merge views, detect loops, maintain consistency, scale to fleets.

PIPELINE

  1. 01Merges egocentric views: RGB streams from multiple robots → one shared 3D map
  2. 02Detects loop closures across agent trajectories with learned feature correspondences
  3. 03Global trajectory optimization: eliminates drift and contradictions
  4. 04Multiple communication topologies: central, peer-to-peer, broadcast

CAPABILITIES

  • FLEET SCALEScales to 5+ agents with real-time or near-real-time fusion→ Warehouse, construction, and inspection fleet mapping
  • HARDWARE FLEXMLX Apple Silicon (M1–M5, ~100ms on M5, 4x M1) + CUDA + CPU→ Deploy on any hardware from laptops to edge devices
  • MULTI-PROTOCOLREST + gRPC APIs for multi-agent message passing protocol→ Integrate with any robotics middleware stack
03ENGINEERING

WHY THIS IS HARD

Collaborative SLAM requires solving data association across agents — is this loop closure real or a false match?

  1. 01Classical descriptor matching is slow and hand-tuned — MAC-Ego3D learns end-to-end correspondences
  2. 02Multi-agent coordination with asynchronous data arrival and partial observability
  3. 03Communication failures must be handled gracefully — agents must continue alone and re-merge
  4. 04iSAM2-style incremental pose graph optimization for real-time global consistency
  5. 05Bounded concurrency per agent and global admission control for fleet scaling

SYNTHESIS wraps these concerns behind a clean async API. Teams integrate without reimplementing the hard optimization core.

04BENCHMARKS

REAL HARDWARE PERFORMANCE

Measured with MAC-Ego3D + collaborative SLAM pipeline:

REAL HARDWARE PERFORMANCE
METRICVALUE
Agent Support5+ simultaneous robots
Loop Closure DetectionReal-time scoring
Pose Graph OptimizationiSAM2-style incremental
Communication TopologiesCentral, peer-to-peer, broadcast
Apple M5 (MLX)~100ms per fusion step (4x M1)
API ProtocolsREST + gRPC
Admission ControlPer-agent and global bounds
Feature ExtractionMAC-Ego3D end-to-end learned
05BUILD STATUS

WHAT'S BUILT TODAY

9/10 COMPONENTS COMPLETE
WHAT'S BUILT TODAY
COMPONENTSTATUSNOTES
Core modelsCOMPLETEMAC-Ego3D backbone + SLAM pipeline, production-ready
API layerIN PROGRESSPublic API, pending dataset infrastructure
REST APICOMPLETEFastAPI with Swagger docs
gRPC ServiceCOMPLETEProtobuf multi-agent protocol
Loop Closure DetectionCOMPLETEMAC-Ego3D egocentric fusion
Trajectory OptimizationCOMPLETEGlobal pose graph refinement
Central TopologyCOMPLETEServer-centric fusion
Peer-to-Peer TopologyCOMPLETEDecentralized agent networks
Admission ControlCOMPLETEPer-agent and global bounds
Quality GatesCOMPLETEruff, pytest, mypy passing
06APPLICATIONS

WHERE SYNTHESIS DEPLOYS

  • APP_01

    FLEET OPERATORS

    Warehouses, inspection, construction fleet mapping with multiple coordinated robots.

  • APP_02

    AUTONOMOUS VEHICLES

    Mapping parking structures and underground facilities with collaborative vehicle fleets.

  • APP_03

    SEARCH & RESCUE

    Coordinating drones across GPS-denied environments for rapid area coverage.

  • APP_04

    DRONE DELIVERY

    Sharing partial maps for route optimization across delivery drone fleets.

  • APP_05

    RESEARCH LABS

    Prototyping multi-robot SLAM without the math — clean API, instant results.

  • APP_06

    UNDERGROUND MINING

    Multi-robot exploration and mapping in tunnels where GPS and comms are unreliable.

07TECHNOLOGY

UNDER THE HOOD

FOUNDATION: MAC-EGO3D (CVPR 2025)

  • Multi-agent collaborative 3D egocentric SLAM
  • Learned loop closure with uncertainty quantification
  • MIT-licensed reference implementation
  • End-to-end feature correspondences (no hand-tuned descriptors)

SYNTHESIS IMPLEMENTATION

FastAPI async REST server
Streaming JSON for real-time fusion updates
gRPC multi-agent protocol
Protobuf message passing between agents
Device abstraction layer
MLX → CUDA → CPU automatic fallback
Embedded iSAM2 solver
Pose graph optimization with bounded concurrency

INTEGRATION POINTS

  • Consumes semantic labels from NEXUS
  • Provides shared maps to HARMONIA
  • Trajectory confidence for planning safety
  • REST + gRPC
  • MLX + CUDA + CPU
08PAPERS

RESEARCH BASIS

  1. [01]MAC-Ego3D — multi-agent collaborative egocentric SLAM (CVPR 2025)