Skip to content
RFL_GLOBAL
中文
  • WAVE 5 // SIMULATION
  • CHIRON + PYGMALION + ERGON
  • TEXT-TO-SCENE

ELYSIUM

LLM-GENERATED ENVIRONMENTS

Generates simulation environments at scale using LLMs (Genie Sim 3.0 approach). Natural language scene descriptions converted into full 3D simulation environments with 5,140+ assets. Dual modes: teleoperation-based and fully automated data collection. 3D reconstruction pipeline for importing real-world assets.

MODULE STATUS: DEVELOPMENT

Asset Library Size

5.1K+

DIVISION
ANIMA
WAVE
W5
DOMAIN
SIMULATION
WAVE 5 // ANIMA SUITE
SIMULATION — LLM-GENERATED ENVIRONMENTS
ELYSIUM // W5 // 018/079
01THE CHALLENGE

BUILDING SIMULATION ENVIRONMENTS DOESN'T SCALE

Robot training requires diverse simulation environments, but creating them is a manual bottleneck. Every new task scenario — a kitchen with specific objects, a warehouse shelf layout, a manufacturing workbench — must be hand-designed by engineers who understand both 3D modeling and robotic task requirements.

The result: labs spend weeks building individual environments while their training pipelines starve for data diversity. Real-world objects lack simulation counterparts. New task domains require starting from scratch. The simulation environment creation process is the hidden bottleneck blocking scalable robot learning.

5,140+
SIMULATION ASSETS
DUAL
COLLECTION MODES
3D
RECONSTRUCTION PIPELINE
02THE SOLUTION

WHAT ELYSIUM DELIVERS

ELYSIUM generates complete simulation environments from natural language descriptions using LLMs. Describe a scene in text, and ELYSIUM produces a fully interactive 3D environment with physics, semantics, and task-relevant object configurations — ready for robot training data collection.

CAPABILITIES

  • Natural language → full 3D simulation environments with physics and semantic annotations
  • 5,140+ asset library with procedural variations for unlimited scene diversity
  • Dual data collection: teleoperation-based for high-quality demos, automated for scale
  • 3D reconstruction pipeline importing real-world objects into simulation as faithful digital twins
03ENGINEERING

WHY THIS IS HARD

Converting natural language into physically valid simulation environments requires solving multiple hard problems:

  1. 01Semantic grounding: LLM scene descriptions must map to physically plausible object placements — not just visually reasonable, but collision-free and gravity-stable
  2. 02Asset diversity: 5,140+ objects must have correct collision meshes, mass properties, friction coefficients, and semantic labels
  3. 03Scene composition: objects must be placed with spatial reasoning — cups on tables, books on shelves, tools within reach of robot arms
  4. 04Dual collection modes: teleoperation requires VR integration and force feedback; automated mode requires task planners and grasp synthesis
  5. 053D reconstruction: real-world objects scanned via RGB-D must produce watertight meshes with accurate physical properties
  6. 06Validation: generated environments must be tested for physical stability, task feasibility, and training signal quality before data collection begins

ELYSIUM combines LLM scene understanding, procedural asset generation, physics validation, and dual-mode data collection into a single automated pipeline.

04BENCHMARKS

ENVIRONMENT GENERATION

Measured across scene generation and data collection pipelines:

ENVIRONMENT GENERATION
METRICVALUE
Asset Library Size5,140+ objects
Collection ModesDual (teleop + automated)
3D ReconstructionRGB-D → sim-ready mesh
Scene GenerationText → 3D environment
05BUILD STATUS

WHAT'S BUILT TODAY

4/7 COMPONENTS COMPLETE
WHAT'S BUILT TODAY
COMPONENTSTATUSNOTES
Asset LibraryCOMPLETE5,140+ objects with collision meshes and physical properties
Scene GenerationCOMPLETELLM-driven text-to-scene with physics validation
Teleoperation ModeIN PROGRESSVR-based demo collection in generated environments
Automated CollectionIN PROGRESSTask planner + grasp synthesis for autonomous data generation
3D ReconstructionCOMPLETERGB-D scan → watertight sim-ready mesh pipeline
Core modelsCOMPLETEScene composition and asset placement models validated
API layerIN PROGRESSREST API for scene generation requests and asset management
06APPLICATIONS

WHERE ELYSIUM DEPLOYS

  • APP_01

    UNLIMITED TRAINING ENVIRONMENTS

    Generate thousands of unique simulation environments from text descriptions — no manual 3D modeling required for robot training.

  • APP_02

    SYNTHETIC DATA GENERATION

    Produce diverse manipulation datasets at scale with dual collection modes — teleoperation for quality, automation for quantity.

  • APP_03

    DIGITAL TWIN CREATION

    Import real-world objects and spaces into simulation via 3D reconstruction — train in faithful digital replicas of deployment environments.