Skip to main content

Primary supervisor

Fucai Ke

Co-supervisors


This project focuses on developing agentic world generation systems for robotic intelligence. Rather than manually constructing simulation environments one by one, the research investigates how AI agents can program, generate, inspect, and refine structured 3D worlds that can serve as scalable training environments for robots.

This project will investigate representations and generative mechanisms that allow agents to construct controllable and physically meaningful environments, combining ideas from procedural generation, structured scene programs, generative models, and agentic AI. The project can span a broad range of environments, from indoor rooms and buildings to outdoor environments, streets, city blocks, and large-scale urban worlds. A key goal is to move toward a programmable world engine for robotics, where diverse environments and their corresponding ground-truth information can be generated automatically for embodied AI training and evaluation.

Potential Applications and Project Directions

  • Game and virtual world generation: automatically creating diverse, editable, and interactive 3D environments.
  • Indoor space and architectural design: generating rooms, buildings, floor plans, and structured indoor environments from high-level requirements.
  • Urban and city-scale generation: designing streets, city blocks, and larger urban environments under spatial and functional constraints.
  • Robotic simulation and training: generating diverse environments and scenarios for training and evaluating embodied agents.
  • Synthetic data generation: automatically producing scenes together with geometry, semantics, spatial relationships, and other ground-truth annotations.
  • Agentic environment design: developing AI agents that can generate, inspect, modify, and iteratively improve complex environments.

Aim/outline

  • Investigate structured representations for programmable and generative 3D worlds.
  • Develop agentic systems that can generate, inspect, modify, and correct scene environments.
  • Explore scalable generation from indoor spaces to outdoor and city-scale environments.
  • Improve the diversity, controllability, physical validity, and semantic consistency of generated worlds.
  • Automatically generate scene information and annotations useful for robotic perception, reasoning, and interaction.
  • Develop a general world-generation framework for training and evaluating future robotic systems.

URLs/references

  • Generative Floor Plan Design with LLMs via Reinforcement Learning with Verifiable Rewards, Findings of ACL 2026.
  • Procedural Scene Programs for Open-Universe Scene Generation: LLM-Free Error Correction via Program Search, SIGGRAPH Asia 2025.
  • Unified Vector Floorplan Generation via Markup Representation, 2026.
  • Infinigen: Infinite Photorealistic Worlds using Procedural Generation, CVPR 2023
  • Infinigen project page: infinigen.org

 

For any inquiries regarding this project, please contact Fucai Ke via fucai.ke@monash.edu

Required knowledge

  • Strong willingness and ability to continuously learn new concepts and technologies.
  • Ability to commit consistent time to the project every week.
  • Good programming skills in Python.
  • Basic knowledge of machine learning, computer vision, robotics, or computer graphics.
  • Experience with PyTorch, generative models, LLMs, reinforcement learning, Blender, or 3D graphics is desirable.
  • Ability to work independently and conduct research systematically.