Hunter Zhang, PhD

Advancing off-road vehicle autonomy.

I am an Assistant Research Professor at Worcester Polytechnic Institute (WPI). My research addresses core challenges in autonomous off-road vehicle mobility through advanced modeling and simulation, intelligent perception in unstructured environments, vehicle control, and intelligent cyber-physical systems.

Research Interests

  • Vehicle Dynamics
  • Multibody Dynamics
  • Terramechanics
  • Vehicle Perception
  • Advanced Vehicle Control
  • Immersive Simulation
  • Sensor and Data Fusion
Hunter Zhang discussing an autonomous ground vehicle with two students in the lab

Real-Time, Physics-Based Co-Simulation for Off-Road Autonomy

Building on years of research in autonomous off-road mobility, I am leading the development of a real-time co-simulation platform at AVMI. The platform integrates physics-based modeling and simulation, autonomy, and mixed-reality testing to support the development and evaluation of the next generation of autonomous off-road vehicles.

Beyond supporting my own research, the platform is designed for other researchers and engineers to use and extend. Its integrated vehicle models, sensor simulations, and autonomy components provide a foundation for users to develop, test, and evaluate their own models and algorithms. This allows them to focus on their specific research questions without having to build and manage the entire simulation system from scratch.

A tan off-road utility vehicle parked on a simulated sandstone ledge above a forested valley

The platform is also heavily used in education: it provides the hands-on foundation for teaching Autonomous Vehicle Engineering, giving students direct experience with vehicle dynamics, sensing, perception, planning, and control in realistic off-road scenarios.

Key features

  • Real-time simulation
  • Physics-based vehicle dynamics and terramechanics
  • Novel virtual driveline concept for modeling independent-wheel-drive vehicles
  • High-fidelity off-road environment creation
  • Physics-based sensor simulation
  • Human-in/on/off-the-loop simulation
  • Full autonomy stack including intelligent perception, mobility-based planning, advanced control
  • Mixed-reality co-simulation
  • Real-synthetic data fusion
  • Real-world, synthetic, and real-synthetic fused data generation
  • Intelligent cyber-physical off-road platform
Architecture diagram: virtual reality, the intelligent autonomy stack, human-in/on/off-the-loop simulation and the cyber-physical vehicle, linked by a 5G data stream around the mixed-reality co-sim environment

Complete Co-Sim Architecture

Integrates four elements into one immersive mixed-reality co-simulation environment: a physics-based virtual world with vehicle dynamics and sensor models, an intelligent autonomy stack, human-in/on/off-the-loop simulation through a physical cockpit and immersive display, and a cyber-physical off-road vehicle platform. A 5G data stream links them in real time, so virtual and real components operate as a single system.

Capabilities of the simulator

Multibody vehicle model with suspension springs and tyre force vectors driving over simulated terrain, with live dynamics plots

Modeling and Simulation of Vehicle System Dynamics

In this system, a standalone simulator was developed for real-time, physics-based simulation of vehicle dynamics at 1 kHz, including nonlinear tire–ground interactions, multibody steering and suspension models, and engine and driveline dynamics. It also incorporates a novel virtual driveline concept for modeling and simulating vehicles with independent wheel-drive motors.

VAE-generated terrain shown in an interactive 3D view with contour lines

High-Fidelity Terrain Generation

Uses a novel integration of a variational autoencoder (VAE) and W–M noise to generate 1 km × 1 km landscapes with detailed surface roughness at spatial resolutions as fine as 2 cm. A newly developed method for quantitatively assessing vehicle topographical mobility provides constraints for the generation process, enabling terrain to be tailored to mobility evaluations of specific vehicle configurations.

Photorealistic simulated off-road ground with vegetation and a moss-covered boulder

Off-Road Environments for Sensor Simulation and Vehicle–Environment Interaction

Creates off-road environments with both photorealism and structural realism in under five minutes. An automated pipeline adds surface textures to the terrain topography and places obstacles in designated regions. Currently, the system supports generation of forest, desert, and mountain environments.

Simulated RGB camera and semantic camera images beside the semantically labelled LiDAR point cloud of an off-road scene

Modeling and Simulation of Vehicle Exteroceptive Sensors

The system supports real-time, physics-based simulation of RGB cameras and 3D LiDAR, with segmentation outputs for both images and point clouds.

Camera image, semantic segmentation, raw LiDAR points and segmented LiDAR points shown as a two-by-two grid

Intelligent Perception

As part of the autonomy stack, the system integrates camera imagery and 3D LiDAR data for semantic scene understanding and depth estimation. A SegFormer-B2 transformer performs image semantic segmentation, while the results are fused with LiDAR measurements to associate identified scene regions with depth information.

Driving camera view beside mobility maps of the terrain that build up as the vehicle explores

Real-Time Topographical Mobility Assessment

The autonomy stack can evaluate vehicle topographical mobility in real time using the point cloud from every single LiDAR sweep, identifying terrain regions the vehicle can traverse without chassis–terrain collision. The resulting topographical mobility assessment informs real-time motion planning.

Vehicle driving through forest with a cost-map inset showing obstacles, the global route and the local trajectory bending around them

Mobility-Aware Motion Planning and Advanced Control

The system supports global and local path planning using predefined go/no-go maps and real-time topographical mobility assessments. A baseline trajectory-following controller with adaptive speed control is currently implemented. A model predictive control (MPC) strategy based on the virtual driveline concept is being integrated into the autonomy stack to enhance the vehicle’s off-road maneuverability.

Aerial view of the simulated vehicle driving through forest, with LiDAR returns and live planning and perception panels

Full Autonomy Stack

The system integrates multimodal perception, real-time topographical mobility assessment, motion planning, and vehicle control into a unified autonomy framework. Together with the vehicle simulator, it forms a complete pipeline for simulating and evaluating autonomous off-road vehicle operation.

The electric off-road UGV on grass, with its LiDAR and stereo camera mounted on the body

Intelligent Cyber-Physical System

A purpose-built electric off-road UGV with independent-wheel-drive motors serves as the physical counterpart of the simulator. It carries cutting-edge sensors and controllers, including a 3D LiDAR, a stereo camera, GNSS/INS, and wheel encoders, with dSPACE embedded computers running perception, planning, and control through a real-time RTMaps/ROS 2 sensor-fusion system. The platform validates simulation-developed autonomy on real terrain and collects real-world data for model training and validation.

Mixed-reality setup at AVMI: a real ground vehicle in the lab, its virtual twin in a simulated copy of the lab, and the combined mixed-reality point cloud

Mixed-Reality Environment

Connects virtual simulation with the cyber-physical vehicle platform through 5G communication, enabling the exchange and fusion of real and simulated sensor data. This environment supports simulation validation and fidelity improvement while expanding the range of operational scenarios available for testing physical vehicles.

A driver at the wheel inside the immersive display cave, the simulated forest wrapped around the screen

Human-in-the-Loop Simulation

A human-in-the-loop driving interface places the operator directly inside the simulation. A professional direct-drive cockpit with up to 25 N·m of force feedback lets the driver feel the simulated tire–terrain forces, while a curved immersive display cave with a 210° field of view or a VR headset presents the environment. The cockpit uses the same steering, throttle, and brake commands as the autonomy controller, so manual driving and autonomy supervision run in one loop.

The simulator driving through a forest with its live semantic segmentation view, while a terminal records ROS 2 bag files

Annotated Synthetic Sensor Data Generation

The simulator doubles as a data generation system, recording synchronized RGB images, pixel-level semantic masks, raw and semantically labeled LiDAR point clouds, and vehicle state, all saved in standard ROS 2 bags. All annotations are generated automatically and exactly by the renderer, with no manual labeling.

A dataset of 1 million synchronized, annotated LiDAR and camera samples with vehicle state information, covering forest, desert, and mountain environments, will be publicly released soon.

Every component is developed in-house, from the vehicle dynamics solver and the physics-based sensor models to the realistic environment generation and the autonomy stack itself. Nothing is adapted from an existing simulator.

Autonomous Systems Laboratory

With $2M in support from the Massachusetts Technology Collaborative (MassTech), I designed and developed a unique Autonomous Systems Laboratory (ASL) within AVMI.

The ASL supports modeling and simulation, engineering design, prototyping, physical and mixed-reality testing, data generation, and operator training for autonomous off-road systems. Its connected physical and virtual platforms enable researchers to develop autonomy algorithms, evaluate vehicle performance, and investigate vehicle–environment interactions.

This laboratory also supports hands-on education, having engaged WPI students across more than six WPI departments, and provides the practical foundation for a developing curriculum in autonomous and off-road vehicle engineering.

Advising at WPI

Since joining WPI in 2023, I have advised more than 100 undergraduate and graduate students across six departments through Major Qualifying Projects (MQPs), independent studies, and directed research.

Selected students’ work

FSAE Club

I also serve as the faculty advisor of WPI Formula SAE, supporting students as they design, build, and compete with an electric formula-style race car in SAE International’s collegiate design competition.

Teaching at WPI

I am passionate about teaching and see contributing to engineering education as a lifelong commitment. I believe students learn engineering best by putting theory into practice, so I bring my research in vehicle dynamics, off-road mobility, autonomy, and simulation into the classroom and laboratory. Through hands-on projects and experiments, I help students connect fundamental concepts with practical challenges and build the skills and confidence to solve engineering problems.

Courses taught

  • ES3011 Control Engineering
  • ME4323 Fundamentals of Vehicle Drivetrain Systems
  • ME4506 Mechanical Vibration
  • Summer program Frontier Mechanical Engineering