Add Newton physics backend with XPBD solver support

[Douglas-Tilley]

Summary

Complete Newton physics backend implementation for the ARK simulator framework. This adds Newton as a fourth backend option alongside PyBullet, MuJoCo, and Genesis.

The Newton backend provides GPU-accelerated physics simulation using Newton's XPBD (Extended Position-Based Dynamics) solver with full integration into ARK's existing architecture.

Files Changed (14 files)

Core Framework Changes (2 files)

File	Description
ark/system/simulation/simulator_node.py	Added Newton backend option to the backend selection logic. When backend_type: "newton" is specified in config, the simulator now imports and instantiates NewtonBackend.
ark/client/comm_handler/subscriber.py	Minor enhancement to subscriber handling for improved message processing.

Newton Backend Implementation (12 files)

File	Description
ark/system/newton/newton_backend.py	Main backend class implementing SimulatorBackend ABC. Handles initialization, physics stepping via XPBD solver, collision detection, and component lifecycle management. Configurable via YAML with gravity, substeps, solver iterations, and joint control parameters.
ark/system/newton/newton_builder.py	Wrapper around Newton's ModelBuilder with comprehensive defensive programming. Provides safe URDF loading, joint configuration defaults, and scene metadata tracking. Ensures robots are loaded before primitives to preserve body indices.
ark/system/newton/newton_robot_driver.py	Robot driver implementing ARK's RobotDriver interface. Supports TARGET_POSITION, TARGET_VELOCITY, and FORCE joint control modes. Publishes joint_state_t messages and subscribes to joint_group_command_t for arm/gripper control.
ark/system/newton/newton_camera_driver.py	Camera sensor driver for RGB and depth image capture. Integrates with Newton's viewer for GPU-accelerated rendering. Publishes images via ARK's standard image channels.
ark/system/newton/newton_lidar_driver.py	LiDAR sensor driver for point cloud generation using Newton's ray-casting capabilities. Configurable scan parameters (range, resolution, FOV).
ark/system/newton/newton_multibody.py	Rigid body management for non-articulated objects (cubes, spheres, etc.). Handles primitive shape creation and ground truth pose publishing.
ark/system/newton/newton_viewer.py	Viewer manager abstracting Newton's visualization. Supports GUI mode (OpenGL viewer) and headless mode. Handles render loop and state visualization.
ark/system/newton/geometry_descriptors.py	Solver-agnostic geometry representation. Defines shape descriptors (box, sphere, capsule, mesh) that can be adapted to different Newton solvers.

Solver Adapters (4 files)

File	Description
ark/system/newton/scene_adapters/__init__.py	Package exports for solver adapters.
ark/system/newton/scene_adapters/base_adapter.py	Abstract base class defining the adapter interface. Adapters handle solver-specific scene building requirements (ground planes, contact parameters, etc.).
ark/system/newton/scene_adapters/xpbd_adapter.py	XPBD solver adapter. Uses Newton's native add_ground_plane() and standard contact handling. Default adapter for most use cases.
ark/system/newton/scene_adapters/mujoco_adapter.py	MuJoCo solver adapter. Works around MuJoCo's ground plane limitations by using explicit box geometry. Useful when MuJoCo solver is preferred for specific simulation characteristics.

Architecture

The Newton backend follows ARK's existing backend pattern:

• Implements SimulatorBackend ABC (same interface as PyBullet/MuJoCo backends)
• Provides NewtonRobotDriver implementing RobotDriver interface
• Integrates via backend_type: "newton" in global config YAML
• Uses LCM pub/sub for joint state/command communication

Configuration Example

simulator:
  name: "franka_newton_simulator"
  backend_type: "newton"
  node_frequency: 240
  config:
    connection_mode: "GUI"
    gravity: [0, 0, -9.81]
    sim_frequency: 120
    substeps: 20
    solver: "xpbd"
    solver_iterations: 10
    device: "cuda:0"

    newton_physics:
      joint_defaults:
        mode: "TARGET_POSITION"
        target_ke: 4000.0
        target_kd: 100.0
        armature: 1.0

Test Plan

• Verified backend_type: "newton" correctly loads NewtonBackend
• Verified Newton IK solver works for pick-and-place sequence
• Verified joint command pipeline (publish command → physics step → state feedback)
• Verified XPBD solver stepping and collision detection
• Verified viewer renders robot and scene correctly
• Tested with Franka Panda robot using keyboard controller

Verification

The Newton backend is 100% Newton-native - it uses only Newton APIs for physics, kinematics, and collision detection. No PyBullet or MuJoCo code is used within the Newton backend implementation.

Future Work (TODO)

• Add Featherstone solver adapter for articulated body dynamics
• Add Semi-Implicit solver adapter for faster but less stable simulations
• Implement force/torque sensor driver
• Add contact force reporting via LCM messages
• Support for soft body simulation (Newton's FEM capabilities)
• Expose parallel environment replication via YAML config

[Refinath]

why did you remove this . please keep latest changes

[Refinath]

Please keep the latest changes exists in the main branch [not only these changes]

[Refinath]

Please update the PR

[cmower]

Hi @Douglas-Tilley thanks so much for this contribution, from the above comments from @Refinath i suspect you may need to rebase your branch. If you can do this asap, I will merge 😄

[Refinath]

I’ve updated the PR with the latest changes from main. However, when I tried running Newton Physics, I encountered the following error:

newton.JointType.TARGET_POSITION
AttributeError: TARGET_POSITION

I’m using Newton version 0.2.0.

[Douglas-Tilley]

Newton has updated and removed this as an Attribute. I will update the code to reflect this.

"JointMode has been removed and it is now possible to set both joint target position and velocity simultaneously using the :attr:newton.Control.joint_target_pos and :attr:newton.Control.joint_target_vel attributes."

I'll rebase and update appropriately