ReactiveGWM · v2

ReactiveGWMFlexible Control and NPC Reactivity
in Game World Models

Zeqing Wang1,2, Danze Chen1,2, Zhaohu Xing1,4, Zizhao Tong1,5, Yinhan Zhang1,6, Xingyi Yang3, Yeying Jin1

1 Tencent, 2 National University of Singapore, 3 The Hong Kong Polytechnic University, 4 The Hong Kong University of Science and Technology (Guangzhou), 5 University of Chinese Academy of Sciences, 6 The Hong Kong University of Science and Technology

Assign control roles to individual characters. Direct the players with actions, and let NPCs respond to interactions through conditional rules.

Paper overview: assigning external control to multiple characters, or assigning a player and an NPC whose rule triggers a jump in response to an approaching fireball.
Control modes are assigned at rollout initialization. The same character can receive external actions or an NPC rule, while all characters evolve together in a shared scene.

Flexible roles in a shared world

Existing game world models typically bind player and NPC roles to fixed characters. This limits multi-character games, where different characters may receive external control and NPCs must react to player-triggered interactions. ReactiveGWM assigns flexible control modes at initialization and jointly simulates externally controlled players and reactive NPCs. Spatial Role Binding grounds learned character handles in initial-frame regions using instance masks. Unified Agency Conditioning encodes player actions and conditional NPC rules into control-token groups, binding each group to the corresponding character handle. We study this interface in two shared-view 2D games: Street Fighter III and Hikou no mizu.

01 / ASSIGN

Choose who is controlled

Initial-frame masks identify each character, including characters with the same appearance. Assign each one a player or NPC role.

02 / CONDITION

Specify actions and rules

Players receive action sequences. NPCs receive an instruction describing a trigger and the response to execute.

03 / GENERATE

Simulate the shared scene

The model generates all characters together and uses causal video context to interpret NPC rules as interactions unfold.

The method

Bind the character. Condition the behavior.

Spatial Role Binding anchors character handles in the first frame. Unified Agency Conditioning connects each character’s actions or rule to its handle through cross-attention.

ReactiveGWM architecture: first-frame masks weight character handles; a causal video transformer reads role-specific player and NPC control tokens, with actor handles bound to cross-attention keys only.

Handles are injected into cross-attention keys; values carry control text. Causal self-attention reads the current and past latent frames. Training uses flow matching over future video latents.

View full-size figure ↗

01 / Generated rollouts

Street Fighter III (SF3)

Two fighters share one scene. External player actions provide the interaction context for the NPC’s conditional response.

10 selected demos

02 / Generated rollouts

Hikou no mizu (HNM)

One selected example for each of nine reaction rules, plus a passive no-op reference, from the 23 September HNM benchmark run.

10 selected demos