GigaBrain-0.7: Scaling Embodied Foundation Models to Emergent Capabilities with a Three-System Architecture

The first family extends the VLM’s native autoregressive interface to robot control by representing continuous actions as discrete token sequences. RT-2 [6] and OpenVLA [39] predict robot-action tokens through next-token prediction, keeping the VLM itself as the action generator. Subsequent work improves the efficiency and expressiveness of action tokenization. FAST [60] exploits temporal redundancy through frequency-domain compression, while learned tokenization methods [44, 19] learn compact discrete representations of continuous action sequences. Galaxea G0.5 [23] further develops this direction with a learned cross-embodiment action tokenizer, allowing reasoning and action tokens to be generated within a single autoregressive Transformer stream. These approaches preserve a direct interface between language modeling and physical action, but sequential token decoding becomes increasingly costly as control frequency, prediction horizon, and action dimensionality grow. OpenVLA-OFT [40] consequently explores parallel decoding, action chunking, and continuous action representations for more efficient downstream adaptation of autoregressive VLA backbones.

A second family separates multimodal reasoning from continuous action generation. Rather than requiring the VLM itself to decode low-level control, these architectures use the pretrained VLM as a multimodal context encoder and attach a separately parameterized action decoder or Action Expert. Representative cascaded or decoupled designs include GR00T N1 [56], Gemini Robotics [25], and Qwen-RobotManip [87]. In this paradigm, vision-language representations are first constructed by the VLM and subsequently consumed by the motor generator through an explicit feature interface. Qwen-RobotManip, for example, employs a flow-matching DiT whose blocks cross-attend to visual and language representations extracted from the final VLM layer. Such modular designs allow the action module to specialize in continuous control, while their effectiveness depends strongly on the information exposed through the VLM–action interface.

A third and increasingly common direction retains a dedicated continuous Action Expert while coupling it more deeply with the vision-language backbone through Mixture-of-Transformers architectures [5, 62, 63, 27, 9, 79, 86, 89]. $\pi_{0}$ [5] established the influential VLM–Action Expert formulation with block-wise causal interaction and flow-matching action generation. The same architectural lineage is extended in $\pi_{0.5}$ [62] and $\pi_{0.7}$ [63] with heterogeneous pretraining, hierarchical task conditioning, observation history, and richer context conditioning. GigaBrain-0 [27] similarly combines a pretrained VLM with a continuous Action Expert in an MoT formulation, together with RGB-D perception, Embodied Chain-of-Thought supervision, and Knowledge Insulation [20]. Wall-OSS-0.5 [86] introduces layer-wise VL and Action Experts with joint attention and end-to-end gradient flow, while HyVLA-0.5 [89] combines an embodied-native MoT backbone with a flow-matching Action Expert and compact temporal memory. Xiaomi-Robotics-0 [9] and Xiaomi-Robotics-1 [79] also adopt VLM–DiT MoT architectures, conditioning flow-matching action generation on VLM representations and KV caches.

Although these models differ in attention routing and gradient coupling, they share a common design principle: a pretrained VLM is paired with a dedicated continuous Action Expert, with different architectures exposing different degrees of interaction between the two components. Several recent systems additionally incorporate visual history into the policy context. HyVLA-0.5 [89] introduces a compact memory encoder for spatiotemporal context, while $\pi_{0.7}$ [63] incorporates multi-frame observation history into context-conditioned policy learning. GigaBrain-0.7 follows the parallel VLM–Action Expert direction for continuous action generation, while introducing separate functional interfaces for understanding and planning, and for future-state prediction and evaluation.

Most VLAs initialize from pretrained vision-language models and subsequently introduce embodied and action supervision, but differ in how strongly these objectives are separated during optimization. Early autoregressive approaches such as RT-2 [6] and OpenVLA [39] adapt pretrained multimodal representations to robot control through action-token prediction. $\pi_{0.5}$ [62] adopts a staged training recipe in which robot actions are represented with discrete tokens during broad pretraining, while a flow-matching Action Expert is introduced during post-training for continuous action generation. Xiaomi-Robotics-0 [9] further decomposes VLA pretraining into two optimization steps: it first trains the VLM jointly on vision-language and robot-trajectory supervision and subsequently freezes the VLM while training the flow-matching DiT. Xiaomi-Robotics-1 [79] similarly adopts an explicit pretraining–post-training recipe, pretraining on over 100,000 hours of real-world UMI trajectories with automatically generated state-transition language and subsequently aligning the learned capabilities to robot embodiments and imperative task instructions.

A complementary direction increasingly unifies vision-language and action objectives within the same VLA pretraining stage. GigaBrain-0 [27] jointly optimizes embodied reasoning, discrete-action prediction, and continuous flow-matching action generation under a unified objective. Wall-OSS-0.5 [86] develops a single-stage gradient-bridged co-training recipe that jointly optimizes multimodal cross-entropy, action-token prediction, and continuous flow matching. Qwen-RobotManip [87] emphasizes cross-source alignment as a prerequisite for scaling heterogeneous manipulation data across different representations, motions, and behaviors. LingBot-VLA 2.0 [78] further scales pretraining to approximately 60,000 hours of embodied data spanning diverse robot configurations and egocentric human experience. GigaBrain-0.7 follows this joint-training direction and scales it to heterogeneous multi-embodiment data, jointly optimizing general vision-language supervision, hierarchical task prediction, discrete action representations, and continuous action generation within the same VLA pretraining stage.

Large-scale pretraining establishes broadly transferable representations and manipulation priors, while post-training specializes these capabilities to target embodiments, tasks, and deployment conditions. $\pi_{0.5}$ [62] introduces continuous flow-matching action generation during post-training and specializes the pretrained model on high-quality task-relevant robot data. OpenVLA-OFT [40] studies efficient downstream fine-tuning of autoregressive VLA backbones through parallel decoding, action chunking, and continuous action prediction. Xiaomi-Robotics-1 [79] uses cross-embodiment robot data during post-training to align UMI-pretrained capabilities with robot embodiments and imperative instructions. HyVLA-0.5 [89] similarly specializes its pretrained model to target robot embodiments through supervised fine-tuning before subsequent reinforcement-learning refinement. Together, these approaches establish post-training as an important bridge between general-purpose pretraining and high-precision downstream robot behavior.

Supervised post-training remains bounded by the state distribution and behavioral quality represented in demonstration data, motivating the use of policy-generated experience for further improvement. Recent approaches therefore extend VLA learning with reinforcement learning over simulated or real-robot rollouts. VLA-RL [52] and SimpleVLA-RL [46] investigate direct reinforcement-learning updates of pretrained VLAs from environment feedback, while Z-1 [10] develops a GRPO-based post-training framework for flow-based VLAs with prefix-based rollout construction and success-aware reward design. RL Token [80] instead exposes a compact representation from a pretrained VLA as an interface for lightweight online actor–critic optimization.

Other approaches make greater use of offline experience, advantage information, or preference supervision. AWR [59] performs advantage-weighted regression over off-policy experience, while $\pi_{0.6}^{*}$ [61] introduces RECAP (RL with Experience and Corrections via Advantage-conditioned Policies), which learns from demonstrations, on-policy experience, and expert corrections through advantage conditioning. GigaBrain-0.5M* [28] introduces RAMP, in which world-model-predicted future states and values condition policy fine-tuning and are iteratively combined with human-in-the-loop rollout experience. HyVLA-0.5 [89] introduces FlowPRO, a critic-free and reward-free PRO-based offline RL method that learns from paired failure and corrective trajectories. Together, these approaches move VLA adaptation beyond static imitation toward closed-loop learning from policy-generated experience and corrective feedback.

One line of work treats world models as scalable data engines that expand the distribution available for policy training. GigaBrain-0 [27] uses world models to synthesize diverse embodied experience through video generation, Real2Real and Sim2Real transfer, viewpoint variation, and human-to-robot transfer. DreamGen [38] similarly uses video world models to generate synthetic interaction trajectories and recovers corresponding robot actions through latent-action or inverse-dynamics models. These approaches primarily use predictive models to increase the diversity and coverage of training experience beyond what can be collected efficiently on physical robots.

A second direction integrates visual dynamics and action prediction within a common generative model. GR-2 [12] combines future video generation with robot-action prediction. DreamZero [83] builds a World-Action Model on a pretrained video diffusion backbone that jointly models future world evolution and robot actions. Unified World Models (UWM) [95] couples video and action diffusion within a unified architecture that supports policy, forward-dynamics, inverse-dynamics, and video-generation modes. GigaWorld-Policy [82] develops an action-centered World-Action Model, using future visual dynamics to provide additional supervision for action learning while allowing explicit future-video generation to be omitted during deployment. Cosmos Policy [41] fine-tunes pretrained video models for visuomotor control and supports robot-action prediction together with future-state and value prediction for planning. Cosmos 3 [57] further extends world modeling toward an omnimodal foundation model for physical AI.

Beyond unified World-Action Models, World-VLA-Loop [50] studies closed-loop refinement between a video world model and a VLA policy. Policy-generated failure rollouts are used to improve the world model, which subsequently provides an updated learned environment for further VLA reinforcement learning. This formulation allows the world model and policy to improve iteratively rather than treating either component as fixed.

A complementary line retains a VLA-oriented policy while using predictive representations, visual futures, or task-progress estimates to improve policy learning and execution. JEPA-VLA [55] integrates video-predictive visual embeddings into existing VLA models to strengthen representations of temporal dynamics and action-relevant visual information. $\pi_{0.7}$ [63] conditions policy behavior on generated visual subgoals together with language, observation history, and episode-level context. GigaBrain-0.5M* [28] introduces RAMP, where a separately pretrained world model predicts future states and task values that condition VLA policy learning and support iterative refinement from human-in-the-loop rollout experience. Feat2Go [69] derives continuous task-progress targets from a pretrained visual world model, trains a value model to estimate this progress, and uses the resulting values for reward shaping during policy optimization.

Together, these works establish complementary roles for predictive models in robot learning, including data generation, dynamics representation, World-Action modeling, policy conditioning, value estimation, and closed-loop policy improvement. GigaBrain-0.7 builds on these directions through a dedicated System 3 for prediction and evaluation. System 3 is pretrained separately for robot-centric future-state prediction and task-progress estimation. Its predicted subgoal images and value-derived progress signals are subsequently provided as additional conditions during task-specific VLA post-training. At inference time, the policy is conditioned on positive progress to guide continuous action generation, without explicit candidate-action scoring or selection. The resulting design coordinates predictive modeling, conditional action generation, and subsequent experience-driven refinement while retaining specialized objectives for understanding, prediction, and action.

The real-robot corpus is assembled from in-house robot deployments and large-scale real-world robot datasets, including [58, 2, 3, 22, 77, 36]. After cleaning and standardization, the real-robot corpus spans 16 robot types, 1,810,101 episodes, 2,077,837,071 frames, and a total of 20,535.65 hours. Maker H01, Maker M01, Agibot-G1, AgileX, Ark, and Galaxea R1 Lite constitute the major data sources, while Franka, UR5, Realman RMC-AIDA-L, and other platforms further extend cross-embodiment and cross-hardware coverage. This distribution provides the basis for learning action patterns across different degrees of freedom, end-effectors, and locomotion modalities. Tab. 2 reports the detailed statistics by robot type.

The UMI corpus is composed of in-house Maker U01 demonstrations and the Jianzhi 10K dataset [26]. After cleaning, it contains 8,251.83 hours, including 6,876.25 hours of in-house data and 1,375.58 hours from Jianzhi 10K. This data source provides large-scale human manipulation demonstrations from near-manipulation viewpoints that visually resemble local observations from wrist-mounted or end-effector-proximal robot cameras. It therefore complements real-robot trajectories with supervision for hand–eye coordination, local spatial geometry, and object-contact relationships.

The EGO corpus contains 2,862.36 hours after cleaning and combines in-house Maker E01 data with EgoDex [35], EgoVerse [64], and World in Your Hands (WiYH) [91]. Because first-person human videos share similar observation viewpoints with robot-mounted cameras, they provide broader object categories, manipulation scenes, and human-interaction priors, strengthening semantic understanding and temporal modeling of real manipulation processes. Tab. 3 summarizes the UMI and EGO data composition.

To complement physically collected trajectories, we further construct 5,607.14 hours of simulation and generated data through two complementary pipelines. The simulation subset contains 1453.92 hours of trajectories collected from configurable physics-based environments, including high-quality trajectories curated from [24, 15, 31, 54, 33], while the world-model-generated subset contributes 4,153.22 hours of data derived from real interaction samples. The two sources target complementary regimes: simulation provides controllable coverage of task and scene configurations that are costly or difficult to reproduce on physical robots, whereas generation expands the visual and contextual diversity of existing real-world interaction data. Together, they increase coverage of long-tail interaction conditions while reducing the need to repeatedly collect rare or safety-sensitive scenarios on physical hardware.

In addition to embodied trajectory data, we construct 271,976,674 VLM image–text/question-answering samples, including 15,234,327 self-collected samples, to preserve foundational vision-language capabilities and strengthen the spatial understanding, object localization, and affordance reasoning required by robotic tasks. The corpus covers image captioning, general visual question answering, multi-image reasoning, region grounding, point prediction, spatial-relation reasoning, robotic task understanding, and affordance learning.

The captioning data include [85, 47, 84, 11, 67, 68], and image–text samples converted from [4], providing supervision for open-vocabulary recognition and image-level semantic description. The general VQA data include [76, 18, 30, 72, 43, 21, 49]. Embodied-specific data include [13, 88, 45, 42, 73, 93, 81], strengthening spatial referring, target localization, manipulation-feasibility judgment, and trajectory-related reasoning in robotic scenes.

We first convert raw data into a unified training-storage format. For LeRobot datasets, all samples are standardized to the LeRobot v3.0 format, improving data-loading efficiency and unifying the organization of episode, frame, observation, action, and language instruction fields. During conversion, we also organize dataset provenance, robot type, task description, camera list, sampling frequency, state dimensionality, action dimensionality, and cleaning flags, enabling downstream training to perform precise filtering and sampling by data source, robot morphology, and task type.

To handle mismatched degrees of freedom and sensor configurations across robot embodiments, we construct a unified robot state and action representation. The dimension ordering follows left arm, right arm, head, waist, and base or legs, while both joint-space and end-effector-space fields are retained to support different supervision signals. Under this representation, joint states, end-effector poses, gripper states, base velocities, and camera observations are organized into a reusable standard format. Missing motion components are represented with masks or placeholder dimensions, avoiding embodiment-specific data formats for individual robot morphologies. Because camera naming and mounting conventions also vary across platforms, preprocessing maps camera fields from each source to standardized camera definitions while retaining the original camera names for traceability.

Raw task instructions often contain inconsistent naming, mixed languages, heterogeneous description granularity, or weak correspondence with the actual task. For example, the same manipulator or gripper may be referred to as left/right/both gripper or hand across different data sources; some Chinese instructions must be aligned with English training corpora; and some samples contain only scene-level words without executable task descriptions. To address these inconsistencies, we adopt an LLM-assisted instruction-standardization pipeline. We first extract instructions using rule-based procedures, rewrite them into canonical task instructions with GLM-5.1 [29], and write the normalized instructions back to the dataset, followed by human spot checks. To improve modeling of long-horizon task structure, we further construct subtask-level annotations. For robot manipulation videos, a multimodal large model temporally segments the manipulation process and generates atomic subtask descriptions; samples with ambiguous task boundaries or unstable model annotations are further corrected through human annotation and review.

The cleaning stage filters or corrects numerical anomalies, prolonged stationary segments, kinematic inconsistencies, and training-anomalous samples.

First, we perform q01/q99 extreme-value filtering. Because quantile normalization is used during training, extreme outliers can substantially stretch the normalization interval and compress normal action signals into a narrow range, thereby destabilizing optimization. We therefore compute the 1st and 99th percentiles independently for each robot type and action dimension and remove anomalous frames or trajectories outside the valid range.

Second, we remove prolonged stationary segments. We compute inter-frame state differences, normalize each dimension by the corresponding joint range of motion, and then calculate the overall $L_{2}$ change magnitude. When the magnitude falls below a predefined threshold, the corresponding frame pair is classified as stationary; consecutive stationary pairs are merged into stationary segments. This procedure reduces the proportion of long, uninformative idle periods and focuses training on effective interaction behaviors.

Third, following Qwen-RobotManip [87], we perform end-effector-pose consistency correction. For each robot type in the training corpus, we collect the corresponding URDF files, align the base coordinate systems, and manually configure the end-effector reference point, thereby standardizing end-effector pose definitions across embodiments.

Finally, we use anomalous training losses as an additional filtering signal. During early pretraining, we record sample-level and data-source-level loss distributions. If particular trajectories persistently produce abnormally high losses or exhibit large loss spikes, we trace them back to the original video, action sequence, and language annotation to identify temporal misalignment, action discontinuities, incorrect instructions, or video corruption. Confirmed anomalous samples are then added to the filtering list.

For UMI demonstrations, we first synchronize multi-camera images, handle poses, gripper states, and host-side timestamps to ensure temporal consistency between visual observations and manipulation trajectories. The acquisition system relies on synchronized multi-camera exposure and triggering, while handle-pose and gripper-opening signals are recorded through wired connections, providing a common temporal basis for subsequent cross-modal alignment, trajectory retargeting, and executability validation.

For quality control, we screen raw clips according to trajectory smoothness, information density, cross-modal consistency, and embodiment executability. We remove samples with pose loss, rapid jitter, prolonged stationary periods, or clear inconsistencies between projected handle motion and visual observations. The retained trajectories are then mapped to the Maker H01 end-effector Tool Center Point (TCP) frame and validated through inverse-kinematics solving, simulation-based executability checks, and sampled real-robot replay. After processing, UMI data are organized into unified training samples containing visual observations, end-effector actions, timestamps, and task-semantic information, and are jointly used with real-robot data for policy learning.

EGO data supplement first-person examples of human manipulation, allowing the model to observe how human hands approach target objects, establish contact, and complete actions such as moving, placing, or opening objects.

We first apply video-level quality filtering and remove clips without clear hand–object interaction, with corrupted frames, prolonged stationary periods, severe occlusion, or viewpoints unsuitable for interpreting the manipulation process. This ensures that retained clips clearly expose the hand, target object, and resulting manipulation outcome.

We then standardize video frames, camera parameters, hand trajectories, and existing action annotations. Timestamps, coordinate systems, and field formats are normalized across data sources so that video observations, hand motion, and action labels can be chronologically organized into training samples consistent with robot trajectories. For data with hand keypoints or hand trajectories, we organize the temporal evolution of hand positions and associate them with contacted or manipulated target objects. For samples without hand- or object-position annotations, we use camera-trajectory estimation, hand-pose estimation, or existing annotations to supplement interaction locations, target objects, and their relative spatial relationships. After processing, EGO samples enter training as current observations paired with language descriptions and future actions or hand-motion trajectories, strengthening the model’s understanding of first-person manipulation processes, hand–object contact relationships, and near-field spatial interactions.

The generation pipeline starts from curated real-world interaction samples and expands their scene and visual conditions through prompt-conditioned video generation. Generation tasks are constructed from the source interaction together with configurable scene and generation conditions, producing additional samples that preserve the underlying interaction context while broadening the visual and environmental coverage of the training distribution. After generation, the resulting samples are organized under the same data management and quality-control framework as the remaining embodied corpus.

In parallel, we construct simulation tasks from configurable robot, object, and scene assets. Each task is instantiated in a physics-based environment and executed to produce temporally consistent simulated interaction data. Compared with physical collection, this pipeline provides a controllable and repeatable source of interaction experience for configurations that are difficult to reproduce frequently or safely on real hardware.

The two pipelines serve complementary purposes. World-model-based generation expands the diversity of real interaction samples while retaining their visual and behavioral context, whereas physics-based simulation provides explicit control over task and scene configurations. Their combination broadens the coverage of long-tail interaction conditions and provides additional experience beyond what can be collected efficiently from physical robots alone.

The VLM corpus combines public image–text/question-answering datasets, converted video–text resources, embodied-specific VLM datasets, and self-collected embodied annotations derived from our trajectory data. We process all sources into a unified, training-ready multimodal dialogue format. The pipeline consists of annotation-format unification, image quality control, and sample-structure validation, ensuring consistency in text fields, image files, visual placeholders, dialogue-role ordering, and task metadata before training.

During annotation-format unification, image descriptions, single-turn question answering, multi-turn question answering, and embodied-specific supervision are mapped into a common dialogue representation according to task type, with appropriate image placeholders retained for samples that require visual input. This step reduces formatting discrepancies across data sources and enables direct integration with the downstream training framework. During image quality control, we check image readability, resolution, aspect ratio, color mode, and content validity, standardizing repairable samples and removing samples that are missing, corrupted, or insufficiently informative. Before final export, we further validate field completeness, image–text path consistency, dialogue-role ordering, and non-empty text constraints, producing structurally consistent and quality-controlled VLM training data.

In addition to aggregating public VLM datasets, we generate self-collected embodied VLM annotations from the cleaned real-robot, UMI, and EGO data. We extract keyframes, short video clips, historical observation windows, and the corresponding task instructions or trajectory context, and organize them into annotation units that can be processed by multimodal models. We then use Qwen3.6-27B [66] for large-scale batch annotation. Following the annotation strategy in [9], the model extracts intermediate structured states including object instances, interactable parts, spatial relations, viewpoint information, task stages, and action intent, from which VLM supervision samples are subsequently derived.

For affordance-prediction tasks, the annotation pipeline focuses on identifying the interactable properties of objects and object parts, including graspable regions, pressable or pullable components, placeable regions, and manipulable objects relevant to the task objective. The multimodal model generates candidate interaction regions and corresponding language descriptions from object states in images or videos. These annotations are further transformed into supervision for target localization, component identification, affordance judgment, and placement-region selection, enabling the model to learn correspondences among object function, manipulation constraints, and locally reachable space.

For spatial-understanding tasks, the annotation pipeline constructs spatial relationships among objects and between objects and the observer. For data with depth, multi-view observations, or camera parameters, geometric information is preferentially used to generate supervision for relative position, proximity, occlusion, containment, and free space. For RGB-only images or first-person videos, candidate spatial relationships are generated by combining visual cues with model reasoning and are subsequently filtered through rule-based consistency checks. This data is primarily used to construct spatial-relation judgment, referring-expression localization, region-grounding, and spatial-feasibility question-answering samples.

For high-level task-planning tasks, annotations are built from real-robot trajectories, UMI near-manipulation demonstrations, and EGO first-person human-manipulation clips, with emphasis on task stages, current subtasks, next-step actions, and task-completion states. The multimodal model performs temporal understanding over continuous clips and generates current-state summaries, subtask labels, action pre/post relationships, and plausible next operations. These structured annotations are further converted into next-step action prediction, task-progress judgment, action-consequence prediction, and goal-consistency question-answering samples, providing supervision for long-horizon task understanding and high-level planning.

To reduce noise from automatic generation, we apply multi-level quality control after data production. Samples are first automatically filtered for geometric consistency, target uniqueness, temporal consistency, and answer-format compliance. Low-confidence samples, non-unique answers, and semantically ambiguous cases are further selected for human spot-check review. Retained samples are then converted into a unified multimodal dialogue format, with data provenance, task type, generation rules, and quality status recorded for downstream sampling, error tracing, and dataset version management.

System 2 receives the current visual observation together with the task instruction. It produces chain-of-thought reasoning that describes the current visual context—identifying relevant objects and their spatial arrangement—followed by a subtask instruction for the immediate manipulation objective. For example, given the instruction “hang the item on the hanger,” System 2 may observe “there are hats, bags…” and decompose the task into the subtask “pick up the red hat” (Fig. 6).

This decomposition provides an intermediate interface between high-level task understanding and low-level action generation. The resulting subtask information is passed to System 1 and also provides task context for System 3, allowing prediction and control to operate on a more explicit representation of the current task stage.

System 3 is the predictive component of GigaBrain-0.7. Built upon GigaWorld-1 [75], it receives the current embodied context—including the current observation, proprioceptive state, and subtask description from System 2 (Fig. 6). Unlike System 2, which reasons over observations that are already available, System 3 explicitly models information about future task evolution. System 3 exposes two complementary signals to System 1.

Future observation. Conditioned on the current embodied context and subtask instruction, the world model generates a short video whose temporal extent is aligned with the subtask horizon of System 1. The last frame of this predicted sequence is extracted as the subgoal image $g_{t}$—a compact visual representation of the physical state expected after short-horizon task progress. It is encoded and provided to System 1 as an additional visual condition, allowing action generation to incorporate an explicit representation of the anticipated scene evolution.

Value-based evaluation. System 3 additionally estimates a scalar value $V_{t}$ representing the current task progress—how far the current execution state has advanced toward subtask completion. While sharing the same world-model backbone as the video pathway, the value pathway attends primarily to visual observations. The value estimate is converted into a binary advantage condition $A_{t}\in\{0,\,1\}$, indicating whether the predicted task progress is increasing or not over the current action segment. As shown in Fig. 6, this value-derived condition is injected into System 1’s prompt, providing a compact evaluation of the current execution state. The derivation of $A_{t}$ from $V_{t}$ is described in Section 5.3.

Together, the subgoal image captures anticipated physical outcomes in visual space, while the advantage provides a scalar evaluation signal for action generation and subsequent experience-based learning.

System 1 receives the current observation and short-term visual memory, the chain-of-thought output and subtask from System 2, proprioceptive state and Robot ID, and predictive signals from System 3 (Fig. 6). These inputs are fused by the VLA to produce action chunks for closed-loop robot execution.

System 1 combines its PaliGemma2 (3B) vision-language backbone with the Action Expert (0.5B) in a Mixture-of-Transformers (MoT) design. Let $\mathbf{H}^{\mathrm{VL}}_{l}$ and $\mathbf{H}^{\mathrm{AE}}_{l}$ denote the hidden states of the vision-language and action-expert streams at layer $l$. The two streams are coupled through joint attention but maintain separate feed-forward parameters:

where $\mathrm{Attn}(\mathbf{Q};\,\mathbf{C})$ denotes attention with queries from $\mathbf{Q}$ and keys/values from context $\mathbf{C}$, $[\cdot,\cdot]$ denotes sequence concatenation, and residual connections and layer normalizations are omitted for clarity. The vision-language stream uses causal self-attention over its own tokens (Eq. 1), preserving the pretrained VLM’s autoregressive interface. The action expert attends bidirectionally to the union of both streams (Eq. 2), enabling grounded action generation informed by the full semantic context without injecting action-specific representations into the VLM attention computation.

Physical interaction is inherently temporal. The same instantaneous observation may correspond to different task states depending on preceding interactions, while temporary occlusions and repeated visual configurations make single-frame perception insufficient for many long-horizon tasks.

GigaBrain-0.7 therefore incorporates short-term visual memory through Temporal-Spatial Blocks inserted into the visual encoder. Given the current observation together with a short history of preceding frames, each block factorizes visual aggregation into temporal and spatial processing. The temporal operation causally aggregates information across recent frames, allowing the current representation to capture how task-relevant regions evolve over time. The subsequent spatial operation models interactions among visual regions within the current observation, integrating the temporally enriched features into a coherent scene representation.

Rather than forwarding all historical visual tokens to the VLA backbone, the Temporal-Spatial Blocks progressively fuse historical information into the current-frame representation. After temporal-spatial processing, past-frame tokens are discarded and only the enriched current-frame tokens are retained. The number of visual tokens entering the VLA backbone therefore remains comparable to the single-frame setting, providing temporal context without naively increasing the high-level context length with the number of history frames.

System 1 supports two complementary action representations. The NTP-based discrete pathway generates discrete action tokens autoregressively through the vision-language head, providing a symbolic action representation compatible with next-token prediction. The FM-based continuous pathway generates action chunks through the Action Expert via conditional flow matching, taking noise-corrupted action representations as input and predicting continuous trajectories for precise robot control.

Both pathways share the same visual-language context through the MoT joint attention but serve different roles: the discrete pathway bridges language understanding with motor control during training, while the continuous pathway provides the executable actions used at deployment.

A central difficulty in VLA learning is balancing pretrained vision-language representations with robot-specific action learning. The VLM must remain sensitive to language, objects, spatial relations, and task semantics, while adapting sufficiently to support physical control.

GigaBrain-0.7 introduces Soft Knowledge Insulation (Soft KI) between the vision-language and continuous-action pathways. Rather than completely isolating semantic representations from action learning [5], Soft KI provides a controlled interface that allows the backbone to develop embodied-aware visual representations while preserving its general-purpose vision-language capabilities.

Scaling a single VLA across heterogeneous robots requires alignment between different state spaces and kinematic configurations. Following the unified robot representation in Section 3, GigaBrain-0.7 provides System 1 with both the proprioceptive state and a Robot ID through a dedicated projection module. The proprioceptive state is element-wise masked by a per-embodiment validity vector—distinguishing absent dimensions from zero-valued states—and projected together with the Robot ID into the shared VLA representation. This corresponds to the “Projection” module in Fig. 6.

The main Action Expert parameters are shared across embodiments, while per-embodiment input and output projections handle differences in native action definitions. This allows different robot platforms to share the core manipulation knowledge while preserving embodiment-specific precision. Rotations are represented using the continuous 6D format [94] throughout the action space.

GigaBrain-0.7 supports two complementary prediction pathways (Section 4.2): an autoregressive next-token prediction (NTP) pathway for language, subtask, and discrete-action outputs, and a flow-matching (FM) pathway for continuous action chunks. Unlike staged training recipes that introduce continuous action generation only in a later training phase, both pathways are optimized throughout GigaBrain-0.7 pre-training.

For an autoregressive target sequence $y_{1:M}$ conditioned on multimodal context $c$, the NTP objective is

where $m_{j}$ masks positions without autoregressive supervision. Depending on the data source, the targets include subtask descriptions, discrete action representations, and general vision-language outputs such as captioning, VQA, spatial reasoning, grounding, and affordance understanding.

For continuous action generation, let $a_{t:t+H}$ denote a ground-truth action chunk and let $\epsilon\sim\mathcal{N}(0,I)$ denote Gaussian noise of the same dimensionality. Following the flow-matching formulation used in prior GigaBrain models [27], we construct

The Action Expert predicts the corresponding flow field and is optimized with

The two pathways are jointly optimized during pre-training:

To reduce interference between continuous-action learning and the pretrained vision-language representation, we apply Soft Knowledge Insulation [20]. Specifically, FM gradients are propagated fully within the Action Expert but attenuated by a coefficient $\alpha_{\mathrm{KI}}$ before entering the vision-language backbone:

This preserves full autoregressive supervision of the vision-language backbone while still allowing continuous-action learning to adapt its representations toward embodied control.

To train the Temporal-Spatial Blocks described in Section 4.2, we provide historical observations alongside the current frame. Historical observations are randomly dropped during training, exposing the model to varying amounts of temporal context—including single-frame inputs—and reducing dependence on a fixed history length.

Each trajectory carries both a task-level instruction $\ell$ and a temporally aligned subtask instruction $\hat{\ell}_{t}$. We mix two supervision modes. In the first, both $\ell$ and $\hat{\ell}_{t}$ are provided and the model directly predicts the corresponding action. In the second, only the task-level instruction is provided, requiring System 2 to autoregressively infer the current subtask before System 1 generates the corresponding action. This couples high-level task reasoning and low-level control within the same pre-training framework. Causal attention masks prevent subtask targets and future actions from leaking into the representations used to predict them.

The pre-training corpus spans embodiments with substantially different morphologies, degrees of freedom, and control conventions. Following the unified representation in Section 3, heterogeneous states and actions are aligned to a common semantic structure while retaining embodiment-specific validity information.

Each embodiment’s proprioceptive state is projected into the shared model space through the embodiment-aware interface described in Section 4.2. Validity masks distinguish unavailable dimensions from valid zero-valued states, and invalid action dimensions are excluded from both flow-matching supervision and noise injection. The Action Expert shares its main Transformer parameters across embodiments, while embodiment-specific input and output projections accommodate differences in action dimensionality and control convention.

For egocentric data, future actions are re-expressed in the coordinate frame of the current observation to provide a stable reference throughout the action chunk. Rotations use the continuous 6D representation defined in Section 4.2.

Starting from GigaWorld-1, we continue video pre-training on robot manipulation data drawn from the real-robot and simulation tiers of the data pyramid (Section 3). The objective is to adapt general visual dynamics toward the state transitions and contact interactions that characterize robotic manipulation.

Conditioned on the current embodied context and subtask instruction, the model generates a future video whose temporal extent matches the action-chunk horizon of System 1. The final predicted frame serves as the subgoal image $g_{t}$—a compact visual representation of the physical state expected after short-horizon task progress.

Future-state prediction tells the policy what may happen, but not whether the current state is favorable for task completion. In a second stage, we equip the model with task-progress estimation.

We extend the video-pre-trained checkpoint into a Mixture-of-Transformers World Value Model that jointly supports future prediction and value estimation. Similar to Viva [53], the value pathway is supervised to estimate task progress—a scalar reflecting how far the current execution state has advanced toward subtask completion, derived from trajectory-level completion annotations in the training data. The value pathway attends primarily to current and historical visual observations when producing the progress estimate $V_{t}$.

System 3 parameters are frozen after this stage. During subsequent VLA post-training, System 3 generates fresh outputs $g_{t}$ and $V_{t}$ for each training sample but receives no gradient updates.

For each post-training sample, System 3 generates future video predictions spanning the same time horizon as the target subtask. The subgoal image $g_{t}$ is encoded through the visual pathway and appended to System 1’s multimodal context, providing the policy with an explicit visual prediction of the near-future state alongside the current observation.

The task-progress estimate from System 3 is converted into a binary advantage condition. For two temporally separated states within the action-chunk interval, we compute the progress difference and discretize it:

where $A_{t}=1$ marks an action segment associated with increasing task progress and $A_{t}=0$ marks non-improving or regressing segments. The advantage is embedded as a conditioning token in the VLA prompt, allowing System 1 to distinguish productive from unproductive behaviors in the training data. At inference time, the policy is conditioned on positive progress, biasing action generation toward behaviors associated with advancing task state—analogous to the advantage-conditioned approach in RAMP [28], but with the advantage derived from a dedicated world value model.

We apply stochastic condition dropout to prevent over-dependence on System 3 outputs. The subgoal image and the value-derived condition are dropped independently, with probabilities 0.50 and 0.15 respectively. The VLA is consequently exposed to four conditioning regimes: neither signal, subgoal only, value only, and both.

Task-specific post-training optimizes the policy through continuous action generation under the augmented context

where $c_{t}^{\mathrm{base}}$ contains the current observation, temporal context, language instruction, proprioceptive state, and embodiment information; $g_{t}$ denotes the predicted subgoal image; and $A_{t}$ denotes the value-derived progress condition.

We optimize the same flow-matching objective defined in Eq. 6, using $c_{t}^{\mathrm{post}}$ in place of the pre-training context $c$. System 3 remains frozen throughout this stage, while Systems 1 and 2 are optimized on task-specific demonstrations.

We deploy the world-model-powered post-trained VLA to collect rollout trajectories and organize them by execution outcome. The resulting dataset contains both successful and failed behavior, providing direct evidence about which actions lead to desirable and undesirable outcomes.

We apply an Advantage-Weighted Regression (AWR) [59] style objective for offline policy refinement. In this part, reward $r_{t}$ is calculated from the progress discriminator and predefined rules, advantage $\tilde{A}_{t}$ is defined as

Where $G_{t}$ and $\tilde{V}_{t}$ denote the cumulative return and value estimate, respectively.

Rollout segments are weighted according to their estimated advantage: successful and higher-progress segments receive greater training weight, while failed or lower-progress behaviors are down-weighted. This allows the policy to learn preferentially from better experience without requiring additional environment interaction during optimization.

The offline stage converts already-collected experience into improved behavior and provides a stronger initialization for online learning, substantially reducing the costly exploration required in the next stage.

After offline refinement, we redeploy the policy and collect on-policy trajectories. When the robot reaches a difficult state, a human operator intervenes and provides corrective actions. The corrective demonstration is recorded as a labeled action segment at the precise failure state and enters the training stream, providing direct supervision on the hardest states of the current policy distribution.

We use an actor–critic framework for online optimization, retaining System 1 as the deployment actor and training a separate critic for value estimation. Rewards combine chunk-level progress differences with terminal sparse signals. Training alternates between rollout collection and model updates:

Each improved policy generates the experience for subsequent iterations, progressively shifting training toward the state distribution encountered during deployment.

The four training stages form a progressive lifecycle: pre-training establishes broad embodied priors; world-model pre-training adds future prediction and progress estimation; post-training specializes these signals for target tasks; and experience-driven reinforcement improves the policy on states produced by its own execution. The online stage can be repeated as new rollout data becomes available, forming a continual rollout–correction–update cycle. A detailed treatment of the human-intervention protocol and the full online RL formulation will be provided in a dedicated future GigaBrain report.

We conduct real-world evaluations on two robot platforms: the AgileX PiPER/PiPER-X, a low-cost dual-arm manipulator widely used for tabletop manipulation benchmarking, and our in-house Maker H01, an embodied-native humanoid platform with a head, waist, and bimanual arms. The two platforms differ substantially in degrees of freedom, control interface, camera configuration, and workspace geometry, providing a rigorous test of cross-embodiment generalization.

For real-world policy evaluation, we compare GigaBrain-0.7 against the following baselines: (i) $\pi_{0.5}$ [62], a state-of-the-art VLA model with heterogeneous co-training and hybrid discrete–continuous supervision; (ii) GigaBrain-0.1, the predecessor model in the GigaBrain series with PaliGemma2 backbone; (iii) Xiaomi-Robotics-1 [79]; (iv) Galaxea G0.5 [23]. These real-world policy baselines are evaluated under the same protocol and task definitions. Separately, the embodied vision-language benchmark evaluation includes Xiaomi-Robotics-0 and G0.5-base, as well as Spirit-v1.5 [70], Wall-OSS-0.5 [86], and Hy-Embodied-0.5-VLA-UMI [89].

Each task is evaluated over a fixed number of trials (typically 10–20 per configuration). We report success rate as the primary metric, defined as the fraction of trials in which the robot completes the full task objective. For language-following evaluations, each task is conditioned on a specific natural-language prompt that specifies attributes such as color, target identity, or spatial direction (e.g., “pick up the red spoon”), and a trial is considered successful only if the correct target is manipulated. We distinguish between in-distribution (ID) tasks whose objects and configurations are represented in the training set and out-of-distribution (OOD) tasks that require generalization to unseen objects, layouts, or instructions.

For post-training evaluations, all models are fine-tuned on task-specific demonstration data collected on the corresponding target platform. The resulting policies are evaluated using the same task definitions and evaluation protocol described above.

We first compare PaliGemma2[71], Qwen3.5[65], and Gemma 4[74] while keeping the dual-stream VLA formulation fixed. The models are evaluated on representative real-robot tasks spanning structured manipulation, language-conditioned object interaction, and long-horizon deformable-object manipulation.

As shown in Tab. 4, increasing backbone size does not lead to uniformly stronger robot performance in the evaluated setting. Gemma 4 performs strongly on the more structured tasks, whereas PaliGemma2 is the only backbone that achieves nonzero success on shirt folding while remaining competitive on fruit manipulation. We therefore use PaliGemma2 as the backbone for the following architecture comparison and data-scaling experiments.

With the backbone fixed to PaliGemma2, we next compare three ways of connecting the vision-language representation to the continuous action expert: Dual Stream, Last-Layer Cross Attention, and Multi-Layer Cross Attention. A dual-stream expert interleaves with the VLM at every layer, attending to concatenated VLM–expert keys and values within a single attention computation, without any separate cross-attention module. A last-layer cross-attention expert instead decouples the action head entirely from the backbone, with every expert block cross-attending only to the VLM’s final-layer hidden states. A multi-layer cross-attention expert maintains a per-layer interface, with expert block i cross-attending to the hidden states of the corresponding VLM layer i. The three variants thus span a spectrum of coupling strength: from per-layer interleaving, through a per-layer interface, to a single terminal representation, progressively decoupling the expert from the backbone at the cost of a narrower information pathway.

Tab. 5 shows a clear trade-off between action-generation capability and computational efficiency. Restricting the action expert to the final VLM representation yields the lowest inference latency, whereas the multi-layer cross-attention variant incurs additional computational cost without improving task success in the evaluated settings. In contrast, the dual-stream architecture achieves the strongest real-robot performance across all three tasks, including nonzero success on the more challenging shirt-folding task, although at a higher inference cost.

Based on these comparisons, we adopt the PaliGemma2-based dual-stream architecture as the default System 1 configuration in the remainder of the experiments. This fixes the model architecture before we study the effect of increasing the scale and diversity of the pretraining data.

We first vary the amount of pretraining data while keeping the model configuration unchanged. As shown in Fig. 8a, larger training sets consistently reach lower validation loss, with the separation between data scales remaining visible throughout optimization. Within the evaluated range, we therefore observe continued improvement as the available pretraining data increases.

We next examine whether the same trend translates to physical execution. Fig. 8b evaluates models trained with progressively larger subsets of the robot data on fruit manipulation and chaotic clothes folding.

Both tasks benefit from additional robot experience, but the scaling behavior differs substantially with task complexity. Fruit manipulation improves relatively smoothly, whereas clothes folding shows a much sharper dependence on data scale. The latter requires sustained interaction with deformable objects and a broader range of intermediate states, suggesting that complex long-horizon skills place substantially greater demands on the coverage of the pretraining distribution. Together with the consistently decreasing validation loss, this trend suggests that scaling does more than improve the training objective: manipulation behaviors that are weak or unreliable at smaller scales become increasingly reliable as embodied experience grows. We view this progression as a direct manifestation of capability emergence with scale.

Finally, we investigate whether scaling data diversity provides benefits beyond increasing robot trajectories alone. Starting from 3,000 hours of robot data, we add equal amounts of UMI or EGO demonstrations, and further evaluate their combination.

As shown in Fig. 8c, both human-data sources improve performance over robot-only pretraining, and the combination of robot, UMI, and EGO data provides the strongest results across the evaluated settings. The improvement persists after task-specific post-training, indicating that the additional human experience changes the quality of the learned pretraining prior rather than providing only an immediate improvement to the base policy.

The different sources also exhibit complementary effects. Robot trajectories provide direct embodiment-specific control supervision, UMI contributes manipulation experience from a near-hand observation and motion interface, and EGO broadens the diversity of first-person human interaction. Their combined benefit supports the central motivation of our data pyramid: effective scaling depends not only on the number of trajectories, but also on bringing heterogeneous forms of physical experience into a common training representation.