A Breakdown of AgiBot’s Embodied AI Tech Stack (2 of 3) (Tech Strategy)

Posted on by

According to Jensen Huang, physical AI and embodied intelligence are the next big wave.

And this means a completely different GenAI and the AI tech stack. You don’t navigate the physical world with text. It’s with vision and action (and language).

I’ve been trying to get smarter about vision-language action (VLA) and world models. That’s the core of physical AI and embodied intelligence. And I’ve found that Agibot is a really good example to study for this.

At the top of the embodied AI tech stack are the tools that businesses and developers actually use to program and use their robots. For AgiBot, that’s two low / no code tools: LinkCraft and LinkSoul.

1. LinkCraft / Lingchuang Platform (灵创平台)

LinkCraft is AgiBot’s no-code motion creation and choreography system. It allows non-technical users to design complex physical movements for AgiBot humanoids. That’s everything from making coffee to doing dance routines. All without writing software code.

In practice, LinkCraft enables:

  • Visual Motion Capture: Users record human movements like martial arts or assembly line work using a standard camera. The AI algorithms then translate those visual frames into joint trajectories for the robot.
  • Timeline Motion Editing: Motion sequences can be trimmed, stitched, and keyframed in an editing interface similar to standard video editing software.
  • Multi-Robot Synchronization: You can also choreograph synchronized actions by multiple AgiBot units simultaneously.

2. LinkSoul / Lingxin Platform (灵心平台)

Lingxin is AgiBot’s no-code interaction and persona customization system. It manages how the robot communicates and relates to humans.

In practice, that means:

  • Voice and Tone Customization: Your robot can have specific brand or character voices. You can use built-in voice presets and voice cloning.
  • Personality and Character Profiles: Users can drag and drop specific personality traits, conversational rules, and facial expressions. You might want different behavior based on the role, such as a museum guide vs. store host vs. concierge.
  • Memory and Enterprise Knowledge: Enterprise documentation can be ingested so the robot builds a persistent memory over time.
  • Behavioral Workflow Mapping: You can integrate spoken dialogue with physical gestures. With specific triggers and task logic.

Ok. Let’s get into the software. That’s the fun stuff.

The 3 Layers of the Robot Compute Architecture

You’ll hear an analogy often about how robots need both a cortex and a cerebellum.

  • The “Cortex” is embodied intelligence’s system for higher level intelligence.
    • Humans think about what we are perceiving and what we are doing. And we have a running dialogue in our heads.
    • That’s the cortex doing visual processing, spatial reasoning, logic and task planning.
    • Robots have a specialized compute architecture for this.
  • The “Cerebellum” is embodied intelligence’s system for rapid and precise control of the robot body.
    • We don’t have to think about breathing or keeping our balance. Or about carrying something in a hand.
    • This is the system for making continuous adjustments of motor commands in real time. For robots, this is important for keeping balance, adjusting grip strength and so on.

That’s a good analogy. But I think it’s better to think in terms of 3 layers.

Layer 1: A Cortex for High-Level Intelligence and Planning

As mentioned, this is high-level perception, natural language comprehension, multimodal vision-language-action (VLA) planning, semantic mapping, and spatial reasoning.

This requires a huge amount of compute. For this, AgiBot uses dedicated AI acceleration hardware (e.g., NVIDIA GPUs / NPUs) on robot. Lots of edge compute. Higher end robots use 2,070 FP16 TFLOPs (or INT8 TOPS). Simpler ones use 500 INT8 TOPS.

This much compute also requires lots of power. Processing high resolution multi camera video feeds consumes lots of power per inference pass.

Real-time VLA inference on robot is at about 10–20 Hz. Larger multimodal transformers can be even slower (2–10 Hz).

Note: AgiBot actually breaks this layer into an additional complementary layer for longer-range planning. They add a separate Visual Language Model (VLM) to enhance the VLA model. The VLM acts as the core spatial backbone and latent planner. So, the VLA might run the grinding beans task but the VLM oversees the entire “making coffee” process.

Layer 2: A Cerebellum for Real-Time Dynamics and Whole-Body Control

This is mid-level motion intelligence, which includes dynamic balance, coordination, reactive policies, and whole-body control. This is the bridge between abstract AI planning and physical hardware sensing and execution.

The computation here is faster (100–500 Hz) but much less intensive. Processing numerical vectors and low-level sensor feedback is less intensive than handling raw visual speeds. It operates on small arrays of floating-point numbers.

This requires a balance between GPUs and more traditional compute (CPUs). This can be by Model Predictive Control (MPC) using CPUs or Diffusion Action Experts (using GPUs).

  • Model Predictive Control (MPC) is an algorithm that maintains balance and trajectory in real time. It uses a mathematical model of the physics of the robot, including mass distribution, gravity, torque capacity, and joint limits. It relies on high-speed numerical solvers for linear algebra, running it on CPUs ensures the execution timing remains steady.
  • Diffusion Action Experts are generative AI models that learn complex action policies. When a robot encounters unpredictable environments, such as a slippery floor or an unexpected push, standard trajectory planners can fail. A Diffusion Action Expert can generate smooth and adaptable action paths.

Layer 3: Fast Reflexes for Low-Level Actuators and Joint Control

These are the specialized microchips located inside the motor housing and joint actuators. These chips translate higher level positional commands into precise raw electrical power fed to electric motors.

These embedded motor drivers and joint microcontrollers (MCUs) can run at 1,000-20,000 Hz. Examples are ARM Cortex-M microcontrollers and FPGAs.

These includes Field-Oriented Control (FOC), proportional-integral-derivative (PID) position loops, and pulse-width modulation (PWM) current/torque loops.

***

Ok. That’s a reasonable summary. If you want to know more of the compute details, I’ve put those below.

The Chips and Other Key Systems

Here are the chips / systems used for computer by layer. Note: AgiBot runs everything on robot during usage (i.e., inference).

For Layer 1 (Cortex), AgiBot uses NVIDIA systems on chip (SoC) hardware platforms.

For the higher functioning robots (such as G2 larger industrial humanoids), AgiBot uses the NVIDIA Jetson SoCs. Plus, AgiBot’s Rhino R1 co-processor.

  • Higher-functioning robots use the NVIDIA Jetson Thor T5000 platform or dual NVIDIA Jetson AGX Orin. For the T5000, there is a Blackwell GPU, several Arm CPUs, memory and visual accelerators.
    • This setup delivers up to 2,070 TFLOPS for FP4 compute (4-bit floating point). The FP16 is slower.
    • This is the AI compute to run real-time Vision-Language-Action (VLA) models and large Visual Language Models (VLMs) locally.
  • This is paired with AgiBot’s proprietary Rhino R1 co-processor. This offloads a lot of the base operations, local perception, and background video streams from the NVIDIA accelerator (which does the heavy VLA inference passes). It provides 500 TOPS of AI compute.

For the mid-range and compact robots (X series small humanoids), they use the NVIDIA Jetson Orin Nx.

  • It has an Nvidia Ampere GPU and 8 core ARM CPUs, providing 157–200 INT8 TOPS.

For Layer 2 (Cerebellum), AgiBot uses the Rockchip RK3588 in a Dual-SoC Configuration. Plus Xyber-DCU (Domain Controller).

Rockchip is a Fujian-based fabless chip designer that manufactures at the Samsung Foundry. AgiBot deploys two RK3588 chips operating on a dedicated main board.

  • The RK3588 uses a licensed 8-core ARM CPU cluster (4 Cortex-A76 + 4  Cortex-A55), an integrated 6 TOPS NPU, and dedicated real-time I/O channels.

This SoC executes high-frequency Model Predictive Control (MPC) linear algebra solvers and joint kinematic transformations. Note: The NPU / GPU cores execute the Diffusion Action Expert policies.

AgiBot’s Xyber-DCU (Domain Control Units) also work at this layer.

For Layer 3 (Fast Reflexes), AgiBot uUses ARM Cortex-M Microcontrollers (MCUs) and FPGAs.

The actuator microcontrollers are the embedded STMicroelectronics STM32 series (high-performance ARM Cortex-M4 / Cortex-M7 MCUs). They are integrated directly into the motor housing.

AgiBot’s proprietary Xyber-Edge joint controllers pair these embedded MCUs with specialized logic (FPGAs or dedicated motion ASICs).

An interesting aspect of this is AgiBot’s proprietary Xyber system.

This is their vertically integrated hardware controller and sub-system architecture. It organizes low latency physical control across the robot body into dedicated embedded controllers.

The Xyber system includes:

  • Xyber-Edge which is a Cerebellum Controller for low-to-mid level, real-time motion execution.
  • Xyber-DCU (Domain Control Unit) for higher motion planning and intelligent decisions
  • Xyber-BMS (Battery Management), which manages power intelligence and safety.
  • PowerFlow joints. The actual hardware that edge controls. These are the physical motors with built-in drivers and small MCUs.

***

Here’s the summary:

  • Layer 1 Cortex is mostly the NVIDIA SoCs plus Rhino (on the flagship models).
  • Layer 2 Cerebellum is Rockchip plus Xyber-DCU.
  • Layer 3 Low Level Actuator Control is Xyber Edge plus STM32 MCUs inside PowerFlow Joints.

Ok. That’s it for this.

In the Part 3, I’ll go into the key capabilities that matter for AgiBot’s embodied intelligence. Which are:

  • Genie Operator (GO)
  • Genie Envisioner (GE)
  • Genie Sim

Those are the “so what” of all this.

Cheers, Jeff

——-

Related articles:

From the Concept Library, concepts for this article are:

  • Robotics

From the Company Library, companies for this article are:

  • AgiBot

——-

I am a consultant & keynote speaker on how to increase digital growth and strengthen digital AI moats.

I am the founder of TechMoat Consulting, a consulting firm specialized in increasing digital growth and strengthening digital AI moats. Get in contact here.

I write (a lot) about digital growth and digital AI strategy (3 best selling books, +2.9M followers on LinkedIn). There is a free book and email newsletter below.

My Moats and Marathons book series is a framework for building and measuring competitive advantages in digital businesses.

This content (articles, podcasts, website info) is not investment, legal or tax advice. The information and opinions from me and any guests may be incorrect. The numbers and information may be wrong. The views expressed may no longer be relevant or accurate. This is not investment advice. Investing is risky. Do your own research.

 

Comments are closed.