JD’s Bet Is a “JoyAI Brain” Plus a “JoyAgent Team” (3 of 3) (Tech Strategy)

Posted on by

In Part 2, I went through half of JD’s suite of AI models (called JoyAI). There are the eight models, which can put into 4 layers.

Voice Interaction (i.e., the ears and voice)

  1. JoyAI-Voice: speech generation model
  2. JoyAI-Talker: voice interaction model

Multimodal Interaction (i.e., the eyes)

  1. JoyAI-Echo: ultra-long audio and video generation model
  2. JoyAI-VL-Interaction: real-time video interaction model
  3. JoyAI-Video-Edit: real-time video editing model
  4. JoyAI-Image-Edit: image editing model

Embodied Intelligence (i.e., the body)

  1. JoyAI-RA: embodied manipulation foundation model

Reasoning (i.e., the brain)

  1. JoyAI-Flash: general-purpose language model

In this article, I’ll go through that last two capabilities, the “body” and the “brain”.

Embodied Intelligence Is the Robot “Body”

Here is a sign posted by JD at the World Robot Conference in Beijing (AI translated).

Within Embodied Intelligence, the JoyAI-RA Model Turns Instructions into Movement

This layer is mostly about JoyAI-RA.

JD calls this its embodied manipulation foundation model. RA probably stands for Robotic Autonomy, which is in the title of JD’s first paper on it.

Basically, the model combines what the camera sees with a language instruction. And it produces the physical actions for the robot to execute.

Why it’s necessary:

  • The other models can talk, watch and generate. But none of them can move a robot. JoyAI-RA is the model that turns the phrase “pick up that box” into actual movement.
  • And JoyAI-RA is built to solve the biggest problem in robotics, which is that there isn’t enough robot training data. So, it is designed to learn from three data sources at once: videos of humans, simulations and real robots.

How it works:

  • The trick is something JD calls dual action alignment.
  • Most human videos have no action labels. So, the model must infer the action from what changed between video frames. That lets ordinary videos of people teach the model how the physical world behaves.
  • Then, where reliable human and robot motion data exists, it maps both into one shared action format.

JoyAI-RA Adds a World Model

What’s interesting here is that JoyAI-RA is one integrated Vision-Language-World-Action model (VLWA). That’s different from a standard VLA.

A standard Vision-Language-Action (VLA) model has two parts:

  • A vision-language model that understands the scene.
  • An action module that outputs movement.

But JoyAI-RA adds a third part, a world model that predicts how the scene will change.

JD calls the result a Vision-Language-World-Action model (VLWA). The first version, JoyAI-RA 0.1, was a standard VLA. Version 0.5 added the world model.

***

In the graphic below, you can see the two other components for the body. But these are supporting capabilities, not models.

EgoLive Is a Dataset of Human Video for Training Robots

EgoLive has 1,680 hours of first-person videos of people doing real work, across 65,866 episodes and 346 tasks. It was recorded with a head-mounted camera in real homes, shops and workplaces.

Why it’s necessary:

  • Robot data is scarce and slow to collect. Having a person remotely operate a robot to create each example doesn’t scale very well.
  • But you can put a camera on a worker’s head and record them doing their normal job, more cheaply and pretty much anywhere. This is the kind of data JoyAI-RA is designed to learn from.

How it works:

  • But having raw video isn’t enough. JD’s pipeline automatically labels it. With hand and camera positions in 3D, depth, object outlines and step-by-step language descriptions of the task.
  • That turns a video of a person into something a robot model can learn from.

How it’s different:

  • JD says EgoLive is the largest open-source annotated dataset of its kind. And JD plans to collect 10 million hours over two years. EgoLive is the first batch released.

JoyAI-Sim Tests Robots Without Robots

This last component, JoyAI-Sim, is a toolchain built around a simulator. Its job is converting data between three forms: robot, simulation and human.

Why it’s necessary:

  • JD says robot models need two things that are hard to scale with physical robots alone. They need trustworthy testing and robot-usable training data.
  • But real-robot trials are slow, hard to repeat and can damage the hardware. Hence you need a sim.

How it works:

  • JoyAI-Sim runs in two directions.
  • For testing, it takes real robot tasks and rebuilds them as digital twins in simulation. So, a model can be evaluated at scale without a physical robot.
  • For training data, it takes first-person human video, lifts it into simulation, checks it against what a robot can physically do, and converts it into robot-ready data.

How it’s different:

  • JoyAI-Echo WM generates videos of the world.
  • JoyAI-Sim rebuilds real tasks as digital twins and checks them against physical constraints. It is the bridge between human data and robot data.

***

Ok, that’s the body., the embodied intelligence layer.

  • JoyAI-RA learns from it and moves the robot.
  • EgoLive is the data.
  • JoyAI-Sim converts the data and tests.

Which brings me to the last model.

JoyAI-LLM Flash is the True “Brain”

The last model is JoyAI-LLM Flash. This is JoyAI’s primary reasoning model.

This is a 48B-parameter Mixture-of-Experts (MoE) large language model. With about 3B activated parameters per token.

This is where the heavy logical thinking happens. It handles chat, short chain-of-thought reasoning and tool use.

Compare this to Alibaba and Tencent, which have lots of LLMs. They have flash models, reasoning models, etc. But as far as I can tell, JD has only one general-purpose LLM, which is Flash.

That’s an interesting approach. JD does not appear to have a tiered general-purpose LLM lineup like OpenAI (e.g., GPT-4o, GPT-4o-mini) or Meta (Llama 8B, 70B, 405B).

Instead, their strategy appears to rely on a single hybrid base model. Which has:

  • Dual Reasoning Modes: Instead of training two separate models such as a fast chat model and a slow reasoning model, JoyAI-LLM Flash supports both a “thinking” mode (short chain-of-thought) and a “non-thinking” mode (direct chat) in one model. It was trained with a new reinforcement learning method JD calls FiberPO.
  • Single Model, Multiple Checkpoints: JD’s Hugging Face page describes the Base model. It says it has quantized versions (FP8, INT8, FP4, GGUF). But they are all the same 48B architecture.

My view is that JoyAI-LLM Flash sits above the embodied intelligence stack. It’s what the others can draw on for more reasoning. So, I think of it as the “Brain”.

Here’s one way to picture how it relates to the JoyAI models for embodied intelligence (from Gemini, not from JD).

Ok. Those are the 8 models covering 5 layers. And within all this, the 3 models I am looking at the most are:

  • JoyAI-Flash LLM
  • JoyAI-VL-Interaction: real-time video interaction model
  • JoyAI-RA

Final Point. JoyAgent Is the Team.

I’ve covered JoyInside and JoyAI. But there is also JoyAgent (specifically open-sourced as JoyAgent-JDGenie).

This is not really part of the embodied AI tech stack (yet). It is a master orchestration framework. It can orchestrate a network of specialized sub-agents to solve complex, multi-step tasks.

So:

  • JoyAI is the suite of AI models.
  • JoyInside handles physical device development and execution.
  • JoyAgent is a workflow engine that can plan, coordinate, and execute multi-step tasks across multiple agents.

For JoyAgent, think supervisor vs workers.

It is the Supervisor Agent that receives the high-level prompt (e.g., “Analyze market trends for US Dollars vs. Gold and create an executive deck”), breaks the objective down into a logical task graph, and then assigns those sub-tasks to specialized Worker Agents.

The Specialized Worker Agents are assigned to narrow roles and run simultaneously or sequentially. They can be specialized, such as:

  • Search and Retrieval Agents: They scrape real-time web data and enterprise databases.
  • Code and Data Agents: They executes Python/SQL scripts to clean and parse incoming numbers.
  • Report Generator Agents: They synthesize analysis into structured text or HTML.
  • PPT/Design Agents: They convert text and chart data into formatted downloadable slides.

In practice, JoyAgent can handle longer-range planning across multiple agents. And I’m assuming this is going to be extended to orchestrating multiple robots and/or devices.

For example, if you say, “Prepare the bedroom for sleep, set an alarm for 7 AM, and give me a 2-minute summary of tomorrow’s schedule,” JoyAgent could (in theory) decompose this into a task graph:

  • Sub-Agent A: Queries calendar/weather APIs for tomorrow’s schedule.
  • Sub-Agent B: Formulates an alarm event.
  • JoyInside Connector: Sends a multi-device execution payload to the home ecosystem (dimming lights, adjusting the smart mattress posture, setting thermostatic controls).

However, JoyAgent today is a high-level cognitive planner, not a low-level robotics path planner.

In Theory, JoyAgent Could Run Your Home

Think about how this could work in consumer products (in theory), like for robots and smart devices in your home.

As discussed, JoyInside helps OEMs create lots of products (appliances, lights, robots, etc.). And then JoyAgent could potentially orchestrate multiple robots and smart devices in a home.

For example, imagine a scenario where a companion robot detects an elderly person falling. JoyAgent could receive the signal, alert a connected J.Zao smart bed/health monitor, and trigger an emergency alert via JD Health.

In Theory, a JoyAgent Could Run a Logistics Warehouse

Think about massive multi-robot fleets moving around a logistics facility. Think of hundreds of autonomous mobile robots all coordinating in a JD fulfillment warehouse.

I didn’t go into the robotics of JD Logistics in these articles. It’s a whole other subject. But it’s pretty great.

***

Ok. That’s it for JoyInside, JoyAI and JoyAgent.

As I mentioned at the beginning, I really underestimated how much JD was doing in embodied AI. I like that they have a unique strategy, which is very different than what I see at other robot businesses.

Cheers, Jeff

——

Related articles:

From the Concept Library, concepts for this article are:

  • Embodied AI

From the Company Library, companies for this article are:

  • JD: JoyAI
  • JD: JoyAgent

——

I am a founder, consultant and keynote speaker specialized in digital and AI agentic strategy.

I am the founder of Agentic Org Now, a firm that helps leadership teams launch AI and agentic initiatives fast.

I write (a lot) about digital and digital AI strategy (3 best selling books, +2.9M followers on LinkedIn). You can get a free book at the email sign-up below.

My Moats and Marathons book series is a framework for building and measuring competitive advantages in digital and AI businesses.

This content (articles, podcasts, website info) is not investment, legal or tax advice. The information and opinions from me and any guests may be incorrect. The numbers and information may be wrong. The views expressed may no longer be relevant or accurate. This is not investment advice. Investing is risky. Do your own research.

Comments are closed.