Steps to reproduce

1. Set up the earth visualization

2. Set up the Google Earth Engine API for self-use

3. Deploy the demo on Cloudflare Workers

Thanks to…

• Three.js - A powerful 3D library for rendering and animating 3D graphics in the browser.
• Three-Globe - A very nice 3D globe visualization library built on top of Three.js.
• Google Earth Engine - A platform for planetary-scale environmental data analysis, it is free for academic and non-commercial use.
• Cloudflare Workers - A serverless platform that allows you to run JavaScript code at the edge of Cloudflare’s global network.

This is a monumental step, but the path is filled with massive hurdles that temper expectations. Let’s break down the landscape, from the core challenges to the elegant architectural solutions that are paving the way forward.

The Big Hurdles: Data and Action

Training effective world models is far more complex than training a language model. The challenges aren’t just about scale; they are fundamental to what we’re asking these models to do.

The Scarcity of the Right Data

While we are swimming in a sea of video data, world models need something very specific: videos paired with actions. To truly learn cause and effect, a model needs to know not just what happened next, but why. What action did an agent take to cause the next frame? For a self-driving car, this could be the steering angle and throttle; for a robot arm, the joint torques. Without this action data, a model might become excellent at predicting the next frame of a passive video, but it will struggle to be a reliable tool for an active agent that needs to plan and control its movements.

The Heterogeneity of Action Spaces

A more profound challenge is the sheer diversity of “action spaces.” For Large Language Models (LLMs), the “action” is simple and unified: predicting the next token. But for physical agents, actions are intrinsically tied to their bodies. A drone’s actions (rotor speeds) are entirely different from a humanoid robot’s (joint movements) or a car’s (steering, acceleration). This makes creating a single, universal world model that works across different agents a monumental task. How can one model learn the physics relevant to both a flying drone and a walking robot? This heterogeneity is a significant barrier to the dream of a general-purpose world model.

Two Paths to Building the World Models: Pixel- vs. Latent-Space

As researchers tackle these challenges, two main architectural philosophies have emerged for how a model should represent and predict the future.

The Pixel-space Approach

The most intuitive approach is to predict the future pixel by pixel. Given the current frame and an action, the model generates the entire next video frame. Early models took this path, aiming for high-fidelity, photorealistic predictions of every detail.

However, this method is computationally ravenous. Why? Because most pixels in a scene are irrelevant for understanding the core dynamics. Does a self-driving car need to predict the exact motion of every leaf on a tree to know it should stop at a red light? This obsession with pixel-level detail makes the models incredibly expensive to train and slow to run, as the compute required scales enormously with both spatial and temporal resolution.

The Efficient Abstraction: Latent-space Models

This brings us to the second, more efficient approach: latent-space models. The guiding principle here is simple: why predict every pixel when you can predict an abstract summary of the future? This is the core idea behind the Joint-Embedding Predictive Architecture (JEPA) championed by Meta AI.

Instead of operating on raw pixels, a JEPA model first learns a compressed, abstract representation—or “embedding”—of the world. It then performs all its predictions within this highly efficient latent space. The goal is not to reconstruct the exact image but to predict the future representation. This approach cleverly sidesteps the need to model unnecessary details like lighting, textures, and background noise, focusing only on the semantic essence of the scene.

JEPA in Action: The DINO-World Case Study

The DINO-world model is a fantastic demonstration of the power of latent-space prediction and highlights a brilliant strategy: decoupling. The authors asked: why force one giant model to learn both visual understanding and physical dynamics simultaneously?

Instead, they took a two-step approach:

1. Leverage a Pre-trained Vision Expert: They start with the frozen visual features from DINOv2, a powerful model already trained to understand the visual world. DINOv2 provides a rich, compressed representation that already captures concepts like objects, depth, and scene layout.

2. Train a Dynamics Predictor: They then train a separate, smaller predictor model that operates exclusively on these abstract DINOv2 features. Its job is not to predict pixels, but to predict what the DINOv2 features of the next frame will look like.

This decoupling provides a massive boost in efficiency. While a pixel-space model like COSMOS can require 4 to 12 billion parameters, the DINO-world predictor achieves strong results with just over 1 billion.

Critically, this approach works remarkably well. DINO-world demonstrates a solid grasp of intuitive physics and excels at downstream tasks like predicting future object segmentation masks and depth maps—all while operating in its efficient, abstract latent space. This validates the core idea that a compressed representation is sufficient for learning the essential dynamics of the world. Rest of the research seems to be converging: 1) Take a strong video model architecture, 2) Plug-in action conditioning modules, 3) Often use something like Diffusion Forcing to refine the predictions but the core engine remains to operate in this very efficient latent space.

Bottom Line: The Future of World Models

The takeaway is clear: the frontier of world models is rapidly shifting. We are moving away from brute-force, pixel-perfect predictions and toward the elegant efficiency of latent-space models. This JEPA-inspired approach allows for building more scalable and capable models that learn the essential dynamics of our world. But what does this mean in practice, and what challenges remain?

World Models as Flexible Simulators, Not a Replacement for Real Data

So, what is the actual impact for building smarter robots and AI agents? The most immediate role for world models is as powerful, flexible simulators. Unlike traditional simulators like MuJoCo or IsaacSim, which are based on hard-coded physics equations, world models learn dynamics directly from real-world video data. This allows them to capture complex, nuanced phenomena that are difficult to model mathematically.

However, they are not a silver bullet that will eliminate the need for real-world data. For high-precision, safety-critical tasks, traditional physics-based simulators remain more reliable. World models are not perfect, and their predictions can degrade over longer time horizons. The future likely involves a hybrid approach, where agents train on a mix of real-world demonstrations, traditional simulation, and imagined scenarios from a world model 3.

Supporting the Next Generation of AI Agents

Perhaps the most exciting role for world models is not as the policy engine itself, but as a critical advisor to one. Think of a generalist agent model (like a VLA) that needs to decide what to do next. Instead of acting blindly, it can use a world model as a short-term “consequence simulator.” The agent can “imagine” the outcome of a few different actions in the world model’s latent space and choose the one that leads to the most desirable future. In this vision, the world model isn’t the decision-maker; it’s the high-speed imagination that enables better, safer decisions.

The Road Ahead: Big Unsolved Problems

Despite the rapid progress, several major challenges stand between us and truly general-purpose world models:

1. Long-Term Consistency: While current models can generate consistent short video clips, their predictions tend to drift and lose coherence over longer sequences. Solving this will likely require new architectures, such as state-space models or retrieval-augmented methods, to maintain a coherent history.

2. Integrating Abstract Reasoning: World models are learning the intuitive “how” of the physical world. However, the abstract “what” and “why”—common sense reasoning—is still the domain of LLMs. A deeper fusion of these two capabilities is needed for models to truly understand and plan complex tasks.

3. Real-Time Performance: For a world model to be useful for an interactive agent, it needs to run in real-time. This remains a significant engineering hurdle, especially for large models that need to run on edge devices like robots.

4. Multi-Agent Dynamics: The real world is inherently multi-agent. Modeling the complex interactions, intentions, and social dynamics between multiple independent agents is a massive open problem that single-agent models do not address.

Citation

1. Assran, M., Duval, Q., Misra, I., Bojanowski, P., Vincent, P., Rabbat, M., … & Ballas, N. (2023). Self-supervised learning from images with a joint-embedding predictive architecture
2. Huang, Siqiao. “Beyond the Hype: How I See World Models Evolving in 2025”. Nemo’s Blog (Oct 2025).
3. Baldassarre, F., Szafraniec, M., Terver, B., Khalidov, V., Massa, F., LeCun, Y., … & Bojanowski, P. (2025). Back to the features: Dino as a foundation for video world models

This post is my distilled set of notes from Thinking Machines’ LoRA without Regret write-up. Their experiments dig into when LoRA behaves like full finetuning, where it breaks, and why it may be the default choice for many post-training pipelines.

LoRA in a Nutshell

LoRA inserts low-rank adapter matrices A and B alongside the original model weights. Because their rank is tiny, we only introduce a small number of new parameters (often around $1\%$ of the number of base model parameters). During training we backpropagate through the frozen base model but only update the adapter weights, effectively capturing task-specific information in a low-dimensional subspace.

When the base weight matrix $W \in \mathbb{R}^{d \times k}$ is frozen, LoRA learns a low-rank update $\Delta W$ expressed as:

with $B \in \mathbb{R}^{d \times r}$ and $A \in \mathbb{R}^{r \times k}$ where $r \ll \min(d, k)$. This keeps the additional parameters to $r\times(d + k)$ rather than $dk$. At inference time, the forward pass simply augments the frozen projection:

so we recover the adapted output without ever modifying the original weights.

Why Should We Care About LoRA?

1. Multi-tenant serving: Load the base model once and hot-swap lightweight adapters to serve domain-specific variants—medical, legal, creative, and more.
2. Lower training overhead: Because gradients and optimizer state only cover the adapters, the hardware and memory requirements drop dramatically compared with full finetuning.
3. Operational simplicity: Adapter checkpoints weigh only a few megabytes, which makes versioning, sharing, and rolling back far easier than dealing with full model shards.
4. Training complexity (simplicity): Backpropagating through the full model is typically the priciest part of training. LoRA saves compute because we only calculate gradients for the tiny A and B matrices. LoRA leads to roughly a 30% reduction in FLOPs per training step when you account for both forward and backward passes.

Can LoRA Match Full Finetuning?

A central theme in the post is “low regret” finetuning—scenarios where LoRA achieves the same downstream performance as updating every parameter. The results split into two regimes:

• Data-rich finetuning: When you have massive datasets, LoRA can run out of capacity. The low rank update simply cannot encode the richer signal you get from truly large-scale finetunes.
• Typical post-training: For instruction tuning, domain adaptation, and similar medium-sized datasets, LoRA keeps pace with full finetuning. As long as the adapters are applied broadly across the network (not just attention blocks), the gap largely disappears.

Training Lessons from the Paper

• Target all major weight matrices: LoRA adapters must cover MLP and MoE layers in addition to attention. Early work focused on attention weights and left a lot of performance on the table; most of the model’s information lives in those feed-forward blocks.
• Batch size sensitivity: Full finetuning tolerates larger batch sizes. LoRA tends to degrade faster as you scale batches, so smaller batches are safer if you want stable convergence.
• Learning rate adjustments: Optimal learning rates ended up around $10\times$ higher than the corresponding full finetunes. Because LoRA initializes the B matrix to zero, the initial gradients are small, and a larger learning rate kickstarts progress across tasks.

Reinforcement Learning is Different

The RL experiments were more positive than the supervised learning experiments. With policy gradient methods, the LoRA adapters matched full finetuning perfectly even with very low ranks (as low as r=1). The authors credit this to the low information density of RL feedback as the training signal is coming from a single episodic reward which is sparser when compared with richly supervised datasets. If the total bits of feedback you can extract are small, then a small adapter has more than enough capacity to memorize the needed adjustments.

Bottom Line

• LoRA delivers near-full-finetuning quality on mainstream post-training tasks while slashing memory and compute costs.
• Cover every major weight matrix—especially MLP and MoE layers—and nudge the learning rate up by about an order of magnitude relative to full finetuning.
• Expect it to be the default choice for instruction tuning, domain specialization, and even RL setups where the signal is sparse.

Citation

• Thinking Machines. (2025). “LoRA without Regret.” https://thinkingmachines.ai/blog/lora/
• Wang, S., Asilis, J., Akgül, Ö. F., Bilgin, E. B., Liu, O., & Neiswanger, W. (2025). “Tina: Tiny reasoning models via lora”.

Highlights

• Real-time CartPole visualisation that runs fully in the browser (no installs).
• Manual keyboard/touch nudges for quick robustness checks.
• Fast turnaround from training to deployment: export a policy, reload the demo, and compare behaviours side by side.
• Designed as a teaching aid for policy-gradient intuition and failure analysis.

Try It Live

Explore the Project

The full source—frontend simulator, training environment, and ONNX export utilities—is available on GitHub. The README walks through setup, training recipes, and deployment tips.

• Code & documentation: github.com/eren-cetin/balancing-bot
• Issues and pull requests welcome if you build new policies or extend the training loop.

1. REINFORCE on CartPole

The policy shipped with the demo is trained using REINFORCE, the vanilla Monte-Carlo policy-gradient method. The objective is to maximise expected return

where a trajectory $\tau$ is a sequence of states $s_t$, actions $a_t$, and rewards $r_t$ collected by rolling out the policy $\pi_\theta(a \mid s)$ in the environment. CartPole provides a reward of $+1$ for every timestep the pole stays balanced, so maximising $J(\theta)$ corresponds to keeping the pole upright as long as possible.

REINFORCE estimates the gradient of this objective with Monte-Carlo returns. For each timestep $t$ in an episode we compute the discounted return

and accumulate the score-function gradient

where $N$ is the number of sampled episodes in a batch and $T_i$ the length of episode $i$. We use $\gamma = 0.99$ to gently prioritise early stability without ignoring late-episode mistakes.

To reduce gradient variance we subtract a baseline $b_t$ (an exponential moving average of returns) before applying the update:

with learning rate $\alpha = 1\times10^{-3}$. In practice we fit a small two-layer MLP policy that outputs the logits for the two CartPole actions (left, right). Episodes terminate when the pole falls or a 500-step cap is reached; stable policies regularly hit the cap.

Training loop at a glance

1. Roll out $N$ CartPole episodes using the current policy.
2. Compute returns $G_t$ and baseline-corrected advantages $A_t = G_t - b_t$.
3. Accumulate $\nabla_\theta \log \pi_\theta(a_t \mid s_t)$ weighted by $A_t$.
4. Apply the gradient step and update the baseline estimate.
5. Repeat until the moving-average return surpasses the 500-step ceiling.

The resulting policy checkpoint is exported to ONNX and bundled with the demo. You can swap in your own checkpoints to compare alternative reward shaping or architecture tweaks side by side.

Technical Stack

• Three.js: 3D graphics library for WebGL rendering
• TypeScript: Type-safe development
• gsplat: Specialized library for Gaussian splatting visualization
• Vite: Modern build tool for fast development

Features

• Interactive orbit controls for scene navigation
• Real-time rendering of 3D Gaussian splat data
• WebGL-accelerated performance
• Responsive design that adapts to different screen sizes
• Support for standard .splat file formats

Live Demo

Experience the interactive 3D Gaussian splat viewer directly in your browser: