Constraint-Aware Diffusion Guidance for Robotics: Real-Time Obstacle Avoidance for Autonomous Racing

Let $x_{0}\in\mathbb{R}^{d}$ denote a noise-free data sample drawn from the data distribution $p_{0}(x)$. A score-based generative model defines a continuous-time diffusion process $\{x_{t}\}_{t\in[0,T]}$, where $t$ denotes the diffusion time, such that $x_{T}$ becomes approximately Gaussian. It is important to note that throughout this paper, we encounter two notions of “time”: here, $t$ refers to the artificial diffusion time governing the processes, while later, $\tau$ will denote the physical time in real-world dynamical systems.

Before introducing the proposed CoDiG framework, we specify the functional forms of the drift and diffusion terms in (1) for concreteness and clarity. It is important to emphasize that the proposed framework does not rely on these specific choices - the following definitions are adopted purely for illustrative purposes and to remain consistent with the experimental setup described later.

We let $\mu=0$, and define the drift term and the diffusion term as

which yields the so-called variance preserving SDE [26], where $g\left(t\right)^{2}=2\beta\left(t\right)$ holds for all $t\in\left[0,T\right]$ such that the marginal variance of $x_{t}$ is preserved over time. This specific choice ensures that the forward process remains stable and tractable for training and sampling, while still allowing for an expressive and well-defined reverse-time generative process. Under this formulation, when the terminal time $T$ is sufficiently large, the forward diffusion process described by (1) converges to a standard Gaussian distribution. As analyzed in Song et al. 2021, the term $\sqrt{2\beta(t)}$ should grow with time, requiring $\beta(t)$ to be strictly increasing.

For simplicity and numerical stability, we normalize the diffusion process to $t\in\left[0,1\right]$. To ensure convergence to a standard Gaussian, the diffusion term $\sqrt{2\beta\left(t\right)}$ must grow rapidly within this interval. In our implementation, we model $\beta\left(t\right)$ as a quadratic function, $\beta\left(t\right)=r_{1}t^{2}+r_{0}$, with parameters detailed in Appendix C.3. In this case, (3) can be reformulated as:

Next, we consider the marginal distribution $p_{t}\left(x_{t}\right)$, which represents the probability distribution of a sample at an intermediate time step, in the absence of constraints. Before incorporating constraints into this distribution, we first introduce the following definitions. Let $c:\mathbb{R}^{d}\times\left[0,1\right]\rightarrow\mathbb{R}^{k}$ denote a time-dependent constraint function, encoding the safety or feasibility requirements of the system. We define the feasible region at time $\tau\geq 0$ as

where the inequality is interpreted element-wise. Naturally, when constraints are introduced, the distribution of interest becomes the conditional distribution:

These constraints may encode different forms of feasibility or safety requirements, depending on the task setting. For example, in autonomous racing, $\mathcal{C}_{\tau}$ refers to the obstacle-free, drivable region of a racing track. While in diffusion-based control policies, $\mathcal{C}_{\tau}$ must account for system dynamics, as the use of visual feedback necessitates control that respects the underlying physical constraints of the system. Here, we use the time subscript $\tau$ to emphasize that such constraints can be time-varying, which is often the case in dynamic or interactive environments. For simplicity, and without loss of clarity, we will omit this subscript in the following when no confusion arises.

Several existing methods attempt to directly model the marginal distribution $p_{t}\left(x_{t}\mid\mathcal{C}\right)$ by injecting the constraint representation into the diffusion model architecture [29]. While effective in big-data domains such as image synthesis, these approaches face significant limitations in the context of robotics:

To overcome these limitations, we leverage the known structure of the constraint $\mathcal{C}$ during sampling to dynamically guide the generation process. We propose an alternative formulation of the constrained distribution $p_{t}\left(x_{t}\mid\mathcal{C}\right)$, which does not require learning the conditional model directly from data:

where $Z_{t}:=\int_{\mathbb{R}^{d}}p_{t}\left(x\right)e^{-\gamma_{t}V\left(x,~\mathcal{C}\right)}~\mathrm{d}x$ is a normalization constant. The barrier function $V:\mathbb{R}^{d}\rightarrow\mathbb{R}_{+}$ assigns large values to infeasible data points, while remaining close to zero within the feasible region. Intuitively, applying the barrier function pushes the probability of infeasible data points toward zero. As a result, the constrained distribution focuses its support almost entirely on the feasible region. We substitute the above formula into the score function and get

where the normalization constant $Z_{t}$ vanishes when taking the gradient of the log-probability, and hence does not affect the reverse-time dynamics. By substituting the right-hand side of (5) into (4), we obtain the modified reverse SDE that incorporates constraint information:

where a constant $\eta\in\left[0,1\right]$ is introduced to accelerate convergence and enhance the stability of the sampling process [30]. We observe that the first term on the right-hand side of (5) corresponds exactly to the unconstrained score function defined in (4). This term can still be approximated by the neural network $s_{\theta}\left(x_{t},t\right)$ trained without considering any constraints. Crucially, the effect of the constraint appears only during the denoising process, in an explicit gradient-based form - as an additive term derived from the constraint potential (e.g., a barrier function). This formulation significantly reduces the need for large amounts of constraint-compliant training data, as the constraint is not encoded in the network itself but instead injected at inference time. Moreover, because the constraint enters the reverse SDE as a differentiable time-varying potential, the framework can naturally accommodate dynamic, time-varying constraints.

It is important to note, as pointed out by Bastek et al. 2025, that applying constraints to data that is close to pure noise in diffusion models is not meaningful. Therefore, during the denoising process - i.e., as $t$ decreases from one to zero - we gradually increase the value of $\gamma_{t}$ starting from zero at $t=1$. This progressive scheduling is crucial for ensuring the stability of the denoising process. The specific design of $\gamma_{t}$ is detailed in Appendix A.

For the considered application, the barrier function is designed as follows:

where the symbol $\mathds{1}\left\{\cdot\right\}$ denotes the indicator function, and the subscript $\left(\cdot\right)_{\text{nominal}}$ refers to the time-optimal solution computed offline in the absence of obstacles, which serves as a reliable reference. Here, $N$ represents the number of discrete points obtained by uniformly sampling along the track center line. In our setting, $\hat{y}$ denotes the lateral displacement and $\widehat{\phi}$ represents the yaw angle in the Frenet coordinate system (see Appendix C.1). The feasible region $\mathcal{C}_{k}$ is also defined in the Frenet frame, capturing the obstacle-free area at each sampled position along the track.

In (7), the first part accounts for time-varying obstacles by guiding the sampling process toward safe, obstacle-free regions. The second part addresses the missing curvature information of the track during training, which is intentionally discarded when transforming trajectories into the local Frenet coordinate system. Beyond promoting near time-optimality without requiring global geometric knowledge of the track, this part also facilitates the generation of dynamically feasible trajectories. The local representation ensures that the resulting motions adhere more closely to the physical and kinematic constraints of the system. The positive constants $\alpha$ and $\epsilon$ are tunable hyperparameters that balance the importance of the two components. Specifically, $\alpha$ modulates the influence of physical safety constraints, while $\epsilon$ regulates the adherence to nominal time-optimality.

It is worth noting that the design of the barrier function is not unique and can be tailored to the specific task. While such customization may require a small amount of tuning effort, it is negligible compared to the cost of collecting expert demonstrations, especially in robotics domains where data is expensive. This makes our framework both flexible and data-efficient.

We train the diffusion model as described in Appendix C.2 for $500$ epochs. During inference, we applied the Euler-Maruyama discretization (8), which corresponds to the discretized version of (6). The denoising process proceeds from $t=1.0$ to $t=0.0$ in $1000$ steps, gradually transforming samples from noise to data. The results are shown in Fig. 1. Fig. 1(a) illustrates the denoising process without using the barrier function, while Fig. 1(b) shows the effect of the proposed constraint-aware guided generation. Each row depicts intermediate generation results at $t=$1\text{\,}\mathrm{s}$,~$0.591\text{\,}\mathrm{s}$,~$0.002\text{\,}\mathrm{s}$$, from left to right. The black points and arrows indicate the evolving trajectories in the $z$-$y$ plane. See Appendix A for the concrete values of the hyperparameters used during inference.

In Fig. 1(a), we observe that the diffusion model learns important features of the data distribution. It respects the fundamental constraints of the racetrack, such as staying within bounds and satisfying the loop closure. However, despite conditioning on constraints during training, the generated samples still fail to consistently avoid obstacles - primarily due to limited training data. Additionally, the lack of curvature information in the local Frenet frame leads to unrealistic and physically implausible results.

In contrast, Fig. 1(b) incorporates the barrier function as described in Sec. 4.1. The guidance significantly improves the sampling process. The model denoises faster (i.e., the samples become structured earlier), the trajectories successfully avoid all obstacles, and the resulting motion aligns well with a physically consistent motion. Moreover, due to the second part in (7) (albeit with a small weight $\epsilon$), the final trajectories closely follow the nominal time-optimal ones, exhibiting near time-optimal properties. For more details on the near time-optimality of the generated trajectories, please refer to Appendix E.

Real-time obstacle avoidance requires not only safe trajectories but also fast replanning. As shown in Sec. 4.2, our diffusion model with a barrier function produces high-quality trajectories after $1000$ denoising steps, but this results in a low sampling frequency of $0.25\text{\,}\mathrm{Hz}$, which is insufficient for real-time racing.

While various acceleration techniques exist [33, 11, 34], we propose a warm-start strategy tailored to robotic control. Unlike standard diffusion generation, which samples each trajectory from pure noise, our proposed warm-start technology perturbs the previous output with small noise and reuses it as the next input. This maintains temporal consistency, reduces trajectory variance, and improves control stability [35]. By promoting smooth transitions between consecutive trajectories, warm-starting significantly lowers the number of denoising steps required and enhances real-time feasibility. A detailed analysis and comparison of results with and without warm-starting are provided in Appendix D.

Through the integration of our warm-start technique, we achieve a sampling frequency of $2.5\text{\,}\mathrm{Hz}$ on a computer equipped with an NVIDIA RTX 4090 GPU. While there is still potential for further acceleration, this performance already satisfies the real-time requirements of obstacle avoidance in racing scenarios.

We successfully deployed CoDiG on our real-world autonomous racing platform for real-time trajectory planning. A tracking model predictive control (TMPC) [36, 37] is employed to follow the planned trajectories. Notably, the TMPC operates without any knowledge of obstacles, relying solely on the reference trajectories for control. Fig. 2 illustrates two representative obstacle avoidance maneuvers during autonomous driving. In both Fig. 2(a) and Fig. 2(b), the red lines represent the trajectories planned by CoDiG, the gray circles denote static obstacles, while the black circles indicate dynamic obstacles. The black dashed lines show the predicted trajectories generated by the TMPC as it attempts to follow the red reference trajectory. Each sequence from left to right captures a complete avoidance cycle:

As shown in the figures, the predicted trajectories from the TMPC closely align with the reference trajectories generated by the diffusion model. This highlights that the planned trajectories are closely aligned with physical feasibility, enabled by the barrier function, which is crucial for effective tracking performance. Additionally, even in the presence of obstacles, the generated trajectories maintain near time-optimality, indicating that the planner does not overly sacrifice efficiency for safety.

Finally, thanks to the warm-start strategy, significant replanning is only triggered when the obstacle actually interferes with the current path. In static conditions, consecutive trajectories remain almost unchanged, ensuring system stability. We conducted five experimental trials, each consisting of $15$ racing laps, across ten different obstacle configurations. The framework achieved a $100\text{\,}\mathrm{\%}$ success rate in obstacle avoidance, demonstrating its robustness and reliability in diverse scenarios.