Forced Variational Integrator Networks with Unit Quaternions
Report | Slides | GitHub repository
From mechanics to network architecture
This course project begins from the existing LieFVIN framework and its accompanying implementation. My aim was first to understand the underlying discrete mechanics, and then to explore a unit-quaternion extension and a sign-invariance constraint. Deep models can approximate complicated dynamics, but physical structure is especially valuable when data are scarce. Variational Integrator Networks (VINs) encode a geometric numerical integrator directly into the architecture. For example, consider the parameterized Lagrangian of a separable Newtonian system, \[ L_\theta(q, \dot{q}) = \frac{1}{2} \dot{q}^\top M_\theta \dot{q} - U_\theta(q) \] for which the velocity-Verlet method gives the explicit update
\[\displaylines{ q_{k+1} = q_k + h \dot{q}_k - \frac{h^2}{2} M_\theta^{-1} \nabla U_\theta(q_k) \\\ \dot{q}_{k+1} = \dot{q}_k - h M_\theta^{-1} \left( \frac{\nabla U_\theta(q_k) + \nabla U_\theta(q_{k+1})}{2} \right) }\]This update can be interpreted as a feed-forward computation graph. More generally, one may define parameterized Lagrangians on manifolds, including matrix Lie groups, and differentiate through the corresponding variational integrator. Discrete variational mechanics therefore supplies a principled way to encode geometric structure in a learned dynamics model.
Forced systems and unit quaternions
Robotic systems also contain damping and control inputs. Forced Variational Integrator Networks (FVINs) extend VINs by learning these external forces. Forced variational integrators are generally neither symplectic nor momentum preserving; the external forcing breaks the corresponding conservation laws. They remain useful because the unforced part of the dynamics still provides a strong structural prior.
The implementation extends LieFVINs from $\mathrm{SO}(3)$-based rotation matrices to the unit-quaternion group $S^3$. Unit quaternions use four coordinates instead of nine, but they double-cover rotations: $q$ and $-q$ represent the same physical attitude. I therefore enforce sign invariance in every learned component,
\[J(q)=J(-q),\qquad U(q)=U(-q),\qquad F^{\pm}(q,u)=F^{\pm}(-q,u).\]Without this constraint, the network may assign different inertia, potential, or forcing to the same rotation. In the planar-pendulum experiment, the sign-invariant model trained more stably and reached a comparable loss in roughly half as many iterations as the rotation-matrix baseline.
A useful hybrid
I also tested a hybrid that retains the $\mathrm{SO}(3)$ state and its efficient Cayley-transform update, but converts each rotation matrix internally through $R\mapsto q$ before applying the learned components. Its convergence was close to that of the direct $S^3$ model while preserving the faster per-iteration update of the original implementation. The main lesson was not simply that quaternions are “better,” but that a compact representation can be introduced inside the model while keeping the numerical structure that makes another representation convenient.