What if we relax these assumptions even further? Let's stop requiring separable action. This leaves q(s)T A(t-s) k(t), where A is an arbitrary translation-invariant function that outputs a square matrix. This means we no longer require any group structure in the output space.

To test, we'll simply take a Gaussian: A(t-s) = \exp(-(t-s)^2 / \sigma^2) I. Our embedding function becomes equivalent to a monotonically decreasing scalar. A quick check shows that this function performs worse than RoPE but still beats the controls. (4/7)