Mean-field VI on a banana

What you are seeing: the simplest variational inference setup, fit to the simplest distribution that breaks it. Target: a Rosenbrock-style "banana" $\log p(x, y) = -((1 - x)^2 + 100 (y - x^2)^2)/20$, a long curved valley. Variational family: a product $q(x, y) = N(\mu_x, \sigma_x^2)\,N(\mu_y, \sigma_y^2)$, axis-aligned Gaussian (mean-field). VI maximizes $\text{ELBO} = E_q[\log p] + H(q)$ by gradient ascent.

The contour lines are the target $\log p$. The orange ellipse is the current $q$. The yellow trace below is the ELBO over training. The "best" Gaussian approximation cannot follow the banana's curvature; it settles into a compact ellipse at the bend.