What is the gradient of a scalar function of a symmetric matrix?

For a real valued function $$\phi $$ ϕ of a matrix argument, the gradient $$\nabla \phi $$ ∇ ϕ is calculated using a standard approach that follows from the definition of a Fréchet derivative for matrix functionals. In cases where the matrix argument is restricted to the space of symmetric matrices, the approach is easily modified to determine that the gradient ought to be $$(\nabla \phi + \nabla \phi ^T)/2$$ ( ∇ ϕ + ∇ ϕ T ) / 2 . However, perusal of research articles in the statistics and electrical engineering communities that deal with the topic of matrix calculus reveal a different approach that leads to a spurious result. In this approach, the gradient of $$\phi $$ ϕ is evaluated by explicitly taking into account the symmetry of the matrix, and this “symmetric gradient" $$\nabla \phi _{sym}$$ ∇ ϕ sym is reported to be related to the gradient $$\nabla \phi $$ ∇ ϕ which is computed by ignoring symmetry as $$\nabla \phi _{sym}= \nabla \phi + \nabla \phi ^T - \nabla \phi \circ I$$ ∇ ϕ sym = ∇ ϕ + ∇ ϕ T - ∇ ϕ ∘ I , where $$\circ $$ ∘ denotes the elementwise Hadamard product of the two matrices and I the identity matrix of the same size as $$\nabla \phi $$ ∇ ϕ . The idea of the “symmetric gradient" has now appeared in several publications, as well as in textbooks and handbooks on matrix calculus which are often cited in this context. One of our important contributions has been to wade through the vague and confusing proofs of the result based on matrix calculus and cast the calculation of the “symmetric gradient” in a rigorous and concrete mathematical setting. After setting up the problem in a finite-dimensional inner-product space, we demonstrate rigorously that $$\nabla \phi _{sym}= (\nabla \phi + \nabla \phi ^T)/2$$ ∇ ϕ sym = ( ∇ ϕ + ∇ ϕ T ) / 2 is the correct relationship. Moreover, our derivation exposes that it is an incorrect lifting from the Euclidean space to the space of symmetric matrices, inconsistent with the underlying inner-product, that leads to the spurious result. We also discuss the implications of using the spurious gradient in different classes of problems, such as those where the gradient itself may be the quantity sought, or as part of an optimization algorithm such as gradient descent. We show that the spurious gradient has a relative error of 100% in the off-diagonal components, which makes it an egregious error if the gradient be a quantity of interest, but fortuitously, it proves to be an ascent direction, so that its use in gradient descent may not lead to major issues.