An algebra of matrices $\A$ with Jacobson radical $\R$ is said to have permutable trace if $\Tr (abc)=\Tr (bac)$ for all $a,b,c$ in $\A$. We show in this paper that in characteristic zero $\A$ has permutable trace if and only if $\A/\R$ is commutative. Generalizing to arbitrary characteristic we find that the result still holds when the trace form of $\A$ is non-degenerate. Finally, in positive characteristic, slightly stronger condition of permutability of the Brauer character is shown to be equivalent to the commutativity of $\A/\R$.