We prove several removable singularity theorems for singular Yang-Mills connections on bundles over Riemannian manifolds of dimensions greater than four. We obtain the local and global removability of singularities for Yang-Mills connections with $L^{\infty}$ or $L^{\frac{n}{2}}$ bounds on their curvature tensors, with weaker assumptions in the $L^{\infty}$ case and stronger assumptions in the $L^{\frac{n}{2}}$ case. With the global gauge construction methods we developed, we also obtain a `stability' result which asserts that the existence of a connection with uniformly small curvature tensor implies that the underlying bundle must be isomorphic to a flat bundle.