Kernel Stein discrepancies (KSDs) measure the quality of a
distributional approximation and can be computed even when the target
density has an intractable normalizing constant. Notable applications
include the diagnosis of approximate MCMC samplers and goodness-of-fit
tests for unnormalized statistical models. The present work analyzes
the convergence control properties of KSDs. We first show that
standard KSDs used for weak convergence control fail to control moment
convergence. To address this limitation, we next provide sufficient
conditions under which alternative diffusion KSDs control both moment
and weak convergence. As an immediate consequence we develop, for each
q>0, the first KSDs known to exactly characterize q-Wasserstein
convergence.